+ PROC_LSST_SUBMIT_PATH=/mmfs1/home/stevengs/dirac/DEEP/submit + bps submit /gscratch/dirac/shared/opt/proc_lsst/pipelines/submit.yaml -b /mmfs1/home/stevengs/dirac/DEEP/repo -i DEEP/20210513/A1c --output-run DEEP/20210513/A1c/science#step0/20240315T134820Z --qgraph pipeline.qgraph lsst.ctrl.bps.drivers INFO: DISCLAIMER: All values regarding memory consumption reported below are approximate and may not accurately reflect actual memory usage by the bps process. lsst.ctrl.bps.drivers INFO: Starting submission process lsst.ctrl.bps.drivers INFO: Initializing execution environment lsst.ctrl.bps.drivers INFO: Initializing execution environment completed: Took 7.7085 seconds; current memory usage: 0.178 Gibyte, delta: 0.041 Gibyte, peak delta: 0.054 Gibyte lsst.ctrl.bps.drivers INFO: Peak memory usage for bps process 0.190 Gibyte (main), 0.000 Gibyte (largest child process) lsst.ctrl.bps.drivers INFO: Starting acquire stage (generating and/or reading quantum graph) lsst.ctrl.bps.pre_transform INFO: Copying quantum graph from 'pipeline.qgraph' lsst.ctrl.bps.pre_transform INFO: Completed copying quantum graph: Took 0.0148 seconds lsst.ctrl.bps.pre_transform INFO: Backing up quantum graph from '/mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/pipeline.qgraph' lsst.ctrl.bps.pre_transform INFO: Completed backing up quantum graph: Took 0.0171 seconds lsst.ctrl.bps.pre_transform INFO: /mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/share/eups/Linux64/ctrl_mpexec/g1ce94f1343+c79f27626b/bin/pipetask --long-log --log-level=VERBOSE update-graph-run /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/pipeline_orig.qgraph DEEP/20210513/A1c/science#step0/20240315T134820Z /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/pipeline.qgraph lsst.ctrl.bps.pre_transform INFO: Reading quantum graph from '/mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/pipeline.qgraph' lsst.ctrl.bps.pre_transform INFO: Completed reading quantum graph: Took 5.9563 seconds lsst.ctrl.bps.drivers INFO: Acquire stage completed: Took 18.7024 seconds; current memory usage: 0.336 Gibyte, delta: 0.158 Gibyte, peak delta: 0.145 Gibyte lsst.ctrl.bps.drivers INFO: Peak memory usage for bps process 0.336 Gibyte (main), 0.332 Gibyte (largest child process) lsst.ctrl.bps.drivers INFO: Starting cluster stage (grouping quanta into jobs) lsst.ctrl.bps.drivers INFO: Cluster stage completed: Took 0.1185 seconds; current memory usage: 0.336 Gibyte, delta: 0.000 Gibyte, peak delta: 0.000 Gibyte lsst.ctrl.bps.drivers INFO: Peak memory usage for bps process 0.336 Gibyte (main), 0.332 Gibyte (largest child process) lsst.ctrl.bps.drivers INFO: ClusteredQuantumGraph contains 5084 cluster(s) lsst.ctrl.bps.drivers INFO: Starting transform stage (creating generic workflow) lsst.ctrl.bps.drivers INFO: Generic workflow name 'DEEP_20210513_A1c_science#step0_20240315T134820Z' lsst.ctrl.bps.drivers INFO: Transform stage completed: Took 0.9727 seconds; current memory usage: 0.336 Gibyte, delta: 0.000 Gibyte, peak delta: 0.000 Gibyte lsst.ctrl.bps.drivers INFO: Peak memory usage for bps process 0.336 Gibyte (main), 0.332 Gibyte (largest child process) lsst.ctrl.bps.drivers INFO: GenericWorkflow contains 5086 job(s) (including final) lsst.ctrl.bps.drivers INFO: Starting prepare stage (creating specific implementation of workflow) parsl.addresses ERROR: Ignoring failure to fetch address from interface eno2 Traceback (most recent call last): File "/mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/lib/python3.11/site-packages/parsl/addresses.py", line 111, in get_all_addresses s_addresses.add(address_by_interface(interface)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/lib/python3.11/site-packages/typeguard/__init__.py", line 1033, in wrapper retval = func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/lib/python3.11/site-packages/parsl/addresses.py", line 93, in address_by_interface return socket.inet_ntoa(fcntl.ioctl( ^^^^^^^^^^^^ OSError: [Errno 99] Cannot assign requested address lsst.ctrl.bps.parsl INFO: Writing workflow with ID=/mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z lsst.ctrl.bps.drivers INFO: Prepare stage completed: Took 1.2510 seconds; current memory usage: 0.341 Gibyte, delta: 0.005 Gibyte, peak delta: 0.005 Gibyte lsst.ctrl.bps.drivers INFO: Peak memory usage for bps process 0.341 Gibyte (main), 0.332 Gibyte (largest child process) lsst.ctrl.bps.drivers INFO: Starting submit stage lsst.ctrl.bps.submit INFO: Submitting run to a workflow management system for execution parsl.dataflow.rundirs DEBUG: Parsl run initializing in rundir: runinfo/000 parsl.dataflow.dflow INFO: Starting DataFlowKernel with config Config( app_cache=True, checkpoint_files=None, checkpoint_mode='task_exit', checkpoint_period=None, executors=[MultiHighThroughputExecutor()], garbage_collect=True, initialize_logging=True, internal_tasks_max_threads=10, max_idletime=120.0, monitoring=None, retries=1, retry_handler=None, run_dir='runinfo', strategy='simple', usage_tracking=False ) parsl.dataflow.dflow INFO: Parsl version: 2023.06.12 parsl.usage_tracking.usage DEBUG: Tracking status: False parsl.dataflow.dflow INFO: Run id is: 5d18103b-d05e-431d-94f4-3bf9638ba436 parsl.dataflow.dflow DEBUG: Considering candidate for workflow name: /mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/lib/python3.11/site-packages/parsl/dataflow/dflow.py parsl.dataflow.dflow DEBUG: Considering candidate for workflow name: /mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/lib/python3.11/site-packages/typeguard/__init__.py parsl.dataflow.dflow DEBUG: Considering candidate for workflow name: /mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/lib/python3.11/site-packages/parsl/dataflow/dflow.py parsl.dataflow.dflow DEBUG: Considering candidate for workflow name: /mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/lib/python3.11/site-packages/typeguard/__init__.py parsl.dataflow.dflow DEBUG: Considering candidate for workflow name: /mmfs1/gscratch/dirac/shared/opt/conda/envs/lsst-scipipe-8.0.0/share/eups/Linux64/ctrl_bps_parsl/g145af14111+5b908e21bc/python/lsst/ctrl/bps/parsl/workflow.py parsl.dataflow.dflow DEBUG: Using workflow.py as workflow name parsl.dataflow.memoization INFO: App caching initialized parsl.dataflow.strategy DEBUG: Scaling strategy: simple parsl.executors.high_throughput.executor DEBUG: Starting queue management thread parsl.executors.high_throughput.executor DEBUG: queue management worker starting parsl.executors.high_throughput.executor DEBUG: Started queue management thread Submit dir: /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z parsl.executors.high_throughput.executor DEBUG: Created management thread: parsl.executors.high_throughput.executor DEBUG: Launch command: process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id={block_id} --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn parsl.executors.high_throughput.executor DEBUG: Starting HighThroughputExecutor with provider: parsl.executors.status_handling INFO: Scaling out by 1 blocks parsl.executors.status_handling INFO: Allocated block ID 0 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-0 2024-03-15 13:49:23 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=0 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-0 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=0 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-0 2024-03-15 13:49:23 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=0 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=0 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 0 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 0 2024-03-15 13:49:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:23 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=0 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 to local proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=0 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 to local parsl.providers.local.local DEBUG: Launching in remote mode 2024-03-15 13:49:23 proc_lsst.multi:170 [INFO] [multi] job_id 91028 proc_lsst.multi INFO: [multi] job_id 91028 2024-03-15 13:49:23 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:23 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:23 proc_lsst.multi:178 [INFO] [multi] provider local accepted submit and returned 91028 proc_lsst.multi INFO: [multi] provider local accepted submit and returned 91028 parsl.executors.status_handling DEBUG: Launched block 0 on executor multi with job ID 91028 parsl.dataflow.job_status_poller DEBUG: Adding executor multi parsl.dataflow.dflow DEBUG: Task 0 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 0 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 0 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 0 has memoization hash 3f870af3d45c9759f4928706318ab496 parsl.dataflow.memoization INFO: Task 0 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee13880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 0 try 0 launched on executor multi with executor id 1 parsl.dataflow.dflow INFO: Standard output for task 0 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8faee67f-132b-42f2-b22f-6f323b61684f_isrForCrosstalkSources_997027_36.stdout parsl.dataflow.dflow INFO: Standard error for task 0 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8faee67f-132b-42f2-b22f-6f323b61684f_isrForCrosstalkSources_997027_36.stderr parsl.dataflow.dflow DEBUG: Task 1 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1 has memoization hash 2285f91fdc50e60cb79b857fd7b01b32 parsl.dataflow.memoization INFO: Task 1 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee13c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1 try 0 launched on executor multi with executor id 2 parsl.dataflow.dflow INFO: Standard output for task 1 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/f9c828f8-1533-49bf-bbf4-31f6b2e3b1cd_isrForCrosstalkSources_997033_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/f9c828f8-1533-49bf-bbf4-31f6b2e3b1cd_isrForCrosstalkSources_997033_14.stderr parsl.dataflow.dflow DEBUG: Task 2 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2 has memoization hash c4387cc0859726aff11e4f26e92e4892 parsl.dataflow.memoization INFO: Task 2 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee13ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2 try 0 launched on executor multi with executor id 3 parsl.dataflow.dflow INFO: Standard output for task 2 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/70f8f7df-bc1d-47fe-aa8d-917f01994001_isrForCrosstalkSources_997032_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/70f8f7df-bc1d-47fe-aa8d-917f01994001_isrForCrosstalkSources_997032_2.stderr parsl.dataflow.dflow DEBUG: Task 3 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3 has memoization hash a9a61dd054367440a832a3ad9bca7af5 parsl.dataflow.memoization INFO: Task 3 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee13d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3 try 0 launched on executor multi with executor id 4 parsl.dataflow.dflow INFO: Standard output for task 3 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b96ee8cf-2a1f-442f-90ec-4e9916c5de70_isrForCrosstalkSources_997001_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b96ee8cf-2a1f-442f-90ec-4e9916c5de70_isrForCrosstalkSources_997001_35.stderr parsl.dataflow.dflow DEBUG: Task 4 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4 has memoization hash 3216ca8a9b19aed1201d12bd12c14c8f parsl.dataflow.memoization INFO: Task 4 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee13f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4 try 0 launched on executor multi with executor id 5 parsl.dataflow.dflow INFO: Standard output for task 4 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/91141d3f-a9b7-4a43-a117-94fd85d92ddb_isrForCrosstalkSources_996995_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/91141d3f-a9b7-4a43-a117-94fd85d92ddb_isrForCrosstalkSources_996995_46.stderr parsl.dataflow.dflow DEBUG: Task 5 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5 has memoization hash 0c287647f8836234b28715b4d7f8aa82 parsl.dataflow.memoization INFO: Task 5 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5 try 0 launched on executor multi with executor id 6 parsl.dataflow.dflow INFO: Standard output for task 5 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/cbc2ed89-621f-475b-b998-9c02920f4515_isrForCrosstalkSources_996965_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/cbc2ed89-621f-475b-b998-9c02920f4515_isrForCrosstalkSources_996965_46.stderr parsl.dataflow.dflow DEBUG: Task 6 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 6 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 6 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 6 has memoization hash 8cd92b448567b79e1586dd8bb650a318 parsl.dataflow.memoization INFO: Task 6 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee13ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 6 try 0 launched on executor multi with executor id 7 parsl.dataflow.dflow INFO: Standard output for task 6 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/170248f1-9a17-471a-a1cf-8412f367c74b_isrForCrosstalkSources_996965_56.stdout parsl.dataflow.dflow INFO: Standard error for task 6 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/170248f1-9a17-471a-a1cf-8412f367c74b_isrForCrosstalkSources_996965_56.stderr parsl.dataflow.dflow DEBUG: Task 7 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 7 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 7 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 7 has memoization hash 46ed9a2e159e063f3f5254380ee15ad0 parsl.dataflow.memoization INFO: Task 7 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee13e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 7 try 0 launched on executor multi with executor id 8 parsl.dataflow.dflow INFO: Standard output for task 7 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5d164c49-eb09-42fa-9ca6-d58f5228685b_isrForCrosstalkSources_997004_54.stdout parsl.dataflow.dflow INFO: Standard error for task 7 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5d164c49-eb09-42fa-9ca6-d58f5228685b_isrForCrosstalkSources_997004_54.stderr parsl.dataflow.dflow DEBUG: Task 8 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 8 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 8 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 8 has memoization hash e1df712d18da8692d9ea9245a6a69b77 parsl.dataflow.memoization INFO: Task 8 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee131a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 8 try 0 launched on executor multi with executor id 9 parsl.dataflow.dflow INFO: Standard output for task 8 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/26cfef11-c7f6-4152-b2b1-94753ce475ad_isrForCrosstalkSources_996992_1.stdout parsl.dataflow.dflow INFO: Standard error for task 8 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/26cfef11-c7f6-4152-b2b1-94753ce475ad_isrForCrosstalkSources_996992_1.stderr parsl.dataflow.dflow DEBUG: Task 9 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 9 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 9 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 9 has memoization hash b48068ecbca8b669488fafb11c857245 parsl.dataflow.memoization INFO: Task 9 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ee139c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 9 try 0 launched on executor multi with executor id 10 parsl.dataflow.dflow INFO: Standard output for task 9 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/917da520-a20f-49da-b8ad-bf5d15f9315e_isrForCrosstalkSources_996982_59.stdout parsl.dataflow.dflow INFO: Standard error for task 9 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/917da520-a20f-49da-b8ad-bf5d15f9315e_isrForCrosstalkSources_996982_59.stderr parsl.dataflow.dflow DEBUG: Task 10 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 10 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 10 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 10 has memoization hash 7034bfd87ee59cbb0819e7a07c3d51bd parsl.dataflow.memoization INFO: Task 10 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 10 try 0 launched on executor multi with executor id 11 parsl.dataflow.dflow INFO: Standard output for task 10 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5bcf6bcc-0b38-41c1-abb1-fb4a31357258_isrForCrosstalkSources_997017_12.stdout parsl.dataflow.dflow INFO: Standard error for task 10 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5bcf6bcc-0b38-41c1-abb1-fb4a31357258_isrForCrosstalkSources_997017_12.stderr parsl.dataflow.dflow DEBUG: Task 11 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 11 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 11 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 11 has memoization hash da51ccefdcedfecca7598ea1960be85a parsl.dataflow.memoization INFO: Task 11 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 11 try 0 launched on executor multi with executor id 12 parsl.dataflow.dflow INFO: Standard output for task 11 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/bb67a28d-d110-4cb7-bf72-78298de41a92_isrForCrosstalkSources_996969_34.stdout parsl.dataflow.dflow INFO: Standard error for task 11 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/bb67a28d-d110-4cb7-bf72-78298de41a92_isrForCrosstalkSources_996969_34.stderr parsl.dataflow.dflow DEBUG: Task 12 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 12 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 12 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 12 has memoization hash 4c26be93964f79c76d6e1daf03d9e51e parsl.dataflow.memoization INFO: Task 12 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6104a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 12 try 0 launched on executor multi with executor id 13 parsl.dataflow.dflow INFO: Standard output for task 12 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/111f589a-79d9-4da6-9f23-946436de9347_isrForCrosstalkSources_996956_62.stdout parsl.dataflow.dflow INFO: Standard error for task 12 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/111f589a-79d9-4da6-9f23-946436de9347_isrForCrosstalkSources_996956_62.stderr parsl.dataflow.dflow DEBUG: Task 13 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 13 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 13 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 13 has memoization hash 12f23acd5743b465efce4ba908d7a22c parsl.dataflow.memoization INFO: Task 13 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6107c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 13 try 0 launched on executor multi with executor id 14 parsl.dataflow.dflow INFO: Standard output for task 13 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/32afa545-8f66-4faa-a15e-c67b94a00320_isrForCrosstalkSources_996975_1.stdout parsl.dataflow.dflow INFO: Standard error for task 13 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/32afa545-8f66-4faa-a15e-c67b94a00320_isrForCrosstalkSources_996975_1.stderr parsl.dataflow.dflow DEBUG: Task 14 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 14 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 14 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 14 has memoization hash 81f903f0c937ca949caabf359162dbca parsl.dataflow.memoization INFO: Task 14 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6109a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 14 try 0 launched on executor multi with executor id 15 parsl.dataflow.dflow INFO: Standard output for task 14 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/5c4d2d03-e29c-4199-baea-e35f0f3ef3a2_isrForCrosstalkSources_996967_23.stdout parsl.dataflow.dflow INFO: Standard error for task 14 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/5c4d2d03-e29c-4199-baea-e35f0f3ef3a2_isrForCrosstalkSources_996967_23.stderr parsl.dataflow.dflow DEBUG: Task 15 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 15 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 15 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 15 has memoization hash c2e523bf3981ebc95272d5ce140c88bb parsl.dataflow.memoization INFO: Task 15 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 15 try 0 launched on executor multi with executor id 16 parsl.dataflow.dflow INFO: Standard output for task 15 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/447f0501-20b3-40a3-9d03-5d4d5e5bfb59_isrForCrosstalkSources_997006_9.stdout parsl.dataflow.dflow INFO: Standard error for task 15 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/447f0501-20b3-40a3-9d03-5d4d5e5bfb59_isrForCrosstalkSources_997006_9.stderr parsl.dataflow.dflow DEBUG: Task 16 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 16 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 16 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 16 has memoization hash f14e8840306656adc97ff5dfbe81f780 parsl.dataflow.memoization INFO: Task 16 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 16 try 0 launched on executor multi with executor id 17 parsl.dataflow.dflow INFO: Standard output for task 16 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9f4c1d8b-91e9-40e0-a979-d4a1359a795f_isrForCrosstalkSources_996995_41.stdout parsl.dataflow.dflow INFO: Standard error for task 16 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9f4c1d8b-91e9-40e0-a979-d4a1359a795f_isrForCrosstalkSources_996995_41.stderr parsl.dataflow.dflow DEBUG: Task 17 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 17 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 17 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 17 has memoization hash cb424b1bb4808962c6e309202b9d6ee3 parsl.dataflow.memoization INFO: Task 17 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 17 try 0 launched on executor multi with executor id 18 parsl.dataflow.dflow INFO: Standard output for task 17 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/3b03cf9c-14a6-4098-87e5-9d2cbacb7bbc_isrForCrosstalkSources_996981_39.stdout parsl.dataflow.dflow INFO: Standard error for task 17 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/3b03cf9c-14a6-4098-87e5-9d2cbacb7bbc_isrForCrosstalkSources_996981_39.stderr parsl.dataflow.dflow DEBUG: Task 18 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 18 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 18 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 18 has memoization hash 474ac1c7bb5b44420c1c17851d9a5add parsl.dataflow.memoization INFO: Task 18 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 18 try 0 launched on executor multi with executor id 19 parsl.dataflow.dflow INFO: Standard output for task 18 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/08b2b231-2f60-4566-b2ff-0373f2ffd571_isrForCrosstalkSources_997032_15.stdout parsl.dataflow.dflow INFO: Standard error for task 18 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/08b2b231-2f60-4566-b2ff-0373f2ffd571_isrForCrosstalkSources_997032_15.stderr parsl.dataflow.dflow DEBUG: Task 19 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 19 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 19 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 19 has memoization hash db74ae9f0a876970c23c390c07b9ee54 parsl.dataflow.memoization INFO: Task 19 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 19 try 0 launched on executor multi with executor id 20 parsl.dataflow.dflow INFO: Standard output for task 19 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/889f548b-d883-45a7-80dc-af175f69d35c_isrForCrosstalkSources_996997_16.stdout parsl.dataflow.dflow INFO: Standard error for task 19 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/889f548b-d883-45a7-80dc-af175f69d35c_isrForCrosstalkSources_996997_16.stderr parsl.dataflow.dflow DEBUG: Task 20 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 20 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 20 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 20 has memoization hash 0fef4e256784c12e0df787d0a3063f9d parsl.dataflow.memoization INFO: Task 20 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 20 try 0 launched on executor multi with executor id 21 parsl.dataflow.dflow INFO: Standard output for task 20 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/13e7e8e5-8559-44ed-bf58-8d913741d954_isrForCrosstalkSources_997016_39.stdout parsl.dataflow.dflow INFO: Standard error for task 20 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/13e7e8e5-8559-44ed-bf58-8d913741d954_isrForCrosstalkSources_997016_39.stderr parsl.dataflow.dflow DEBUG: Task 21 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 21 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 21 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 21 has memoization hash d442684d45ccb25cc306cdcedcc16877 parsl.dataflow.memoization INFO: Task 21 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 21 try 0 launched on executor multi with executor id 22 parsl.dataflow.dflow INFO: Standard output for task 21 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/bf2640a4-a504-4aa6-b5ec-0f0fd2d42b63_isrForCrosstalkSources_997011_21.stdout parsl.dataflow.dflow INFO: Standard error for task 21 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/bf2640a4-a504-4aa6-b5ec-0f0fd2d42b63_isrForCrosstalkSources_997011_21.stderr parsl.dataflow.dflow DEBUG: Task 22 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 22 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 22 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 22 has memoization hash de687cddcffba15a565a147fe345b35e parsl.dataflow.memoization INFO: Task 22 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 22 try 0 launched on executor multi with executor id 23 parsl.dataflow.dflow INFO: Standard output for task 22 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/990e58e9-4394-4a84-b6ce-b7960cc72c83_isrForCrosstalkSources_997023_49.stdout parsl.dataflow.dflow INFO: Standard error for task 22 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/990e58e9-4394-4a84-b6ce-b7960cc72c83_isrForCrosstalkSources_997023_49.stderr parsl.dataflow.dflow DEBUG: Task 23 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 23 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 23 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 23 has memoization hash 903b8693c585c7f592f9b0b286961b85 parsl.dataflow.memoization INFO: Task 23 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 23 try 0 launched on executor multi with executor id 24 parsl.dataflow.dflow INFO: Standard output for task 23 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/92f9b283-80d2-47d0-ad23-0680444f4226_isrForCrosstalkSources_996959_54.stdout parsl.dataflow.dflow INFO: Standard error for task 23 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/92f9b283-80d2-47d0-ad23-0680444f4226_isrForCrosstalkSources_996959_54.stderr parsl.dataflow.dflow DEBUG: Task 24 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 24 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 24 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 24 has memoization hash 501c93a680b428fa1d9357c6e35abb03 parsl.dataflow.memoization INFO: Task 24 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 24 try 0 launched on executor multi with executor id 25 parsl.dataflow.dflow INFO: Standard output for task 24 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9bd3c513-b556-4e37-affd-38a7beef8ef3_isrForCrosstalkSources_997023_56.stdout parsl.dataflow.dflow INFO: Standard error for task 24 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9bd3c513-b556-4e37-affd-38a7beef8ef3_isrForCrosstalkSources_997023_56.stderr parsl.dataflow.dflow DEBUG: Task 25 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 25 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 25 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 25 has memoization hash 55dcd0cf2d234694c57ca05450f108c6 parsl.dataflow.memoization INFO: Task 25 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 25 try 0 launched on executor multi with executor id 26 parsl.dataflow.dflow INFO: Standard output for task 25 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/df05ae00-5c52-4ec8-b82c-4b1850b20a5d_isrForCrosstalkSources_996997_52.stdout parsl.dataflow.dflow INFO: Standard error for task 25 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/df05ae00-5c52-4ec8-b82c-4b1850b20a5d_isrForCrosstalkSources_996997_52.stderr parsl.dataflow.dflow DEBUG: Task 26 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 26 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 26 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 26 has memoization hash 25387a4d7a3e966e12711ccb8ef066ff parsl.dataflow.memoization INFO: Task 26 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 26 try 0 launched on executor multi with executor id 27 parsl.dataflow.dflow INFO: Standard output for task 26 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/70c568aa-4f83-47f5-9b59-d09b787ec11f_isrForCrosstalkSources_997009_31.stdout parsl.dataflow.dflow INFO: Standard error for task 26 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/70c568aa-4f83-47f5-9b59-d09b787ec11f_isrForCrosstalkSources_997009_31.stderr parsl.dataflow.dflow DEBUG: Task 27 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 27 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 27 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 27 has memoization hash 7a36ee2f0f4e894903b20f520cb3692f parsl.dataflow.memoization INFO: Task 27 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 27 try 0 launched on executor multi with executor id 28 parsl.dataflow.dflow INFO: Standard output for task 27 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/0b3ed825-85da-4e93-85d1-07f4c7c98c4f_isrForCrosstalkSources_996952_61.stdout parsl.dataflow.dflow INFO: Standard error for task 27 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/0b3ed825-85da-4e93-85d1-07f4c7c98c4f_isrForCrosstalkSources_996952_61.stderr parsl.dataflow.dflow DEBUG: Task 28 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 28 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 28 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 28 has memoization hash 66a784096ccb4b9dd0e8231f8de6e2f1 parsl.dataflow.memoization INFO: Task 28 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 28 try 0 launched on executor multi with executor id 29 parsl.dataflow.dflow INFO: Standard output for task 28 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b9dac3b1-014b-4a3b-b3ed-24111a092ee9_isrForCrosstalkSources_997021_61.stdout parsl.dataflow.dflow INFO: Standard error for task 28 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b9dac3b1-014b-4a3b-b3ed-24111a092ee9_isrForCrosstalkSources_997021_61.stderr parsl.dataflow.dflow DEBUG: Task 29 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 29 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 29 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 29 has memoization hash faf6965c005e545f450965b71b8843f5 parsl.dataflow.memoization INFO: Task 29 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6111c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 29 try 0 launched on executor multi with executor id 30 parsl.dataflow.dflow INFO: Standard output for task 29 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/94a8a546-d420-4c8e-ae1a-ef2396509c79_isrForCrosstalkSources_996967_62.stdout parsl.dataflow.dflow INFO: Standard error for task 29 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/94a8a546-d420-4c8e-ae1a-ef2396509c79_isrForCrosstalkSources_996967_62.stderr parsl.dataflow.dflow DEBUG: Task 30 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 30 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 30 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 30 has memoization hash 172ee54fbc0ffc33a34ad9c8e197ae9e parsl.dataflow.memoization INFO: Task 30 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 30 try 0 launched on executor multi with executor id 31 parsl.dataflow.dflow INFO: Standard output for task 30 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a65b1606-0079-4b6a-a114-1485315fa59b_isrForCrosstalkSources_997006_60.stdout parsl.dataflow.dflow INFO: Standard error for task 30 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a65b1606-0079-4b6a-a114-1485315fa59b_isrForCrosstalkSources_997006_60.stderr parsl.dataflow.dflow DEBUG: Task 31 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 31 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 31 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 31 has memoization hash e764d14e5f925a8272be0f0607163bb5 parsl.dataflow.memoization INFO: Task 31 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 31 try 0 launched on executor multi with executor id 32 parsl.dataflow.dflow INFO: Standard output for task 31 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/8c852584-a025-4036-a50e-f9f5765cff47_isrForCrosstalkSources_996998_38.stdout parsl.dataflow.dflow INFO: Standard error for task 31 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/8c852584-a025-4036-a50e-f9f5765cff47_isrForCrosstalkSources_996998_38.stderr parsl.dataflow.dflow DEBUG: Task 32 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 32 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 32 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 32 has memoization hash 86a1a89abe1bba127d064894a9e7e983 parsl.dataflow.memoization INFO: Task 32 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6113a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 32 try 0 launched on executor multi with executor id 33 parsl.dataflow.dflow INFO: Standard output for task 32 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/610ca1be-a28e-4d64-8d65-42fe24bc15ea_isrForCrosstalkSources_996974_42.stdout parsl.dataflow.dflow INFO: Standard error for task 32 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/610ca1be-a28e-4d64-8d65-42fe24bc15ea_isrForCrosstalkSources_996974_42.stderr parsl.dataflow.dflow DEBUG: Task 33 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 33 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 33 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 33 has memoization hash fedbd5207f369e92a36e9718aabf8136 parsl.dataflow.memoization INFO: Task 33 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 33 try 0 launched on executor multi with executor id 34 parsl.dataflow.dflow INFO: Standard output for task 33 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/326f25b8-c6ce-467c-a9e9-4a7a4db54d38_isrForCrosstalkSources_997019_26.stdout parsl.dataflow.dflow INFO: Standard error for task 33 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/326f25b8-c6ce-467c-a9e9-4a7a4db54d38_isrForCrosstalkSources_997019_26.stderr parsl.dataflow.dflow DEBUG: Task 34 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 34 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 34 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 34 has memoization hash 4218e543b8cf5752278c24efab7fa480 parsl.dataflow.memoization INFO: Task 34 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6114e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 34 try 0 launched on executor multi with executor id 35 parsl.dataflow.dflow INFO: Standard output for task 34 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/23df6717-b6f0-4e8f-816b-28c9388e0cab_isrForCrosstalkSources_997010_15.stdout parsl.dataflow.dflow INFO: Standard error for task 34 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/23df6717-b6f0-4e8f-816b-28c9388e0cab_isrForCrosstalkSources_997010_15.stderr parsl.dataflow.dflow DEBUG: Task 35 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 35 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 35 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 35 has memoization hash 5dc95572a6e766df170dcfb5c369036b parsl.dataflow.memoization INFO: Task 35 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 35 try 0 launched on executor multi with executor id 36 parsl.dataflow.dflow INFO: Standard output for task 35 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/37a5e84f-b245-4095-9c6b-988bcb217c79_isrForCrosstalkSources_997015_51.stdout parsl.dataflow.dflow INFO: Standard error for task 35 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/37a5e84f-b245-4095-9c6b-988bcb217c79_isrForCrosstalkSources_997015_51.stderr parsl.dataflow.dflow DEBUG: Task 36 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 36 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 36 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 36 has memoization hash 12b4c1a9aefebdc13dbfdf6dac3212af parsl.dataflow.memoization INFO: Task 36 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 36 try 0 launched on executor multi with executor id 37 parsl.dataflow.dflow INFO: Standard output for task 36 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7e888150-9a69-411c-8ba4-e88da8177835_isrForCrosstalkSources_997033_33.stdout parsl.dataflow.dflow INFO: Standard error for task 36 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7e888150-9a69-411c-8ba4-e88da8177835_isrForCrosstalkSources_997033_33.stderr parsl.dataflow.dflow DEBUG: Task 37 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 37 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 37 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 37 has memoization hash 3cfc13b94be90dd65e11109952cacdfb parsl.dataflow.memoization INFO: Task 37 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6116c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 37 try 0 launched on executor multi with executor id 38 parsl.dataflow.dflow INFO: Standard output for task 37 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/deaecffd-bd08-469a-b168-55190f404c2c_isrForCrosstalkSources_996980_54.stdout parsl.dataflow.dflow INFO: Standard error for task 37 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/deaecffd-bd08-469a-b168-55190f404c2c_isrForCrosstalkSources_996980_54.stderr parsl.dataflow.dflow DEBUG: Task 38 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 38 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 38 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 38 has memoization hash ad81832faa76c990fe606945513d8fed parsl.dataflow.memoization INFO: Task 38 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 38 try 0 launched on executor multi with executor id 39 parsl.dataflow.dflow INFO: Standard output for task 38 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/df3895d8-03a7-45c7-8a90-67639359d2d4_isrForCrosstalkSources_997029_55.stdout parsl.dataflow.dflow INFO: Standard error for task 38 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/df3895d8-03a7-45c7-8a90-67639359d2d4_isrForCrosstalkSources_997029_55.stderr parsl.dataflow.dflow DEBUG: Task 39 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 39 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 39 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 39 has memoization hash 1ebf8d3fe58a908192d5e677d51ea34b parsl.dataflow.memoization INFO: Task 39 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 39 try 0 launched on executor multi with executor id 40 parsl.dataflow.dflow INFO: Standard output for task 39 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/79b4c3d3-649c-43f5-9de0-cd892fe7e17e_isrForCrosstalkSources_996999_18.stdout parsl.dataflow.dflow INFO: Standard error for task 39 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/79b4c3d3-649c-43f5-9de0-cd892fe7e17e_isrForCrosstalkSources_996999_18.stderr parsl.dataflow.dflow DEBUG: Task 40 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 40 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 40 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 40 has memoization hash 1e894828ed8545a7c7452e403d85739d parsl.dataflow.memoization INFO: Task 40 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 40 try 0 launched on executor multi with executor id 41 parsl.dataflow.dflow INFO: Standard output for task 40 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/dafa327f-7bbc-44b3-a932-1fa003ec5b77_isrForCrosstalkSources_997006_16.stdout parsl.dataflow.dflow INFO: Standard error for task 40 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/dafa327f-7bbc-44b3-a932-1fa003ec5b77_isrForCrosstalkSources_997006_16.stderr parsl.dataflow.dflow DEBUG: Task 41 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 41 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 41 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 41 has memoization hash e4538b73023eb13e4267f620caa69e7b parsl.dataflow.memoization INFO: Task 41 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6118a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 41 try 0 launched on executor multi with executor id 42 parsl.dataflow.dflow INFO: Standard output for task 41 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/44cb1c17-7beb-44ba-9027-df768437db73_isrForCrosstalkSources_997019_53.stdout parsl.dataflow.dflow INFO: Standard error for task 41 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/44cb1c17-7beb-44ba-9027-df768437db73_isrForCrosstalkSources_997019_53.stderr parsl.dataflow.dflow DEBUG: Task 42 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 42 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 42 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 42 has memoization hash 7b1ac4d50fda7cc0fe10f1c1ef5e2b51 parsl.dataflow.memoization INFO: Task 42 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6119e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 42 try 0 launched on executor multi with executor id 43 parsl.dataflow.dflow INFO: Standard output for task 42 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cc0e9506-2a62-40c7-a44f-a946384a7aff_isrForCrosstalkSources_996957_62.stdout parsl.dataflow.dflow INFO: Standard error for task 42 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cc0e9506-2a62-40c7-a44f-a946384a7aff_isrForCrosstalkSources_996957_62.stderr parsl.dataflow.dflow DEBUG: Task 43 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 43 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 43 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 43 has memoization hash 716e44646e9ce03a86a4eaeee802854b parsl.dataflow.memoization INFO: Task 43 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 43 try 0 launched on executor multi with executor id 44 parsl.dataflow.dflow INFO: Standard output for task 43 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ad277931-73d9-4f90-bb23-11ac940d84c4_isrForCrosstalkSources_997013_21.stdout parsl.dataflow.dflow INFO: Standard error for task 43 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ad277931-73d9-4f90-bb23-11ac940d84c4_isrForCrosstalkSources_997013_21.stderr parsl.dataflow.dflow DEBUG: Task 44 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 44 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 44 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 44 has memoization hash ca647dfe59ab359eac5625903413a2b4 parsl.dataflow.memoization INFO: Task 44 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 44 try 0 launched on executor multi with executor id 45 parsl.dataflow.dflow INFO: Standard output for task 44 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6394f85e-884d-4744-a1ed-fa04bd304750_isrForCrosstalkSources_997006_57.stdout parsl.dataflow.dflow INFO: Standard error for task 44 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6394f85e-884d-4744-a1ed-fa04bd304750_isrForCrosstalkSources_997006_57.stderr parsl.dataflow.dflow DEBUG: Task 45 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 45 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 45 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 45 has memoization hash 56ee85fb9d0b1b8cda1a0132be2066d7 parsl.dataflow.memoization INFO: Task 45 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 45 try 0 launched on executor multi with executor id 46 parsl.dataflow.dflow INFO: Standard output for task 45 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6ace84a6-4415-4e10-8211-57bb438090f5_isrForCrosstalkSources_996953_48.stdout parsl.dataflow.dflow INFO: Standard error for task 45 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6ace84a6-4415-4e10-8211-57bb438090f5_isrForCrosstalkSources_996953_48.stderr parsl.dataflow.dflow DEBUG: Task 46 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 46 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 46 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 46 has memoization hash e6fbec2255a166632a87d32829ef3c55 parsl.dataflow.memoization INFO: Task 46 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 46 try 0 launched on executor multi with executor id 47 parsl.dataflow.dflow INFO: Standard output for task 46 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/cb78d91f-8720-4ece-96ce-530da8cb8af7_isrForCrosstalkSources_996970_54.stdout parsl.dataflow.dflow INFO: Standard error for task 46 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/cb78d91f-8720-4ece-96ce-530da8cb8af7_isrForCrosstalkSources_996970_54.stderr parsl.dataflow.dflow DEBUG: Task 47 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 47 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 47 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 47 has memoization hash 2aa11ba4689a206627063423f62d6c72 parsl.dataflow.memoization INFO: Task 47 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 47 try 0 launched on executor multi with executor id 48 parsl.dataflow.dflow INFO: Standard output for task 47 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/fd3afc48-2174-43a5-a4f5-5c617441e437_isrForCrosstalkSources_996952_16.stdout parsl.dataflow.dflow INFO: Standard error for task 47 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/fd3afc48-2174-43a5-a4f5-5c617441e437_isrForCrosstalkSources_996952_16.stderr parsl.dataflow.dflow DEBUG: Task 48 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 48 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 48 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 48 has memoization hash 7dec3c3bc322ab6fe13f582e220650bd parsl.dataflow.memoization INFO: Task 48 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 48 try 0 launched on executor multi with executor id 49 parsl.dataflow.dflow INFO: Standard output for task 48 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/83741167-45e4-40e2-b9b5-83a4dc357a74_isrForCrosstalkSources_997024_16.stdout parsl.dataflow.dflow INFO: Standard error for task 48 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/83741167-45e4-40e2-b9b5-83a4dc357a74_isrForCrosstalkSources_997024_16.stderr parsl.dataflow.dflow DEBUG: Task 49 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 49 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 49 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 49 has memoization hash c38f08ddad2aa0f2b7dbf0ce821b20b0 parsl.dataflow.memoization INFO: Task 49 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 49 try 0 launched on executor multi with executor id 50 parsl.dataflow.dflow INFO: Standard output for task 49 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e7d3441f-d0df-486f-a43a-a5584f80bdfe_isrForCrosstalkSources_996986_33.stdout parsl.dataflow.dflow INFO: Standard error for task 49 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e7d3441f-d0df-486f-a43a-a5584f80bdfe_isrForCrosstalkSources_996986_33.stderr parsl.dataflow.dflow DEBUG: Task 50 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 50 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 50 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 50 has memoization hash 7384ddf5d5ff407879202a1ebbee4ff0 parsl.dataflow.memoization INFO: Task 50 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 50 try 0 launched on executor multi with executor id 51 parsl.dataflow.dflow INFO: Standard output for task 50 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/7a2cecc4-ede4-4ff7-b882-8f286eb281d9_isrForCrosstalkSources_997023_4.stdout parsl.dataflow.dflow INFO: Standard error for task 50 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/7a2cecc4-ede4-4ff7-b882-8f286eb281d9_isrForCrosstalkSources_997023_4.stderr parsl.dataflow.dflow DEBUG: Task 51 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 51 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 51 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 51 has memoization hash 1fac930ff3766b69fb7094a53bc6f4ad parsl.dataflow.memoization INFO: Task 51 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f611ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 51 try 0 launched on executor multi with executor id 52 parsl.dataflow.dflow INFO: Standard output for task 51 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4d3f6cb0-286b-4204-b762-a2207a53f17c_isrForCrosstalkSources_996976_57.stdout parsl.dataflow.dflow INFO: Standard error for task 51 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4d3f6cb0-286b-4204-b762-a2207a53f17c_isrForCrosstalkSources_996976_57.stderr parsl.dataflow.dflow DEBUG: Task 52 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 52 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 52 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 52 has memoization hash 52a84d834c0871421b45085927321eb6 parsl.dataflow.memoization INFO: Task 52 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 52 try 0 launched on executor multi with executor id 53 parsl.dataflow.dflow INFO: Standard output for task 52 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d5aba35f-2244-4118-8cdb-432c12d078c9_isrForCrosstalkSources_996998_15.stdout parsl.dataflow.dflow INFO: Standard error for task 52 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d5aba35f-2244-4118-8cdb-432c12d078c9_isrForCrosstalkSources_996998_15.stderr parsl.dataflow.dflow DEBUG: Task 53 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 53 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 53 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 53 has memoization hash 5624af16cbdd7273e01ab51ba0e7d854 parsl.dataflow.memoization INFO: Task 53 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6120c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 53 try 0 launched on executor multi with executor id 54 parsl.dataflow.dflow INFO: Standard output for task 53 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/48231e4e-5ebc-497f-af14-07b3e91435b4_isrForCrosstalkSources_997003_10.stdout parsl.dataflow.dflow INFO: Standard error for task 53 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/48231e4e-5ebc-497f-af14-07b3e91435b4_isrForCrosstalkSources_997003_10.stderr parsl.dataflow.dflow DEBUG: Task 54 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 54 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 54 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 54 has memoization hash f0c02f37f5c331c0a4e24e14877cc3a8 parsl.dataflow.memoization INFO: Task 54 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 54 try 0 launched on executor multi with executor id 55 parsl.dataflow.dflow INFO: Standard output for task 54 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d6676acd-ed5c-46cc-8fb8-3927f419cb29_isrForCrosstalkSources_996954_47.stdout parsl.dataflow.dflow INFO: Standard error for task 54 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d6676acd-ed5c-46cc-8fb8-3927f419cb29_isrForCrosstalkSources_996954_47.stderr parsl.dataflow.dflow DEBUG: Task 55 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 55 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 55 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 55 has memoization hash 89360378947b4e11242b73e46dddedc0 parsl.dataflow.memoization INFO: Task 55 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6122a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 55 try 0 launched on executor multi with executor id 56 parsl.dataflow.dflow INFO: Standard output for task 55 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5ecf84a0-34af-413b-a968-60c2a32bf7d7_isrForCrosstalkSources_996972_14.stdout parsl.dataflow.dflow INFO: Standard error for task 55 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5ecf84a0-34af-413b-a968-60c2a32bf7d7_isrForCrosstalkSources_996972_14.stderr parsl.dataflow.dflow DEBUG: Task 56 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 56 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 56 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 56 has memoization hash 39f9043e9a368a32b4c0de18b432e96f parsl.dataflow.memoization INFO: Task 56 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 56 try 0 launched on executor multi with executor id 57 parsl.dataflow.dflow INFO: Standard output for task 56 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/c2af5f87-30ed-4a52-aa57-992997ddaf0c_isrForCrosstalkSources_996957_28.stdout parsl.dataflow.dflow INFO: Standard error for task 56 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/c2af5f87-30ed-4a52-aa57-992997ddaf0c_isrForCrosstalkSources_996957_28.stderr parsl.dataflow.dflow DEBUG: Task 57 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 57 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 57 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 57 has memoization hash 88e231a0bed8bf613290396e88971743 parsl.dataflow.memoization INFO: Task 57 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 57 try 0 launched on executor multi with executor id 58 parsl.dataflow.dflow INFO: Standard output for task 57 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2958b64f-c2dc-40d4-8796-69628c667b2c_isrForCrosstalkSources_996961_7.stdout parsl.dataflow.dflow INFO: Standard error for task 57 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2958b64f-c2dc-40d4-8796-69628c667b2c_isrForCrosstalkSources_996961_7.stderr parsl.dataflow.dflow DEBUG: Task 58 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 58 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 58 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 58 has memoization hash 9a696ccae5463aeb776215e52af2d250 parsl.dataflow.memoization INFO: Task 58 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6123e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 58 try 0 launched on executor multi with executor id 59 parsl.dataflow.dflow INFO: Standard output for task 58 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/f8a165e6-dbda-4ae2-8c1e-c45544442a8d_isrForCrosstalkSources_997030_50.stdout parsl.dataflow.dflow INFO: Standard error for task 58 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/f8a165e6-dbda-4ae2-8c1e-c45544442a8d_isrForCrosstalkSources_997030_50.stderr parsl.dataflow.dflow DEBUG: Task 59 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 59 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 59 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 59 has memoization hash de81bf7989da01f9e647aa6a156105fd parsl.dataflow.memoization INFO: Task 59 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 59 try 0 launched on executor multi with executor id 60 parsl.dataflow.dflow INFO: Standard output for task 59 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/909d631e-91aa-46d3-90b7-69df88074b22_isrForCrosstalkSources_996985_29.stdout parsl.dataflow.dflow INFO: Standard error for task 59 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/909d631e-91aa-46d3-90b7-69df88074b22_isrForCrosstalkSources_996985_29.stderr parsl.dataflow.dflow DEBUG: Task 60 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 60 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 60 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 60 has memoization hash 808162cf48b228888bbe608d793e3f4d parsl.dataflow.memoization INFO: Task 60 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6125c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 60 try 0 launched on executor multi with executor id 61 parsl.dataflow.dflow INFO: Standard output for task 60 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7be3be1c-591a-404e-a97e-5dc03be6ff53_isrForCrosstalkSources_996988_60.stdout parsl.dataflow.dflow INFO: Standard error for task 60 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7be3be1c-591a-404e-a97e-5dc03be6ff53_isrForCrosstalkSources_996988_60.stderr parsl.dataflow.dflow DEBUG: Task 61 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 61 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 61 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 61 has memoization hash b51d62babfbac966bb1c77eea6688e00 parsl.dataflow.memoization INFO: Task 61 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 61 try 0 launched on executor multi with executor id 62 parsl.dataflow.dflow INFO: Standard output for task 61 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/69443999-cf7a-43f8-a553-0d0318e26210_isrForCrosstalkSources_996987_13.stdout parsl.dataflow.dflow INFO: Standard error for task 61 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/69443999-cf7a-43f8-a553-0d0318e26210_isrForCrosstalkSources_996987_13.stderr parsl.dataflow.dflow DEBUG: Task 62 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 62 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 62 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 62 has memoization hash db6494c2816370f599141b311d7b7070 parsl.dataflow.memoization INFO: Task 62 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 62 try 0 launched on executor multi with executor id 63 parsl.dataflow.dflow INFO: Standard output for task 62 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/4e73a223-b353-4064-af7f-edbb56ff7ebb_isrForCrosstalkSources_996990_51.stdout parsl.dataflow.dflow INFO: Standard error for task 62 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/4e73a223-b353-4064-af7f-edbb56ff7ebb_isrForCrosstalkSources_996990_51.stderr parsl.dataflow.dflow DEBUG: Task 63 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 63 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 63 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 63 has memoization hash 156a1da3b892693f7958238c16b69ea8 parsl.dataflow.memoization INFO: Task 63 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 63 try 0 launched on executor multi with executor id 64 parsl.dataflow.dflow INFO: Standard output for task 63 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4f3eb4fa-529c-4eb4-9c2a-0877743a8c3e_isrForCrosstalkSources_996972_53.stdout parsl.dataflow.dflow INFO: Standard error for task 63 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4f3eb4fa-529c-4eb4-9c2a-0877743a8c3e_isrForCrosstalkSources_996972_53.stderr parsl.dataflow.dflow DEBUG: Task 64 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 64 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 64 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 64 has memoization hash 2052354eca088925aed54a0976586778 parsl.dataflow.memoization INFO: Task 64 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6127a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 64 try 0 launched on executor multi with executor id 65 parsl.dataflow.dflow INFO: Standard output for task 64 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9b0e8c43-d55a-4f22-a2f1-0148b97488bf_isrForCrosstalkSources_996995_30.stdout parsl.dataflow.dflow INFO: Standard error for task 64 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9b0e8c43-d55a-4f22-a2f1-0148b97488bf_isrForCrosstalkSources_996995_30.stderr parsl.dataflow.dflow DEBUG: Task 65 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 65 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 65 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 65 has memoization hash 2746b14ef84dded70d0475724c088349 parsl.dataflow.memoization INFO: Task 65 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6128e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 65 try 0 launched on executor multi with executor id 66 parsl.dataflow.dflow INFO: Standard output for task 65 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/59177452-c882-473e-9915-c875c09447f1_isrForCrosstalkSources_996962_14.stdout parsl.dataflow.dflow INFO: Standard error for task 65 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/59177452-c882-473e-9915-c875c09447f1_isrForCrosstalkSources_996962_14.stderr parsl.dataflow.dflow DEBUG: Task 66 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 66 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 66 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 66 has memoization hash 24417ff488aa2d8840063824b662e186 parsl.dataflow.memoization INFO: Task 66 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 66 try 0 launched on executor multi with executor id 67 parsl.dataflow.dflow INFO: Standard output for task 66 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/6e2e4b77-0c69-4e4f-bc75-9e9eb4410f73_isrForCrosstalkSources_996997_60.stdout parsl.dataflow.dflow INFO: Standard error for task 66 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/6e2e4b77-0c69-4e4f-bc75-9e9eb4410f73_isrForCrosstalkSources_996997_60.stderr parsl.dataflow.dflow DEBUG: Task 67 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 67 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 67 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 67 has memoization hash b184ad672cb8d70b65499339b1bb8797 parsl.dataflow.memoization INFO: Task 67 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 67 try 0 launched on executor multi with executor id 68 parsl.dataflow.dflow INFO: Standard output for task 67 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/12fc536e-a669-4f54-a111-b97702bd8768_isrForCrosstalkSources_997025_35.stdout parsl.dataflow.dflow INFO: Standard error for task 67 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/12fc536e-a669-4f54-a111-b97702bd8768_isrForCrosstalkSources_997025_35.stderr parsl.dataflow.dflow DEBUG: Task 68 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 68 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 68 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 68 has memoization hash 32506fa13caa0c33b29a0a45f1964549 parsl.dataflow.memoization INFO: Task 68 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 68 try 0 launched on executor multi with executor id 69 parsl.dataflow.dflow INFO: Standard output for task 68 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/640e5fd2-32e1-4010-a436-5facde608175_isrForCrosstalkSources_996989_52.stdout parsl.dataflow.dflow INFO: Standard error for task 68 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/640e5fd2-32e1-4010-a436-5facde608175_isrForCrosstalkSources_996989_52.stderr parsl.dataflow.dflow DEBUG: Task 69 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 69 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 69 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 69 has memoization hash c62fd04979093720bfbde4c42b1018a9 parsl.dataflow.memoization INFO: Task 69 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 69 try 0 launched on executor multi with executor id 70 parsl.dataflow.dflow INFO: Standard output for task 69 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/9d681ffa-be5d-4793-9e75-8e09d80ec554_isrForCrosstalkSources_996967_36.stdout parsl.dataflow.dflow INFO: Standard error for task 69 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/9d681ffa-be5d-4793-9e75-8e09d80ec554_isrForCrosstalkSources_996967_36.stderr parsl.dataflow.dflow DEBUG: Task 70 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 70 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 70 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 70 has memoization hash 83422729fa032ccaaa245dafcaee1dea parsl.dataflow.memoization INFO: Task 70 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 70 try 0 launched on executor multi with executor id 71 parsl.dataflow.dflow INFO: Standard output for task 70 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/03d74eeb-7388-4196-b3c2-eca8e38af562_isrForCrosstalkSources_996963_39.stdout parsl.dataflow.dflow INFO: Standard error for task 70 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/03d74eeb-7388-4196-b3c2-eca8e38af562_isrForCrosstalkSources_996963_39.stderr parsl.dataflow.dflow DEBUG: Task 71 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 71 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 71 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 71 has memoization hash 71b92a704c533b6a3740248bde0b55b0 parsl.dataflow.memoization INFO: Task 71 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 71 try 0 launched on executor multi with executor id 72 parsl.dataflow.dflow INFO: Standard output for task 71 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9829c48a-7666-4daf-bd6b-889343b5947a_isrForCrosstalkSources_996959_19.stdout parsl.dataflow.dflow INFO: Standard error for task 71 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9829c48a-7666-4daf-bd6b-889343b5947a_isrForCrosstalkSources_996959_19.stderr parsl.dataflow.dflow DEBUG: Task 72 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 72 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 72 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 72 has memoization hash 949cddc32175547066349f8b6f8052c1 parsl.dataflow.memoization INFO: Task 72 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 72 try 0 launched on executor multi with executor id 73 parsl.dataflow.dflow INFO: Standard output for task 72 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/0fe94d2e-e048-4623-b59e-fa3db23dd0ff_isrForCrosstalkSources_996961_60.stdout parsl.dataflow.dflow INFO: Standard error for task 72 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/0fe94d2e-e048-4623-b59e-fa3db23dd0ff_isrForCrosstalkSources_996961_60.stderr parsl.dataflow.dflow DEBUG: Task 73 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 73 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 73 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 73 has memoization hash 1e191b40a9a762701a3633d80e49e596 parsl.dataflow.memoization INFO: Task 73 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 73 try 0 launched on executor multi with executor id 74 parsl.dataflow.dflow INFO: Standard output for task 73 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/6eb2a6d8-9dc7-4a18-bfe4-24f427f53143_isrForCrosstalkSources_996955_18.stdout parsl.dataflow.dflow INFO: Standard error for task 73 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/6eb2a6d8-9dc7-4a18-bfe4-24f427f53143_isrForCrosstalkSources_996955_18.stderr parsl.dataflow.dflow DEBUG: Task 74 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 74 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 74 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 74 has memoization hash 69d14c7c37778c5d380bf2b2728efc45 parsl.dataflow.memoization INFO: Task 74 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 74 try 0 launched on executor multi with executor id 75 parsl.dataflow.dflow INFO: Standard output for task 74 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f9dcf0e7-4ef2-461d-be65-b784bb59d0f9_isrForCrosstalkSources_996956_40.stdout parsl.dataflow.dflow INFO: Standard error for task 74 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f9dcf0e7-4ef2-461d-be65-b784bb59d0f9_isrForCrosstalkSources_996956_40.stderr parsl.dataflow.dflow DEBUG: Task 75 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 75 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 75 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 75 has memoization hash 7519557c116e2d80b6f869239c48ae74 parsl.dataflow.memoization INFO: Task 75 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 75 try 0 launched on executor multi with executor id 76 parsl.dataflow.dflow INFO: Standard output for task 75 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e3a1fcd6-62f4-4a4f-8812-e26eb6eb2f85_isrForCrosstalkSources_996964_39.stdout parsl.dataflow.dflow INFO: Standard error for task 75 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e3a1fcd6-62f4-4a4f-8812-e26eb6eb2f85_isrForCrosstalkSources_996964_39.stderr parsl.dataflow.dflow DEBUG: Task 76 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 76 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 76 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 76 has memoization hash 30756c6d0c39ff1833d9827633bb9b9f parsl.dataflow.memoization INFO: Task 76 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 76 try 0 launched on executor multi with executor id 77 parsl.dataflow.dflow INFO: Standard output for task 76 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/473f451a-0148-4778-b462-2df0b128c66b_isrForCrosstalkSources_997017_51.stdout parsl.dataflow.dflow INFO: Standard error for task 76 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/473f451a-0148-4778-b462-2df0b128c66b_isrForCrosstalkSources_997017_51.stderr parsl.dataflow.dflow DEBUG: Task 77 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 77 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 77 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 77 has memoization hash 22438b48dbfd3964bfe0515db98125d2 parsl.dataflow.memoization INFO: Task 77 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f612fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 77 try 0 launched on executor multi with executor id 78 parsl.dataflow.dflow INFO: Standard output for task 77 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c7189b5b-d3d8-45a3-a092-b42a7508d4f7_isrForCrosstalkSources_997007_3.stdout parsl.dataflow.dflow INFO: Standard error for task 77 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c7189b5b-d3d8-45a3-a092-b42a7508d4f7_isrForCrosstalkSources_997007_3.stderr parsl.dataflow.dflow DEBUG: Task 78 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 78 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 78 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 78 has memoization hash b32fcd4e429a2aa663dd5d74319f6208 parsl.dataflow.memoization INFO: Task 78 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 78 try 0 launched on executor multi with executor id 79 parsl.dataflow.dflow INFO: Standard output for task 78 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3553abe3-a95f-4a7c-94e5-ed8741102e5b_isrForCrosstalkSources_996999_22.stdout parsl.dataflow.dflow INFO: Standard error for task 78 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3553abe3-a95f-4a7c-94e5-ed8741102e5b_isrForCrosstalkSources_996999_22.stderr parsl.dataflow.dflow DEBUG: Task 79 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 79 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 79 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 79 has memoization hash fc5454b2edc55f06aae59960fa824120 parsl.dataflow.memoization INFO: Task 79 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 79 try 0 launched on executor multi with executor id 80 parsl.dataflow.dflow INFO: Standard output for task 79 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f728ee0b-a439-46f5-aa3d-c61572dfca6e_isrForCrosstalkSources_996985_46.stdout parsl.dataflow.dflow INFO: Standard error for task 79 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f728ee0b-a439-46f5-aa3d-c61572dfca6e_isrForCrosstalkSources_996985_46.stderr parsl.dataflow.dflow DEBUG: Task 80 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 80 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 80 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 80 has memoization hash d89ef47f9446491fc282a71bf7c43c9f parsl.dataflow.memoization INFO: Task 80 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 80 try 0 launched on executor multi with executor id 81 parsl.dataflow.dflow INFO: Standard output for task 80 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f5f9a6ed-9d3c-4bc8-801b-a183ab1ca3d9_isrForCrosstalkSources_996974_49.stdout parsl.dataflow.dflow INFO: Standard error for task 80 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f5f9a6ed-9d3c-4bc8-801b-a183ab1ca3d9_isrForCrosstalkSources_996974_49.stderr parsl.dataflow.dflow DEBUG: Task 81 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 81 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 81 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 81 has memoization hash 6541b0f4c45dace5711c5a533b4d7892 parsl.dataflow.memoization INFO: Task 81 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6131a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 81 try 0 launched on executor multi with executor id 82 parsl.dataflow.dflow INFO: Standard output for task 81 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/be373964-0175-4859-8ca0-0b6785553f61_isrForCrosstalkSources_996997_44.stdout parsl.dataflow.dflow INFO: Standard error for task 81 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/be373964-0175-4859-8ca0-0b6785553f61_isrForCrosstalkSources_996997_44.stderr parsl.dataflow.dflow DEBUG: Task 82 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 82 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 82 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 82 has memoization hash 6acd51c273c5a43adb8c109237428ad5 parsl.dataflow.memoization INFO: Task 82 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6132e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 82 try 0 launched on executor multi with executor id 83 parsl.dataflow.dflow INFO: Standard output for task 82 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/413b8973-9a62-49f6-87b4-12dad535329b_isrForCrosstalkSources_996994_10.stdout parsl.dataflow.dflow INFO: Standard error for task 82 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/413b8973-9a62-49f6-87b4-12dad535329b_isrForCrosstalkSources_996994_10.stderr parsl.dataflow.dflow DEBUG: Task 83 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 83 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 83 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 83 has memoization hash ed850339454c26f707311d93f129d08c parsl.dataflow.memoization INFO: Task 83 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 83 try 0 launched on executor multi with executor id 84 parsl.dataflow.dflow INFO: Standard output for task 83 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/3413f2aa-e21b-4836-9858-b90e177b40a1_isrForCrosstalkSources_997016_1.stdout parsl.dataflow.dflow INFO: Standard error for task 83 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/3413f2aa-e21b-4836-9858-b90e177b40a1_isrForCrosstalkSources_997016_1.stderr parsl.dataflow.dflow DEBUG: Task 84 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 84 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 84 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 84 has memoization hash d7852832d28ab1af13aa9ac2e3193638 parsl.dataflow.memoization INFO: Task 84 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 84 try 0 launched on executor multi with executor id 85 parsl.dataflow.dflow INFO: Standard output for task 84 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/fcf1a4a4-7426-47d1-a768-0840f8574399_isrForCrosstalkSources_996987_60.stdout parsl.dataflow.dflow INFO: Standard error for task 84 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/fcf1a4a4-7426-47d1-a768-0840f8574399_isrForCrosstalkSources_996987_60.stderr parsl.dataflow.dflow DEBUG: Task 85 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 85 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 85 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 85 has memoization hash dfebdaa6ff146f98fd16751ea2440e26 parsl.dataflow.memoization INFO: Task 85 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 85 try 0 launched on executor multi with executor id 86 parsl.dataflow.dflow INFO: Standard output for task 85 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c17cbef9-aacb-4752-b80d-aaf1397dc7a5_isrForCrosstalkSources_996973_6.stdout parsl.dataflow.dflow INFO: Standard error for task 85 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c17cbef9-aacb-4752-b80d-aaf1397dc7a5_isrForCrosstalkSources_996973_6.stderr parsl.dataflow.dflow DEBUG: Task 86 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 86 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 86 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 86 has memoization hash ae916a5d24b029cfe7d936b4c9118c38 parsl.dataflow.memoization INFO: Task 86 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6134c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 86 try 0 launched on executor multi with executor id 87 parsl.dataflow.dflow INFO: Standard output for task 86 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/f40c8a2a-a00d-4128-ba7e-86ac8c2c78ee_isrForCrosstalkSources_996994_33.stdout parsl.dataflow.dflow INFO: Standard error for task 86 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/f40c8a2a-a00d-4128-ba7e-86ac8c2c78ee_isrForCrosstalkSources_996994_33.stderr parsl.dataflow.dflow DEBUG: Task 87 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 87 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 87 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 87 has memoization hash 40f99d5bf38e8bae1e6dda20fe3c10e1 parsl.dataflow.memoization INFO: Task 87 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 87 try 0 launched on executor multi with executor id 88 parsl.dataflow.dflow INFO: Standard output for task 87 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/090c00a5-f758-4475-8db0-4788e643dec5_isrForCrosstalkSources_996966_37.stdout parsl.dataflow.dflow INFO: Standard error for task 87 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/090c00a5-f758-4475-8db0-4788e643dec5_isrForCrosstalkSources_996966_37.stderr parsl.dataflow.dflow DEBUG: Task 88 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 88 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 88 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 88 has memoization hash 4130b967586a26def3207597bb2363a0 parsl.dataflow.memoization INFO: Task 88 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6136a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 88 try 0 launched on executor multi with executor id 89 parsl.dataflow.dflow INFO: Standard output for task 88 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b5d7e425-8206-4210-bc8f-c2b01237f1c5_isrForCrosstalkSources_997013_29.stdout parsl.dataflow.dflow INFO: Standard error for task 88 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b5d7e425-8206-4210-bc8f-c2b01237f1c5_isrForCrosstalkSources_997013_29.stderr parsl.dataflow.dflow DEBUG: Task 89 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 89 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 89 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 89 has memoization hash 98bd46cf91df4efd9757be9592a7233d parsl.dataflow.memoization INFO: Task 89 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 89 try 0 launched on executor multi with executor id 90 parsl.dataflow.dflow INFO: Standard output for task 89 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4f4fdb78-bf2a-4794-b643-7ad566e145bf_isrForCrosstalkSources_996998_53.stdout parsl.dataflow.dflow INFO: Standard error for task 89 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4f4fdb78-bf2a-4794-b643-7ad566e145bf_isrForCrosstalkSources_996998_53.stderr parsl.dataflow.dflow DEBUG: Task 90 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 90 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 90 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 90 has memoization hash 60c06ff9ab5f117d0b6062494a0416e5 parsl.dataflow.memoization INFO: Task 90 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 90 try 0 launched on executor multi with executor id 91 parsl.dataflow.dflow INFO: Standard output for task 90 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/89c97da1-ca50-4d5b-91ee-6c35ce8c1782_isrForCrosstalkSources_997013_42.stdout parsl.dataflow.dflow INFO: Standard error for task 90 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/89c97da1-ca50-4d5b-91ee-6c35ce8c1782_isrForCrosstalkSources_997013_42.stderr parsl.dataflow.dflow DEBUG: Task 91 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 91 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 91 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 91 has memoization hash 14d18171bc6da7b22ac2b1b40351f486 parsl.dataflow.memoization INFO: Task 91 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6137e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 91 try 0 launched on executor multi with executor id 92 parsl.dataflow.dflow INFO: Standard output for task 91 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/311616ee-1fdc-4ed0-8bed-a2fe97500bf3_isrForCrosstalkSources_997002_7.stdout parsl.dataflow.dflow INFO: Standard error for task 91 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/311616ee-1fdc-4ed0-8bed-a2fe97500bf3_isrForCrosstalkSources_997002_7.stderr parsl.dataflow.dflow DEBUG: Task 92 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 92 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 92 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 92 has memoization hash 5a5b0997e6a7e9ea0e11df2c3ccf5168 parsl.dataflow.memoization INFO: Task 92 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 92 try 0 launched on executor multi with executor id 93 parsl.dataflow.dflow INFO: Standard output for task 92 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/4c3cdf26-9c39-4ed0-a85e-55fcfa850bf0_isrForCrosstalkSources_996985_48.stdout parsl.dataflow.dflow INFO: Standard error for task 92 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/4c3cdf26-9c39-4ed0-a85e-55fcfa850bf0_isrForCrosstalkSources_996985_48.stderr parsl.dataflow.dflow DEBUG: Task 93 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 93 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 93 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 93 has memoization hash ec66f11c5a5eb1bdb22198fafc6740b0 parsl.dataflow.memoization INFO: Task 93 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f6139c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 93 try 0 launched on executor multi with executor id 94 parsl.dataflow.dflow INFO: Standard output for task 93 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f5e93a6f-21e7-4f8e-8ce1-3ad0cef885eb_isrForCrosstalkSources_996995_6.stdout parsl.dataflow.dflow INFO: Standard error for task 93 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f5e93a6f-21e7-4f8e-8ce1-3ad0cef885eb_isrForCrosstalkSources_996995_6.stderr parsl.dataflow.dflow DEBUG: Task 94 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 94 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 94 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 94 has memoization hash c720224bcd1628d7cd915ce373ce121f parsl.dataflow.memoization INFO: Task 94 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 94 try 0 launched on executor multi with executor id 95 parsl.dataflow.dflow INFO: Standard output for task 94 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/ba482c0f-9b9e-4250-a394-5e26435cf00a_isrForCrosstalkSources_996956_49.stdout parsl.dataflow.dflow INFO: Standard error for task 94 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/ba482c0f-9b9e-4250-a394-5e26435cf00a_isrForCrosstalkSources_996956_49.stderr parsl.dataflow.dflow DEBUG: Task 95 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 95 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 95 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 95 has memoization hash 228f88c1316bf83f563229f0cd3f0979 parsl.dataflow.memoization INFO: Task 95 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 95 try 0 launched on executor multi with executor id 96 parsl.dataflow.dflow INFO: Standard output for task 95 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fb31705d-fe5f-4fd0-9d9e-49f24db30404_isrForCrosstalkSources_997002_47.stdout parsl.dataflow.dflow INFO: Standard error for task 95 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fb31705d-fe5f-4fd0-9d9e-49f24db30404_isrForCrosstalkSources_997002_47.stderr parsl.dataflow.dflow DEBUG: Task 96 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 96 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 96 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 96 has memoization hash 84e11ddf8a92673c0295eb86b67ea2e3 parsl.dataflow.memoization INFO: Task 96 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 96 try 0 launched on executor multi with executor id 97 parsl.dataflow.dflow INFO: Standard output for task 96 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/634a01cf-9a39-4517-8e26-5a0b3661da09_isrForCrosstalkSources_997000_57.stdout parsl.dataflow.dflow INFO: Standard error for task 96 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/634a01cf-9a39-4517-8e26-5a0b3661da09_isrForCrosstalkSources_997000_57.stderr parsl.dataflow.dflow DEBUG: Task 97 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 97 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 97 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 97 has memoization hash f7b3b0bb89f59e4c22f07098924c97ad parsl.dataflow.memoization INFO: Task 97 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 97 try 0 launched on executor multi with executor id 98 parsl.dataflow.dflow INFO: Standard output for task 97 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/b0896d75-450f-4787-ad4e-1d15df974675_isrForCrosstalkSources_997005_59.stdout parsl.dataflow.dflow INFO: Standard error for task 97 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/b0896d75-450f-4787-ad4e-1d15df974675_isrForCrosstalkSources_997005_59.stderr parsl.dataflow.dflow DEBUG: Task 98 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 98 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 98 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 98 has memoization hash e872d28080a105935b50f629e6e5239c parsl.dataflow.memoization INFO: Task 98 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 98 try 0 launched on executor multi with executor id 99 parsl.dataflow.dflow INFO: Standard output for task 98 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68ae1b2a-a115-43b7-a1a8-ca55ebe8e510_isrForCrosstalkSources_996985_1.stdout parsl.dataflow.dflow INFO: Standard error for task 98 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68ae1b2a-a115-43b7-a1a8-ca55ebe8e510_isrForCrosstalkSources_996985_1.stderr parsl.dataflow.dflow DEBUG: Task 99 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 99 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 99 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 99 has memoization hash 02435cea2230a42a584e4534491fcfea parsl.dataflow.memoization INFO: Task 99 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 99 try 0 launched on executor multi with executor id 100 parsl.dataflow.dflow INFO: Standard output for task 99 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/23f02009-c9ce-4a48-8e50-dfcf0d9eb09d_isrForCrosstalkSources_997032_27.stdout parsl.dataflow.dflow INFO: Standard error for task 99 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/23f02009-c9ce-4a48-8e50-dfcf0d9eb09d_isrForCrosstalkSources_997032_27.stderr parsl.dataflow.dflow DEBUG: Task 100 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 100 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 100 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 100 has memoization hash c6ac99ccea5ceb3b411ddb831f1dd423 parsl.dataflow.memoization INFO: Task 100 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 100 try 0 launched on executor multi with executor id 101 parsl.dataflow.dflow INFO: Standard output for task 100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/1feacbec-b11a-4b57-aef9-ce30e05fa82d_isrForCrosstalkSources_996986_25.stdout parsl.dataflow.dflow INFO: Standard error for task 100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/1feacbec-b11a-4b57-aef9-ce30e05fa82d_isrForCrosstalkSources_996986_25.stderr parsl.dataflow.dflow DEBUG: Task 101 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 101 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 101 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 101 has memoization hash 10c29c90f743db4cdd64b6f4c7a570ce parsl.dataflow.memoization INFO: Task 101 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 101 try 0 launched on executor multi with executor id 102 parsl.dataflow.dflow INFO: Standard output for task 101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/713bdaac-b205-4418-b9fc-37a401f36211_isrForCrosstalkSources_996971_25.stdout parsl.dataflow.dflow INFO: Standard error for task 101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/713bdaac-b205-4418-b9fc-37a401f36211_isrForCrosstalkSources_996971_25.stderr parsl.dataflow.dflow DEBUG: Task 102 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 102 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 102 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 102 has memoization hash b7dd9fe157d4da1ed96f6d811d7c9180 parsl.dataflow.memoization INFO: Task 102 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f610a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 102 try 0 launched on executor multi with executor id 103 parsl.dataflow.dflow INFO: Standard output for task 102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/e1eb5408-aa7e-4e5e-b929-d0ae45bf0fc4_isrForCrosstalkSources_996973_58.stdout parsl.dataflow.dflow INFO: Standard error for task 102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/e1eb5408-aa7e-4e5e-b929-d0ae45bf0fc4_isrForCrosstalkSources_996973_58.stderr parsl.dataflow.dflow DEBUG: Task 103 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 103 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 103 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 103 has memoization hash c7cca8de6c3860073b94944dd1fb530d parsl.dataflow.memoization INFO: Task 103 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8300e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 103 try 0 launched on executor multi with executor id 104 parsl.dataflow.dflow INFO: Standard output for task 103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2f725ab0-a311-4628-9450-5727c62351a8_isrForCrosstalkSources_996974_9.stdout parsl.dataflow.dflow INFO: Standard error for task 103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2f725ab0-a311-4628-9450-5727c62351a8_isrForCrosstalkSources_996974_9.stderr parsl.dataflow.dflow DEBUG: Task 104 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 104 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 104 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 104 has memoization hash 6d02dac4c6d80d39dd6061510a600792 parsl.dataflow.memoization INFO: Task 104 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 104 try 0 launched on executor multi with executor id 105 parsl.dataflow.dflow INFO: Standard output for task 104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ddc5b242-a550-4d1b-8c05-a26f3d5e99f0_isrForCrosstalkSources_997020_55.stdout parsl.dataflow.dflow INFO: Standard error for task 104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ddc5b242-a550-4d1b-8c05-a26f3d5e99f0_isrForCrosstalkSources_997020_55.stderr parsl.dataflow.dflow DEBUG: Task 105 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 105 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 105 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 105 has memoization hash cbbab1d9fb35a1602592d332ec62ccc8 parsl.dataflow.memoization INFO: Task 105 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 105 try 0 launched on executor multi with executor id 106 parsl.dataflow.dflow INFO: Standard output for task 105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a3fa0d90-e685-4bf4-a223-07737d8ea342_isrForCrosstalkSources_997011_59.stdout parsl.dataflow.dflow INFO: Standard error for task 105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a3fa0d90-e685-4bf4-a223-07737d8ea342_isrForCrosstalkSources_997011_59.stderr parsl.dataflow.dflow DEBUG: Task 106 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 106 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 106 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 106 has memoization hash b48ca35e1ee98a3a2207cf8dc602664b parsl.dataflow.memoization INFO: Task 106 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8302c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 106 try 0 launched on executor multi with executor id 107 parsl.dataflow.dflow INFO: Standard output for task 106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/312595b0-4d49-47fe-ad5b-58c589ac4314_isrForCrosstalkSources_996981_17.stdout parsl.dataflow.dflow INFO: Standard error for task 106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/312595b0-4d49-47fe-ad5b-58c589ac4314_isrForCrosstalkSources_996981_17.stderr parsl.dataflow.dflow DEBUG: Task 107 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 107 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 107 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 107 has memoization hash 96f7b996723e623ed9b9cf4e34214a8e parsl.dataflow.memoization INFO: Task 107 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 107 try 0 launched on executor multi with executor id 108 parsl.dataflow.dflow INFO: Standard output for task 107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cf20fafe-976e-438a-9032-f0a9f0656692_isrForCrosstalkSources_997024_8.stdout parsl.dataflow.dflow INFO: Standard error for task 107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cf20fafe-976e-438a-9032-f0a9f0656692_isrForCrosstalkSources_997024_8.stderr parsl.dataflow.dflow DEBUG: Task 108 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 108 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 108 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 108 has memoization hash fdb727f5d1be7c9f04870efabaf56f15 parsl.dataflow.memoization INFO: Task 108 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 108 try 0 launched on executor multi with executor id 109 parsl.dataflow.dflow INFO: Standard output for task 108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/927cb936-48c8-4e0b-acaf-7ecb9ade929a_isrForCrosstalkSources_997032_22.stdout parsl.dataflow.dflow INFO: Standard error for task 108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/927cb936-48c8-4e0b-acaf-7ecb9ade929a_isrForCrosstalkSources_997032_22.stderr parsl.dataflow.dflow DEBUG: Task 109 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 109 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 109 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 109 has memoization hash 44b6f21012b0113e651e1a8089b661ab parsl.dataflow.memoization INFO: Task 109 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 109 try 0 launched on executor multi with executor id 110 parsl.dataflow.dflow INFO: Standard output for task 109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/a9d09f05-fb54-4d94-ae6a-60a08518964f_isrForCrosstalkSources_997024_38.stdout parsl.dataflow.dflow INFO: Standard error for task 109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/a9d09f05-fb54-4d94-ae6a-60a08518964f_isrForCrosstalkSources_997024_38.stderr parsl.dataflow.dflow DEBUG: Task 110 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 110 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 110 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 110 has memoization hash 44a4d70c87b069c97ba0f14566cfebb1 parsl.dataflow.memoization INFO: Task 110 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8304a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 110 try 0 launched on executor multi with executor id 111 parsl.dataflow.dflow INFO: Standard output for task 110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b00777c1-8bd8-43c9-9fb8-0ae58f88e4ad_isrForCrosstalkSources_996997_39.stdout parsl.dataflow.dflow INFO: Standard error for task 110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b00777c1-8bd8-43c9-9fb8-0ae58f88e4ad_isrForCrosstalkSources_996997_39.stderr parsl.dataflow.dflow DEBUG: Task 111 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 111 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 111 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 111 has memoization hash dfc0522b952a3e3d069affbefdfd887f parsl.dataflow.memoization INFO: Task 111 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8305e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 111 try 0 launched on executor multi with executor id 112 parsl.dataflow.dflow INFO: Standard output for task 111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/60f490e6-1613-48c6-8926-c7483f90bce0_isrForCrosstalkSources_996959_9.stdout parsl.dataflow.dflow INFO: Standard error for task 111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/60f490e6-1613-48c6-8926-c7483f90bce0_isrForCrosstalkSources_996959_9.stderr parsl.dataflow.dflow DEBUG: Task 112 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 112 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 112 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 112 has memoization hash f74c13535b10e8202fa718b0c32bf32b parsl.dataflow.memoization INFO: Task 112 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 112 try 0 launched on executor multi with executor id 113 parsl.dataflow.dflow INFO: Standard output for task 112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/d34c8067-d669-48f6-9d33-7a07ccd43b45_isrForCrosstalkSources_997018_12.stdout parsl.dataflow.dflow INFO: Standard error for task 112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/d34c8067-d669-48f6-9d33-7a07ccd43b45_isrForCrosstalkSources_997018_12.stderr parsl.dataflow.dflow DEBUG: Task 113 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 113 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 113 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 113 has memoization hash e500889f3f8a804a0c99b9dff9c8471a parsl.dataflow.memoization INFO: Task 113 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 113 try 0 launched on executor multi with executor id 114 parsl.dataflow.dflow INFO: Standard output for task 113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0227d6d3-7f5f-4c71-a332-033cc5919fa5_isrForCrosstalkSources_996981_31.stdout parsl.dataflow.dflow INFO: Standard error for task 113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0227d6d3-7f5f-4c71-a332-033cc5919fa5_isrForCrosstalkSources_996981_31.stderr parsl.dataflow.dflow DEBUG: Task 114 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 114 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 114 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 114 has memoization hash b08306ee7d2d0de3b48c7c08d68fbb2d parsl.dataflow.memoization INFO: Task 114 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 114 try 0 launched on executor multi with executor id 115 parsl.dataflow.dflow INFO: Standard output for task 114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c54f1ece-7134-447c-8224-c94a960a5f24_isrForCrosstalkSources_996961_8.stdout parsl.dataflow.dflow INFO: Standard error for task 114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c54f1ece-7134-447c-8224-c94a960a5f24_isrForCrosstalkSources_996961_8.stderr parsl.dataflow.dflow DEBUG: Task 115 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 115 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 115 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 115 has memoization hash 3f2ce15a3850b6c9b5e056a9479b7934 parsl.dataflow.memoization INFO: Task 115 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8307c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 115 try 0 launched on executor multi with executor id 116 parsl.dataflow.dflow INFO: Standard output for task 115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/49f57fd2-7bff-4a57-8f9f-e2bafcd2306c_isrForCrosstalkSources_996953_46.stdout parsl.dataflow.dflow INFO: Standard error for task 115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/49f57fd2-7bff-4a57-8f9f-e2bafcd2306c_isrForCrosstalkSources_996953_46.stderr parsl.dataflow.dflow DEBUG: Task 116 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 116 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 116 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 116 has memoization hash 313b75e61a7ee74e56cedc1d85289299 parsl.dataflow.memoization INFO: Task 116 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 116 try 0 launched on executor multi with executor id 117 parsl.dataflow.dflow INFO: Standard output for task 116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/97c692f5-10bd-436d-a790-f79e9a1be7a7_isrForCrosstalkSources_997033_32.stdout parsl.dataflow.dflow INFO: Standard error for task 116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/97c692f5-10bd-436d-a790-f79e9a1be7a7_isrForCrosstalkSources_997033_32.stderr parsl.dataflow.dflow DEBUG: Task 117 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 117 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 117 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 117 has memoization hash 54ecdb941fc14267235d747f6a5573ea parsl.dataflow.memoization INFO: Task 117 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8309a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 117 try 0 launched on executor multi with executor id 118 parsl.dataflow.dflow INFO: Standard output for task 117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5916c9a8-54db-450c-9356-3fa2dcd8dac3_isrForCrosstalkSources_996968_47.stdout parsl.dataflow.dflow INFO: Standard error for task 117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5916c9a8-54db-450c-9356-3fa2dcd8dac3_isrForCrosstalkSources_996968_47.stderr parsl.dataflow.dflow DEBUG: Task 118 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 118 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 118 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 118 has memoization hash f9e658452d15f672201398ca64d017a6 parsl.dataflow.memoization INFO: Task 118 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 118 try 0 launched on executor multi with executor id 119 parsl.dataflow.dflow INFO: Standard output for task 118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1fc92050-7c34-4fbe-9768-f6eed0cc0552_isrForCrosstalkSources_996964_6.stdout parsl.dataflow.dflow INFO: Standard error for task 118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1fc92050-7c34-4fbe-9768-f6eed0cc0552_isrForCrosstalkSources_996964_6.stderr parsl.dataflow.dflow DEBUG: Task 119 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 119 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 119 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 119 has memoization hash 2eb95725fde4673e05523ea19725c37f parsl.dataflow.memoization INFO: Task 119 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 119 try 0 launched on executor multi with executor id 120 parsl.dataflow.dflow INFO: Standard output for task 119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/5d437e0a-16b3-4132-b251-b349a07a79a3_isrForCrosstalkSources_997011_8.stdout parsl.dataflow.dflow INFO: Standard error for task 119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/5d437e0a-16b3-4132-b251-b349a07a79a3_isrForCrosstalkSources_997011_8.stderr parsl.dataflow.dflow DEBUG: Task 120 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 120 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 120 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 120 has memoization hash 017a23a876f1168300e799395c40f2a4 parsl.dataflow.memoization INFO: Task 120 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 120 try 0 launched on executor multi with executor id 121 parsl.dataflow.dflow INFO: Standard output for task 120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c20d9e14-bc0b-4073-9420-9f208149ff81_isrForCrosstalkSources_996993_17.stdout parsl.dataflow.dflow INFO: Standard error for task 120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c20d9e14-bc0b-4073-9420-9f208149ff81_isrForCrosstalkSources_996993_17.stderr parsl.dataflow.dflow DEBUG: Task 121 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 121 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 121 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 121 has memoization hash 89da1e49d15cafe7c4a28b5e90f95d5c parsl.dataflow.memoization INFO: Task 121 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 121 try 0 launched on executor multi with executor id 122 parsl.dataflow.dflow INFO: Standard output for task 121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/535d81ae-f011-4dd8-9aaa-8e88d0911479_isrForCrosstalkSources_996971_6.stdout parsl.dataflow.dflow INFO: Standard error for task 121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/535d81ae-f011-4dd8-9aaa-8e88d0911479_isrForCrosstalkSources_996971_6.stderr parsl.dataflow.dflow DEBUG: Task 122 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 122 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 122 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 122 has memoization hash 8905255393440e4252dc1680a0ce7fb0 parsl.dataflow.memoization INFO: Task 122 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 122 try 0 launched on executor multi with executor id 123 parsl.dataflow.dflow INFO: Standard output for task 122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c90b80d7-a453-4f38-9f87-e5fd13fb895d_isrForCrosstalkSources_996955_56.stdout parsl.dataflow.dflow INFO: Standard error for task 122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c90b80d7-a453-4f38-9f87-e5fd13fb895d_isrForCrosstalkSources_996955_56.stderr parsl.dataflow.dflow DEBUG: Task 123 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 123 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 123 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 123 has memoization hash bcbfc096cdb79d11231f2333da52aee9 parsl.dataflow.memoization INFO: Task 123 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 123 try 0 launched on executor multi with executor id 124 parsl.dataflow.dflow INFO: Standard output for task 123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/2894e442-8d1f-4dc4-8dc4-aa8ba218152b_isrForCrosstalkSources_997018_10.stdout parsl.dataflow.dflow INFO: Standard error for task 123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/2894e442-8d1f-4dc4-8dc4-aa8ba218152b_isrForCrosstalkSources_997018_10.stderr parsl.dataflow.dflow DEBUG: Task 124 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 124 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 124 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 124 has memoization hash 7ef6d4b4ebfa09226f340042a85609d4 parsl.dataflow.memoization INFO: Task 124 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 124 try 0 launched on executor multi with executor id 125 parsl.dataflow.dflow INFO: Standard output for task 124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/88aaceea-5b6c-4d88-ba64-918da8291703_isrForCrosstalkSources_997014_44.stdout parsl.dataflow.dflow INFO: Standard error for task 124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/88aaceea-5b6c-4d88-ba64-918da8291703_isrForCrosstalkSources_997014_44.stderr parsl.dataflow.dflow DEBUG: Task 125 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 125 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 125 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 125 has memoization hash ddc470688bb5443e40390d40defa3861 parsl.dataflow.memoization INFO: Task 125 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 125 try 0 launched on executor multi with executor id 126 parsl.dataflow.dflow INFO: Standard output for task 125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/561e693e-8aad-4ff6-991e-1ba5b3fd03a4_isrForCrosstalkSources_996958_20.stdout parsl.dataflow.dflow INFO: Standard error for task 125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/561e693e-8aad-4ff6-991e-1ba5b3fd03a4_isrForCrosstalkSources_996958_20.stderr parsl.dataflow.dflow DEBUG: Task 126 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 126 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 126 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 126 has memoization hash befc43c9a5dd9829750e73f3c0f47219 parsl.dataflow.memoization INFO: Task 126 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 126 try 0 launched on executor multi with executor id 127 parsl.dataflow.dflow INFO: Standard output for task 126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/5a72cd91-3565-4858-bac6-f1eaf38c4097_isrForCrosstalkSources_997007_5.stdout parsl.dataflow.dflow INFO: Standard error for task 126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/5a72cd91-3565-4858-bac6-f1eaf38c4097_isrForCrosstalkSources_997007_5.stderr parsl.dataflow.dflow DEBUG: Task 127 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 127 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 127 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 127 has memoization hash 5b6c9bf5bbc4ceed02661321fd8443d1 parsl.dataflow.memoization INFO: Task 127 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 127 try 0 launched on executor multi with executor id 128 parsl.dataflow.dflow INFO: Standard output for task 127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c6f89d07-c99e-4b6b-96cb-b584ef2ca7ea_isrForCrosstalkSources_996966_12.stdout parsl.dataflow.dflow INFO: Standard error for task 127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c6f89d07-c99e-4b6b-96cb-b584ef2ca7ea_isrForCrosstalkSources_996966_12.stderr parsl.dataflow.dflow DEBUG: Task 128 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 128 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 128 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 128 has memoization hash 2af846a7ae6d55a32aaad5738fc65c99 parsl.dataflow.memoization INFO: Task 128 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 128 try 0 launched on executor multi with executor id 129 parsl.dataflow.dflow INFO: Standard output for task 128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/0368705d-b686-4046-b784-b80fdc9f8144_isrForCrosstalkSources_996995_34.stdout parsl.dataflow.dflow INFO: Standard error for task 128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/0368705d-b686-4046-b784-b80fdc9f8144_isrForCrosstalkSources_996995_34.stderr parsl.dataflow.dflow DEBUG: Task 129 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 129 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 129 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 129 has memoization hash 081cf076af672236374bdbd20fbde6d1 parsl.dataflow.memoization INFO: Task 129 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8311c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 129 try 0 launched on executor multi with executor id 130 parsl.dataflow.dflow INFO: Standard output for task 129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7728f295-4697-4a29-a43d-700ad042b218_isrForCrosstalkSources_996993_23.stdout parsl.dataflow.dflow INFO: Standard error for task 129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7728f295-4697-4a29-a43d-700ad042b218_isrForCrosstalkSources_996993_23.stderr parsl.dataflow.dflow DEBUG: Task 130 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 130 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 130 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 130 has memoization hash ae96883ada8d87d9a5eb1a763495e625 parsl.dataflow.memoization INFO: Task 130 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 130 try 0 launched on executor multi with executor id 131 parsl.dataflow.dflow INFO: Standard output for task 130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/8f474223-1f82-4826-882d-b9a1159e4239_isrForCrosstalkSources_996988_11.stdout parsl.dataflow.dflow INFO: Standard error for task 130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/8f474223-1f82-4826-882d-b9a1159e4239_isrForCrosstalkSources_996988_11.stderr parsl.dataflow.dflow DEBUG: Task 131 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 131 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 131 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 131 has memoization hash 7690c3080b987e708e4dc4bea8fd87da parsl.dataflow.memoization INFO: Task 131 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 131 try 0 launched on executor multi with executor id 132 parsl.dataflow.dflow INFO: Standard output for task 131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/8ae25595-3d99-40f6-948d-80483b43fbbe_isrForCrosstalkSources_997001_6.stdout parsl.dataflow.dflow INFO: Standard error for task 131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/8ae25595-3d99-40f6-948d-80483b43fbbe_isrForCrosstalkSources_997001_6.stderr parsl.dataflow.dflow DEBUG: Task 132 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 132 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 132 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 132 has memoization hash 75ed1ca17c42010fc18caac007f934f1 parsl.dataflow.memoization INFO: Task 132 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 132 try 0 launched on executor multi with executor id 133 parsl.dataflow.dflow INFO: Standard output for task 132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/08dbb5d2-b3c3-48cd-892b-79f7d83cbd2b_isrForCrosstalkSources_996967_6.stdout parsl.dataflow.dflow INFO: Standard error for task 132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/08dbb5d2-b3c3-48cd-892b-79f7d83cbd2b_isrForCrosstalkSources_996967_6.stderr parsl.dataflow.dflow DEBUG: Task 133 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 133 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 133 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 133 has memoization hash 8c6f2ec66510fd6dcef1d4ffa8e016a5 parsl.dataflow.memoization INFO: Task 133 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8313a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 133 try 0 launched on executor multi with executor id 134 parsl.dataflow.dflow INFO: Standard output for task 133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/29bdc3b1-9e77-4273-8118-1ecc82846ff8_isrForCrosstalkSources_997011_30.stdout parsl.dataflow.dflow INFO: Standard error for task 133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/29bdc3b1-9e77-4273-8118-1ecc82846ff8_isrForCrosstalkSources_997011_30.stderr parsl.dataflow.dflow DEBUG: Task 134 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 134 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 134 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 134 has memoization hash 16cb31d47eacd0a1e97daf08784bca80 parsl.dataflow.memoization INFO: Task 134 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8314e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 134 try 0 launched on executor multi with executor id 135 parsl.dataflow.dflow INFO: Standard output for task 134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c645529b-1d87-4ea8-8b0a-e76be9278e2d_isrForCrosstalkSources_997028_41.stdout parsl.dataflow.dflow INFO: Standard error for task 134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c645529b-1d87-4ea8-8b0a-e76be9278e2d_isrForCrosstalkSources_997028_41.stderr parsl.dataflow.dflow DEBUG: Task 135 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 135 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 135 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 135 has memoization hash 8ea6580294160ce3f87a204dbf554f79 parsl.dataflow.memoization INFO: Task 135 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 135 try 0 launched on executor multi with executor id 136 parsl.dataflow.dflow INFO: Standard output for task 135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6fb54150-36fb-4395-9006-250fae2766da_isrForCrosstalkSources_996970_7.stdout parsl.dataflow.dflow INFO: Standard error for task 135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6fb54150-36fb-4395-9006-250fae2766da_isrForCrosstalkSources_996970_7.stderr parsl.dataflow.dflow DEBUG: Task 136 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 136 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 136 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 136 has memoization hash 647ad5b9bce1e28d2af0f243e5cfd7b2 parsl.dataflow.memoization INFO: Task 136 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 136 try 0 launched on executor multi with executor id 137 parsl.dataflow.dflow INFO: Standard output for task 136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e20e2a1f-bec9-428c-b83e-df1322496f06_isrForCrosstalkSources_997030_30.stdout parsl.dataflow.dflow INFO: Standard error for task 136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e20e2a1f-bec9-428c-b83e-df1322496f06_isrForCrosstalkSources_997030_30.stderr parsl.dataflow.dflow DEBUG: Task 137 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 137 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 137 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 137 has memoization hash 3fb5ad27a8c0238565f34d1a221bb56b parsl.dataflow.memoization INFO: Task 137 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 137 try 0 launched on executor multi with executor id 138 parsl.dataflow.dflow INFO: Standard output for task 137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/a02fa3a0-c388-4203-adeb-93f60b127369_isrForCrosstalkSources_996986_45.stdout parsl.dataflow.dflow INFO: Standard error for task 137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/a02fa3a0-c388-4203-adeb-93f60b127369_isrForCrosstalkSources_996986_45.stderr parsl.dataflow.dflow DEBUG: Task 138 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 138 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 138 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 138 has memoization hash 2232c367642f242dc113a32ce04e07cf parsl.dataflow.memoization INFO: Task 138 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8316c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 138 try 0 launched on executor multi with executor id 139 parsl.dataflow.dflow INFO: Standard output for task 138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/8842e99a-dc56-44ab-b4b0-d18e7bff5e3d_isrForCrosstalkSources_997026_28.stdout parsl.dataflow.dflow INFO: Standard error for task 138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/8842e99a-dc56-44ab-b4b0-d18e7bff5e3d_isrForCrosstalkSources_997026_28.stderr parsl.dataflow.dflow DEBUG: Task 139 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 139 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 139 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 139 has memoization hash 91cdaf77891e8e19906c577571b6b246 parsl.dataflow.memoization INFO: Task 139 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 139 try 0 launched on executor multi with executor id 140 parsl.dataflow.dflow INFO: Standard output for task 139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/6f559da1-75aa-4fbe-960a-ca09e2c005ad_isrForCrosstalkSources_997027_53.stdout parsl.dataflow.dflow INFO: Standard error for task 139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/6f559da1-75aa-4fbe-960a-ca09e2c005ad_isrForCrosstalkSources_997027_53.stderr parsl.dataflow.dflow DEBUG: Task 140 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 140 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 140 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 140 has memoization hash b44001936dca3645f2a37ad37f5b19ca parsl.dataflow.memoization INFO: Task 140 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 140 try 0 launched on executor multi with executor id 141 parsl.dataflow.dflow INFO: Standard output for task 140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5b4d8ad4-77a6-4b6c-8323-76761489a847_isrForCrosstalkSources_997004_21.stdout parsl.dataflow.dflow INFO: Standard error for task 140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5b4d8ad4-77a6-4b6c-8323-76761489a847_isrForCrosstalkSources_997004_21.stderr parsl.dataflow.dflow DEBUG: Task 141 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 141 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 141 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 141 has memoization hash 1f54e356a3c8d3b30d873a716978c209 parsl.dataflow.memoization INFO: Task 141 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8318a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 141 try 0 launched on executor multi with executor id 142 parsl.dataflow.dflow INFO: Standard output for task 141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2f63142-3bf6-4932-a1f6-8ddb9bf7b9f3_isrForCrosstalkSources_996996_30.stdout parsl.dataflow.dflow INFO: Standard error for task 141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2f63142-3bf6-4932-a1f6-8ddb9bf7b9f3_isrForCrosstalkSources_996996_30.stderr parsl.dataflow.dflow DEBUG: Task 142 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 142 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 142 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 142 has memoization hash b96a2e3fab51e7e1507ecdd35da20f58 parsl.dataflow.memoization INFO: Task 142 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 142 try 0 launched on executor multi with executor id 143 parsl.dataflow.dflow INFO: Standard output for task 142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/4992e519-b6ad-4351-865f-2aaf8f246934_isrForCrosstalkSources_997011_19.stdout parsl.dataflow.dflow INFO: Standard error for task 142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/4992e519-b6ad-4351-865f-2aaf8f246934_isrForCrosstalkSources_997011_19.stderr parsl.dataflow.dflow DEBUG: Task 143 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 143 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 143 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 143 has memoization hash ac351d43ca23f5603f8e21a0722837ae parsl.dataflow.memoization INFO: Task 143 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8319e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 143 try 0 launched on executor multi with executor id 144 parsl.dataflow.dflow INFO: Standard output for task 143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/de93b5d7-484a-4ac0-ad41-3c65bdf9ada6_isrForCrosstalkSources_997010_12.stdout parsl.dataflow.dflow INFO: Standard error for task 143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/de93b5d7-484a-4ac0-ad41-3c65bdf9ada6_isrForCrosstalkSources_997010_12.stderr parsl.dataflow.dflow DEBUG: Task 144 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 144 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 144 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 144 has memoization hash e4932da250504f251deac864007f5754 parsl.dataflow.memoization INFO: Task 144 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 144 try 0 launched on executor multi with executor id 145 parsl.dataflow.dflow INFO: Standard output for task 144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6f089fc4-fed1-42d4-acac-62c995c2ed38_isrForCrosstalkSources_997030_42.stdout parsl.dataflow.dflow INFO: Standard error for task 144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6f089fc4-fed1-42d4-acac-62c995c2ed38_isrForCrosstalkSources_997030_42.stderr parsl.dataflow.dflow DEBUG: Task 145 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 145 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 145 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 145 has memoization hash 4896f1712bc9b730a48da4ac455bd601 parsl.dataflow.memoization INFO: Task 145 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 145 try 0 launched on executor multi with executor id 146 parsl.dataflow.dflow INFO: Standard output for task 145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/df91b8a8-4823-4d77-a7f0-c1b884e4180c_isrForCrosstalkSources_996977_1.stdout parsl.dataflow.dflow INFO: Standard error for task 145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/df91b8a8-4823-4d77-a7f0-c1b884e4180c_isrForCrosstalkSources_996977_1.stderr parsl.dataflow.dflow DEBUG: Task 146 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 146 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 146 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 146 has memoization hash ff6400bcaa1b87766f9522cf1e9223df parsl.dataflow.memoization INFO: Task 146 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 146 try 0 launched on executor multi with executor id 147 parsl.dataflow.dflow INFO: Standard output for task 146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d1e3a898-fa97-4af6-8eda-606fd9a4ac1e_isrForCrosstalkSources_996981_15.stdout parsl.dataflow.dflow INFO: Standard error for task 146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d1e3a898-fa97-4af6-8eda-606fd9a4ac1e_isrForCrosstalkSources_996981_15.stderr parsl.dataflow.dflow DEBUG: Task 147 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 147 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 147 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 147 has memoization hash 62cadbb1cf5b7bdfb567faaaac9d4b28 parsl.dataflow.memoization INFO: Task 147 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 147 try 0 launched on executor multi with executor id 148 parsl.dataflow.dflow INFO: Standard output for task 147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6d091807-6efc-47fa-bbee-03ee1f42c416_isrForCrosstalkSources_996954_24.stdout parsl.dataflow.dflow INFO: Standard error for task 147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6d091807-6efc-47fa-bbee-03ee1f42c416_isrForCrosstalkSources_996954_24.stderr parsl.dataflow.dflow DEBUG: Task 148 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 148 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 148 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 148 has memoization hash 719e1e3d8e8f96354cb688a7a48ad157 parsl.dataflow.memoization INFO: Task 148 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 148 try 0 launched on executor multi with executor id 149 parsl.dataflow.dflow INFO: Standard output for task 148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cd606719-65e8-4950-b601-2ce5de073951_isrForCrosstalkSources_996988_34.stdout parsl.dataflow.dflow INFO: Standard error for task 148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cd606719-65e8-4950-b601-2ce5de073951_isrForCrosstalkSources_996988_34.stderr parsl.dataflow.dflow DEBUG: Task 149 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 149 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 149 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 149 has memoization hash 089da2324b153f0141e2c2afef6f8aa0 parsl.dataflow.memoization INFO: Task 149 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:24 proc_lsst.multi:146 [INFO] found job 91028 in provider local proc_lsst.multi INFO: found job 91028 in provider local parsl.dataflow.dflow INFO: Parsl task 149 try 0 launched on executor multi with executor id 150 parsl.dataflow.dflow INFO: Standard output for task 149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/06e39138-0364-4976-b42e-1cd1e3fdb857_isrForCrosstalkSources_997019_6.stdout parsl.dataflow.dflow INFO: Standard error for task 149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/06e39138-0364-4976-b42e-1cd1e3fdb857_isrForCrosstalkSources_997019_6.stderr parsl.dataflow.dflow DEBUG: Task 150 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 150 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 150 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 150 has memoization hash 4e03e0cc7c660eaeb8a1cea44766a1f1 parsl.dataflow.memoization INFO: Task 150 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 150 try 0 launched on executor multi with executor id 151 parsl.dataflow.dflow INFO: Standard output for task 150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0f10034e-739c-481a-bf0b-f8002daa4336_isrForCrosstalkSources_996966_19.stdout parsl.dataflow.dflow INFO: Standard error for task 150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0f10034e-739c-481a-bf0b-f8002daa4336_isrForCrosstalkSources_996966_19.stderr parsl.dataflow.dflow DEBUG: Task 151 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 151 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 151 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 151 has memoization hash 891f9dc6e928a38c9ed8c4d61499d03a parsl.dataflow.memoization INFO: Task 151 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f830040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 151 try 0 launched on executor multi with executor id 152 parsl.dataflow.dflow INFO: Standard output for task 151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/68b8b0d5-3f3a-4a33-b3f5-02e4118e3259_isrForCrosstalkSources_997017_58.stdout parsl.dataflow.dflow INFO: Standard error for task 151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/68b8b0d5-3f3a-4a33-b3f5-02e4118e3259_isrForCrosstalkSources_997017_58.stderr parsl.dataflow.dflow DEBUG: Task 152 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 152 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 152 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 152 has memoization hash 4175e819d1c4b1ac7a0cd4bb6aa75bcc parsl.dataflow.memoization INFO: Task 152 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 152 try 0 launched on executor multi with executor id 153 parsl.dataflow.dflow INFO: Standard output for task 152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/691bd4be-0e78-44b0-8cd0-a404553b5e77_isrForCrosstalkSources_996979_50.stdout parsl.dataflow.dflow INFO: Standard error for task 152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/691bd4be-0e78-44b0-8cd0-a404553b5e77_isrForCrosstalkSources_996979_50.stderr parsl.dataflow.dflow DEBUG: Task 153 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 153 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 153 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 153 has memoization hash a80fcedb89f95b5d8551f02e3d32139a parsl.dataflow.memoization INFO: Task 153 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 153 try 0 launched on executor multi with executor id 154 parsl.dataflow.dflow INFO: Standard output for task 153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/98cdd415-559d-427d-b2bf-7cccd22af008_isrForCrosstalkSources_996957_20.stdout parsl.dataflow.dflow INFO: Standard error for task 153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/98cdd415-559d-427d-b2bf-7cccd22af008_isrForCrosstalkSources_996957_20.stderr parsl.dataflow.dflow DEBUG: Task 154 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 154 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 154 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 154 has memoization hash 9b61aec4126b43017be9ef39847cdb12 parsl.dataflow.memoization INFO: Task 154 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8323e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 154 try 0 launched on executor multi with executor id 155 parsl.dataflow.dflow INFO: Standard output for task 154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/052b7fb1-2b82-4b49-b08b-3766999ea9dc_isrForCrosstalkSources_997010_4.stdout parsl.dataflow.dflow INFO: Standard error for task 154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/052b7fb1-2b82-4b49-b08b-3766999ea9dc_isrForCrosstalkSources_997010_4.stderr parsl.dataflow.dflow DEBUG: Task 155 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 155 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 155 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 155 has memoization hash 7b1de88b01e92ed3914fd861185c5b41 parsl.dataflow.memoization INFO: Task 155 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 155 try 0 launched on executor multi with executor id 156 parsl.dataflow.dflow INFO: Standard output for task 155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4dfecde3-bc23-4e54-89d0-808dbc6a9180_isrForCrosstalkSources_997014_25.stdout parsl.dataflow.dflow INFO: Standard error for task 155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4dfecde3-bc23-4e54-89d0-808dbc6a9180_isrForCrosstalkSources_997014_25.stderr parsl.dataflow.dflow DEBUG: Task 156 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 156 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 156 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 156 has memoization hash 9339e53f034261383ac5922e5ef18e85 parsl.dataflow.memoization INFO: Task 156 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.strategy DEBUG: general strategy starting with strategy_type simple for 1 executors parsl.dataflow.strategy DEBUG: Strategizing for executor multi parsl.dataflow.strategy DEBUG: Slot ratio calculation: active_slots = 1, active_tasks = 157 parsl.dataflow.strategy DEBUG: Executor multi has 157 active tasks, 1/0 running/pending blocks, and 2 connected workers parsl.dataflow.strategy DEBUG: Strategy case 2: slots are overloaded - (slot_ratio = active_slots/active_tasks) < parallelism parsl.dataflow.strategy DEBUG: Strategy case 2b: active_blocks 1 < max_blocks 312 so scaling out parsl.dataflow.strategy DEBUG: Requesting 39 more blocks parsl.executors.status_handling INFO: Scaling out by 39 blocks parsl.executors.status_handling INFO: Allocated block ID 1 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-1 2024-03-15 13:49:24 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=1 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-1 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=1 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-1 2024-03-15 13:49:24 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=1 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=1 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:24 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=1 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=1 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 0 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 0 2024-03-15 13:49:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:24 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=1 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=1 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 156 try 0 launched on executor multi with executor id 157 parsl.dataflow.dflow INFO: Standard output for task 156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ffef8d4f-a8ef-4ef0-b3d3-a8a260f90c76_isrForCrosstalkSources_996992_12.stdout parsl.dataflow.dflow INFO: Standard error for task 156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ffef8d4f-a8ef-4ef0-b3d3-a8a260f90c76_isrForCrosstalkSources_996992_12.stderr parsl.dataflow.dflow DEBUG: Task 157 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 157 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 157 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 157 has memoization hash 1a93608426730b2f2398031ac83dfcb4 parsl.dataflow.memoization INFO: Task 157 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8322a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 157 try 0 launched on executor multi with executor id 158 parsl.dataflow.dflow INFO: Standard output for task 157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6c125359-a2ee-4c71-91a7-8f558060209a_isrForCrosstalkSources_996991_27.stdout parsl.dataflow.dflow INFO: Standard error for task 157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6c125359-a2ee-4c71-91a7-8f558060209a_isrForCrosstalkSources_996991_27.stderr parsl.dataflow.dflow DEBUG: Task 158 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 158 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 158 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 158 has memoization hash d261534fb99ddfd9698a1efe871f92a1 parsl.dataflow.memoization INFO: Task 158 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8327a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 158 try 0 launched on executor multi with executor id 159 parsl.dataflow.dflow INFO: Standard output for task 158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a6b3769d-308e-4a89-89e8-c3c408d52fd7_isrForCrosstalkSources_997028_25.stdout parsl.dataflow.dflow INFO: Standard error for task 158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a6b3769d-308e-4a89-89e8-c3c408d52fd7_isrForCrosstalkSources_997028_25.stderr parsl.dataflow.dflow DEBUG: Task 159 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 159 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 159 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 159 has memoization hash bf1839026f6ab4d26db4f541dc64b9ae parsl.dataflow.memoization INFO: Task 159 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8328e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 159 try 0 launched on executor multi with executor id 160 parsl.dataflow.dflow INFO: Standard output for task 159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a42815b2-f9ef-4c90-98e8-89219df51fa5_isrForCrosstalkSources_996960_61.stdout parsl.dataflow.dflow INFO: Standard error for task 159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a42815b2-f9ef-4c90-98e8-89219df51fa5_isrForCrosstalkSources_996960_61.stderr parsl.dataflow.dflow DEBUG: Task 160 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 160 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 160 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 160 has memoization hash 3947926d625e7a0d46324a17d2c2f252 parsl.dataflow.memoization INFO: Task 160 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8325c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 160 try 0 launched on executor multi with executor id 161 parsl.dataflow.dflow INFO: Standard output for task 160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/72281006-32e4-4a47-afb2-8a1e2bd7b200_isrForCrosstalkSources_997023_25.stdout parsl.dataflow.dflow INFO: Standard error for task 160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/72281006-32e4-4a47-afb2-8a1e2bd7b200_isrForCrosstalkSources_997023_25.stderr parsl.dataflow.dflow DEBUG: Task 161 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 161 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 161 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 161 has memoization hash 0e9db247aca2dc3f7049e1ceef78f76f parsl.dataflow.memoization INFO: Task 161 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 161 try 0 launched on executor multi with executor id 162 parsl.dataflow.dflow INFO: Standard output for task 161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a75d7578-4a6c-47f5-9d3a-567514e54484_isrForCrosstalkSources_997020_17.stdout parsl.dataflow.dflow INFO: Standard error for task 161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a75d7578-4a6c-47f5-9d3a-567514e54484_isrForCrosstalkSources_997020_17.stderr parsl.dataflow.dflow DEBUG: Task 162 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 162 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 162 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 162 has memoization hash 98848b0cf8b685837a51472cf168f4eb parsl.dataflow.memoization INFO: Task 162 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 162 try 0 launched on executor multi with executor id 163 parsl.dataflow.dflow INFO: Standard output for task 162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/91194d3f-ab3a-4c7c-b15d-8ef43dd0a053_isrForCrosstalkSources_997007_47.stdout parsl.dataflow.dflow INFO: Standard error for task 162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/91194d3f-ab3a-4c7c-b15d-8ef43dd0a053_isrForCrosstalkSources_997007_47.stderr parsl.dataflow.dflow DEBUG: Task 163 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 163 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 163 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 163 has memoization hash 16e9e94051bc6a64ed2a2c88ffcd74d2 parsl.dataflow.memoization INFO: Task 163 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 163 try 0 launched on executor multi with executor id 164 parsl.dataflow.dflow INFO: Standard output for task 163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c9466315-984f-4244-97e9-84a96678f209_isrForCrosstalkSources_997004_46.stdout parsl.dataflow.dflow INFO: Standard error for task 163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c9466315-984f-4244-97e9-84a96678f209_isrForCrosstalkSources_997004_46.stderr parsl.dataflow.dflow DEBUG: Task 164 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 164 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 164 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 164 has memoization hash 57ae08323e82a36bcadf0741c10b3e8f parsl.dataflow.memoization INFO: Task 164 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 164 try 0 launched on executor multi with executor id 165 parsl.dataflow.dflow INFO: Standard output for task 164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6c7c978c-9fb5-4d4e-a3ae-e5e8c23e99d4_isrForCrosstalkSources_997015_14.stdout parsl.dataflow.dflow INFO: Standard error for task 164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6c7c978c-9fb5-4d4e-a3ae-e5e8c23e99d4_isrForCrosstalkSources_997015_14.stderr parsl.dataflow.dflow DEBUG: Task 165 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 165 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 165 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 165 has memoization hash 5d76c3f7062eb12f5a7881b4b6368958 parsl.dataflow.memoization INFO: Task 165 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 165 try 0 launched on executor multi with executor id 166 parsl.dataflow.dflow INFO: Standard output for task 165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/25cec907-7085-4326-bfdc-d3044413eb32_isrForCrosstalkSources_997001_44.stdout parsl.dataflow.dflow INFO: Standard error for task 165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/25cec907-7085-4326-bfdc-d3044413eb32_isrForCrosstalkSources_997001_44.stderr parsl.dataflow.dflow DEBUG: Task 166 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 166 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 166 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 166 has memoization hash dc81ef507d756b2185fd0bc0efb10b1f parsl.dataflow.memoization INFO: Task 166 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 166 try 0 launched on executor multi with executor id 167 parsl.dataflow.dflow INFO: Standard output for task 166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4f6f9ac1-71f7-4a2e-8ec8-4e2cedcb32ee_isrForCrosstalkSources_996957_41.stdout parsl.dataflow.dflow INFO: Standard error for task 166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4f6f9ac1-71f7-4a2e-8ec8-4e2cedcb32ee_isrForCrosstalkSources_996957_41.stderr parsl.dataflow.dflow DEBUG: Task 167 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 167 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 167 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 167 has memoization hash 36d699eadf9dd43631e0d1d41a1ef365 parsl.dataflow.memoization INFO: Task 167 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 167 try 0 launched on executor multi with executor id 168 parsl.dataflow.dflow INFO: Standard output for task 167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/b8492d18-1513-453a-8998-11171d6c5a77_isrForCrosstalkSources_997010_32.stdout parsl.dataflow.dflow INFO: Standard error for task 167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/b8492d18-1513-453a-8998-11171d6c5a77_isrForCrosstalkSources_997010_32.stderr parsl.dataflow.dflow DEBUG: Task 168 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 168 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 168 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 168 has memoization hash 0f5f91b8238a5e5ffe2bc9d132b93c5c parsl.dataflow.memoization INFO: Task 168 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 168 try 0 launched on executor multi with executor id 169 parsl.dataflow.dflow INFO: Standard output for task 168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ae288daa-28b0-45ed-92c2-200d38e254e2_isrForCrosstalkSources_997001_11.stdout parsl.dataflow.dflow INFO: Standard error for task 168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ae288daa-28b0-45ed-92c2-200d38e254e2_isrForCrosstalkSources_997001_11.stderr parsl.dataflow.dflow DEBUG: Task 169 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 169 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 169 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 169 has memoization hash af63338d9d54838c0e4dede95f98cdf7 parsl.dataflow.memoization INFO: Task 169 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 169 try 0 launched on executor multi with executor id 170 parsl.dataflow.dflow INFO: Standard output for task 169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/72e3cc6b-a7f0-4391-b3d4-18ed5436e7d3_isrForCrosstalkSources_996975_3.stdout parsl.dataflow.dflow INFO: Standard error for task 169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/72e3cc6b-a7f0-4391-b3d4-18ed5436e7d3_isrForCrosstalkSources_996975_3.stderr parsl.dataflow.dflow DEBUG: Task 170 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 170 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 170 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 170 has memoization hash 343637131db0fbb3301c2f9518f8dbe6 parsl.dataflow.memoization INFO: Task 170 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 170 try 0 launched on executor multi with executor id 171 parsl.dataflow.dflow INFO: Standard output for task 170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/6ff0f0a0-d431-4b71-8d4d-f9c9dfada8d9_isrForCrosstalkSources_997032_23.stdout parsl.dataflow.dflow INFO: Standard error for task 170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/6ff0f0a0-d431-4b71-8d4d-f9c9dfada8d9_isrForCrosstalkSources_997032_23.stderr parsl.dataflow.dflow DEBUG: Task 171 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 171 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 171 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 171 has memoization hash 4226ca2739d84620f9c72cb7e20ee11e parsl.dataflow.memoization INFO: Task 171 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 171 try 0 launched on executor multi with executor id 172 parsl.dataflow.dflow INFO: Standard output for task 171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a92e28f5-8e86-4eab-afe5-9b14c5788efc_isrForCrosstalkSources_996991_36.stdout parsl.dataflow.dflow INFO: Standard error for task 171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a92e28f5-8e86-4eab-afe5-9b14c5788efc_isrForCrosstalkSources_996991_36.stderr parsl.dataflow.dflow DEBUG: Task 172 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 172 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 172 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 172 has memoization hash b26fa66067e887989cca6efbd42d655c parsl.dataflow.memoization INFO: Task 172 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 172 try 0 launched on executor multi with executor id 173 parsl.dataflow.dflow INFO: Standard output for task 172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ba629af3-22c5-41b1-8a52-7faa9e1a76cc_isrForCrosstalkSources_996989_24.stdout parsl.dataflow.dflow INFO: Standard error for task 172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ba629af3-22c5-41b1-8a52-7faa9e1a76cc_isrForCrosstalkSources_996989_24.stderr parsl.dataflow.dflow DEBUG: Task 173 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 173 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 173 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 173 has memoization hash 580765732597dcc6eb3106dca326611e parsl.dataflow.memoization INFO: Task 173 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 173 try 0 launched on executor multi with executor id 174 parsl.dataflow.dflow INFO: Standard output for task 173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0fb71618-4ad0-43c2-8ab6-cb9281d8e677_isrForCrosstalkSources_996981_46.stdout parsl.dataflow.dflow INFO: Standard error for task 173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0fb71618-4ad0-43c2-8ab6-cb9281d8e677_isrForCrosstalkSources_996981_46.stderr parsl.dataflow.dflow DEBUG: Task 174 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 174 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 174 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 174 has memoization hash 55b57fa48e7e84c2923b4213d9bb8807 parsl.dataflow.memoization INFO: Task 174 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8332e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 174 try 0 launched on executor multi with executor id 175 parsl.dataflow.dflow INFO: Standard output for task 174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1f916f9f-f0d5-47d1-b081-1b5ce79545fa_isrForCrosstalkSources_996961_37.stdout parsl.dataflow.dflow INFO: Standard error for task 174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1f916f9f-f0d5-47d1-b081-1b5ce79545fa_isrForCrosstalkSources_996961_37.stderr parsl.dataflow.dflow DEBUG: Task 175 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 175 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 175 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 175 has memoization hash 7655762cf5455708f64744eace6f0446 parsl.dataflow.memoization INFO: Task 175 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8331a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 175 try 0 launched on executor multi with executor id 176 parsl.dataflow.dflow INFO: Standard output for task 175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7e8da971-fe03-4c86-9ac6-d6fa3cc42049_isrForCrosstalkSources_997025_41.stdout parsl.dataflow.dflow INFO: Standard error for task 175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7e8da971-fe03-4c86-9ac6-d6fa3cc42049_isrForCrosstalkSources_997025_41.stderr parsl.dataflow.dflow DEBUG: Task 176 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 176 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 176 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 176 has memoization hash da3adf4370de9d4d2043de243158dfcf parsl.dataflow.memoization INFO: Task 176 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 176 try 0 launched on executor multi with executor id 177 parsl.dataflow.dflow INFO: Standard output for task 176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/36374751-9c82-44d4-bec2-a000b959ddee_isrForCrosstalkSources_997013_33.stdout parsl.dataflow.dflow INFO: Standard error for task 176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/36374751-9c82-44d4-bec2-a000b959ddee_isrForCrosstalkSources_997013_33.stderr parsl.dataflow.dflow DEBUG: Task 177 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 177 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 177 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 177 has memoization hash 6cf8a73b6e376a4995f5c2b097e6481e parsl.dataflow.memoization INFO: Task 177 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 177 try 0 launched on executor multi with executor id 178 parsl.dataflow.dflow INFO: Standard output for task 177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/94b04152-18e3-43c1-90bc-228e5dfd667e_isrForCrosstalkSources_997000_4.stdout parsl.dataflow.dflow INFO: Standard error for task 177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/94b04152-18e3-43c1-90bc-228e5dfd667e_isrForCrosstalkSources_997000_4.stderr parsl.dataflow.dflow DEBUG: Task 178 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 178 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 178 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 178 has memoization hash 77a3bdc70711847a73b60d055674d111 parsl.dataflow.memoization INFO: Task 178 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 178 try 0 launched on executor multi with executor id 179 parsl.dataflow.dflow INFO: Standard output for task 178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cbc9d2ac-f7cb-4e7f-bd88-cfc94e2c908a_isrForCrosstalkSources_996987_25.stdout parsl.dataflow.dflow INFO: Standard error for task 178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cbc9d2ac-f7cb-4e7f-bd88-cfc94e2c908a_isrForCrosstalkSources_996987_25.stderr parsl.dataflow.dflow DEBUG: Task 179 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 179 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 179 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 179 has memoization hash aca2244c169b3a93408a7fd80e80fd57 parsl.dataflow.memoization INFO: Task 179 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 179 try 0 launched on executor multi with executor id 180 parsl.dataflow.dflow INFO: Standard output for task 179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6326830d-48df-4e3f-a29f-d10e3bc692b2_isrForCrosstalkSources_997005_40.stdout parsl.dataflow.dflow INFO: Standard error for task 179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6326830d-48df-4e3f-a29f-d10e3bc692b2_isrForCrosstalkSources_997005_40.stderr parsl.dataflow.dflow DEBUG: Task 180 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 180 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 180 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 180 has memoization hash 9b08beb40bb82ab9e77c9fea7fc0cd72 parsl.dataflow.memoization INFO: Task 180 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8334c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 180 try 0 launched on executor multi with executor id 181 parsl.dataflow.dflow INFO: Standard output for task 180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/edea1a4c-a085-48f4-b1c7-f2ff3ac53ce9_isrForCrosstalkSources_996992_59.stdout parsl.dataflow.dflow INFO: Standard error for task 180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/edea1a4c-a085-48f4-b1c7-f2ff3ac53ce9_isrForCrosstalkSources_996992_59.stderr parsl.dataflow.dflow DEBUG: Task 181 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 181 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 181 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 181 has memoization hash 162516838ef924313cb81b2df49dd734 parsl.dataflow.memoization INFO: Task 181 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 181 try 0 launched on executor multi with executor id 182 parsl.dataflow.dflow INFO: Standard output for task 181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/85becbf1-4997-432a-a255-3106f2ceeb59_isrForCrosstalkSources_997008_10.stdout parsl.dataflow.dflow INFO: Standard error for task 181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/85becbf1-4997-432a-a255-3106f2ceeb59_isrForCrosstalkSources_997008_10.stderr parsl.dataflow.dflow DEBUG: Task 182 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 182 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 182 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 182 has memoization hash a5aeabe9d252ed917fc60cfe81a84807 parsl.dataflow.memoization INFO: Task 182 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 182 try 0 launched on executor multi with executor id 183 parsl.dataflow.dflow INFO: Standard output for task 182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/21d6cd6f-ca52-455d-8430-24406bb059f4_isrForCrosstalkSources_996979_13.stdout parsl.dataflow.dflow INFO: Standard error for task 182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/21d6cd6f-ca52-455d-8430-24406bb059f4_isrForCrosstalkSources_996979_13.stderr parsl.dataflow.dflow DEBUG: Task 183 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 183 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 183 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 183 has memoization hash 9de42ceeec49018997bf3443797eab80 parsl.dataflow.memoization INFO: Task 183 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 183 try 0 launched on executor multi with executor id 184 parsl.dataflow.dflow INFO: Standard output for task 183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/663042e3-d7a9-447a-93b0-9616d248e102_isrForCrosstalkSources_996971_18.stdout parsl.dataflow.dflow INFO: Standard error for task 183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/663042e3-d7a9-447a-93b0-9616d248e102_isrForCrosstalkSources_996971_18.stderr parsl.dataflow.dflow DEBUG: Task 184 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 184 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 184 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 184 has memoization hash 59be94b79ac548efbaaee7e1f9630698 parsl.dataflow.memoization INFO: Task 184 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 184 try 0 launched on executor multi with executor id 185 parsl.dataflow.dflow INFO: Standard output for task 184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/698c1247-3c49-4cc8-9185-20585f921e92_isrForCrosstalkSources_997011_29.stdout parsl.dataflow.dflow INFO: Standard error for task 184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/698c1247-3c49-4cc8-9185-20585f921e92_isrForCrosstalkSources_997011_29.stderr parsl.dataflow.dflow DEBUG: Task 185 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 185 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 185 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 185 has memoization hash fb9c77c88a1cda83554bc0b8633e2f5a parsl.dataflow.memoization INFO: Task 185 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8336a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 185 try 0 launched on executor multi with executor id 186 parsl.dataflow.dflow INFO: Standard output for task 185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c66e81af-6767-4964-a8a1-7e2198a539a5_isrForCrosstalkSources_997009_44.stdout parsl.dataflow.dflow INFO: Standard error for task 185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c66e81af-6767-4964-a8a1-7e2198a539a5_isrForCrosstalkSources_997009_44.stderr parsl.dataflow.dflow DEBUG: Task 186 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 186 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 186 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 186 has memoization hash a0e8a0d3c1b960efa0584d34f5bf6e9f parsl.dataflow.memoization INFO: Task 186 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 186 try 0 launched on executor multi with executor id 187 parsl.dataflow.dflow INFO: Standard output for task 186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/98d002a0-ad9e-40a7-b405-f459b42fff9d_isrForCrosstalkSources_996972_42.stdout parsl.dataflow.dflow INFO: Standard error for task 186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/98d002a0-ad9e-40a7-b405-f459b42fff9d_isrForCrosstalkSources_996972_42.stderr parsl.dataflow.dflow DEBUG: Task 187 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 187 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 187 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 187 has memoization hash 0ffa87f5b02062890f2d85a6c54f4951 parsl.dataflow.memoization INFO: Task 187 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 187 try 0 launched on executor multi with executor id 188 parsl.dataflow.dflow INFO: Standard output for task 187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/1e909c76-4b95-40d0-83f4-2347c5aa7e78_isrForCrosstalkSources_996978_26.stdout parsl.dataflow.dflow INFO: Standard error for task 187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/1e909c76-4b95-40d0-83f4-2347c5aa7e78_isrForCrosstalkSources_996978_26.stderr parsl.dataflow.dflow DEBUG: Task 188 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 188 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 188 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 188 has memoization hash 174d3fadf13e9aa4edc30e6d6d985c2b parsl.dataflow.memoization INFO: Task 188 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8337e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 188 try 0 launched on executor multi with executor id 189 parsl.dataflow.dflow INFO: Standard output for task 188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/907a24b9-edb6-48c2-800e-1a80edeb3ed8_isrForCrosstalkSources_997023_37.stdout parsl.dataflow.dflow INFO: Standard error for task 188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/907a24b9-edb6-48c2-800e-1a80edeb3ed8_isrForCrosstalkSources_997023_37.stderr parsl.dataflow.dflow DEBUG: Task 189 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 189 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 189 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 189 has memoization hash 675f24d4e504f58a42a11569e9034717 parsl.dataflow.memoization INFO: Task 189 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f8339c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 189 try 0 launched on executor multi with executor id 190 parsl.dataflow.dflow INFO: Standard output for task 189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8e23b2b4-0650-4fd9-a434-28c369740624_isrForCrosstalkSources_996992_61.stdout parsl.dataflow.dflow INFO: Standard error for task 189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8e23b2b4-0650-4fd9-a434-28c369740624_isrForCrosstalkSources_996992_61.stderr parsl.dataflow.dflow DEBUG: Task 190 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 190 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 190 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 190 has memoization hash 453ada7ac66350dd46169a41a8da9e63 parsl.dataflow.memoization INFO: Task 190 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 190 try 0 launched on executor multi with executor id 191 parsl.dataflow.dflow INFO: Standard output for task 190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/ff50f9a6-5a45-4588-8ddf-a7456ecfd891_isrForCrosstalkSources_996964_11.stdout parsl.dataflow.dflow INFO: Standard error for task 190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/ff50f9a6-5a45-4588-8ddf-a7456ecfd891_isrForCrosstalkSources_996964_11.stderr parsl.dataflow.dflow DEBUG: Task 191 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 191 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 191 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 191 has memoization hash 99e40de344335f66aa08fcfa0e18dcdd parsl.dataflow.memoization INFO: Task 191 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 191 try 0 launched on executor multi with executor id 192 parsl.dataflow.dflow INFO: Standard output for task 191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/57d48a80-211b-4a83-aa5b-b2534effb0f8_isrForCrosstalkSources_996958_40.stdout parsl.dataflow.dflow INFO: Standard error for task 191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/57d48a80-211b-4a83-aa5b-b2534effb0f8_isrForCrosstalkSources_996958_40.stderr parsl.dataflow.dflow DEBUG: Task 192 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 192 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 192 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 192 has memoization hash da0ec276c5a7e0fd2338f77b93e80166 parsl.dataflow.memoization INFO: Task 192 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 192 try 0 launched on executor multi with executor id 193 parsl.dataflow.dflow INFO: Standard output for task 192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/824ec742-e9a7-482a-8837-6030989ac3ba_isrForCrosstalkSources_997009_53.stdout parsl.dataflow.dflow INFO: Standard error for task 192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/824ec742-e9a7-482a-8837-6030989ac3ba_isrForCrosstalkSources_997009_53.stderr parsl.dataflow.dflow DEBUG: Task 193 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 193 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 193 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 193 has memoization hash 615ebfae913549c19d3a579998ab8bb3 parsl.dataflow.memoization INFO: Task 193 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 193 try 0 launched on executor multi with executor id 194 parsl.dataflow.dflow INFO: Standard output for task 193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/0fa42b75-b50a-4215-bbaf-88d67312f47e_isrForCrosstalkSources_996968_34.stdout parsl.dataflow.dflow INFO: Standard error for task 193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/0fa42b75-b50a-4215-bbaf-88d67312f47e_isrForCrosstalkSources_996968_34.stderr parsl.dataflow.dflow DEBUG: Task 194 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 194 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 194 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 194 has memoization hash b19bb335261d3311069dce4eb5b1bea3 parsl.dataflow.memoization INFO: Task 194 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 194 try 0 launched on executor multi with executor id 195 parsl.dataflow.dflow INFO: Standard output for task 194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b5c0b3ae-f330-430a-915f-08894805ecbc_isrForCrosstalkSources_996979_32.stdout parsl.dataflow.dflow INFO: Standard error for task 194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b5c0b3ae-f330-430a-915f-08894805ecbc_isrForCrosstalkSources_996979_32.stderr parsl.dataflow.dflow DEBUG: Task 195 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 195 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 195 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 195 has memoization hash cf3978ec6b9bdc02ce1bc529a59428fb parsl.dataflow.memoization INFO: Task 195 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 195 try 0 launched on executor multi with executor id 196 parsl.dataflow.dflow INFO: Standard output for task 195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/8222bb63-d060-434c-ae88-0fd195716ab9_isrForCrosstalkSources_997016_23.stdout parsl.dataflow.dflow INFO: Standard error for task 195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/8222bb63-d060-434c-ae88-0fd195716ab9_isrForCrosstalkSources_997016_23.stderr parsl.dataflow.dflow DEBUG: Task 196 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 196 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 196 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 196 has memoization hash a58a24be9478df45b649ee9f74308dfc parsl.dataflow.memoization INFO: Task 196 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 196 try 0 launched on executor multi with executor id 197 parsl.dataflow.dflow INFO: Standard output for task 196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/861eed98-0eda-4f4e-bcd5-fbb28ca90ab9_isrForCrosstalkSources_997001_41.stdout parsl.dataflow.dflow INFO: Standard error for task 196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/861eed98-0eda-4f4e-bcd5-fbb28ca90ab9_isrForCrosstalkSources_997001_41.stderr parsl.dataflow.dflow DEBUG: Task 197 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 197 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 197 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 197 has memoization hash 6067ec93f11024e6eb0a6a1db9d97cf0 parsl.dataflow.memoization INFO: Task 197 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f833f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 197 try 0 launched on executor multi with executor id 198 parsl.dataflow.dflow INFO: Standard output for task 197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/38126090-ee38-44cc-a1d5-2fc112bd5ee2_isrForCrosstalkSources_996959_29.stdout parsl.dataflow.dflow INFO: Standard error for task 197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/38126090-ee38-44cc-a1d5-2fc112bd5ee2_isrForCrosstalkSources_996959_29.stderr parsl.dataflow.dflow DEBUG: Task 198 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 198 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 198 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 198 has memoization hash 7071e299ad74ea1780de0524a1257664 parsl.dataflow.memoization INFO: Task 198 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 198 try 0 launched on executor multi with executor id 199 parsl.dataflow.dflow INFO: Standard output for task 198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/38f5b633-aded-4239-aaf0-ef076e45f384_isrForCrosstalkSources_996977_17.stdout parsl.dataflow.dflow INFO: Standard error for task 198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/38f5b633-aded-4239-aaf0-ef076e45f384_isrForCrosstalkSources_996977_17.stderr parsl.dataflow.dflow DEBUG: Task 199 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 199 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 199 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 199 has memoization hash 28a2cbf5abc6d5fbc460adb0c96619e9 parsl.dataflow.memoization INFO: Task 199 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 199 try 0 launched on executor multi with executor id 200 parsl.dataflow.dflow INFO: Standard output for task 199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/00ea58bd-c629-4c43-92ff-bc0b4cf9a346_isrForCrosstalkSources_996955_40.stdout parsl.dataflow.dflow INFO: Standard error for task 199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/00ea58bd-c629-4c43-92ff-bc0b4cf9a346_isrForCrosstalkSources_996955_40.stderr parsl.dataflow.dflow DEBUG: Task 200 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 200 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 200 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 200 has memoization hash 53aa750a2d0510725b672a6f34149ce6 parsl.dataflow.memoization INFO: Task 200 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3200e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 200 try 0 launched on executor multi with executor id 201 parsl.dataflow.dflow INFO: Standard output for task 200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/e4d662a7-2b07-477f-a180-6ca1f734492e_isrForCrosstalkSources_996953_53.stdout parsl.dataflow.dflow INFO: Standard error for task 200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/e4d662a7-2b07-477f-a180-6ca1f734492e_isrForCrosstalkSources_996953_53.stderr parsl.dataflow.dflow DEBUG: Task 201 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 201 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 201 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 201 has memoization hash 4cff44260fa0472e3fb4b27a966888eb parsl.dataflow.memoization INFO: Task 201 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 201 try 0 launched on executor multi with executor id 202 parsl.dataflow.dflow INFO: Standard output for task 201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/a61d530b-1df7-4b3a-9b10-50027d302b76_isrForCrosstalkSources_996955_58.stdout parsl.dataflow.dflow INFO: Standard error for task 201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/a61d530b-1df7-4b3a-9b10-50027d302b76_isrForCrosstalkSources_996955_58.stderr parsl.dataflow.dflow DEBUG: Task 202 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 202 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 202 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 202 has memoization hash 7011f325e19ea4eaa6401b49a35a48e3 parsl.dataflow.memoization INFO: Task 202 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3202c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 202 try 0 launched on executor multi with executor id 203 parsl.dataflow.dflow INFO: Standard output for task 202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5bf3606d-8cb9-4807-88e4-f1e83489c52d_isrForCrosstalkSources_996978_1.stdout parsl.dataflow.dflow INFO: Standard error for task 202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5bf3606d-8cb9-4807-88e4-f1e83489c52d_isrForCrosstalkSources_996978_1.stderr parsl.dataflow.dflow DEBUG: Task 203 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 203 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 203 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 203 has memoization hash 6a75121d40e10ecfa88ace7822883bfc parsl.dataflow.memoization INFO: Task 203 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 203 try 0 launched on executor multi with executor id 204 parsl.dataflow.dflow INFO: Standard output for task 203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/bcf53adb-bcf6-4309-a5c2-9a9ba74f6d11_isrForCrosstalkSources_997018_3.stdout parsl.dataflow.dflow INFO: Standard error for task 203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/bcf53adb-bcf6-4309-a5c2-9a9ba74f6d11_isrForCrosstalkSources_997018_3.stderr parsl.dataflow.dflow DEBUG: Task 204 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 204 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 204 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 204 has memoization hash f796598e7df0116186703165ae7dabd2 parsl.dataflow.memoization INFO: Task 204 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3204a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 204 try 0 launched on executor multi with executor id 205 parsl.dataflow.dflow INFO: Standard output for task 204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/32f1524a-d913-4245-8937-f328edcaf359_isrForCrosstalkSources_996996_62.stdout parsl.dataflow.dflow INFO: Standard error for task 204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/32f1524a-d913-4245-8937-f328edcaf359_isrForCrosstalkSources_996996_62.stderr parsl.dataflow.dflow DEBUG: Task 205 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 205 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 205 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 205 has memoization hash ca610664c20339218c0b176dcc00bc56 parsl.dataflow.memoization INFO: Task 205 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 205 try 0 launched on executor multi with executor id 206 parsl.dataflow.dflow INFO: Standard output for task 205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d6923b37-1691-492a-8e83-14cb32163c0e_isrForCrosstalkSources_997010_2.stdout parsl.dataflow.dflow INFO: Standard error for task 205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d6923b37-1691-492a-8e83-14cb32163c0e_isrForCrosstalkSources_997010_2.stderr parsl.dataflow.dflow DEBUG: Task 206 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 206 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 206 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 206 has memoization hash b0399c2fae9a830a309dc1ae0e45141b parsl.dataflow.memoization INFO: Task 206 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:25 proc_lsst.multi:170 [INFO] [multi] job_id 17074833 proc_lsst.multi INFO: [multi] job_id 17074833 2024-03-15 13:49:25 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:25 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:25 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074833 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074833 parsl.executors.status_handling DEBUG: Launched block 1 on executor multi with job ID 17074833 parsl.executors.status_handling INFO: Allocated block ID 2 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-2 2024-03-15 13:49:25 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=2 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-2 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=2 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-2 2024-03-15 13:49:25 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=2 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=2 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:25 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=2 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=2 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:25 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=2 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=2 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 206 try 0 launched on executor multi with executor id 207 parsl.dataflow.dflow INFO: Standard output for task 206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b122600e-48f2-4f40-89de-4dea222b8b69_isrForCrosstalkSources_997014_26.stdout parsl.dataflow.dflow INFO: Standard error for task 206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b122600e-48f2-4f40-89de-4dea222b8b69_isrForCrosstalkSources_997014_26.stderr parsl.dataflow.dflow DEBUG: Task 207 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 207 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 207 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 207 has memoization hash b70329c9703f18d84652c3714a403425 parsl.dataflow.memoization INFO: Task 207 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 207 try 0 launched on executor multi with executor id 208 parsl.dataflow.dflow INFO: Standard output for task 207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/a7f428f2-d86c-42ba-9ac4-dc7a5bb4ef9d_isrForCrosstalkSources_997012_29.stdout parsl.dataflow.dflow INFO: Standard error for task 207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/a7f428f2-d86c-42ba-9ac4-dc7a5bb4ef9d_isrForCrosstalkSources_997012_29.stderr parsl.dataflow.dflow DEBUG: Task 208 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 208 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 208 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 208 has memoization hash 1e06632dfc2263f146b2781953db204e parsl.dataflow.memoization INFO: Task 208 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 208 try 0 launched on executor multi with executor id 209 parsl.dataflow.dflow INFO: Standard output for task 208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c02b27ec-960e-4d5d-a1c6-f7894f011a86_isrForCrosstalkSources_997009_27.stdout parsl.dataflow.dflow INFO: Standard error for task 208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c02b27ec-960e-4d5d-a1c6-f7894f011a86_isrForCrosstalkSources_997009_27.stderr parsl.dataflow.dflow DEBUG: Task 209 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 209 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 209 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 209 has memoization hash a6aa24f646ad22b15a717de88bda04c7 parsl.dataflow.memoization INFO: Task 209 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 209 try 0 launched on executor multi with executor id 210 parsl.dataflow.dflow INFO: Standard output for task 209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/26bf42e5-790c-437b-a627-cffacd6469c6_isrForCrosstalkSources_996974_59.stdout parsl.dataflow.dflow INFO: Standard error for task 209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/26bf42e5-790c-437b-a627-cffacd6469c6_isrForCrosstalkSources_996974_59.stderr parsl.dataflow.dflow DEBUG: Task 210 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 210 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 210 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 210 has memoization hash d29d0aff63cd9712beb3c5e7629e4de6 parsl.dataflow.memoization INFO: Task 210 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3205e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 210 try 0 launched on executor multi with executor id 211 parsl.dataflow.dflow INFO: Standard output for task 210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9baccfe5-a714-4d2c-ad41-e673fcc31c24_isrForCrosstalkSources_997030_15.stdout parsl.dataflow.dflow INFO: Standard error for task 210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9baccfe5-a714-4d2c-ad41-e673fcc31c24_isrForCrosstalkSources_997030_15.stderr parsl.dataflow.dflow DEBUG: Task 211 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 211 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 211 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 211 has memoization hash f868324bf8f200bf6e66353046931804 parsl.dataflow.memoization INFO: Task 211 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 211 try 0 launched on executor multi with executor id 212 parsl.dataflow.dflow INFO: Standard output for task 211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ebce626d-fe78-4726-a4fe-876083e84c42_isrForCrosstalkSources_996984_8.stdout parsl.dataflow.dflow INFO: Standard error for task 211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ebce626d-fe78-4726-a4fe-876083e84c42_isrForCrosstalkSources_996984_8.stderr parsl.dataflow.dflow DEBUG: Task 212 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 212 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 212 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 212 has memoization hash 19f7e3d4abcb761f4f114ca5f58ebf69 parsl.dataflow.memoization INFO: Task 212 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 212 try 0 launched on executor multi with executor id 213 parsl.dataflow.dflow INFO: Standard output for task 212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9e0b4ce5-0157-4d59-88c3-0e45a4679ac8_isrForCrosstalkSources_997000_40.stdout parsl.dataflow.dflow INFO: Standard error for task 212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9e0b4ce5-0157-4d59-88c3-0e45a4679ac8_isrForCrosstalkSources_997000_40.stderr parsl.dataflow.dflow DEBUG: Task 213 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 213 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 213 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 213 has memoization hash 66d88b43e0802936a1077a0dc9030b74 parsl.dataflow.memoization INFO: Task 213 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3207c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 213 try 0 launched on executor multi with executor id 214 parsl.dataflow.dflow INFO: Standard output for task 213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d280ab18-ca0d-4559-b6c1-8f09257bdac5_isrForCrosstalkSources_996989_32.stdout parsl.dataflow.dflow INFO: Standard error for task 213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d280ab18-ca0d-4559-b6c1-8f09257bdac5_isrForCrosstalkSources_996989_32.stderr parsl.dataflow.dflow DEBUG: Task 214 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 214 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 214 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 214 has memoization hash d700f23c9b37e67950ef9165a1077f4e parsl.dataflow.memoization INFO: Task 214 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 214 try 0 launched on executor multi with executor id 215 parsl.dataflow.dflow INFO: Standard output for task 214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/8bc3e6f8-b54a-4995-8b14-46cfb9bf3907_isrForCrosstalkSources_996979_22.stdout parsl.dataflow.dflow INFO: Standard error for task 214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/8bc3e6f8-b54a-4995-8b14-46cfb9bf3907_isrForCrosstalkSources_996979_22.stderr parsl.dataflow.dflow DEBUG: Task 215 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 215 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 215 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 215 has memoization hash d28b8b0a243ac02fae9e192501bf8451 parsl.dataflow.memoization INFO: Task 215 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 215 try 0 launched on executor multi with executor id 216 parsl.dataflow.dflow INFO: Standard output for task 215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/6521fa8f-688a-48e8-8788-2c3efd5cbe7d_isrForCrosstalkSources_997009_54.stdout parsl.dataflow.dflow INFO: Standard error for task 215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/6521fa8f-688a-48e8-8788-2c3efd5cbe7d_isrForCrosstalkSources_997009_54.stderr parsl.dataflow.dflow DEBUG: Task 216 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 216 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 216 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 216 has memoization hash 03ca2db348e8a9ff52c086f0041e61ae parsl.dataflow.memoization INFO: Task 216 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3209a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 216 try 0 launched on executor multi with executor id 217 parsl.dataflow.dflow INFO: Standard output for task 216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a12d938f-09ce-493e-b783-3e86db9b3bbe_isrForCrosstalkSources_997002_4.stdout parsl.dataflow.dflow INFO: Standard error for task 216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a12d938f-09ce-493e-b783-3e86db9b3bbe_isrForCrosstalkSources_997002_4.stderr parsl.dataflow.dflow DEBUG: Task 217 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 217 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 217 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 217 has memoization hash 70149189fb63353c16c97aa994fa261d parsl.dataflow.memoization INFO: Task 217 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 217 try 0 launched on executor multi with executor id 218 parsl.dataflow.dflow INFO: Standard output for task 217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9faecc48-548e-4c20-ab9e-b97d8aafa4b0_isrForCrosstalkSources_996979_42.stdout parsl.dataflow.dflow INFO: Standard error for task 217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9faecc48-548e-4c20-ab9e-b97d8aafa4b0_isrForCrosstalkSources_996979_42.stderr parsl.dataflow.dflow DEBUG: Task 218 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 218 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 218 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 218 has memoization hash fa080f4f687e708178877df345f08c94 parsl.dataflow.memoization INFO: Task 218 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 218 try 0 launched on executor multi with executor id 219 parsl.dataflow.dflow INFO: Standard output for task 218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/3d5fcbe0-a7d1-4cd1-8bb7-973ec8ee9f4b_isrForCrosstalkSources_996983_56.stdout parsl.dataflow.dflow INFO: Standard error for task 218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/3d5fcbe0-a7d1-4cd1-8bb7-973ec8ee9f4b_isrForCrosstalkSources_996983_56.stderr parsl.dataflow.dflow DEBUG: Task 219 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 219 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 219 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 219 has memoization hash 04126ecbf6096815ac350be09b286f91 parsl.dataflow.memoization INFO: Task 219 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 219 try 0 launched on executor multi with executor id 220 parsl.dataflow.dflow INFO: Standard output for task 219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/aebd8aa1-ee70-4aa4-af2d-3b9cd7ce30a5_isrForCrosstalkSources_996988_2.stdout parsl.dataflow.dflow INFO: Standard error for task 219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/aebd8aa1-ee70-4aa4-af2d-3b9cd7ce30a5_isrForCrosstalkSources_996988_2.stderr parsl.dataflow.dflow DEBUG: Task 220 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 220 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 220 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 220 has memoization hash e452580b6ac95732f3c5a534f3a7d123 parsl.dataflow.memoization INFO: Task 220 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 220 try 0 launched on executor multi with executor id 221 parsl.dataflow.dflow INFO: Standard output for task 220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/3ef2c260-4344-4df1-ae18-989d41eb165a_isrForCrosstalkSources_996960_33.stdout parsl.dataflow.dflow INFO: Standard error for task 220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/3ef2c260-4344-4df1-ae18-989d41eb165a_isrForCrosstalkSources_996960_33.stderr parsl.dataflow.dflow DEBUG: Task 221 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 221 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 221 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 221 has memoization hash 1adf07e8060934e2539e1ea0cfa08b21 parsl.dataflow.memoization INFO: Task 221 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 221 try 0 launched on executor multi with executor id 222 parsl.dataflow.dflow INFO: Standard output for task 221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/eed6f060-1708-4240-a19b-57f7b7be73b8_isrForCrosstalkSources_997018_58.stdout parsl.dataflow.dflow INFO: Standard error for task 221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/eed6f060-1708-4240-a19b-57f7b7be73b8_isrForCrosstalkSources_997018_58.stderr parsl.dataflow.dflow DEBUG: Task 222 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 222 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 222 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 222 has memoization hash 2102614bb2dfc1931daedd0bd701a17f parsl.dataflow.memoization INFO: Task 222 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 222 try 0 launched on executor multi with executor id 223 parsl.dataflow.dflow INFO: Standard output for task 222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/60315d7c-2c16-42be-a7ef-ba4381bb6fee_isrForCrosstalkSources_997001_50.stdout parsl.dataflow.dflow INFO: Standard error for task 222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/60315d7c-2c16-42be-a7ef-ba4381bb6fee_isrForCrosstalkSources_997001_50.stderr parsl.dataflow.dflow DEBUG: Task 223 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 223 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 223 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 223 has memoization hash 7b4b6200289e0032f6b8c580e0597751 parsl.dataflow.memoization INFO: Task 223 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 223 try 0 launched on executor multi with executor id 224 parsl.dataflow.dflow INFO: Standard output for task 223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/149f058e-94f3-400a-82b5-c7398c9d85a7_isrForCrosstalkSources_996955_16.stdout parsl.dataflow.dflow INFO: Standard error for task 223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/149f058e-94f3-400a-82b5-c7398c9d85a7_isrForCrosstalkSources_996955_16.stderr parsl.dataflow.dflow DEBUG: Task 224 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 224 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 224 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 224 has memoization hash 629d184a9664d2ac5f1072c11a6af1ae parsl.dataflow.memoization INFO: Task 224 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 224 try 0 launched on executor multi with executor id 225 parsl.dataflow.dflow INFO: Standard output for task 224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/748b5c14-a716-450e-8a82-0e7cdd64dd71_isrForCrosstalkSources_996981_42.stdout parsl.dataflow.dflow INFO: Standard error for task 224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/748b5c14-a716-450e-8a82-0e7cdd64dd71_isrForCrosstalkSources_996981_42.stderr parsl.dataflow.dflow DEBUG: Task 225 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 225 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 225 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 225 has memoization hash b6f8b7c3debebc63d70c9c05e8c3fd65 parsl.dataflow.memoization INFO: Task 225 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 225 try 0 launched on executor multi with executor id 226 parsl.dataflow.dflow INFO: Standard output for task 225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/381fc8f5-6dcb-4aa7-b750-dfbec75730be_isrForCrosstalkSources_996962_46.stdout parsl.dataflow.dflow INFO: Standard error for task 225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/381fc8f5-6dcb-4aa7-b750-dfbec75730be_isrForCrosstalkSources_996962_46.stderr parsl.dataflow.dflow DEBUG: Task 226 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 226 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 226 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 226 has memoization hash 1a6dbd126123c016fdcc7b230e99249c parsl.dataflow.memoization INFO: Task 226 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 226 try 0 launched on executor multi with executor id 227 parsl.dataflow.dflow INFO: Standard output for task 226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/52bb9479-8fca-48b8-8ea8-a15c8d674041_isrForCrosstalkSources_996978_48.stdout parsl.dataflow.dflow INFO: Standard error for task 226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/52bb9479-8fca-48b8-8ea8-a15c8d674041_isrForCrosstalkSources_996978_48.stderr parsl.dataflow.dflow DEBUG: Task 227 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 227 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 227 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 227 has memoization hash e17caeea55401d35ea005a66fc0f9377 parsl.dataflow.memoization INFO: Task 227 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 227 try 0 launched on executor multi with executor id 228 parsl.dataflow.dflow INFO: Standard output for task 227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/08db3d1b-60b4-42bd-9caa-91fcb641702b_isrForCrosstalkSources_997022_39.stdout parsl.dataflow.dflow INFO: Standard error for task 227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/08db3d1b-60b4-42bd-9caa-91fcb641702b_isrForCrosstalkSources_997022_39.stderr parsl.dataflow.dflow DEBUG: Task 228 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 228 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 228 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 228 has memoization hash 3a56e0a492c0789c1917b33bfa3ea46d parsl.dataflow.memoization INFO: Task 228 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 228 try 0 launched on executor multi with executor id 229 parsl.dataflow.dflow INFO: Standard output for task 228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ba909b15-9026-4149-8204-3f75d927fba1_isrForCrosstalkSources_997028_51.stdout parsl.dataflow.dflow INFO: Standard error for task 228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ba909b15-9026-4149-8204-3f75d927fba1_isrForCrosstalkSources_997028_51.stderr parsl.dataflow.dflow DEBUG: Task 229 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 229 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 229 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 229 has memoization hash aa14f99580db2f4709d3a144c98fa006 parsl.dataflow.memoization INFO: Task 229 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 229 try 0 launched on executor multi with executor id 230 parsl.dataflow.dflow INFO: Standard output for task 229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d18e2f1-c5f7-441a-a410-9afd64c56533_isrForCrosstalkSources_996995_13.stdout parsl.dataflow.dflow INFO: Standard error for task 229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d18e2f1-c5f7-441a-a410-9afd64c56533_isrForCrosstalkSources_996995_13.stderr parsl.dataflow.dflow DEBUG: Task 230 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 230 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 230 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 230 has memoization hash 59525b57241fcbfab13957ef3427ca0d parsl.dataflow.memoization INFO: Task 230 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 230 try 0 launched on executor multi with executor id 231 parsl.dataflow.dflow INFO: Standard output for task 230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2047f377-5696-4cc0-a960-e62106da2012_isrForCrosstalkSources_996963_56.stdout parsl.dataflow.dflow INFO: Standard error for task 230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2047f377-5696-4cc0-a960-e62106da2012_isrForCrosstalkSources_996963_56.stderr parsl.dataflow.dflow DEBUG: Task 231 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 231 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 231 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 231 has memoization hash 8cb2109cad5b055accd79fe5705ce04d parsl.dataflow.memoization INFO: Task 231 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3211c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 231 try 0 launched on executor multi with executor id 232 parsl.dataflow.dflow INFO: Standard output for task 231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/01750578-37b3-4ee0-886b-660673030f26_isrForCrosstalkSources_996999_41.stdout parsl.dataflow.dflow INFO: Standard error for task 231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/01750578-37b3-4ee0-886b-660673030f26_isrForCrosstalkSources_996999_41.stderr parsl.dataflow.dflow DEBUG: Task 232 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 232 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 232 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 232 has memoization hash fda67be4134b5527d38faf1e8d90a9a8 parsl.dataflow.memoization INFO: Task 232 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 232 try 0 launched on executor multi with executor id 233 parsl.dataflow.dflow INFO: Standard output for task 232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/f9148d69-7064-46f7-bb6c-cc6934f84e30_isrForCrosstalkSources_996972_47.stdout parsl.dataflow.dflow INFO: Standard error for task 232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/f9148d69-7064-46f7-bb6c-cc6934f84e30_isrForCrosstalkSources_996972_47.stderr parsl.dataflow.dflow DEBUG: Task 233 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 233 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 233 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 233 has memoization hash 95fd83785dddae727e4b24301c0f4a7c parsl.dataflow.memoization INFO: Task 233 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3213a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 233 try 0 launched on executor multi with executor id 234 parsl.dataflow.dflow INFO: Standard output for task 233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/b6e245f6-8783-4616-b5fa-4ddb8629554e_isrForCrosstalkSources_996984_12.stdout parsl.dataflow.dflow INFO: Standard error for task 233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/b6e245f6-8783-4616-b5fa-4ddb8629554e_isrForCrosstalkSources_996984_12.stderr parsl.dataflow.dflow DEBUG: Task 234 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 234 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 234 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 234 has memoization hash 384e9c2a284cdf852df48d0a4e314169 parsl.dataflow.memoization INFO: Task 234 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 234 try 0 launched on executor multi with executor id 235 parsl.dataflow.dflow INFO: Standard output for task 234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7b5d6531-eb25-4098-8d4f-f76df8e7815b_isrForCrosstalkSources_996994_55.stdout parsl.dataflow.dflow INFO: Standard error for task 234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7b5d6531-eb25-4098-8d4f-f76df8e7815b_isrForCrosstalkSources_996994_55.stderr parsl.dataflow.dflow DEBUG: Task 235 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 235 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 235 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 235 has memoization hash 79ddf84cc8530898c2f80cf358ef59d6 parsl.dataflow.memoization INFO: Task 235 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 235 try 0 launched on executor multi with executor id 236 parsl.dataflow.dflow INFO: Standard output for task 235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/25c06052-4d50-44ad-8d86-a27b2d1e60d3_isrForCrosstalkSources_996952_11.stdout parsl.dataflow.dflow INFO: Standard error for task 235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/25c06052-4d50-44ad-8d86-a27b2d1e60d3_isrForCrosstalkSources_996952_11.stderr parsl.dataflow.dflow DEBUG: Task 236 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 236 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 236 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 236 has memoization hash 4ec9f0095d47e0069b16f88adc3a92ad parsl.dataflow.memoization INFO: Task 236 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3214e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 236 try 0 launched on executor multi with executor id 237 parsl.dataflow.dflow INFO: Standard output for task 236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6fea21bd-ca55-45b9-b0a7-df4d312fd860_isrForCrosstalkSources_996973_55.stdout parsl.dataflow.dflow INFO: Standard error for task 236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6fea21bd-ca55-45b9-b0a7-df4d312fd860_isrForCrosstalkSources_996973_55.stderr parsl.dataflow.dflow DEBUG: Task 237 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 237 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 237 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 237 has memoization hash 33596125a2388c43d4eb1b271db48b0d parsl.dataflow.memoization INFO: Task 237 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 237 try 0 launched on executor multi with executor id 238 parsl.dataflow.dflow INFO: Standard output for task 237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/175059bf-3ac9-4819-a41d-785d359cc57a_isrForCrosstalkSources_996977_36.stdout parsl.dataflow.dflow INFO: Standard error for task 237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/175059bf-3ac9-4819-a41d-785d359cc57a_isrForCrosstalkSources_996977_36.stderr parsl.dataflow.dflow DEBUG: Task 238 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 238 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 238 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 238 has memoization hash ea8e14c9873ec7fd15ea2678558960c3 parsl.dataflow.memoization INFO: Task 238 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3216c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 238 try 0 launched on executor multi with executor id 239 parsl.dataflow.dflow INFO: Standard output for task 238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a5002ea6-e10a-479f-9adf-1c2e402fc927_isrForCrosstalkSources_997022_58.stdout parsl.dataflow.dflow INFO: Standard error for task 238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a5002ea6-e10a-479f-9adf-1c2e402fc927_isrForCrosstalkSources_997022_58.stderr parsl.dataflow.dflow DEBUG: Task 239 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 239 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 239 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 239 has memoization hash 32ddf94374ae06c0e2320c70848e223a parsl.dataflow.memoization INFO: Task 239 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 239 try 0 launched on executor multi with executor id 240 parsl.dataflow.dflow INFO: Standard output for task 239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8f9e39b3-f675-4ef0-9ca6-04db25318ea9_isrForCrosstalkSources_996987_16.stdout parsl.dataflow.dflow INFO: Standard error for task 239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8f9e39b3-f675-4ef0-9ca6-04db25318ea9_isrForCrosstalkSources_996987_16.stderr parsl.dataflow.dflow DEBUG: Task 240 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 240 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 240 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 240 has memoization hash 0b4ce257fbe133a5f78bc1377aecb4a6 parsl.dataflow.memoization INFO: Task 240 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 240 try 0 launched on executor multi with executor id 241 parsl.dataflow.dflow INFO: Standard output for task 240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/26dfde53-6b63-4331-bd29-de377a156cc7_isrForCrosstalkSources_996989_17.stdout parsl.dataflow.dflow INFO: Standard error for task 240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/26dfde53-6b63-4331-bd29-de377a156cc7_isrForCrosstalkSources_996989_17.stderr parsl.dataflow.dflow DEBUG: Task 241 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 241 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 241 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 241 has memoization hash fbed5d1a9d3d1b4a448672d4250f4992 parsl.dataflow.memoization INFO: Task 241 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 241 try 0 launched on executor multi with executor id 242 parsl.dataflow.dflow INFO: Standard output for task 241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5cc8ab78-194d-427a-8342-ff11bd79f27b_isrForCrosstalkSources_997009_23.stdout parsl.dataflow.dflow INFO: Standard error for task 241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5cc8ab78-194d-427a-8342-ff11bd79f27b_isrForCrosstalkSources_997009_23.stderr parsl.dataflow.dflow DEBUG: Task 242 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 242 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 242 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 242 has memoization hash 372bc94cec64d3b676ef64c95ceb88e6 parsl.dataflow.memoization INFO: Task 242 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3218a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 242 try 0 launched on executor multi with executor id 243 parsl.dataflow.dflow INFO: Standard output for task 242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/6f091bc4-e4ca-4b55-9547-cdb1c9762bb1_isrForCrosstalkSources_997013_44.stdout parsl.dataflow.dflow INFO: Standard error for task 242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/6f091bc4-e4ca-4b55-9547-cdb1c9762bb1_isrForCrosstalkSources_997013_44.stderr parsl.dataflow.dflow DEBUG: Task 243 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 243 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 243 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 243 has memoization hash 8943f4d3a2bd1a2dc35ede559c4247f0 parsl.dataflow.memoization INFO: Task 243 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3219e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 243 try 0 launched on executor multi with executor id 244 parsl.dataflow.dflow INFO: Standard output for task 243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/805116a2-5f8f-46c1-ba05-1ee417c9d055_isrForCrosstalkSources_996980_3.stdout parsl.dataflow.dflow INFO: Standard error for task 243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/805116a2-5f8f-46c1-ba05-1ee417c9d055_isrForCrosstalkSources_996980_3.stderr parsl.dataflow.dflow DEBUG: Task 244 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 244 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 244 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 244 has memoization hash 1c63e96bc6c5bb8c30d50ed5b574606e parsl.dataflow.memoization INFO: Task 244 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 244 try 0 launched on executor multi with executor id 245 parsl.dataflow.dflow INFO: Standard output for task 244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6b58120-be70-46ad-8a8f-524f652ad0dc_isrForCrosstalkSources_997026_37.stdout parsl.dataflow.dflow INFO: Standard error for task 244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6b58120-be70-46ad-8a8f-524f652ad0dc_isrForCrosstalkSources_997026_37.stderr parsl.dataflow.dflow DEBUG: Task 245 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 245 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 245 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 245 has memoization hash 79debdb013333d6e166ea51ac138d558 parsl.dataflow.memoization INFO: Task 245 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 245 try 0 launched on executor multi with executor id 246 parsl.dataflow.dflow INFO: Standard output for task 245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8bddbbf3-f769-4bf8-ac9f-dde7886b26a2_isrForCrosstalkSources_996997_13.stdout parsl.dataflow.dflow INFO: Standard error for task 245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8bddbbf3-f769-4bf8-ac9f-dde7886b26a2_isrForCrosstalkSources_996997_13.stderr parsl.dataflow.dflow DEBUG: Task 246 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 246 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 246 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 246 has memoization hash c826b85f9ad8b9132348ee2a5a364d47 parsl.dataflow.memoization INFO: Task 246 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 246 try 0 launched on executor multi with executor id 247 parsl.dataflow.dflow INFO: Standard output for task 246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/25d00663-629f-4429-b6b9-6f63891b6535_isrForCrosstalkSources_996971_58.stdout parsl.dataflow.dflow INFO: Standard error for task 246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/25d00663-629f-4429-b6b9-6f63891b6535_isrForCrosstalkSources_996971_58.stderr parsl.dataflow.dflow DEBUG: Task 247 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 247 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 247 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 247 has memoization hash bd4829bf741d360f680d9b1405cb6a52 parsl.dataflow.memoization INFO: Task 247 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 247 try 0 launched on executor multi with executor id 248 parsl.dataflow.dflow INFO: Standard output for task 247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1470b61e-fbcc-4bc3-b26f-c8502fb1a874_isrForCrosstalkSources_996953_16.stdout parsl.dataflow.dflow INFO: Standard error for task 247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1470b61e-fbcc-4bc3-b26f-c8502fb1a874_isrForCrosstalkSources_996953_16.stderr parsl.dataflow.dflow DEBUG: Task 248 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 248 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 248 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 248 has memoization hash b345b3727df70b0ad69e6b3cb4f426a1 parsl.dataflow.memoization INFO: Task 248 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 248 try 0 launched on executor multi with executor id 249 parsl.dataflow.dflow INFO: Standard output for task 248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d5941b02-cead-42d9-9978-d1d7b14a6f94_isrForCrosstalkSources_996964_31.stdout parsl.dataflow.dflow INFO: Standard error for task 248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d5941b02-cead-42d9-9978-d1d7b14a6f94_isrForCrosstalkSources_996964_31.stderr parsl.dataflow.dflow DEBUG: Task 249 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 249 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 249 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 249 has memoization hash 2f51a39c4c5a91d513f4982a4b23343c parsl.dataflow.memoization INFO: Task 249 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 249 try 0 launched on executor multi with executor id 250 parsl.dataflow.dflow INFO: Standard output for task 249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/42cb7ee4-7a50-43a0-9cdf-d450652b698b_isrForCrosstalkSources_997018_13.stdout parsl.dataflow.dflow INFO: Standard error for task 249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/42cb7ee4-7a50-43a0-9cdf-d450652b698b_isrForCrosstalkSources_997018_13.stderr parsl.dataflow.dflow DEBUG: Task 250 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 250 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 250 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 250 has memoization hash 8e2b8d96ba4acc2fc1e6e9b01b030ee7 parsl.dataflow.memoization INFO: Task 250 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 250 try 0 launched on executor multi with executor id 251 parsl.dataflow.dflow INFO: Standard output for task 250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b59a07f0-93cb-4e49-b00e-612509551427_isrForCrosstalkSources_996964_50.stdout parsl.dataflow.dflow INFO: Standard error for task 250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b59a07f0-93cb-4e49-b00e-612509551427_isrForCrosstalkSources_996964_50.stderr parsl.dataflow.dflow DEBUG: Task 251 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 251 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 251 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 251 has memoization hash 0d55b4c0c528959940214b6a035a06ea parsl.dataflow.memoization INFO: Task 251 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 251 try 0 launched on executor multi with executor id 252 parsl.dataflow.dflow INFO: Standard output for task 251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5d898322-acdc-4504-9773-827f1686de6b_isrForCrosstalkSources_997015_20.stdout parsl.dataflow.dflow INFO: Standard error for task 251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5d898322-acdc-4504-9773-827f1686de6b_isrForCrosstalkSources_997015_20.stderr parsl.dataflow.dflow DEBUG: Task 252 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 252 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 252 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 252 has memoization hash 7ba7d329ef5546ae9dff44f8a9e2f762 parsl.dataflow.memoization INFO: Task 252 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 252 try 0 launched on executor multi with executor id 253 parsl.dataflow.dflow INFO: Standard output for task 252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/be5ce00b-7fcc-496b-95f2-cc7afd54bb2a_isrForCrosstalkSources_997016_52.stdout parsl.dataflow.dflow INFO: Standard error for task 252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/be5ce00b-7fcc-496b-95f2-cc7afd54bb2a_isrForCrosstalkSources_997016_52.stderr parsl.dataflow.dflow DEBUG: Task 253 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 253 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 253 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 253 has memoization hash 67754eb06f86511b45927aed61fc2719 parsl.dataflow.memoization INFO: Task 253 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f321ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 253 try 0 launched on executor multi with executor id 254 parsl.dataflow.dflow INFO: Standard output for task 253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9cd0b273-d235-484c-89b6-4f66a963c42c_isrForCrosstalkSources_997018_23.stdout parsl.dataflow.dflow INFO: Standard error for task 253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9cd0b273-d235-484c-89b6-4f66a963c42c_isrForCrosstalkSources_997018_23.stderr parsl.dataflow.dflow DEBUG: Task 254 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 254 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 254 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 254 has memoization hash 2c319615f95ca25d662ea5b0f324103c parsl.dataflow.memoization INFO: Task 254 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 254 try 0 launched on executor multi with executor id 255 parsl.dataflow.dflow INFO: Standard output for task 254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7f40e489-6b40-486d-9b70-fc619769c339_isrForCrosstalkSources_997003_42.stdout parsl.dataflow.dflow INFO: Standard error for task 254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7f40e489-6b40-486d-9b70-fc619769c339_isrForCrosstalkSources_997003_42.stderr parsl.dataflow.dflow DEBUG: Task 255 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 255 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 255 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 255 has memoization hash 617733def177ee5a2c34d99cb2e675df parsl.dataflow.memoization INFO: Task 255 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 255 try 0 launched on executor multi with executor id 256 parsl.dataflow.dflow INFO: Standard output for task 255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7abcfe4a-23cc-46e9-8f8e-ce3eb3644f61_isrForCrosstalkSources_996963_44.stdout parsl.dataflow.dflow INFO: Standard error for task 255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7abcfe4a-23cc-46e9-8f8e-ce3eb3644f61_isrForCrosstalkSources_996963_44.stderr parsl.dataflow.dflow DEBUG: Task 256 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 256 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 256 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 256 has memoization hash 222b82b6dcd4ef2eb2f88d9eaad52119 parsl.dataflow.memoization INFO: Task 256 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3220c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 256 try 0 launched on executor multi with executor id 257 parsl.dataflow.dflow INFO: Standard output for task 256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/f47c0440-05fc-4688-8b6f-387b9a6ed24e_isrForCrosstalkSources_997027_56.stdout parsl.dataflow.dflow INFO: Standard error for task 256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/f47c0440-05fc-4688-8b6f-387b9a6ed24e_isrForCrosstalkSources_997027_56.stderr parsl.dataflow.dflow DEBUG: Task 257 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 257 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 257 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 257 has memoization hash 4ca440b4819bdffacd5ad05b1bb76c11 parsl.dataflow.memoization INFO: Task 257 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3222a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 257 try 0 launched on executor multi with executor id 258 parsl.dataflow.dflow INFO: Standard output for task 257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/0e092664-2ad7-4a09-a7c0-6658d160b688_isrForCrosstalkSources_997006_23.stdout parsl.dataflow.dflow INFO: Standard error for task 257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/0e092664-2ad7-4a09-a7c0-6658d160b688_isrForCrosstalkSources_997006_23.stderr parsl.dataflow.dflow DEBUG: Task 258 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 258 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 258 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 258 has memoization hash 0a02f8b3e9fa909034c6177e88193852 parsl.dataflow.memoization INFO: Task 258 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 258 try 0 launched on executor multi with executor id 259 parsl.dataflow.dflow INFO: Standard output for task 258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/26726060-6e46-4252-9cbf-4de8cd15743e_isrForCrosstalkSources_996968_19.stdout parsl.dataflow.dflow INFO: Standard error for task 258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/26726060-6e46-4252-9cbf-4de8cd15743e_isrForCrosstalkSources_996968_19.stderr parsl.dataflow.dflow DEBUG: Task 259 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 259 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 259 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 259 has memoization hash 8ab0c6a14501bebc7446fb693fbb67a9 parsl.dataflow.memoization INFO: Task 259 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 259 try 0 launched on executor multi with executor id 260 parsl.dataflow.dflow INFO: Standard output for task 259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e337ee50-4942-473e-b0ff-594148b84cb1_isrForCrosstalkSources_996964_60.stdout parsl.dataflow.dflow INFO: Standard error for task 259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e337ee50-4942-473e-b0ff-594148b84cb1_isrForCrosstalkSources_996964_60.stderr parsl.dataflow.dflow DEBUG: Task 260 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 260 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 260 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 260 has memoization hash 2e9a6f86fc2f1f7d234e654d8f955e27 parsl.dataflow.memoization INFO: Task 260 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3223e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 260 try 0 launched on executor multi with executor id 261 parsl.dataflow.dflow INFO: Standard output for task 260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/809e2f5c-3505-4776-a3a3-d6c1121c4b0f_isrForCrosstalkSources_997031_33.stdout parsl.dataflow.dflow INFO: Standard error for task 260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/809e2f5c-3505-4776-a3a3-d6c1121c4b0f_isrForCrosstalkSources_997031_33.stderr parsl.dataflow.dflow DEBUG: Task 261 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 261 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 261 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 261 has memoization hash 359f7e4f79f659a0bcfa635c3cb5c3b8 parsl.dataflow.memoization INFO: Task 261 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 261 try 0 launched on executor multi with executor id 262 parsl.dataflow.dflow INFO: Standard output for task 261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/cdfdd9db-4267-4129-8345-2dd6730b7d66_isrForCrosstalkSources_996974_14.stdout parsl.dataflow.dflow INFO: Standard error for task 261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/cdfdd9db-4267-4129-8345-2dd6730b7d66_isrForCrosstalkSources_996974_14.stderr parsl.dataflow.dflow DEBUG: Task 262 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 262 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 262 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 262 has memoization hash a06c27d38217e8daad2ac9e680daff2f parsl.dataflow.memoization INFO: Task 262 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3225c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 262 try 0 launched on executor multi with executor id 263 parsl.dataflow.dflow INFO: Standard output for task 262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ede64a10-8ac0-4604-a5c2-ce362df85691_isrForCrosstalkSources_996984_60.stdout parsl.dataflow.dflow INFO: Standard error for task 262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ede64a10-8ac0-4604-a5c2-ce362df85691_isrForCrosstalkSources_996984_60.stderr parsl.dataflow.dflow DEBUG: Task 263 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 263 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 263 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 263 has memoization hash 186ba9a8c465002b63b9cfa35da3189e parsl.dataflow.memoization INFO: Task 263 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 263 try 0 launched on executor multi with executor id 264 parsl.dataflow.dflow INFO: Standard output for task 263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/f6ca5d89-39d4-4dfb-a98c-ee098e3516dd_isrForCrosstalkSources_997025_14.stdout parsl.dataflow.dflow INFO: Standard error for task 263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/f6ca5d89-39d4-4dfb-a98c-ee098e3516dd_isrForCrosstalkSources_997025_14.stderr parsl.dataflow.dflow DEBUG: Task 264 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 264 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 264 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 264 has memoization hash c0c44537fee4c489efaec0a89cc503b7 parsl.dataflow.memoization INFO: Task 264 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 264 try 0 launched on executor multi with executor id 265 parsl.dataflow.dflow INFO: Standard output for task 264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/9a68c496-3ddc-45da-a795-0cdacff15800_isrForCrosstalkSources_996993_12.stdout parsl.dataflow.dflow INFO: Standard error for task 264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/9a68c496-3ddc-45da-a795-0cdacff15800_isrForCrosstalkSources_996993_12.stderr parsl.dataflow.dflow DEBUG: Task 265 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 265 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 265 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 265 has memoization hash c57fb4c01450148c6ae9210eca5b4b70 parsl.dataflow.memoization INFO: Task 265 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 265 try 0 launched on executor multi with executor id 266 parsl.dataflow.dflow INFO: Standard output for task 265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bd753d5c-9054-4cbb-9739-38e8fd85a0f0_isrForCrosstalkSources_997026_50.stdout parsl.dataflow.dflow INFO: Standard error for task 265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bd753d5c-9054-4cbb-9739-38e8fd85a0f0_isrForCrosstalkSources_997026_50.stderr parsl.dataflow.dflow DEBUG: Task 266 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 266 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 266 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 266 has memoization hash ddab5ba099ec90b611e50d7ae67e1db4 parsl.dataflow.memoization INFO: Task 266 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 266 try 0 launched on executor multi with executor id 267 parsl.dataflow.dflow INFO: Standard output for task 266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/41dad024-76b7-40b3-9b1f-6b780acc8155_isrForCrosstalkSources_997024_2.stdout parsl.dataflow.dflow INFO: Standard error for task 266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/41dad024-76b7-40b3-9b1f-6b780acc8155_isrForCrosstalkSources_997024_2.stderr parsl.dataflow.dflow DEBUG: Task 267 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 267 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 267 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 267 has memoization hash 17fed749da56c1a8b2ec009f52c35a7a parsl.dataflow.memoization INFO: Task 267 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3228e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 267 try 0 launched on executor multi with executor id 268 parsl.dataflow.dflow INFO: Standard output for task 267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/42f37f7d-1380-4cc2-a37e-4a67a95720df_isrForCrosstalkSources_996974_2.stdout parsl.dataflow.dflow INFO: Standard error for task 267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/42f37f7d-1380-4cc2-a37e-4a67a95720df_isrForCrosstalkSources_996974_2.stderr parsl.dataflow.dflow DEBUG: Task 268 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 268 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 268 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 268 has memoization hash 8f8bfacc356b85f197175ecdc5ba0cb1 parsl.dataflow.memoization INFO: Task 268 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 268 try 0 launched on executor multi with executor id 269 parsl.dataflow.dflow INFO: Standard output for task 268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/c9fe3b9a-9a7d-4ada-a26f-2872e1133012_isrForCrosstalkSources_997024_50.stdout parsl.dataflow.dflow INFO: Standard error for task 268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/c9fe3b9a-9a7d-4ada-a26f-2872e1133012_isrForCrosstalkSources_997024_50.stderr parsl.dataflow.dflow DEBUG: Task 269 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 269 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 269 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 269 has memoization hash 4503b09844d870624cbd4ffe474a56a5 parsl.dataflow.memoization INFO: Task 269 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 269 try 0 launched on executor multi with executor id 270 parsl.dataflow.dflow INFO: Standard output for task 269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8db7c8d6-52cd-428e-90ce-4614923a728d_isrForCrosstalkSources_996956_1.stdout parsl.dataflow.dflow INFO: Standard error for task 269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8db7c8d6-52cd-428e-90ce-4614923a728d_isrForCrosstalkSources_996956_1.stderr parsl.dataflow.dflow DEBUG: Task 270 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 270 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 270 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 270 has memoization hash 58b6bfe98c836554d90f8abb0a292145 parsl.dataflow.memoization INFO: Task 270 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3227a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 270 try 0 launched on executor multi with executor id 271 parsl.dataflow.dflow INFO: Standard output for task 270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f02032fb-8646-4bc6-bcb1-22941638a8c6_isrForCrosstalkSources_996967_5.stdout parsl.dataflow.dflow INFO: Standard error for task 270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f02032fb-8646-4bc6-bcb1-22941638a8c6_isrForCrosstalkSources_996967_5.stderr parsl.dataflow.dflow DEBUG: Task 271 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 271 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 271 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 271 has memoization hash a704ce1130d1340ba8e62724323c2b53 parsl.dataflow.memoization INFO: Task 271 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 271 try 0 launched on executor multi with executor id 272 parsl.dataflow.dflow INFO: Standard output for task 271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/148988df-1d3b-425c-98f9-68e5802b745e_isrForCrosstalkSources_997019_45.stdout parsl.dataflow.dflow INFO: Standard error for task 271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/148988df-1d3b-425c-98f9-68e5802b745e_isrForCrosstalkSources_997019_45.stderr parsl.dataflow.dflow DEBUG: Task 272 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 272 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 272 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 272 has memoization hash d6dc7d1d4fe33171a22eed4d0e1f1fab parsl.dataflow.memoization INFO: Task 272 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 272 try 0 launched on executor multi with executor id 273 parsl.dataflow.dflow INFO: Standard output for task 272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/eabc891b-6eb4-4fb1-a73d-ea51cfba0980_isrForCrosstalkSources_996985_28.stdout parsl.dataflow.dflow INFO: Standard error for task 272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/eabc891b-6eb4-4fb1-a73d-ea51cfba0980_isrForCrosstalkSources_996985_28.stderr parsl.dataflow.dflow DEBUG: Task 273 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 273 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 273 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 273 has memoization hash c5dfc0aa5a088ee0514fbe670713f75b parsl.dataflow.memoization INFO: Task 273 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 273 try 0 launched on executor multi with executor id 274 parsl.dataflow.dflow INFO: Standard output for task 273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/2947c406-6abf-47a8-90c7-808be10d49f9_isrForCrosstalkSources_996964_48.stdout parsl.dataflow.dflow INFO: Standard error for task 273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/2947c406-6abf-47a8-90c7-808be10d49f9_isrForCrosstalkSources_996964_48.stderr parsl.dataflow.dflow DEBUG: Task 274 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 274 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 274 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 274 has memoization hash 257b09a1bc456b581f9481118616633d parsl.dataflow.memoization INFO: Task 274 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 274 try 0 launched on executor multi with executor id 275 parsl.dataflow.dflow INFO: Standard output for task 274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/43a80d84-5b6c-48dc-971b-11a2be08c058_isrForCrosstalkSources_996982_24.stdout parsl.dataflow.dflow INFO: Standard error for task 274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/43a80d84-5b6c-48dc-971b-11a2be08c058_isrForCrosstalkSources_996982_24.stderr parsl.dataflow.dflow DEBUG: Task 275 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 275 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 275 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 275 has memoization hash e54cb63b52c8772ddc4caebe88926f19 parsl.dataflow.memoization INFO: Task 275 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 275 try 0 launched on executor multi with executor id 276 parsl.dataflow.dflow INFO: Standard output for task 275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/15d3d7b1-c8b3-40b2-9a1f-ac2367fc81b4_isrForCrosstalkSources_996981_50.stdout parsl.dataflow.dflow INFO: Standard error for task 275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/15d3d7b1-c8b3-40b2-9a1f-ac2367fc81b4_isrForCrosstalkSources_996981_50.stderr parsl.dataflow.dflow DEBUG: Task 276 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 276 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 276 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 276 has memoization hash 3ac323996d56c9891513285f26068483 parsl.dataflow.memoization INFO: Task 276 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 276 try 0 launched on executor multi with executor id 277 parsl.dataflow.dflow INFO: Standard output for task 276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/42da37ee-6d16-4181-b2da-8b1b6a0afb31_isrForCrosstalkSources_996999_40.stdout parsl.dataflow.dflow INFO: Standard error for task 276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/42da37ee-6d16-4181-b2da-8b1b6a0afb31_isrForCrosstalkSources_996999_40.stderr parsl.dataflow.dflow DEBUG: Task 277 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 277 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 277 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 277 has memoization hash 79578570f3ea4c44a4fe9f3d77fc585f parsl.dataflow.memoization INFO: Task 277 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3239c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 277 try 0 launched on executor multi with executor id 278 parsl.dataflow.dflow INFO: Standard output for task 277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/60357f1f-262c-4927-9d30-55d5f63a3dfd_isrForCrosstalkSources_997016_21.stdout parsl.dataflow.dflow INFO: Standard error for task 277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/60357f1f-262c-4927-9d30-55d5f63a3dfd_isrForCrosstalkSources_997016_21.stderr parsl.dataflow.dflow DEBUG: Task 278 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 278 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 278 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 278 has memoization hash a5a4e539995d1b6a1b7efb8b2e0e2068 parsl.dataflow.memoization INFO: Task 278 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 278 try 0 launched on executor multi with executor id 279 parsl.dataflow.dflow INFO: Standard output for task 278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4a82e811-0542-476e-8b03-e136f26c4bcc_isrForCrosstalkSources_996958_19.stdout parsl.dataflow.dflow INFO: Standard error for task 278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4a82e811-0542-476e-8b03-e136f26c4bcc_isrForCrosstalkSources_996958_19.stderr parsl.dataflow.dflow DEBUG: Task 279 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 279 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 279 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 279 has memoization hash ed23756b42d32ff04d9d4eefd9a821e0 parsl.dataflow.memoization INFO: Task 279 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3237e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 279 try 0 launched on executor multi with executor id 280 parsl.dataflow.dflow INFO: Standard output for task 279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/5229ad77-8198-4398-bb20-e2a6d383ea7b_isrForCrosstalkSources_997001_47.stdout parsl.dataflow.dflow INFO: Standard error for task 279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/5229ad77-8198-4398-bb20-e2a6d383ea7b_isrForCrosstalkSources_997001_47.stderr parsl.dataflow.dflow DEBUG: Task 280 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 280 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 280 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 280 has memoization hash a99a57fad86ae4723401231e8e2dbf49 parsl.dataflow.memoization INFO: Task 280 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 280 try 0 launched on executor multi with executor id 281 parsl.dataflow.dflow INFO: Standard output for task 280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/e33f2f9f-50df-4b20-bcaf-48b2ac89ae17_isrForCrosstalkSources_996982_58.stdout parsl.dataflow.dflow INFO: Standard error for task 280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/e33f2f9f-50df-4b20-bcaf-48b2ac89ae17_isrForCrosstalkSources_996982_58.stderr parsl.dataflow.dflow DEBUG: Task 281 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 281 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 281 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 281 has memoization hash b794c74685f0a3f95d0fcfa2bafee76d parsl.dataflow.memoization INFO: Task 281 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 281 try 0 launched on executor multi with executor id 282 parsl.dataflow.dflow INFO: Standard output for task 281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7bb51dc0-b331-42c1-b725-5be15f17f12c_isrForCrosstalkSources_996953_20.stdout parsl.dataflow.dflow INFO: Standard error for task 281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7bb51dc0-b331-42c1-b725-5be15f17f12c_isrForCrosstalkSources_996953_20.stderr parsl.dataflow.dflow DEBUG: Task 282 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 282 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 282 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 282 has memoization hash 9aee59e323c5265e79cc4bb5d186afdf parsl.dataflow.memoization INFO: Task 282 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3236a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 282 try 0 launched on executor multi with executor id 283 parsl.dataflow.dflow INFO: Standard output for task 282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/f4a7ec81-9649-401e-95d3-475724e9862f_isrForCrosstalkSources_996992_9.stdout parsl.dataflow.dflow INFO: Standard error for task 282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/f4a7ec81-9649-401e-95d3-475724e9862f_isrForCrosstalkSources_996992_9.stderr parsl.dataflow.dflow DEBUG: Task 283 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 283 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 283 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 283 has memoization hash 813cfad2acdb8b942c9bbf72973bfb89 parsl.dataflow.memoization INFO: Task 283 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 283 try 0 launched on executor multi with executor id 284 parsl.dataflow.dflow INFO: Standard output for task 283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/33e8cb2b-c64b-4355-9bc5-6bf51b736caa_isrForCrosstalkSources_997024_22.stdout parsl.dataflow.dflow INFO: Standard error for task 283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/33e8cb2b-c64b-4355-9bc5-6bf51b736caa_isrForCrosstalkSources_997024_22.stderr parsl.dataflow.dflow DEBUG: Task 284 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 284 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 284 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 284 has memoization hash fb05a796998b21a70773a3d2244470db parsl.dataflow.memoization INFO: Task 284 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3234c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 284 try 0 launched on executor multi with executor id 285 parsl.dataflow.dflow INFO: Standard output for task 284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/97c880b6-6aab-4dba-a1ce-7983aca71977_isrForCrosstalkSources_997032_52.stdout parsl.dataflow.dflow INFO: Standard error for task 284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/97c880b6-6aab-4dba-a1ce-7983aca71977_isrForCrosstalkSources_997032_52.stderr parsl.dataflow.dflow DEBUG: Task 285 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 285 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 285 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 285 has memoization hash 1a513d9d7e9a597ca02cd6b891f7f297 parsl.dataflow.memoization INFO: Task 285 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 285 try 0 launched on executor multi with executor id 286 parsl.dataflow.dflow INFO: Standard output for task 285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/8608d81f-e6ec-41b8-bb86-f56b4aae798e_isrForCrosstalkSources_996955_35.stdout parsl.dataflow.dflow INFO: Standard error for task 285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/8608d81f-e6ec-41b8-bb86-f56b4aae798e_isrForCrosstalkSources_996955_35.stderr parsl.dataflow.dflow DEBUG: Task 286 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 286 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 286 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 286 has memoization hash 647d68b5881b58318541b09214d64858 parsl.dataflow.memoization INFO: Task 286 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 286 try 0 launched on executor multi with executor id 287 parsl.dataflow.dflow INFO: Standard output for task 286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/7ec61a31-b64d-49b8-a7e4-7565ad6d3993_isrForCrosstalkSources_996969_40.stdout parsl.dataflow.dflow INFO: Standard error for task 286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/7ec61a31-b64d-49b8-a7e4-7565ad6d3993_isrForCrosstalkSources_996969_40.stderr parsl.dataflow.dflow DEBUG: Task 287 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 287 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 287 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 287 has memoization hash 1e0a68973eac4023f3367bc3205ee6b9 parsl.dataflow.memoization INFO: Task 287 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 287 try 0 launched on executor multi with executor id 288 parsl.dataflow.dflow INFO: Standard output for task 287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/cf160bab-2be0-4346-9d4d-7a0796f76d19_isrForCrosstalkSources_996996_3.stdout parsl.dataflow.dflow INFO: Standard error for task 287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/cf160bab-2be0-4346-9d4d-7a0796f76d19_isrForCrosstalkSources_996996_3.stderr parsl.dataflow.dflow DEBUG: Task 288 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 288 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 288 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 288 has memoization hash cd9858d95cb255c2a1e7ea747fad3373 parsl.dataflow.memoization INFO: Task 288 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3232e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 288 try 0 launched on executor multi with executor id 289 parsl.dataflow.dflow INFO: Standard output for task 288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/92a506a5-f9a6-434f-abc5-eca1ded66ce3_isrForCrosstalkSources_996980_16.stdout parsl.dataflow.dflow INFO: Standard error for task 288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/92a506a5-f9a6-434f-abc5-eca1ded66ce3_isrForCrosstalkSources_996980_16.stderr parsl.dataflow.dflow DEBUG: Task 289 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 289 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 289 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 289 has memoization hash 7fad3ecdd6ef8b2dd4cfd52dbb60e649 parsl.dataflow.memoization INFO: Task 289 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f3231a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 289 try 0 launched on executor multi with executor id 290 parsl.dataflow.dflow INFO: Standard output for task 289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/26fd3b20-7453-4c84-b14e-0d119a50a364_isrForCrosstalkSources_996961_19.stdout parsl.dataflow.dflow INFO: Standard error for task 289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/26fd3b20-7453-4c84-b14e-0d119a50a364_isrForCrosstalkSources_996961_19.stderr parsl.dataflow.dflow DEBUG: Task 290 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 290 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 290 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 290 has memoization hash 6ce6f66b38b676417ca16780db959460 parsl.dataflow.memoization INFO: Task 290 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 290 try 0 launched on executor multi with executor id 291 parsl.dataflow.dflow INFO: Standard output for task 290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5d955431-4aa5-45ea-98ad-39fdb0e4582b_isrForCrosstalkSources_997021_6.stdout parsl.dataflow.dflow INFO: Standard error for task 290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5d955431-4aa5-45ea-98ad-39fdb0e4582b_isrForCrosstalkSources_997021_6.stderr parsl.dataflow.dflow DEBUG: Task 291 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 291 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 291 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 291 has memoization hash 8dac4cb474610faaa734db80fde8f15a parsl.dataflow.memoization INFO: Task 291 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 291 try 0 launched on executor multi with executor id 292 parsl.dataflow.dflow INFO: Standard output for task 291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/1455e419-9c40-43e6-9398-c9732750e85f_isrForCrosstalkSources_996996_44.stdout parsl.dataflow.dflow INFO: Standard error for task 291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/1455e419-9c40-43e6-9398-c9732750e85f_isrForCrosstalkSources_996996_44.stderr parsl.dataflow.dflow DEBUG: Task 292 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 292 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 292 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 292 has memoization hash 5a6f4379975340f0e3e48eb45c9044ba parsl.dataflow.memoization INFO: Task 292 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 292 try 0 launched on executor multi with executor id 293 parsl.dataflow.dflow INFO: Standard output for task 292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/8b1f04c0-5783-400b-ac13-0ccc4cf0871b_isrForCrosstalkSources_997023_7.stdout parsl.dataflow.dflow INFO: Standard error for task 292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/8b1f04c0-5783-400b-ac13-0ccc4cf0871b_isrForCrosstalkSources_997023_7.stderr parsl.dataflow.dflow DEBUG: Task 293 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 293 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 293 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 293 has memoization hash c4bf04bc74f230ad0df9771f428e8eee parsl.dataflow.memoization INFO: Task 293 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 293 try 0 launched on executor multi with executor id 294 parsl.dataflow.dflow INFO: Standard output for task 293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/f1bd9f09-a515-47e4-bdb8-c6a9f809c847_isrForCrosstalkSources_996998_35.stdout parsl.dataflow.dflow INFO: Standard error for task 293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/f1bd9f09-a515-47e4-bdb8-c6a9f809c847_isrForCrosstalkSources_996998_35.stderr parsl.dataflow.dflow DEBUG: Task 294 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 294 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 294 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 294 has memoization hash bd8fdcde4acb54fb4185315a769ab8e3 parsl.dataflow.memoization INFO: Task 294 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 294 try 0 launched on executor multi with executor id 295 parsl.dataflow.dflow INFO: Standard output for task 294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/10a0aa23-cec9-42e8-9fd3-27340c34706d_isrForCrosstalkSources_997009_39.stdout parsl.dataflow.dflow INFO: Standard error for task 294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/10a0aa23-cec9-42e8-9fd3-27340c34706d_isrForCrosstalkSources_997009_39.stderr parsl.dataflow.dflow DEBUG: Task 295 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 295 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 295 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 295 has memoization hash 65fae1543b33042fc7b5b71e898f1035 parsl.dataflow.memoization INFO: Task 295 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 295 try 0 launched on executor multi with executor id 296 parsl.dataflow.dflow INFO: Standard output for task 295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/67a4415e-e372-439e-9a84-8da03b166aae_isrForCrosstalkSources_997002_56.stdout parsl.dataflow.dflow INFO: Standard error for task 295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/67a4415e-e372-439e-9a84-8da03b166aae_isrForCrosstalkSources_997002_56.stderr parsl.dataflow.dflow DEBUG: Task 296 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 296 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 296 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 296 has memoization hash 5e4b56eb8d98cc46bc135b7c508a531b parsl.dataflow.memoization INFO: Task 296 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 296 try 0 launched on executor multi with executor id 297 parsl.dataflow.dflow INFO: Standard output for task 296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b5cbfa28-8d3f-4388-9eb8-2a071f9a6b4b_isrForCrosstalkSources_997030_37.stdout parsl.dataflow.dflow INFO: Standard error for task 296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b5cbfa28-8d3f-4388-9eb8-2a071f9a6b4b_isrForCrosstalkSources_997030_37.stderr parsl.dataflow.dflow DEBUG: Task 297 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 297 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 297 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 297 has memoization hash cf36490ce00c06a8ee61ab17dbb5c9ef parsl.dataflow.memoization INFO: Task 297 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 297 try 0 launched on executor multi with executor id 298 parsl.dataflow.dflow INFO: Standard output for task 297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/15e16b83-35db-4eba-b530-5d2ac4fa91e6_isrForCrosstalkSources_997033_7.stdout parsl.dataflow.dflow INFO: Standard error for task 297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/15e16b83-35db-4eba-b530-5d2ac4fa91e6_isrForCrosstalkSources_997033_7.stderr parsl.dataflow.dflow DEBUG: Task 298 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 298 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 298 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 298 has memoization hash fa944bfb773fc83b0612c3a91ee27c79 parsl.dataflow.memoization INFO: Task 298 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 298 try 0 launched on executor multi with executor id 299 parsl.dataflow.dflow INFO: Standard output for task 298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/806ef374-d179-4a99-8858-d6ed95a132dc_isrForCrosstalkSources_997009_2.stdout parsl.dataflow.dflow INFO: Standard error for task 298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/806ef374-d179-4a99-8858-d6ed95a132dc_isrForCrosstalkSources_997009_2.stderr parsl.dataflow.dflow DEBUG: Task 299 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 299 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 299 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 299 has memoization hash 6fce926caebc5b49a1fdb5b431fe562a parsl.dataflow.memoization INFO: Task 299 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f323f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 299 try 0 launched on executor multi with executor id 300 parsl.dataflow.dflow INFO: Standard output for task 299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/25a91652-7212-4dc7-9072-068886b8998a_isrForCrosstalkSources_996978_14.stdout parsl.dataflow.dflow INFO: Standard error for task 299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/25a91652-7212-4dc7-9072-068886b8998a_isrForCrosstalkSources_996978_14.stderr parsl.dataflow.dflow DEBUG: Task 300 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 300 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 300 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 300 has memoization hash 02dba0f375c97d57e62141e9fdd1e67f parsl.dataflow.memoization INFO: Task 300 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f322b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 300 try 0 launched on executor multi with executor id 301 parsl.dataflow.dflow INFO: Standard output for task 300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/06abc97e-f9ec-4ac0-8337-0217ccac31ce_isrForCrosstalkSources_997013_50.stdout parsl.dataflow.dflow INFO: Standard error for task 300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/06abc97e-f9ec-4ac0-8337-0217ccac31ce_isrForCrosstalkSources_997013_50.stderr parsl.dataflow.dflow DEBUG: Task 301 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 301 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 301 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 301 has memoization hash 2e045d044b4d2c9b5a5c421eafc7c2e2 parsl.dataflow.memoization INFO: Task 301 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee40e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 301 try 0 launched on executor multi with executor id 302 parsl.dataflow.dflow INFO: Standard output for task 301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/2750800c-8105-418d-b7ce-7b84c35f7b5e_isrForCrosstalkSources_996958_42.stdout parsl.dataflow.dflow INFO: Standard error for task 301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/2750800c-8105-418d-b7ce-7b84c35f7b5e_isrForCrosstalkSources_996958_42.stderr parsl.dataflow.dflow DEBUG: Task 302 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 302 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 302 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 302 has memoization hash 9743ca0c78f6ee03ac294da821510239 parsl.dataflow.memoization INFO: Task 302 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 302 try 0 launched on executor multi with executor id 303 parsl.dataflow.dflow INFO: Standard output for task 302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5b772235-96e5-48bc-a150-4f1a1f7b41dc_isrForCrosstalkSources_996988_33.stdout parsl.dataflow.dflow INFO: Standard error for task 302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5b772235-96e5-48bc-a150-4f1a1f7b41dc_isrForCrosstalkSources_996988_33.stderr parsl.dataflow.dflow DEBUG: Task 303 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 303 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 303 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 303 has memoization hash b0440ed9ddca6de162479a70d5d8fcf1 parsl.dataflow.memoization INFO: Task 303 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 303 try 0 launched on executor multi with executor id 304 parsl.dataflow.dflow INFO: Standard output for task 303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f894cbe2-5a66-4572-9fad-1ab6da42dc05_isrForCrosstalkSources_996996_61.stdout parsl.dataflow.dflow INFO: Standard error for task 303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f894cbe2-5a66-4572-9fad-1ab6da42dc05_isrForCrosstalkSources_996996_61.stderr parsl.dataflow.dflow DEBUG: Task 304 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 304 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 304 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 304 has memoization hash cd687c82e1cb1af40089135613cce284 parsl.dataflow.memoization INFO: Task 304 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 304 try 0 launched on executor multi with executor id 305 parsl.dataflow.dflow INFO: Standard output for task 304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/95793988-dc05-44ee-b9b5-25efdcf76041_isrForCrosstalkSources_997033_28.stdout parsl.dataflow.dflow INFO: Standard error for task 304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/95793988-dc05-44ee-b9b5-25efdcf76041_isrForCrosstalkSources_997033_28.stderr parsl.dataflow.dflow DEBUG: Task 305 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 305 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 305 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 305 has memoization hash a985c76fdbeca7189839c2e771f46016 parsl.dataflow.memoization INFO: Task 305 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee42c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 305 try 0 launched on executor multi with executor id 306 parsl.dataflow.dflow INFO: Standard output for task 305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b19cde89-c54f-4ec7-9bed-264a079e9174_isrForCrosstalkSources_996959_37.stdout parsl.dataflow.dflow INFO: Standard error for task 305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b19cde89-c54f-4ec7-9bed-264a079e9174_isrForCrosstalkSources_996959_37.stderr parsl.dataflow.dflow DEBUG: Task 306 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 306 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 306 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 306 has memoization hash c83a51311bb1a8773265708a6fcc3630 parsl.dataflow.memoization INFO: Task 306 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 306 try 0 launched on executor multi with executor id 307 parsl.dataflow.dflow INFO: Standard output for task 306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/502e7b26-9649-44d4-a942-1cabda176125_isrForCrosstalkSources_997010_37.stdout parsl.dataflow.dflow INFO: Standard error for task 306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/502e7b26-9649-44d4-a942-1cabda176125_isrForCrosstalkSources_997010_37.stderr parsl.dataflow.dflow DEBUG: Task 307 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 307 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 307 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 307 has memoization hash be60a12af860dc69f96ab9bd81939f6b parsl.dataflow.memoization INFO: Task 307 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee44a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 307 try 0 launched on executor multi with executor id 308 parsl.dataflow.dflow INFO: Standard output for task 307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/6b0ec8a7-4460-4545-8076-02e38f06dd9c_isrForCrosstalkSources_996995_45.stdout parsl.dataflow.dflow INFO: Standard error for task 307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/6b0ec8a7-4460-4545-8076-02e38f06dd9c_isrForCrosstalkSources_996995_45.stderr parsl.dataflow.dflow DEBUG: Task 308 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 308 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 308 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 308 has memoization hash d31eed44970efd10ec961bc58b95baec parsl.dataflow.memoization INFO: Task 308 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 308 try 0 launched on executor multi with executor id 309 parsl.dataflow.dflow INFO: Standard output for task 308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/24221d94-3b36-4570-9bf1-b14b41c25c62_isrForCrosstalkSources_997000_26.stdout parsl.dataflow.dflow INFO: Standard error for task 308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/24221d94-3b36-4570-9bf1-b14b41c25c62_isrForCrosstalkSources_997000_26.stderr parsl.dataflow.dflow DEBUG: Task 309 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 309 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 309 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 309 has memoization hash 812c18b65396aa5948fc66e8db6ee90d parsl.dataflow.memoization INFO: Task 309 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 309 try 0 launched on executor multi with executor id 310 parsl.dataflow.dflow INFO: Standard output for task 309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/971d351f-baf1-4211-901a-4234aba57741_isrForCrosstalkSources_997013_37.stdout parsl.dataflow.dflow INFO: Standard error for task 309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/971d351f-baf1-4211-901a-4234aba57741_isrForCrosstalkSources_997013_37.stderr parsl.dataflow.dflow DEBUG: Task 310 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 310 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 310 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 310 has memoization hash 649d19308603b58084ff4fc82ac1de17 parsl.dataflow.memoization INFO: Task 310 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee45e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 310 try 0 launched on executor multi with executor id 311 parsl.dataflow.dflow INFO: Standard output for task 310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/5a5a0e4f-bd4b-42e0-ae70-1b62035ff0d9_isrForCrosstalkSources_996973_33.stdout parsl.dataflow.dflow INFO: Standard error for task 310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/5a5a0e4f-bd4b-42e0-ae70-1b62035ff0d9_isrForCrosstalkSources_996973_33.stderr parsl.dataflow.dflow DEBUG: Task 311 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 311 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 311 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 311 has memoization hash 4c9f7f0c033fe21f0f7cc969203b3b8f parsl.dataflow.memoization INFO: Task 311 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 311 try 0 launched on executor multi with executor id 312 parsl.dataflow.dflow INFO: Standard output for task 311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ee691298-c8cf-45fe-9744-d108ffdfd7ce_isrForCrosstalkSources_996990_19.stdout parsl.dataflow.dflow INFO: Standard error for task 311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ee691298-c8cf-45fe-9744-d108ffdfd7ce_isrForCrosstalkSources_996990_19.stderr parsl.dataflow.dflow DEBUG: Task 312 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 312 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 312 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 312 has memoization hash 256b8220249676098857eba0e6969bac parsl.dataflow.memoization INFO: Task 312 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee47c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 312 try 0 launched on executor multi with executor id 313 parsl.dataflow.dflow INFO: Standard output for task 312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/49cf2ac5-1e07-4e87-afd7-3a93be4513bd_isrForCrosstalkSources_996993_9.stdout parsl.dataflow.dflow INFO: Standard error for task 312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/49cf2ac5-1e07-4e87-afd7-3a93be4513bd_isrForCrosstalkSources_996993_9.stderr parsl.dataflow.dflow DEBUG: Task 313 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 313 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 313 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 313 has memoization hash 529cf4fe7d12b31bf12d43e948f8e56c parsl.dataflow.memoization INFO: Task 313 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 313 try 0 launched on executor multi with executor id 314 parsl.dataflow.dflow INFO: Standard output for task 313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b9c4b36d-d690-4fa1-88fc-236cd7f58ad9_isrForCrosstalkSources_996966_2.stdout parsl.dataflow.dflow INFO: Standard error for task 313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b9c4b36d-d690-4fa1-88fc-236cd7f58ad9_isrForCrosstalkSources_996966_2.stderr parsl.dataflow.dflow DEBUG: Task 314 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 314 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 314 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 314 has memoization hash 6bcfc38c518841b9f5e5f327a8c77e43 parsl.dataflow.memoization INFO: Task 314 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee49a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 314 try 0 launched on executor multi with executor id 315 parsl.dataflow.dflow INFO: Standard output for task 314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/6d520c28-a4cb-4900-a665-c27b0ce63ff5_isrForCrosstalkSources_996965_7.stdout parsl.dataflow.dflow INFO: Standard error for task 314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/6d520c28-a4cb-4900-a665-c27b0ce63ff5_isrForCrosstalkSources_996965_7.stderr parsl.dataflow.dflow DEBUG: Task 315 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 315 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 315 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 315 has memoization hash 86eda3650cbf752d473cf883cb96bdf9 parsl.dataflow.memoization INFO: Task 315 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 315 try 0 launched on executor multi with executor id 316 parsl.dataflow.dflow INFO: Standard output for task 315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d52954e8-d53e-4546-9111-fab797915dc2_isrForCrosstalkSources_997024_44.stdout parsl.dataflow.dflow INFO: Standard error for task 315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d52954e8-d53e-4546-9111-fab797915dc2_isrForCrosstalkSources_997024_44.stderr parsl.dataflow.dflow DEBUG: Task 316 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 316 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 316 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 316 has memoization hash d731e1164f6f4929e877b2b2cf935082 parsl.dataflow.memoization INFO: Task 316 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 316 try 0 launched on executor multi with executor id 317 parsl.dataflow.dflow INFO: Standard output for task 316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4a868a22-8428-4bb3-b7a1-cc014f182a8d_isrForCrosstalkSources_996959_21.stdout parsl.dataflow.dflow INFO: Standard error for task 316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4a868a22-8428-4bb3-b7a1-cc014f182a8d_isrForCrosstalkSources_996959_21.stderr parsl.dataflow.dflow DEBUG: Task 317 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 317 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 317 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 317 has memoization hash 452f5583b5c63bad998274b3f5af776e parsl.dataflow.memoization INFO: Task 317 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 317 try 0 launched on executor multi with executor id 318 parsl.dataflow.dflow INFO: Standard output for task 317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/75d9d059-cfe1-4014-a507-14b029c2877c_isrForCrosstalkSources_996971_51.stdout parsl.dataflow.dflow INFO: Standard error for task 317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/75d9d059-cfe1-4014-a507-14b029c2877c_isrForCrosstalkSources_996971_51.stderr parsl.dataflow.dflow DEBUG: Task 318 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 318 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 318 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 318 has memoization hash bd8c5950835aa0c067071828d9128a71 parsl.dataflow.memoization INFO: Task 318 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 318 try 0 launched on executor multi with executor id 319 parsl.dataflow.dflow INFO: Standard output for task 318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ad04b079-2c07-4108-8ecc-56988dd77fd8_isrForCrosstalkSources_997021_41.stdout parsl.dataflow.dflow INFO: Standard error for task 318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ad04b079-2c07-4108-8ecc-56988dd77fd8_isrForCrosstalkSources_997021_41.stderr parsl.dataflow.dflow DEBUG: Task 319 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 319 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 319 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 319 has memoization hash f33cfc68f32e548955ff8daad3104a41 parsl.dataflow.memoization INFO: Task 319 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 319 try 0 launched on executor multi with executor id 320 parsl.dataflow.dflow INFO: Standard output for task 319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/92bfe988-877c-4db7-bd77-997b56741a6e_isrForCrosstalkSources_996980_14.stdout parsl.dataflow.dflow INFO: Standard error for task 319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/92bfe988-877c-4db7-bd77-997b56741a6e_isrForCrosstalkSources_996980_14.stderr parsl.dataflow.dflow DEBUG: Task 320 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 320 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 320 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 320 has memoization hash 9b8cb7b7b0ee6e593b3a1dc2e4c7d3ae parsl.dataflow.memoization INFO: Task 320 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:25 proc_lsst.multi:170 [INFO] [multi] job_id 17074834 proc_lsst.multi INFO: [multi] job_id 17074834 2024-03-15 13:49:25 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 2 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 2 2024-03-15 13:49:25 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:25 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074834 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074834 parsl.executors.status_handling DEBUG: Launched block 2 on executor multi with job ID 17074834 parsl.executors.status_handling INFO: Allocated block ID 3 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-3 2024-03-15 13:49:25 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=3 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-3 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=3 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-3 2024-03-15 13:49:25 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=3 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=3 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:25 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=3 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=3 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 2 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 2 2024-03-15 13:49:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:25 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=3 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=3 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 320 try 0 launched on executor multi with executor id 321 parsl.dataflow.dflow INFO: Standard output for task 320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/55a154e0-7a41-4f67-b3e1-69f61df98cff_isrForCrosstalkSources_997026_13.stdout parsl.dataflow.dflow INFO: Standard error for task 320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/55a154e0-7a41-4f67-b3e1-69f61df98cff_isrForCrosstalkSources_997026_13.stderr parsl.dataflow.dflow DEBUG: Task 321 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 321 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 321 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 321 has memoization hash c697db1f7b7fd7f66997205663b46a5b parsl.dataflow.memoization INFO: Task 321 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 321 try 0 launched on executor multi with executor id 322 parsl.dataflow.dflow INFO: Standard output for task 321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/18db146c-a3fa-4cad-aa43-37d914da22ef_isrForCrosstalkSources_996964_27.stdout parsl.dataflow.dflow INFO: Standard error for task 321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/18db146c-a3fa-4cad-aa43-37d914da22ef_isrForCrosstalkSources_996964_27.stderr parsl.dataflow.dflow DEBUG: Task 322 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 322 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 322 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 322 has memoization hash 6de8ce80192e5f687e6564721094f00e parsl.dataflow.memoization INFO: Task 322 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 322 try 0 launched on executor multi with executor id 323 parsl.dataflow.dflow INFO: Standard output for task 322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/2ac8b7e5-9b5d-4147-a0a1-c5dee24afe1e_isrForCrosstalkSources_997024_9.stdout parsl.dataflow.dflow INFO: Standard error for task 322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/2ac8b7e5-9b5d-4147-a0a1-c5dee24afe1e_isrForCrosstalkSources_997024_9.stderr parsl.dataflow.dflow DEBUG: Task 323 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 323 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 323 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 323 has memoization hash da3f4c431b19871b5de706653ec2899b parsl.dataflow.memoization INFO: Task 323 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 323 try 0 launched on executor multi with executor id 324 parsl.dataflow.dflow INFO: Standard output for task 323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/3e7182ca-683e-42d5-8ff4-040b078571b9_isrForCrosstalkSources_997006_43.stdout parsl.dataflow.dflow INFO: Standard error for task 323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/3e7182ca-683e-42d5-8ff4-040b078571b9_isrForCrosstalkSources_997006_43.stderr parsl.dataflow.dflow DEBUG: Task 324 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 324 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 324 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 324 has memoization hash a183924b70827f8672d624135c3b5ff2 parsl.dataflow.memoization INFO: Task 324 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 324 try 0 launched on executor multi with executor id 325 parsl.dataflow.dflow INFO: Standard output for task 324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f848d1c7-6460-49b2-884c-0656f7213aa5_isrForCrosstalkSources_997020_23.stdout parsl.dataflow.dflow INFO: Standard error for task 324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f848d1c7-6460-49b2-884c-0656f7213aa5_isrForCrosstalkSources_997020_23.stderr parsl.dataflow.dflow DEBUG: Task 325 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 325 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 325 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 325 has memoization hash 692d45bba5169341c1c7d0bb0e8bfc2e parsl.dataflow.memoization INFO: Task 325 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 325 try 0 launched on executor multi with executor id 326 parsl.dataflow.dflow INFO: Standard output for task 325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/156726e6-f63e-4090-bb28-f5731baa03f6_isrForCrosstalkSources_997004_1.stdout parsl.dataflow.dflow INFO: Standard error for task 325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/156726e6-f63e-4090-bb28-f5731baa03f6_isrForCrosstalkSources_997004_1.stderr parsl.dataflow.dflow DEBUG: Task 326 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 326 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 326 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 326 has memoization hash 6612c521f0fd1dcd31e815849ad1ec8e parsl.dataflow.memoization INFO: Task 326 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 326 try 0 launched on executor multi with executor id 327 parsl.dataflow.dflow INFO: Standard output for task 326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/431248ba-297d-49dd-b723-e4790b36b983_isrForCrosstalkSources_997012_19.stdout parsl.dataflow.dflow INFO: Standard error for task 326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/431248ba-297d-49dd-b723-e4790b36b983_isrForCrosstalkSources_997012_19.stderr parsl.dataflow.dflow DEBUG: Task 327 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 327 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 327 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 327 has memoization hash 6c10e125fd5eded6fd98afd3eb6dc0af parsl.dataflow.memoization INFO: Task 327 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 327 try 0 launched on executor multi with executor id 328 parsl.dataflow.dflow INFO: Standard output for task 327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/1dc0914c-8f9d-485b-a290-827b9c7145a5_isrForCrosstalkSources_996969_54.stdout parsl.dataflow.dflow INFO: Standard error for task 327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/1dc0914c-8f9d-485b-a290-827b9c7145a5_isrForCrosstalkSources_996969_54.stderr parsl.dataflow.dflow DEBUG: Task 328 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 328 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 328 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 328 has memoization hash 085a3996af57262ee0623c2f4398876d parsl.dataflow.memoization INFO: Task 328 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 328 try 0 launched on executor multi with executor id 329 parsl.dataflow.dflow INFO: Standard output for task 328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/3d773dd3-7824-4498-89cd-1c1570ed63b9_isrForCrosstalkSources_996979_21.stdout parsl.dataflow.dflow INFO: Standard error for task 328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/3d773dd3-7824-4498-89cd-1c1570ed63b9_isrForCrosstalkSources_996979_21.stderr parsl.dataflow.dflow DEBUG: Task 329 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 329 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 329 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 329 has memoization hash e4bc214ab529dee10668e37d85c6c88c parsl.dataflow.memoization INFO: Task 329 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee51c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 329 try 0 launched on executor multi with executor id 330 parsl.dataflow.dflow INFO: Standard output for task 329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/7a8ed3ab-a07a-40a8-8770-d7bfb0ea0330_isrForCrosstalkSources_996998_62.stdout parsl.dataflow.dflow INFO: Standard error for task 329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/7a8ed3ab-a07a-40a8-8770-d7bfb0ea0330_isrForCrosstalkSources_996998_62.stderr parsl.dataflow.dflow DEBUG: Task 330 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 330 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 330 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 330 has memoization hash d4952c9cca1fdc9b168794f2a39f583f parsl.dataflow.memoization INFO: Task 330 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 330 try 0 launched on executor multi with executor id 331 parsl.dataflow.dflow INFO: Standard output for task 330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a7c854dd-f037-4b8c-a63e-1177f5a02c08_isrForCrosstalkSources_996978_61.stdout parsl.dataflow.dflow INFO: Standard error for task 330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a7c854dd-f037-4b8c-a63e-1177f5a02c08_isrForCrosstalkSources_996978_61.stderr parsl.dataflow.dflow DEBUG: Task 331 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 331 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 331 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 331 has memoization hash faddcc7faae2607eca5c36febbf56933 parsl.dataflow.memoization INFO: Task 331 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 331 try 0 launched on executor multi with executor id 332 parsl.dataflow.dflow INFO: Standard output for task 331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/763d1a83-5316-4e41-9392-9e20fb73966e_isrForCrosstalkSources_997005_7.stdout parsl.dataflow.dflow INFO: Standard error for task 331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/763d1a83-5316-4e41-9392-9e20fb73966e_isrForCrosstalkSources_997005_7.stderr parsl.dataflow.dflow DEBUG: Task 332 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 332 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 332 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 332 has memoization hash 2f3decd4136131fe8d76d189a714600d parsl.dataflow.memoization INFO: Task 332 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 332 try 0 launched on executor multi with executor id 333 parsl.dataflow.dflow INFO: Standard output for task 332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d43c7b07-1bb2-4c15-b36e-bc6d5955a7c0_isrForCrosstalkSources_997004_34.stdout parsl.dataflow.dflow INFO: Standard error for task 332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d43c7b07-1bb2-4c15-b36e-bc6d5955a7c0_isrForCrosstalkSources_997004_34.stderr parsl.dataflow.dflow DEBUG: Task 333 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 333 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 333 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 333 has memoization hash 74c0dd7177fdaa47d74d57a743c81335 parsl.dataflow.memoization INFO: Task 333 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 333 try 0 launched on executor multi with executor id 334 parsl.dataflow.dflow INFO: Standard output for task 333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/18953af5-9b64-4037-b488-69c23e5486e1_isrForCrosstalkSources_997028_24.stdout parsl.dataflow.dflow INFO: Standard error for task 333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/18953af5-9b64-4037-b488-69c23e5486e1_isrForCrosstalkSources_997028_24.stderr parsl.dataflow.dflow DEBUG: Task 334 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 334 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 334 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 334 has memoization hash 5d3909ed4ad20076db1715f4576e25cb parsl.dataflow.memoization INFO: Task 334 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 334 try 0 launched on executor multi with executor id 335 parsl.dataflow.dflow INFO: Standard output for task 334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c6e55c69-c054-46d0-a6ba-ca4027cad1f1_isrForCrosstalkSources_997000_37.stdout parsl.dataflow.dflow INFO: Standard error for task 334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c6e55c69-c054-46d0-a6ba-ca4027cad1f1_isrForCrosstalkSources_997000_37.stderr parsl.dataflow.dflow DEBUG: Task 335 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 335 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 335 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 335 has memoization hash a6cf0fd709e9c431a7cc74a5011f3508 parsl.dataflow.memoization INFO: Task 335 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 335 try 0 launched on executor multi with executor id 336 parsl.dataflow.dflow INFO: Standard output for task 335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/c936e6ea-b16e-4e1a-8a45-2ce3fd7297aa_isrForCrosstalkSources_996978_31.stdout parsl.dataflow.dflow INFO: Standard error for task 335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/c936e6ea-b16e-4e1a-8a45-2ce3fd7297aa_isrForCrosstalkSources_996978_31.stderr parsl.dataflow.dflow DEBUG: Task 336 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 336 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 336 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 336 has memoization hash beee3b6f7f7ac419880d28fc965b93b8 parsl.dataflow.memoization INFO: Task 336 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 336 try 0 launched on executor multi with executor id 337 parsl.dataflow.dflow INFO: Standard output for task 336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2ae830b2-8627-4675-ae6e-3af9b2725501_isrForCrosstalkSources_996980_8.stdout parsl.dataflow.dflow INFO: Standard error for task 336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2ae830b2-8627-4675-ae6e-3af9b2725501_isrForCrosstalkSources_996980_8.stderr parsl.dataflow.dflow DEBUG: Task 337 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 337 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 337 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 337 has memoization hash b4c355b8b03cd6f215dcdc573290e8a7 parsl.dataflow.memoization INFO: Task 337 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee54e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 337 try 0 launched on executor multi with executor id 338 parsl.dataflow.dflow INFO: Standard output for task 337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/02c6bedf-36e4-4c8d-85ef-41939d7aa451_isrForCrosstalkSources_997010_27.stdout parsl.dataflow.dflow INFO: Standard error for task 337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/02c6bedf-36e4-4c8d-85ef-41939d7aa451_isrForCrosstalkSources_997010_27.stderr parsl.dataflow.dflow DEBUG: Task 338 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 338 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 338 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 338 has memoization hash f42cada4f323915025d7988d59480859 parsl.dataflow.memoization INFO: Task 338 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 338 try 0 launched on executor multi with executor id 339 parsl.dataflow.dflow INFO: Standard output for task 338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/de64a921-098e-4d5d-8c78-b98efdb21da1_isrForCrosstalkSources_997027_21.stdout parsl.dataflow.dflow INFO: Standard error for task 338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/de64a921-098e-4d5d-8c78-b98efdb21da1_isrForCrosstalkSources_997027_21.stderr parsl.dataflow.dflow DEBUG: Task 339 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 339 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 339 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 339 has memoization hash ba83912d5f5400220d904b54eec20a55 parsl.dataflow.memoization INFO: Task 339 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 339 try 0 launched on executor multi with executor id 340 parsl.dataflow.dflow INFO: Standard output for task 339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4d4bb956-f0dc-4b3d-9490-2f847703aec8_isrForCrosstalkSources_997020_2.stdout parsl.dataflow.dflow INFO: Standard error for task 339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4d4bb956-f0dc-4b3d-9490-2f847703aec8_isrForCrosstalkSources_997020_2.stderr parsl.dataflow.dflow DEBUG: Task 340 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 340 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 340 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 340 has memoization hash d142f69abbfad850d4c8e70c35734427 parsl.dataflow.memoization INFO: Task 340 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 340 try 0 launched on executor multi with executor id 341 parsl.dataflow.dflow INFO: Standard output for task 340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/862b2700-ce51-42f2-acc5-24c286107113_isrForCrosstalkSources_997031_55.stdout parsl.dataflow.dflow INFO: Standard error for task 340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/862b2700-ce51-42f2-acc5-24c286107113_isrForCrosstalkSources_997031_55.stderr parsl.dataflow.dflow DEBUG: Task 341 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 341 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 341 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 341 has memoization hash 1cea99f427da98b9991fb3ece98066dc parsl.dataflow.memoization INFO: Task 341 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee56c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 341 try 0 launched on executor multi with executor id 342 parsl.dataflow.dflow INFO: Standard output for task 341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/1db2cd6b-03ff-4912-a01e-2d31bb7e402f_isrForCrosstalkSources_997006_41.stdout parsl.dataflow.dflow INFO: Standard error for task 341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/1db2cd6b-03ff-4912-a01e-2d31bb7e402f_isrForCrosstalkSources_997006_41.stderr parsl.dataflow.dflow DEBUG: Task 342 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 342 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 342 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 342 has memoization hash 1246d912ba18b32fe40766dbd3311a21 parsl.dataflow.memoization INFO: Task 342 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 342 try 0 launched on executor multi with executor id 343 parsl.dataflow.dflow INFO: Standard output for task 342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/53fc4f8e-57dd-4304-ae4c-7b3b95adf092_isrForCrosstalkSources_996983_44.stdout parsl.dataflow.dflow INFO: Standard error for task 342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/53fc4f8e-57dd-4304-ae4c-7b3b95adf092_isrForCrosstalkSources_996983_44.stderr parsl.dataflow.dflow DEBUG: Task 343 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 343 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 343 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 343 has memoization hash 0814f1d4263ab4834653f570fec3fd4c parsl.dataflow.memoization INFO: Task 343 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 343 try 0 launched on executor multi with executor id 344 parsl.dataflow.dflow INFO: Standard output for task 343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e02b4e3b-7bee-464c-aced-86df7821d630_isrForCrosstalkSources_996965_13.stdout parsl.dataflow.dflow INFO: Standard error for task 343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e02b4e3b-7bee-464c-aced-86df7821d630_isrForCrosstalkSources_996965_13.stderr parsl.dataflow.dflow DEBUG: Task 344 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 344 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 344 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 344 has memoization hash 5a4632fc283eab22f161007446d44e00 parsl.dataflow.memoization INFO: Task 344 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee59e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 344 try 0 launched on executor multi with executor id 345 parsl.dataflow.dflow INFO: Standard output for task 344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/00a89453-6805-48e8-955f-40e987b6c18c_isrForCrosstalkSources_997030_53.stdout parsl.dataflow.dflow INFO: Standard error for task 344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/00a89453-6805-48e8-955f-40e987b6c18c_isrForCrosstalkSources_997030_53.stderr parsl.dataflow.dflow DEBUG: Task 345 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 345 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 345 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 345 has memoization hash 9d4b94ec3212263a1997fc6dd491fdeb parsl.dataflow.memoization INFO: Task 345 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 345 try 0 launched on executor multi with executor id 346 parsl.dataflow.dflow INFO: Standard output for task 345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/da1de5af-941f-42df-99ba-3ed268c4f19e_isrForCrosstalkSources_996996_45.stdout parsl.dataflow.dflow INFO: Standard error for task 345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/da1de5af-941f-42df-99ba-3ed268c4f19e_isrForCrosstalkSources_996996_45.stderr parsl.dataflow.dflow DEBUG: Task 346 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 346 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 346 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 346 has memoization hash 38b8fbcbc8a35a6b2026032065c6dce7 parsl.dataflow.memoization INFO: Task 346 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee58a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 346 try 0 launched on executor multi with executor id 347 parsl.dataflow.dflow INFO: Standard output for task 346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/14f81efe-6d05-4054-a2d0-686d364c8633_isrForCrosstalkSources_997029_36.stdout parsl.dataflow.dflow INFO: Standard error for task 346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/14f81efe-6d05-4054-a2d0-686d364c8633_isrForCrosstalkSources_997029_36.stderr parsl.dataflow.dflow DEBUG: Task 347 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 347 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 347 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 347 has memoization hash d7d3b781ad5992eaac6b0848a9f74be2 parsl.dataflow.memoization INFO: Task 347 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 347 try 0 launched on executor multi with executor id 348 parsl.dataflow.dflow INFO: Standard output for task 347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/427b348e-8c29-4262-9824-ee22283ef5b9_isrForCrosstalkSources_996958_14.stdout parsl.dataflow.dflow INFO: Standard error for task 347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/427b348e-8c29-4262-9824-ee22283ef5b9_isrForCrosstalkSources_996958_14.stderr parsl.dataflow.dflow DEBUG: Task 348 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 348 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 348 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 348 has memoization hash ec9858d33efea5f3a7425f0917ff61dc parsl.dataflow.memoization INFO: Task 348 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 348 try 0 launched on executor multi with executor id 349 parsl.dataflow.dflow INFO: Standard output for task 348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/93c27a30-f3d7-4eba-8dfa-d2630a7e12e6_isrForCrosstalkSources_997028_26.stdout parsl.dataflow.dflow INFO: Standard error for task 348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/93c27a30-f3d7-4eba-8dfa-d2630a7e12e6_isrForCrosstalkSources_997028_26.stderr parsl.dataflow.dflow DEBUG: Task 349 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 349 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 349 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 349 has memoization hash 4da613a5a19a449880da36e042bb7469 parsl.dataflow.memoization INFO: Task 349 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 349 try 0 launched on executor multi with executor id 350 parsl.dataflow.dflow INFO: Standard output for task 349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5ea8ebce-7771-462d-93af-ac613467575d_isrForCrosstalkSources_996964_33.stdout parsl.dataflow.dflow INFO: Standard error for task 349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5ea8ebce-7771-462d-93af-ac613467575d_isrForCrosstalkSources_996964_33.stderr parsl.dataflow.dflow DEBUG: Task 350 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 350 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 350 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 350 has memoization hash f170ee67c9ed6b81962ba28e94a6aa65 parsl.dataflow.memoization INFO: Task 350 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 350 try 0 launched on executor multi with executor id 351 parsl.dataflow.dflow INFO: Standard output for task 350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f88678b7-44c3-4cf6-8d9a-36ee13c420fa_isrForCrosstalkSources_996974_40.stdout parsl.dataflow.dflow INFO: Standard error for task 350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f88678b7-44c3-4cf6-8d9a-36ee13c420fa_isrForCrosstalkSources_996974_40.stderr parsl.dataflow.dflow DEBUG: Task 351 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 351 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 351 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 351 has memoization hash 139ce654e78d11789a7c521b7f363830 parsl.dataflow.memoization INFO: Task 351 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 351 try 0 launched on executor multi with executor id 352 parsl.dataflow.dflow INFO: Standard output for task 351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/7e19d65e-1294-4bee-90ed-28ffb8c4295d_isrForCrosstalkSources_996979_27.stdout parsl.dataflow.dflow INFO: Standard error for task 351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/7e19d65e-1294-4bee-90ed-28ffb8c4295d_isrForCrosstalkSources_996979_27.stderr parsl.dataflow.dflow DEBUG: Task 352 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 352 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 352 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 352 has memoization hash 1e852add73bc342a26f5b499de398f5c parsl.dataflow.memoization INFO: Task 352 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 352 try 0 launched on executor multi with executor id 353 parsl.dataflow.dflow INFO: Standard output for task 352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/cdf68f38-7cee-4af2-bbbf-afdc4c953136_isrForCrosstalkSources_997000_13.stdout parsl.dataflow.dflow INFO: Standard error for task 352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/cdf68f38-7cee-4af2-bbbf-afdc4c953136_isrForCrosstalkSources_997000_13.stderr parsl.dataflow.dflow DEBUG: Task 353 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 353 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 353 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 353 has memoization hash 49e0043c0cf379f5085ca48c053f64b6 parsl.dataflow.memoization INFO: Task 353 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 353 try 0 launched on executor multi with executor id 354 parsl.dataflow.dflow INFO: Standard output for task 353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a23ba9fb-ab82-49ca-b1da-6d811ca5ba89_isrForCrosstalkSources_997027_35.stdout parsl.dataflow.dflow INFO: Standard error for task 353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a23ba9fb-ab82-49ca-b1da-6d811ca5ba89_isrForCrosstalkSources_997027_35.stderr parsl.dataflow.dflow DEBUG: Task 354 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 354 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 354 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 354 has memoization hash f69ab3438ee785ff34a505c1ff44e510 parsl.dataflow.memoization INFO: Task 354 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 354 try 0 launched on executor multi with executor id 355 parsl.dataflow.dflow INFO: Standard output for task 354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7d55a850-ad4b-4e26-b003-d92d02bd97b4_isrForCrosstalkSources_996975_46.stdout parsl.dataflow.dflow INFO: Standard error for task 354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7d55a850-ad4b-4e26-b003-d92d02bd97b4_isrForCrosstalkSources_996975_46.stderr parsl.dataflow.dflow DEBUG: Task 355 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 355 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 355 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 355 has memoization hash 86cb0bb80369ce8109f180ebf653cb88 parsl.dataflow.memoization INFO: Task 355 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 355 try 0 launched on executor multi with executor id 356 parsl.dataflow.dflow INFO: Standard output for task 355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/41a96e08-d826-4473-85aa-f669be62655e_isrForCrosstalkSources_997001_55.stdout parsl.dataflow.dflow INFO: Standard error for task 355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/41a96e08-d826-4473-85aa-f669be62655e_isrForCrosstalkSources_997001_55.stderr parsl.dataflow.dflow DEBUG: Task 356 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 356 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 356 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 356 has memoization hash a988378ee20c76337d03016d299e698d parsl.dataflow.memoization INFO: Task 356 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee60c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 356 try 0 launched on executor multi with executor id 357 parsl.dataflow.dflow INFO: Standard output for task 356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3214c2a6-2d96-4309-8e01-6f9ee887f0bb_isrForCrosstalkSources_996967_45.stdout parsl.dataflow.dflow INFO: Standard error for task 356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3214c2a6-2d96-4309-8e01-6f9ee887f0bb_isrForCrosstalkSources_996967_45.stderr parsl.dataflow.dflow DEBUG: Task 357 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 357 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 357 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 357 has memoization hash 342c2c18ab4d70d053c6afea80a92f3b parsl.dataflow.memoization INFO: Task 357 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 357 try 0 launched on executor multi with executor id 358 parsl.dataflow.dflow INFO: Standard output for task 357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/98686114-5397-477b-bc63-4aee57c7cd1d_isrForCrosstalkSources_996990_4.stdout parsl.dataflow.dflow INFO: Standard error for task 357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/98686114-5397-477b-bc63-4aee57c7cd1d_isrForCrosstalkSources_996990_4.stderr parsl.dataflow.dflow DEBUG: Task 358 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 358 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 358 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 358 has memoization hash 00b3ddc3e2c73704eee7be86a79eefec parsl.dataflow.memoization INFO: Task 358 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 358 try 0 launched on executor multi with executor id 359 parsl.dataflow.dflow INFO: Standard output for task 358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ccb8a67a-1906-475c-bfaa-73528e8dce19_isrForCrosstalkSources_997006_4.stdout parsl.dataflow.dflow INFO: Standard error for task 358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ccb8a67a-1906-475c-bfaa-73528e8dce19_isrForCrosstalkSources_997006_4.stderr parsl.dataflow.dflow DEBUG: Task 359 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 359 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 359 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 359 has memoization hash b1b5f309a45ca6bca48b4a8f1570605c parsl.dataflow.memoization INFO: Task 359 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 359 try 0 launched on executor multi with executor id 360 parsl.dataflow.dflow INFO: Standard output for task 359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7f2a4993-825a-49c2-b114-194438753b02_isrForCrosstalkSources_997005_3.stdout parsl.dataflow.dflow INFO: Standard error for task 359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7f2a4993-825a-49c2-b114-194438753b02_isrForCrosstalkSources_997005_3.stderr parsl.dataflow.dflow DEBUG: Task 360 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 360 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 360 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 360 has memoization hash e9fad27054875687b613e3a6e7fe2edd parsl.dataflow.memoization INFO: Task 360 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee62a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 360 try 0 launched on executor multi with executor id 361 parsl.dataflow.dflow INFO: Standard output for task 360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/ff1114c1-134c-4c97-8191-c4433b60a442_isrForCrosstalkSources_996999_42.stdout parsl.dataflow.dflow INFO: Standard error for task 360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/ff1114c1-134c-4c97-8191-c4433b60a442_isrForCrosstalkSources_996999_42.stderr parsl.dataflow.dflow DEBUG: Task 361 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 361 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 361 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 361 has memoization hash b203f4bc1dd820a741ac2b5b3a21d5b9 parsl.dataflow.memoization INFO: Task 361 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 361 try 0 launched on executor multi with executor id 362 parsl.dataflow.dflow INFO: Standard output for task 361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a7e86d17-b1bb-4454-aec6-f0d0b01f666f_isrForCrosstalkSources_996995_59.stdout parsl.dataflow.dflow INFO: Standard error for task 361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a7e86d17-b1bb-4454-aec6-f0d0b01f666f_isrForCrosstalkSources_996995_59.stderr parsl.dataflow.dflow DEBUG: Task 362 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 362 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 362 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 362 has memoization hash 3795ee2fe1255f5fc27dae55773497c6 parsl.dataflow.memoization INFO: Task 362 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 362 try 0 launched on executor multi with executor id 363 parsl.dataflow.dflow INFO: Standard output for task 362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9a584c27-ef90-40ca-ae7a-2dfacb4282c3_isrForCrosstalkSources_996997_54.stdout parsl.dataflow.dflow INFO: Standard error for task 362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9a584c27-ef90-40ca-ae7a-2dfacb4282c3_isrForCrosstalkSources_996997_54.stderr parsl.dataflow.dflow DEBUG: Task 363 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 363 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 363 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 363 has memoization hash 3fa1df46f7f4f2905405a6c0c4dbfc25 parsl.dataflow.memoization INFO: Task 363 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee63e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 363 try 0 launched on executor multi with executor id 364 parsl.dataflow.dflow INFO: Standard output for task 363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6ed0b3ab-34b3-4b1a-acbc-9b54d1933054_isrForCrosstalkSources_997017_35.stdout parsl.dataflow.dflow INFO: Standard error for task 363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6ed0b3ab-34b3-4b1a-acbc-9b54d1933054_isrForCrosstalkSources_997017_35.stderr parsl.dataflow.dflow DEBUG: Task 364 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 364 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 364 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 364 has memoization hash d5dd7c0d1a2accbd69fbd74ce720455d parsl.dataflow.memoization INFO: Task 364 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 364 try 0 launched on executor multi with executor id 365 parsl.dataflow.dflow INFO: Standard output for task 364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f8cb6052-6200-421b-a1e9-a05c634bf83a_isrForCrosstalkSources_996957_16.stdout parsl.dataflow.dflow INFO: Standard error for task 364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f8cb6052-6200-421b-a1e9-a05c634bf83a_isrForCrosstalkSources_996957_16.stderr parsl.dataflow.dflow DEBUG: Task 365 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 365 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 365 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 365 has memoization hash d0313878f53db3426d0d971194b56c41 parsl.dataflow.memoization INFO: Task 365 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee65c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 365 try 0 launched on executor multi with executor id 366 parsl.dataflow.dflow INFO: Standard output for task 365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/9602c128-92e1-47f6-a22b-be46fa03497d_isrForCrosstalkSources_997012_47.stdout parsl.dataflow.dflow INFO: Standard error for task 365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/9602c128-92e1-47f6-a22b-be46fa03497d_isrForCrosstalkSources_997012_47.stderr parsl.dataflow.dflow DEBUG: Task 366 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 366 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 366 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 366 has memoization hash 6106e30cf8233facf2694ea35cfaf9a9 parsl.dataflow.memoization INFO: Task 366 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 366 try 0 launched on executor multi with executor id 367 parsl.dataflow.dflow INFO: Standard output for task 366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5edb0088-5009-4954-aa76-8a4f023c3ea2_isrForCrosstalkSources_997015_27.stdout parsl.dataflow.dflow INFO: Standard error for task 366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5edb0088-5009-4954-aa76-8a4f023c3ea2_isrForCrosstalkSources_997015_27.stderr parsl.dataflow.dflow DEBUG: Task 367 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 367 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 367 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 367 has memoization hash 1ad1c9acd3938eba01dbf601e26b6b5b parsl.dataflow.memoization INFO: Task 367 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee67a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 367 try 0 launched on executor multi with executor id 368 parsl.dataflow.dflow INFO: Standard output for task 367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/93cea72d-386c-463b-839d-eff3a878d75f_isrForCrosstalkSources_997025_31.stdout parsl.dataflow.dflow INFO: Standard error for task 367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/93cea72d-386c-463b-839d-eff3a878d75f_isrForCrosstalkSources_997025_31.stderr parsl.dataflow.dflow DEBUG: Task 368 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 368 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 368 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 368 has memoization hash b17e9caa6cfc6094238ea6111e869ea9 parsl.dataflow.memoization INFO: Task 368 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 368 try 0 launched on executor multi with executor id 369 parsl.dataflow.dflow INFO: Standard output for task 368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b900ab46-53ef-4b75-8ac8-693ec175ad86_isrForCrosstalkSources_996986_51.stdout parsl.dataflow.dflow INFO: Standard error for task 368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b900ab46-53ef-4b75-8ac8-693ec175ad86_isrForCrosstalkSources_996986_51.stderr parsl.dataflow.dflow DEBUG: Task 369 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 369 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 369 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 369 has memoization hash 14f48a5c75db8b997cb081618ae0f7ed parsl.dataflow.memoization INFO: Task 369 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 369 try 0 launched on executor multi with executor id 370 parsl.dataflow.dflow INFO: Standard output for task 369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/b4c71dd2-cd89-464c-9ddf-4315a57fd6d1_isrForCrosstalkSources_996976_15.stdout parsl.dataflow.dflow INFO: Standard error for task 369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/b4c71dd2-cd89-464c-9ddf-4315a57fd6d1_isrForCrosstalkSources_996976_15.stderr parsl.dataflow.dflow DEBUG: Task 370 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 370 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 370 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 370 has memoization hash 49c90d5c8a89675415bd9336358dc774 parsl.dataflow.memoization INFO: Task 370 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee68e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 370 try 0 launched on executor multi with executor id 371 parsl.dataflow.dflow INFO: Standard output for task 370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c5080bfe-5c5a-48b7-bb4d-d1fdcb27437d_isrForCrosstalkSources_997015_49.stdout parsl.dataflow.dflow INFO: Standard error for task 370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c5080bfe-5c5a-48b7-bb4d-d1fdcb27437d_isrForCrosstalkSources_997015_49.stderr parsl.dataflow.dflow DEBUG: Task 371 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 371 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 371 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 371 has memoization hash b96e51ca187965183e22c9fb83b051d8 parsl.dataflow.memoization INFO: Task 371 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 371 try 0 launched on executor multi with executor id 372 parsl.dataflow.dflow INFO: Standard output for task 371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/a94e9bdd-d543-47d1-9163-c2bbeb0ecab8_isrForCrosstalkSources_996974_38.stdout parsl.dataflow.dflow INFO: Standard error for task 371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/a94e9bdd-d543-47d1-9163-c2bbeb0ecab8_isrForCrosstalkSources_996974_38.stderr parsl.dataflow.dflow DEBUG: Task 372 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 372 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 372 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 372 has memoization hash 190dab8374da269c46e346a6e9d0aa2e parsl.dataflow.memoization INFO: Task 372 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 372 try 0 launched on executor multi with executor id 373 parsl.dataflow.dflow INFO: Standard output for task 372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5ca73b03-10c7-4dbf-a5b6-52fd51df1b62_isrForCrosstalkSources_996954_37.stdout parsl.dataflow.dflow INFO: Standard error for task 372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5ca73b03-10c7-4dbf-a5b6-52fd51df1b62_isrForCrosstalkSources_996954_37.stderr parsl.dataflow.dflow DEBUG: Task 373 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 373 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 373 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 373 has memoization hash 29af126b42696e614217862e6709b417 parsl.dataflow.memoization INFO: Task 373 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 373 try 0 launched on executor multi with executor id 374 parsl.dataflow.dflow INFO: Standard output for task 373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cb7b7737-fb9d-4e08-88f8-c3ae5c691902_isrForCrosstalkSources_996956_20.stdout parsl.dataflow.dflow INFO: Standard error for task 373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cb7b7737-fb9d-4e08-88f8-c3ae5c691902_isrForCrosstalkSources_996956_20.stderr parsl.dataflow.dflow DEBUG: Task 374 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 374 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 374 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 374 has memoization hash 0caa8baa2b79914e6f686835f5718945 parsl.dataflow.memoization INFO: Task 374 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 374 try 0 launched on executor multi with executor id 375 parsl.dataflow.dflow INFO: Standard output for task 374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/5424dd0c-a806-4c03-8069-5f3103612956_isrForCrosstalkSources_996960_2.stdout parsl.dataflow.dflow INFO: Standard error for task 374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/5424dd0c-a806-4c03-8069-5f3103612956_isrForCrosstalkSources_996960_2.stderr parsl.dataflow.dflow DEBUG: Task 375 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 375 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 375 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 375 has memoization hash 7246597ef2e1695c9585a780d4f82e1c parsl.dataflow.memoization INFO: Task 375 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 375 try 0 launched on executor multi with executor id 376 parsl.dataflow.dflow INFO: Standard output for task 375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/3a199a8f-002d-4815-b55f-55b3a846c8cd_isrForCrosstalkSources_997009_37.stdout parsl.dataflow.dflow INFO: Standard error for task 375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/3a199a8f-002d-4815-b55f-55b3a846c8cd_isrForCrosstalkSources_997009_37.stderr parsl.dataflow.dflow DEBUG: Task 376 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 376 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 376 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 376 has memoization hash 41b1e6c05781edefa7b1f5a91d87ef8a parsl.dataflow.memoization INFO: Task 376 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 376 try 0 launched on executor multi with executor id 377 parsl.dataflow.dflow INFO: Standard output for task 376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/8253f94b-f87b-4f8f-8c70-12217e130cee_isrForCrosstalkSources_997009_9.stdout parsl.dataflow.dflow INFO: Standard error for task 376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/8253f94b-f87b-4f8f-8c70-12217e130cee_isrForCrosstalkSources_997009_9.stderr parsl.dataflow.dflow DEBUG: Task 377 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 377 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 377 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 377 has memoization hash 14739be2fc107f7dbe24777ef826b698 parsl.dataflow.memoization INFO: Task 377 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 377 try 0 launched on executor multi with executor id 378 parsl.dataflow.dflow INFO: Standard output for task 377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9cf20c12-5098-47f8-baf7-4dc28e47499f_isrForCrosstalkSources_997015_31.stdout parsl.dataflow.dflow INFO: Standard error for task 377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9cf20c12-5098-47f8-baf7-4dc28e47499f_isrForCrosstalkSources_997015_31.stderr parsl.dataflow.dflow DEBUG: Task 378 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 378 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 378 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 378 has memoization hash 73f6ccc86b5dac0a3d3cf41e424b1ee5 parsl.dataflow.memoization INFO: Task 378 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 378 try 0 launched on executor multi with executor id 379 parsl.dataflow.dflow INFO: Standard output for task 378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/d2bc552a-a2e3-4392-84d4-d829947b6cc6_isrForCrosstalkSources_996970_44.stdout parsl.dataflow.dflow INFO: Standard error for task 378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/d2bc552a-a2e3-4392-84d4-d829947b6cc6_isrForCrosstalkSources_996970_44.stderr parsl.dataflow.dflow DEBUG: Task 379 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 379 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 379 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 379 has memoization hash 8708f228d5c463fd017aee236ede5918 parsl.dataflow.memoization INFO: Task 379 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 379 try 0 launched on executor multi with executor id 380 parsl.dataflow.dflow INFO: Standard output for task 379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/086bcd9d-8998-4682-a358-215ea498d581_isrForCrosstalkSources_997028_27.stdout parsl.dataflow.dflow INFO: Standard error for task 379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/086bcd9d-8998-4682-a358-215ea498d581_isrForCrosstalkSources_997028_27.stderr parsl.dataflow.dflow DEBUG: Task 380 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 380 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 380 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 380 has memoization hash f1ab3f268bf60cf6cc1d33f2fdc91344 parsl.dataflow.memoization INFO: Task 380 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 380 try 0 launched on executor multi with executor id 381 parsl.dataflow.dflow INFO: Standard output for task 380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/6eb3d3f4-d163-4d92-b542-a803b7c89ce7_isrForCrosstalkSources_997011_58.stdout parsl.dataflow.dflow INFO: Standard error for task 380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/6eb3d3f4-d163-4d92-b542-a803b7c89ce7_isrForCrosstalkSources_997011_58.stderr parsl.dataflow.dflow DEBUG: Task 381 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 381 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 381 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 381 has memoization hash 3bf57cc8f1f4b253fcd24273d9e40f49 parsl.dataflow.memoization INFO: Task 381 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee6de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 381 try 0 launched on executor multi with executor id 382 parsl.dataflow.dflow INFO: Standard output for task 381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9d1c647e-537e-4dfb-900e-8464de1c1999_isrForCrosstalkSources_996986_46.stdout parsl.dataflow.dflow INFO: Standard error for task 381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9d1c647e-537e-4dfb-900e-8464de1c1999_isrForCrosstalkSources_996986_46.stderr parsl.dataflow.dflow DEBUG: Task 382 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 382 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 382 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 382 has memoization hash f8fbcd7e93cab00167a9228b97b914e3 parsl.dataflow.memoization INFO: Task 382 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 382 try 0 launched on executor multi with executor id 383 parsl.dataflow.dflow INFO: Standard output for task 382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f8d7a9f4-d4ee-4868-af98-45551c2ac61a_isrForCrosstalkSources_996989_58.stdout parsl.dataflow.dflow INFO: Standard error for task 382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f8d7a9f4-d4ee-4868-af98-45551c2ac61a_isrForCrosstalkSources_996989_58.stderr parsl.dataflow.dflow DEBUG: Task 383 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 383 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 383 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 383 has memoization hash 2fbb6699ff41e9a4d8ab3e42901c383d parsl.dataflow.memoization INFO: Task 383 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 383 try 0 launched on executor multi with executor id 384 parsl.dataflow.dflow INFO: Standard output for task 383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/139216da-99ad-441e-9013-c3ee7f4168a8_isrForCrosstalkSources_996985_4.stdout parsl.dataflow.dflow INFO: Standard error for task 383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/139216da-99ad-441e-9013-c3ee7f4168a8_isrForCrosstalkSources_996985_4.stderr parsl.dataflow.dflow DEBUG: Task 384 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 384 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 384 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 384 has memoization hash 607a10a99a763eab444dee318a7be5fb parsl.dataflow.memoization INFO: Task 384 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 384 try 0 launched on executor multi with executor id 385 parsl.dataflow.dflow INFO: Standard output for task 384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/81db0716-d415-4105-8e8f-f15c0e390155_isrForCrosstalkSources_996982_43.stdout parsl.dataflow.dflow INFO: Standard error for task 384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/81db0716-d415-4105-8e8f-f15c0e390155_isrForCrosstalkSources_996982_43.stderr parsl.dataflow.dflow DEBUG: Task 385 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 385 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 385 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 385 has memoization hash a416872033b5f405e42cf5eecb87e4f9 parsl.dataflow.memoization INFO: Task 385 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 385 try 0 launched on executor multi with executor id 386 parsl.dataflow.dflow INFO: Standard output for task 385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/295a0b95-fe7f-425e-b198-99444d66f6ac_isrForCrosstalkSources_996993_15.stdout parsl.dataflow.dflow INFO: Standard error for task 385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/295a0b95-fe7f-425e-b198-99444d66f6ac_isrForCrosstalkSources_996993_15.stderr parsl.dataflow.dflow DEBUG: Task 386 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 386 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 386 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 386 has memoization hash 4219856e7be1cf0e650920420defbe97 parsl.dataflow.memoization INFO: Task 386 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 386 try 0 launched on executor multi with executor id 387 parsl.dataflow.dflow INFO: Standard output for task 386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4c6dc2dc-4686-4533-94fe-a6f415c7e004_isrForCrosstalkSources_996963_20.stdout parsl.dataflow.dflow INFO: Standard error for task 386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4c6dc2dc-4686-4533-94fe-a6f415c7e004_isrForCrosstalkSources_996963_20.stderr parsl.dataflow.dflow DEBUG: Task 387 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 387 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 387 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 387 has memoization hash 73be8686466734aaf43463373d794824 parsl.dataflow.memoization INFO: Task 387 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 387 try 0 launched on executor multi with executor id 388 parsl.dataflow.dflow INFO: Standard output for task 387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/1848fb79-937f-4091-b602-e29cfdb08520_isrForCrosstalkSources_996989_39.stdout parsl.dataflow.dflow INFO: Standard error for task 387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/1848fb79-937f-4091-b602-e29cfdb08520_isrForCrosstalkSources_996989_39.stderr parsl.dataflow.dflow DEBUG: Task 388 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 388 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 388 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 388 has memoization hash ef8f22c304041896e7a0320195e0a173 parsl.dataflow.memoization INFO: Task 388 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 388 try 0 launched on executor multi with executor id 389 parsl.dataflow.dflow INFO: Standard output for task 388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/138dde35-d0c5-4619-ba67-e42b613258e5_isrForCrosstalkSources_997016_6.stdout parsl.dataflow.dflow INFO: Standard error for task 388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/138dde35-d0c5-4619-ba67-e42b613258e5_isrForCrosstalkSources_997016_6.stderr parsl.dataflow.dflow DEBUG: Task 389 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 389 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 389 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 389 has memoization hash d92c31c4033826930c243f0cd9e8dea8 parsl.dataflow.memoization INFO: Task 389 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 389 try 0 launched on executor multi with executor id 390 parsl.dataflow.dflow INFO: Standard output for task 389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b40c1e6b-8887-4cd2-80e3-cc75a37222f7_isrForCrosstalkSources_996953_54.stdout parsl.dataflow.dflow INFO: Standard error for task 389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b40c1e6b-8887-4cd2-80e3-cc75a37222f7_isrForCrosstalkSources_996953_54.stderr parsl.dataflow.dflow DEBUG: Task 390 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 390 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 390 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 390 has memoization hash 13035f0819202abf550e5c432d4439b1 parsl.dataflow.memoization INFO: Task 390 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee79c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 390 try 0 launched on executor multi with executor id 391 parsl.dataflow.dflow INFO: Standard output for task 390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/87d17837-2a1e-4ff1-9fe7-cc9fa76f8a0e_isrForCrosstalkSources_996962_56.stdout parsl.dataflow.dflow INFO: Standard error for task 390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/87d17837-2a1e-4ff1-9fe7-cc9fa76f8a0e_isrForCrosstalkSources_996962_56.stderr parsl.dataflow.dflow DEBUG: Task 391 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 391 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 391 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 391 has memoization hash 17d69e65795b972c0161df0e15d7f409 parsl.dataflow.memoization INFO: Task 391 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 391 try 0 launched on executor multi with executor id 392 parsl.dataflow.dflow INFO: Standard output for task 391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/915567b0-e432-4cf8-8a17-ea81bee3a1dd_isrForCrosstalkSources_996973_61.stdout parsl.dataflow.dflow INFO: Standard error for task 391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/915567b0-e432-4cf8-8a17-ea81bee3a1dd_isrForCrosstalkSources_996973_61.stderr parsl.dataflow.dflow DEBUG: Task 392 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 392 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 392 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 392 has memoization hash 015a804ece89641b5274b0b7da808acc parsl.dataflow.memoization INFO: Task 392 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee77e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 392 try 0 launched on executor multi with executor id 393 parsl.dataflow.dflow INFO: Standard output for task 392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c73a5aa7-32a9-45d5-84e3-dc4296555aeb_isrForCrosstalkSources_997020_61.stdout parsl.dataflow.dflow INFO: Standard error for task 392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c73a5aa7-32a9-45d5-84e3-dc4296555aeb_isrForCrosstalkSources_997020_61.stderr parsl.dataflow.dflow DEBUG: Task 393 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 393 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 393 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 393 has memoization hash 68c54f743b57454bc178c53185cc32c7 parsl.dataflow.memoization INFO: Task 393 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 393 try 0 launched on executor multi with executor id 394 parsl.dataflow.dflow INFO: Standard output for task 393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/62075e10-8559-4741-91fc-cd380c814fa9_isrForCrosstalkSources_996953_39.stdout parsl.dataflow.dflow INFO: Standard error for task 393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/62075e10-8559-4741-91fc-cd380c814fa9_isrForCrosstalkSources_996953_39.stderr parsl.dataflow.dflow DEBUG: Task 394 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 394 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 394 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 394 has memoization hash 21fa5901467dd679cc0bd5302fe6381f parsl.dataflow.memoization INFO: Task 394 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 394 try 0 launched on executor multi with executor id 395 parsl.dataflow.dflow INFO: Standard output for task 394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/255c352a-2b28-4045-a08c-2ad06a7707ae_isrForCrosstalkSources_996996_1.stdout parsl.dataflow.dflow INFO: Standard error for task 394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/255c352a-2b28-4045-a08c-2ad06a7707ae_isrForCrosstalkSources_996996_1.stderr parsl.dataflow.dflow DEBUG: Task 395 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 395 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 395 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 395 has memoization hash 996296723a7e63ea33f6bcbb479d0f79 parsl.dataflow.memoization INFO: Task 395 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee76a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 395 try 0 launched on executor multi with executor id 396 parsl.dataflow.dflow INFO: Standard output for task 395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/90aece87-522d-4d20-9f11-3896ffeca0c4_isrForCrosstalkSources_997008_37.stdout parsl.dataflow.dflow INFO: Standard error for task 395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/90aece87-522d-4d20-9f11-3896ffeca0c4_isrForCrosstalkSources_997008_37.stderr parsl.dataflow.dflow DEBUG: Task 396 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 396 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 396 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 396 has memoization hash 68b4e636ca9dd1dbea89c87524f6992d parsl.dataflow.memoization INFO: Task 396 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 396 try 0 launched on executor multi with executor id 397 parsl.dataflow.dflow INFO: Standard output for task 396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/12578100-1175-4848-be84-7895207b47a6_isrForCrosstalkSources_996993_7.stdout parsl.dataflow.dflow INFO: Standard error for task 396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/12578100-1175-4848-be84-7895207b47a6_isrForCrosstalkSources_996993_7.stderr parsl.dataflow.dflow DEBUG: Task 397 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 397 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 397 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 397 has memoization hash a84f9a84048a0df48678c0dca5ad0119 parsl.dataflow.memoization INFO: Task 397 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee74c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 397 try 0 launched on executor multi with executor id 398 parsl.dataflow.dflow INFO: Standard output for task 397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/4b1bf8c7-4eac-4c1d-ad4d-d6cfe64f28f7_isrForCrosstalkSources_997002_34.stdout parsl.dataflow.dflow INFO: Standard error for task 397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/4b1bf8c7-4eac-4c1d-ad4d-d6cfe64f28f7_isrForCrosstalkSources_997002_34.stderr parsl.dataflow.dflow DEBUG: Task 398 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 398 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 398 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 398 has memoization hash 706550b039fe747f2e558f2dc4a85982 parsl.dataflow.memoization INFO: Task 398 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 398 try 0 launched on executor multi with executor id 399 parsl.dataflow.dflow INFO: Standard output for task 398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/2ea902a6-4abf-4984-ae58-bc9dabbf3dcc_isrForCrosstalkSources_997004_3.stdout parsl.dataflow.dflow INFO: Standard error for task 398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/2ea902a6-4abf-4984-ae58-bc9dabbf3dcc_isrForCrosstalkSources_997004_3.stderr parsl.dataflow.dflow DEBUG: Task 399 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 399 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 399 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 399 has memoization hash 0869e8ba57a975788c3b3914c94c9933 parsl.dataflow.memoization INFO: Task 399 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee72e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 399 try 0 launched on executor multi with executor id 400 parsl.dataflow.dflow INFO: Standard output for task 399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/aeaa9001-549c-4e30-907f-0c55d5929858_isrForCrosstalkSources_997031_31.stdout parsl.dataflow.dflow INFO: Standard error for task 399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/aeaa9001-549c-4e30-907f-0c55d5929858_isrForCrosstalkSources_997031_31.stderr parsl.dataflow.dflow DEBUG: Task 400 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 400 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 400 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 400 has memoization hash bf57cfabd435d383160b2a7b0205de28 parsl.dataflow.memoization INFO: Task 400 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee7380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 400 try 0 launched on executor multi with executor id 401 parsl.dataflow.dflow INFO: Standard output for task 400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e11427b0-ff32-4ac2-983a-8a99def333c2_isrForCrosstalkSources_997016_47.stdout parsl.dataflow.dflow INFO: Standard error for task 400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e11427b0-ff32-4ac2-983a-8a99def333c2_isrForCrosstalkSources_997016_47.stderr parsl.dataflow.dflow DEBUG: Task 401 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 401 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 401 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 401 has memoization hash 4995552f850c0d76350f5e86ed851a11 parsl.dataflow.memoization INFO: Task 401 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee71a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 401 try 0 launched on executor multi with executor id 402 parsl.dataflow.dflow INFO: Standard output for task 401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/ca5d2855-207d-44e6-ac7e-13f39a79e2b1_isrForCrosstalkSources_996993_8.stdout parsl.dataflow.dflow INFO: Standard error for task 401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/ca5d2855-207d-44e6-ac7e-13f39a79e2b1_isrForCrosstalkSources_996993_8.stderr parsl.dataflow.dflow DEBUG: Task 402 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 402 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 402 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 402 has memoization hash cd4d836737fc191adf94f59e9511dc7b parsl.dataflow.memoization INFO: Task 402 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee53a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 402 try 0 launched on executor multi with executor id 403 parsl.dataflow.dflow INFO: Standard output for task 402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/19122548-755b-47f0-b2e3-f6beb6dbca90_isrForCrosstalkSources_996971_48.stdout parsl.dataflow.dflow INFO: Standard error for task 402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/19122548-755b-47f0-b2e3-f6beb6dbca90_isrForCrosstalkSources_996971_48.stderr parsl.dataflow.dflow DEBUG: Task 403 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 403 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 403 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 403 has memoization hash 876c35efbf5caff5e28e6a21b343bf57 parsl.dataflow.memoization INFO: Task 403 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803200e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 403 try 0 launched on executor multi with executor id 404 parsl.dataflow.dflow INFO: Standard output for task 403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/eee1f0d9-c023-4992-8a2d-30df552cbd56_isrForCrosstalkSources_996969_20.stdout parsl.dataflow.dflow INFO: Standard error for task 403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/eee1f0d9-c023-4992-8a2d-30df552cbd56_isrForCrosstalkSources_996969_20.stderr parsl.dataflow.dflow DEBUG: Task 404 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 404 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 404 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 404 has memoization hash cb4fab36a36e35bd8c818f40d8093cbc parsl.dataflow.memoization INFO: Task 404 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 404 try 0 launched on executor multi with executor id 405 parsl.dataflow.dflow INFO: Standard output for task 404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/79e78b6c-4c09-4813-a3f7-13d2ef5f496b_isrForCrosstalkSources_996982_28.stdout parsl.dataflow.dflow INFO: Standard error for task 404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/79e78b6c-4c09-4813-a3f7-13d2ef5f496b_isrForCrosstalkSources_996982_28.stderr parsl.dataflow.dflow DEBUG: Task 405 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 405 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 405 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 405 has memoization hash d59679ee2c532bfa60a03c8a4db5cfef parsl.dataflow.memoization INFO: Task 405 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 405 try 0 launched on executor multi with executor id 406 parsl.dataflow.dflow INFO: Standard output for task 405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5db3e851-b800-49e5-b65a-564c3bb83ae3_isrForCrosstalkSources_996988_44.stdout parsl.dataflow.dflow INFO: Standard error for task 405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5db3e851-b800-49e5-b65a-564c3bb83ae3_isrForCrosstalkSources_996988_44.stderr parsl.dataflow.dflow DEBUG: Task 406 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 406 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 406 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 406 has memoization hash 17446a552ac61f565b7a06db30effa12 parsl.dataflow.memoization INFO: Task 406 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803202c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 406 try 0 launched on executor multi with executor id 407 parsl.dataflow.dflow INFO: Standard output for task 406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/49723508-8c4e-43c7-a9a7-5af1071bebda_isrForCrosstalkSources_996966_50.stdout parsl.dataflow.dflow INFO: Standard error for task 406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/49723508-8c4e-43c7-a9a7-5af1071bebda_isrForCrosstalkSources_996966_50.stderr parsl.dataflow.dflow DEBUG: Task 407 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 407 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 407 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 407 has memoization hash 43322b718f09b9f0201e90c543c4e811 parsl.dataflow.memoization INFO: Task 407 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 407 try 0 launched on executor multi with executor id 408 parsl.dataflow.dflow INFO: Standard output for task 407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3d2d3d57-e959-4b96-a2d8-d8ed442bad6c_isrForCrosstalkSources_997014_33.stdout parsl.dataflow.dflow INFO: Standard error for task 407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3d2d3d57-e959-4b96-a2d8-d8ed442bad6c_isrForCrosstalkSources_997014_33.stderr parsl.dataflow.dflow DEBUG: Task 408 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 408 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 408 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 408 has memoization hash 8b5f772317f42fd0b31949ed57a46ab0 parsl.dataflow.memoization INFO: Task 408 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 408 try 0 launched on executor multi with executor id 409 parsl.dataflow.dflow INFO: Standard output for task 408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/49cb4143-7ce9-41d3-b006-ad48067cfb94_isrForCrosstalkSources_996982_2.stdout parsl.dataflow.dflow INFO: Standard error for task 408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/49cb4143-7ce9-41d3-b006-ad48067cfb94_isrForCrosstalkSources_996982_2.stderr parsl.dataflow.dflow DEBUG: Task 409 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 409 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 409 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 409 has memoization hash 1fac137baf39836c98bd5c8553b383ab parsl.dataflow.memoization INFO: Task 409 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 409 try 0 launched on executor multi with executor id 410 parsl.dataflow.dflow INFO: Standard output for task 409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/48fd3b63-6e94-4aa6-a2e8-c16173953525_isrForCrosstalkSources_996994_40.stdout parsl.dataflow.dflow INFO: Standard error for task 409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/48fd3b63-6e94-4aa6-a2e8-c16173953525_isrForCrosstalkSources_996994_40.stderr parsl.dataflow.dflow DEBUG: Task 410 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 410 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 410 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 410 has memoization hash ad3fc185422c2d977995e9ebc9a8fa56 parsl.dataflow.memoization INFO: Task 410 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803204a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 410 try 0 launched on executor multi with executor id 411 parsl.dataflow.dflow INFO: Standard output for task 410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/39f24c1d-b87c-4452-87b3-f9add6ede7cd_isrForCrosstalkSources_996983_37.stdout parsl.dataflow.dflow INFO: Standard error for task 410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/39f24c1d-b87c-4452-87b3-f9add6ede7cd_isrForCrosstalkSources_996983_37.stderr parsl.dataflow.dflow DEBUG: Task 411 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 411 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 411 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 411 has memoization hash 953132cf488e1093bd06fc35b8b06c4a parsl.dataflow.memoization INFO: Task 411 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803205e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 411 try 0 launched on executor multi with executor id 412 parsl.dataflow.dflow INFO: Standard output for task 411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/314d9662-5014-4a4e-8368-5571bb4b37fa_isrForCrosstalkSources_997019_60.stdout parsl.dataflow.dflow INFO: Standard error for task 411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/314d9662-5014-4a4e-8368-5571bb4b37fa_isrForCrosstalkSources_997019_60.stderr parsl.dataflow.dflow DEBUG: Task 412 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 412 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 412 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 412 has memoization hash 93e2dc1a9c910a0cb5507b44f6c5bc28 parsl.dataflow.memoization INFO: Task 412 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 412 try 0 launched on executor multi with executor id 413 parsl.dataflow.dflow INFO: Standard output for task 412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0e1523cf-e13c-4c50-bd69-a3f3261227f4_isrForCrosstalkSources_996982_37.stdout parsl.dataflow.dflow INFO: Standard error for task 412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0e1523cf-e13c-4c50-bd69-a3f3261227f4_isrForCrosstalkSources_996982_37.stderr parsl.dataflow.dflow DEBUG: Task 413 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 413 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 413 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 413 has memoization hash afa18dcf0ab7b7620d78ea003ae198bc parsl.dataflow.memoization INFO: Task 413 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 413 try 0 launched on executor multi with executor id 414 parsl.dataflow.dflow INFO: Standard output for task 413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/332e8064-b22f-45af-b977-0da0c58e5d04_isrForCrosstalkSources_997027_47.stdout parsl.dataflow.dflow INFO: Standard error for task 413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/332e8064-b22f-45af-b977-0da0c58e5d04_isrForCrosstalkSources_997027_47.stderr parsl.dataflow.dflow DEBUG: Task 414 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 414 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 414 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 414 has memoization hash 968228c4293965b1ed1d300f4291aa77 parsl.dataflow.memoization INFO: Task 414 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 414 try 0 launched on executor multi with executor id 415 parsl.dataflow.dflow INFO: Standard output for task 414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e745762c-840b-4549-a325-9645bfaa6008_isrForCrosstalkSources_996998_54.stdout parsl.dataflow.dflow INFO: Standard error for task 414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e745762c-840b-4549-a325-9645bfaa6008_isrForCrosstalkSources_996998_54.stderr parsl.dataflow.dflow DEBUG: Task 415 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 415 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 415 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 415 has memoization hash 4fcaa08f6276e1b719c35707f6928b39 parsl.dataflow.memoization INFO: Task 415 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803207c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 415 try 0 launched on executor multi with executor id 416 parsl.dataflow.dflow INFO: Standard output for task 415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/90a52a52-facf-41a9-8559-13b679cdfd22_isrForCrosstalkSources_996982_9.stdout parsl.dataflow.dflow INFO: Standard error for task 415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/90a52a52-facf-41a9-8559-13b679cdfd22_isrForCrosstalkSources_996982_9.stderr parsl.dataflow.dflow DEBUG: Task 416 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 416 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 416 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 416 has memoization hash 1eec9fa68dd9d8f6a9f73f3506cb0b29 parsl.dataflow.memoization INFO: Task 416 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 416 try 0 launched on executor multi with executor id 417 parsl.dataflow.dflow INFO: Standard output for task 416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/67bdda8a-a3b1-4bb0-8518-928c201c2cf5_isrForCrosstalkSources_997019_7.stdout parsl.dataflow.dflow INFO: Standard error for task 416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/67bdda8a-a3b1-4bb0-8518-928c201c2cf5_isrForCrosstalkSources_997019_7.stderr parsl.dataflow.dflow DEBUG: Task 417 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 417 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 417 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 417 has memoization hash d37d10c5103c2df9d5b3b6eddfe61be7 parsl.dataflow.memoization INFO: Task 417 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803209a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 417 try 0 launched on executor multi with executor id 418 parsl.dataflow.dflow INFO: Standard output for task 417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/be2ee2f1-8d87-409f-b6d4-24f4fd890576_isrForCrosstalkSources_996962_54.stdout parsl.dataflow.dflow INFO: Standard error for task 417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/be2ee2f1-8d87-409f-b6d4-24f4fd890576_isrForCrosstalkSources_996962_54.stderr parsl.dataflow.dflow DEBUG: Task 418 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 418 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 418 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 418 has memoization hash d73c458e558c2c8022bba37572dbba75 parsl.dataflow.memoization INFO: Task 418 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 418 try 0 launched on executor multi with executor id 419 parsl.dataflow.dflow INFO: Standard output for task 418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/f293f229-929b-4450-9236-c993e74267e3_isrForCrosstalkSources_996970_1.stdout parsl.dataflow.dflow INFO: Standard error for task 418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/f293f229-929b-4450-9236-c993e74267e3_isrForCrosstalkSources_996970_1.stderr parsl.dataflow.dflow DEBUG: Task 419 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 419 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 419 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 419 has memoization hash 5fe5be9cfebcc883486f940e05597a4c parsl.dataflow.memoization INFO: Task 419 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 419 try 0 launched on executor multi with executor id 420 parsl.dataflow.dflow INFO: Standard output for task 419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6568113f-62e5-4f62-b1dd-cd50e81c8f76_isrForCrosstalkSources_996971_36.stdout parsl.dataflow.dflow INFO: Standard error for task 419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6568113f-62e5-4f62-b1dd-cd50e81c8f76_isrForCrosstalkSources_996971_36.stderr parsl.dataflow.dflow DEBUG: Task 420 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 420 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 420 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 420 has memoization hash ab8fcf4f2a0e108fd8d2323812f622d6 parsl.dataflow.memoization INFO: Task 420 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 420 try 0 launched on executor multi with executor id 421 parsl.dataflow.dflow INFO: Standard output for task 420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6b1b3eac-ac77-40f3-886a-a1a1d0b982f2_isrForCrosstalkSources_996954_5.stdout parsl.dataflow.dflow INFO: Standard error for task 420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6b1b3eac-ac77-40f3-886a-a1a1d0b982f2_isrForCrosstalkSources_996954_5.stderr parsl.dataflow.dflow DEBUG: Task 421 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 421 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 421 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 421 has memoization hash dc195c1b957e04eaee931d337ec1f315 parsl.dataflow.memoization INFO: Task 421 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 421 try 0 launched on executor multi with executor id 422 parsl.dataflow.dflow INFO: Standard output for task 421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f28b7acf-b031-41ad-8313-597d9435fd0b_isrForCrosstalkSources_997015_17.stdout parsl.dataflow.dflow INFO: Standard error for task 421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f28b7acf-b031-41ad-8313-597d9435fd0b_isrForCrosstalkSources_997015_17.stderr parsl.dataflow.dflow DEBUG: Task 422 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 422 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 422 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 422 has memoization hash 39eaccfa068a63ed0e39d694b3b50799 parsl.dataflow.memoization INFO: Task 422 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 422 try 0 launched on executor multi with executor id 423 parsl.dataflow.dflow INFO: Standard output for task 422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1ef549f7-cc1a-46d6-a08e-6519b49d53fc_isrForCrosstalkSources_996953_58.stdout parsl.dataflow.dflow INFO: Standard error for task 422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1ef549f7-cc1a-46d6-a08e-6519b49d53fc_isrForCrosstalkSources_996953_58.stderr parsl.dataflow.dflow DEBUG: Task 423 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 423 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 423 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 423 has memoization hash 916eb831cb60f1c9cf4193c7363ed1f5 parsl.dataflow.memoization INFO: Task 423 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 423 try 0 launched on executor multi with executor id 424 parsl.dataflow.dflow INFO: Standard output for task 423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/f7e49f93-89d9-44f8-bf82-9c817fa262b8_isrForCrosstalkSources_996970_48.stdout parsl.dataflow.dflow INFO: Standard error for task 423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/f7e49f93-89d9-44f8-bf82-9c817fa262b8_isrForCrosstalkSources_996970_48.stderr parsl.dataflow.dflow DEBUG: Task 424 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 424 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 424 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 424 has memoization hash dd513f36f3b46f5aa65cd268589e2774 parsl.dataflow.memoization INFO: Task 424 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 424 try 0 launched on executor multi with executor id 425 parsl.dataflow.dflow INFO: Standard output for task 424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ae61f202-06d7-42d5-a338-1c3ee05c73ea_isrForCrosstalkSources_997009_7.stdout parsl.dataflow.dflow INFO: Standard error for task 424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ae61f202-06d7-42d5-a338-1c3ee05c73ea_isrForCrosstalkSources_997009_7.stderr parsl.dataflow.dflow DEBUG: Task 425 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 425 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 425 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 425 has memoization hash 2993cc9a88801b42486a1aea5ce6ed40 parsl.dataflow.memoization INFO: Task 425 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 425 try 0 launched on executor multi with executor id 426 parsl.dataflow.dflow INFO: Standard output for task 425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/04732ca5-fc1c-4efb-b596-092f425c12e4_isrForCrosstalkSources_996984_1.stdout parsl.dataflow.dflow INFO: Standard error for task 425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/04732ca5-fc1c-4efb-b596-092f425c12e4_isrForCrosstalkSources_996984_1.stderr parsl.dataflow.dflow DEBUG: Task 426 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 426 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 426 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 426 has memoization hash 04c8a741637c858bec899805f7e3056f parsl.dataflow.memoization INFO: Task 426 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 426 try 0 launched on executor multi with executor id 427 parsl.dataflow.dflow INFO: Standard output for task 426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bcc2cac7-583d-4586-b178-b7de50485e66_isrForCrosstalkSources_997030_14.stdout parsl.dataflow.dflow INFO: Standard error for task 426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bcc2cac7-583d-4586-b178-b7de50485e66_isrForCrosstalkSources_997030_14.stderr parsl.dataflow.dflow DEBUG: Task 427 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 427 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 427 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 427 has memoization hash bde7db16c69d5e7e2f6a3e17315a6bb5 parsl.dataflow.memoization INFO: Task 427 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 427 try 0 launched on executor multi with executor id 428 parsl.dataflow.dflow INFO: Standard output for task 427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c132cd6b-e85c-492b-8cc6-a04eba5b4140_isrForCrosstalkSources_997033_36.stdout parsl.dataflow.dflow INFO: Standard error for task 427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c132cd6b-e85c-492b-8cc6-a04eba5b4140_isrForCrosstalkSources_997033_36.stderr parsl.dataflow.dflow DEBUG: Task 428 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 428 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 428 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 428 has memoization hash 89daa0b95ae442cf09a8b2c7c1a58e32 parsl.dataflow.memoization INFO: Task 428 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 428 try 0 launched on executor multi with executor id 429 parsl.dataflow.dflow INFO: Standard output for task 428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/36897564-02c4-4abd-b29f-f06b530d34db_isrForCrosstalkSources_996952_41.stdout parsl.dataflow.dflow INFO: Standard error for task 428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/36897564-02c4-4abd-b29f-f06b530d34db_isrForCrosstalkSources_996952_41.stderr parsl.dataflow.dflow DEBUG: Task 429 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 429 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 429 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 429 has memoization hash d773a43be4caece6d493f418842a02f0 parsl.dataflow.memoization INFO: Task 429 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803211c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 429 try 0 launched on executor multi with executor id 430 parsl.dataflow.dflow INFO: Standard output for task 429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1821a837-9fa4-448d-abb3-b411c8e3e669_isrForCrosstalkSources_997000_50.stdout parsl.dataflow.dflow INFO: Standard error for task 429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1821a837-9fa4-448d-abb3-b411c8e3e669_isrForCrosstalkSources_997000_50.stderr parsl.dataflow.dflow DEBUG: Task 430 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 430 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 430 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 430 has memoization hash 874b9a5e772c9ce7bb3ec4a09dbe58ca parsl.dataflow.memoization INFO: Task 430 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 430 try 0 launched on executor multi with executor id 431 parsl.dataflow.dflow INFO: Standard output for task 430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d5d767b4-2c8c-4d64-afeb-2abe5be11f86_isrForCrosstalkSources_996965_16.stdout parsl.dataflow.dflow INFO: Standard error for task 430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d5d767b4-2c8c-4d64-afeb-2abe5be11f86_isrForCrosstalkSources_996965_16.stderr parsl.dataflow.dflow DEBUG: Task 431 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 431 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 431 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 431 has memoization hash 85da958fa0d213aed1e8aa83f24bdbda parsl.dataflow.memoization INFO: Task 431 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:26 proc_lsst.multi:170 [INFO] [multi] job_id 17074835 proc_lsst.multi INFO: [multi] job_id 17074835 2024-03-15 13:49:26 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 3 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 3 2024-03-15 13:49:26 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:26 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074835 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074835 parsl.executors.status_handling DEBUG: Launched block 3 on executor multi with job ID 17074835 parsl.executors.status_handling INFO: Allocated block ID 4 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-4 2024-03-15 13:49:26 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=4 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-4 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=4 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-4 2024-03-15 13:49:26 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=4 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=4 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:26 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=4 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=4 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 3 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 3 2024-03-15 13:49:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:26 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=4 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=4 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 431 try 0 launched on executor multi with executor id 432 parsl.dataflow.dflow INFO: Standard output for task 431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1dc7fb8-9eaa-4904-a448-36265344c293_isrForCrosstalkSources_996975_41.stdout parsl.dataflow.dflow INFO: Standard error for task 431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1dc7fb8-9eaa-4904-a448-36265344c293_isrForCrosstalkSources_996975_41.stderr parsl.dataflow.dflow DEBUG: Task 432 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 432 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 432 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 432 has memoization hash 861dc4dd88d5521d477ee5487d735b1f parsl.dataflow.memoization INFO: Task 432 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f320ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 432 try 0 launched on executor multi with executor id 433 parsl.dataflow.dflow INFO: Standard output for task 432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c3e04b10-3653-4912-9fbf-ae46c97a4229_isrForCrosstalkSources_996980_30.stdout parsl.dataflow.dflow INFO: Standard error for task 432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c3e04b10-3653-4912-9fbf-ae46c97a4229_isrForCrosstalkSources_996980_30.stderr parsl.dataflow.dflow DEBUG: Task 433 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 433 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 433 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 433 has memoization hash 8a02f0920ad2ab254a4758073d24c29f parsl.dataflow.memoization INFO: Task 433 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f613f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 433 try 0 launched on executor multi with executor id 434 parsl.dataflow.dflow INFO: Standard output for task 433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/eb2f8e1c-2fca-4ce9-b8a5-71c22c252368_isrForCrosstalkSources_997002_62.stdout parsl.dataflow.dflow INFO: Standard error for task 433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/eb2f8e1c-2fca-4ce9-b8a5-71c22c252368_isrForCrosstalkSources_997002_62.stderr parsl.dataflow.dflow DEBUG: Task 434 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 434 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 434 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 434 has memoization hash 6ee15ac2938c8ad2a0073acf8748f982 parsl.dataflow.memoization INFO: Task 434 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780320040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 434 try 0 launched on executor multi with executor id 435 parsl.dataflow.dflow INFO: Standard output for task 434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0c5e499c-22c8-47f0-9f95-0ede9ccc4b6b_isrForCrosstalkSources_996985_45.stdout parsl.dataflow.dflow INFO: Standard error for task 434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0c5e499c-22c8-47f0-9f95-0ede9ccc4b6b_isrForCrosstalkSources_996985_45.stderr parsl.dataflow.dflow DEBUG: Task 435 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 435 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 435 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 435 has memoization hash 3cd004ca21811571ac87c88e42383d6f parsl.dataflow.memoization INFO: Task 435 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 435 try 0 launched on executor multi with executor id 436 parsl.dataflow.dflow INFO: Standard output for task 435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/544da854-c8ca-423d-840f-08d67be6f8fe_isrForCrosstalkSources_997030_36.stdout parsl.dataflow.dflow INFO: Standard error for task 435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/544da854-c8ca-423d-840f-08d67be6f8fe_isrForCrosstalkSources_997030_36.stderr parsl.dataflow.dflow DEBUG: Task 436 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 436 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 436 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 436 has memoization hash 7493e9666506f7940b2aea3c155ea1c4 parsl.dataflow.memoization INFO: Task 436 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803213a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 436 try 0 launched on executor multi with executor id 437 parsl.dataflow.dflow INFO: Standard output for task 436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/5ce7879d-ac23-45de-a9fa-54051b095bfe_isrForCrosstalkSources_996974_44.stdout parsl.dataflow.dflow INFO: Standard error for task 436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/5ce7879d-ac23-45de-a9fa-54051b095bfe_isrForCrosstalkSources_996974_44.stderr parsl.dataflow.dflow DEBUG: Task 437 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 437 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 437 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 437 has memoization hash 6096affa12cc9bd540d8c524fe15a8f1 parsl.dataflow.memoization INFO: Task 437 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803214e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 437 try 0 launched on executor multi with executor id 438 parsl.dataflow.dflow INFO: Standard output for task 437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/8ffcef67-eb65-41b1-bec4-124a5e887d62_isrForCrosstalkSources_996957_24.stdout parsl.dataflow.dflow INFO: Standard error for task 437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/8ffcef67-eb65-41b1-bec4-124a5e887d62_isrForCrosstalkSources_996957_24.stderr parsl.dataflow.dflow DEBUG: Task 438 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 438 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 438 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 438 has memoization hash 243265f0298e9dcc1cea7080f6dfbc1e parsl.dataflow.memoization INFO: Task 438 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 438 try 0 launched on executor multi with executor id 439 parsl.dataflow.dflow INFO: Standard output for task 438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/cc49b279-2ad7-4a89-8097-9f4738d8b697_isrForCrosstalkSources_996977_3.stdout parsl.dataflow.dflow INFO: Standard error for task 438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/cc49b279-2ad7-4a89-8097-9f4738d8b697_isrForCrosstalkSources_996977_3.stderr parsl.dataflow.dflow DEBUG: Task 439 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 439 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 439 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 439 has memoization hash 5d1a351a3e83f303e666a55ad74ac85e parsl.dataflow.memoization INFO: Task 439 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 439 try 0 launched on executor multi with executor id 440 parsl.dataflow.dflow INFO: Standard output for task 439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/8916bcc2-541b-4060-9089-cbfcade307f3_isrForCrosstalkSources_997029_38.stdout parsl.dataflow.dflow INFO: Standard error for task 439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/8916bcc2-541b-4060-9089-cbfcade307f3_isrForCrosstalkSources_997029_38.stderr parsl.dataflow.dflow DEBUG: Task 440 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 440 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 440 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 440 has memoization hash 7ba93b23b955de5be838f28581509629 parsl.dataflow.memoization INFO: Task 440 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 440 try 0 launched on executor multi with executor id 441 parsl.dataflow.dflow INFO: Standard output for task 440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7148144a-392f-4a80-a265-417b57333701_isrForCrosstalkSources_997021_34.stdout parsl.dataflow.dflow INFO: Standard error for task 440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7148144a-392f-4a80-a265-417b57333701_isrForCrosstalkSources_997021_34.stderr parsl.dataflow.dflow DEBUG: Task 441 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 441 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 441 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 441 has memoization hash b60db28fb80f631c377120afddfc6296 parsl.dataflow.memoization INFO: Task 441 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803216c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 441 try 0 launched on executor multi with executor id 442 parsl.dataflow.dflow INFO: Standard output for task 441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/64679921-e376-49c7-9d8b-71b5a677bccd_isrForCrosstalkSources_997011_39.stdout parsl.dataflow.dflow INFO: Standard error for task 441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/64679921-e376-49c7-9d8b-71b5a677bccd_isrForCrosstalkSources_997011_39.stderr parsl.dataflow.dflow DEBUG: Task 442 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 442 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 442 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 442 has memoization hash 392403ebe05483418eac9f879defdc10 parsl.dataflow.memoization INFO: Task 442 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 442 try 0 launched on executor multi with executor id 443 parsl.dataflow.dflow INFO: Standard output for task 442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2f04a853-e565-4e31-8626-be5a77461991_isrForCrosstalkSources_996990_17.stdout parsl.dataflow.dflow INFO: Standard error for task 442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2f04a853-e565-4e31-8626-be5a77461991_isrForCrosstalkSources_996990_17.stderr parsl.dataflow.dflow DEBUG: Task 443 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 443 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 443 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 443 has memoization hash 61f223e1fff7c98a9990b7b07aac9363 parsl.dataflow.memoization INFO: Task 443 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803218a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 443 try 0 launched on executor multi with executor id 444 parsl.dataflow.dflow INFO: Standard output for task 443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ff8baf52-48e6-4cae-9e7a-827a773ca579_isrForCrosstalkSources_996978_62.stdout parsl.dataflow.dflow INFO: Standard error for task 443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ff8baf52-48e6-4cae-9e7a-827a773ca579_isrForCrosstalkSources_996978_62.stderr parsl.dataflow.dflow DEBUG: Task 444 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 444 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 444 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 444 has memoization hash 2bfa2ba89b5b22e887146aa1fc2d9060 parsl.dataflow.memoization INFO: Task 444 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803219e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 444 try 0 launched on executor multi with executor id 445 parsl.dataflow.dflow INFO: Standard output for task 444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/fbf25cf2-728f-438c-a6a9-f3493160ee90_isrForCrosstalkSources_997024_17.stdout parsl.dataflow.dflow INFO: Standard error for task 444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/fbf25cf2-728f-438c-a6a9-f3493160ee90_isrForCrosstalkSources_997024_17.stderr parsl.dataflow.dflow DEBUG: Task 445 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 445 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 445 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 445 has memoization hash 0dc84dc7e6712e987878bfb38a824217 parsl.dataflow.memoization INFO: Task 445 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 445 try 0 launched on executor multi with executor id 446 parsl.dataflow.dflow INFO: Standard output for task 445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f95bede1-ba37-43ba-bd0f-e036d5c76d2b_isrForCrosstalkSources_996973_45.stdout parsl.dataflow.dflow INFO: Standard error for task 445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f95bede1-ba37-43ba-bd0f-e036d5c76d2b_isrForCrosstalkSources_996973_45.stderr parsl.dataflow.dflow DEBUG: Task 446 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 446 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 446 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 446 has memoization hash 042b15b7dd7ee9e6c53f7ac8f6365074 parsl.dataflow.memoization INFO: Task 446 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 446 try 0 launched on executor multi with executor id 447 parsl.dataflow.dflow INFO: Standard output for task 446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d628f2d0-73a6-4105-9fa9-d6262a6102e1_isrForCrosstalkSources_996979_38.stdout parsl.dataflow.dflow INFO: Standard error for task 446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d628f2d0-73a6-4105-9fa9-d6262a6102e1_isrForCrosstalkSources_996979_38.stderr parsl.dataflow.dflow DEBUG: Task 447 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 447 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 447 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 447 has memoization hash 77b0663ec1baeba707fb9c52a2934c6a parsl.dataflow.memoization INFO: Task 447 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 447 try 0 launched on executor multi with executor id 448 parsl.dataflow.dflow INFO: Standard output for task 447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/006c7fb8-0360-467f-b7fa-b9ebc239574a_isrForCrosstalkSources_996958_18.stdout parsl.dataflow.dflow INFO: Standard error for task 447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/006c7fb8-0360-467f-b7fa-b9ebc239574a_isrForCrosstalkSources_996958_18.stderr parsl.dataflow.dflow DEBUG: Task 448 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 448 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 448 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 448 has memoization hash ab0ce5053521ca4a2f9814a976f433a4 parsl.dataflow.memoization INFO: Task 448 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 448 try 0 launched on executor multi with executor id 449 parsl.dataflow.dflow INFO: Standard output for task 448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/ba7cebf9-9e09-4ee1-9a23-6a3540c31411_isrForCrosstalkSources_996970_47.stdout parsl.dataflow.dflow INFO: Standard error for task 448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/ba7cebf9-9e09-4ee1-9a23-6a3540c31411_isrForCrosstalkSources_996970_47.stderr parsl.dataflow.dflow DEBUG: Task 449 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 449 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 449 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 449 has memoization hash 1347d6fec49b7c6ba191e9b95285e01e parsl.dataflow.memoization INFO: Task 449 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 449 try 0 launched on executor multi with executor id 450 parsl.dataflow.dflow INFO: Standard output for task 449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/9b2ad1c3-53e6-4963-b215-23f97f1d4010_isrForCrosstalkSources_997031_50.stdout parsl.dataflow.dflow INFO: Standard error for task 449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/9b2ad1c3-53e6-4963-b215-23f97f1d4010_isrForCrosstalkSources_997031_50.stderr parsl.dataflow.dflow DEBUG: Task 450 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 450 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 450 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 450 has memoization hash 3c2f4931784ac67e31a45482c7d2a838 parsl.dataflow.memoization INFO: Task 450 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 450 try 0 launched on executor multi with executor id 451 parsl.dataflow.dflow INFO: Standard output for task 450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5fce2064-4a2d-4563-9c08-cd5fdcace021_isrForCrosstalkSources_997009_36.stdout parsl.dataflow.dflow INFO: Standard error for task 450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5fce2064-4a2d-4563-9c08-cd5fdcace021_isrForCrosstalkSources_997009_36.stderr parsl.dataflow.dflow DEBUG: Task 451 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 451 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 451 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 451 has memoization hash a07735057b04752142984fcb04b9df15 parsl.dataflow.memoization INFO: Task 451 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 451 try 0 launched on executor multi with executor id 452 parsl.dataflow.dflow INFO: Standard output for task 451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/65b2979d-83d6-47db-b504-3495a5114c60_isrForCrosstalkSources_997022_51.stdout parsl.dataflow.dflow INFO: Standard error for task 451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/65b2979d-83d6-47db-b504-3495a5114c60_isrForCrosstalkSources_997022_51.stderr parsl.dataflow.dflow DEBUG: Task 452 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 452 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 452 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 452 has memoization hash e00a6003664f37e5dd64bd320d52c0f3 parsl.dataflow.memoization INFO: Task 452 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 452 try 0 launched on executor multi with executor id 453 parsl.dataflow.dflow INFO: Standard output for task 452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7b74e025-b9ae-4d53-bc7b-63a14ce3209e_isrForCrosstalkSources_996968_39.stdout parsl.dataflow.dflow INFO: Standard error for task 452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7b74e025-b9ae-4d53-bc7b-63a14ce3209e_isrForCrosstalkSources_996968_39.stderr parsl.dataflow.dflow DEBUG: Task 453 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 453 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 453 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 453 has memoization hash 6bcfebd8e6868660be4712ba50a12bd4 parsl.dataflow.memoization INFO: Task 453 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 453 try 0 launched on executor multi with executor id 454 parsl.dataflow.dflow INFO: Standard output for task 453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/02988f69-6519-43a0-b3a9-f563965aee5c_isrForCrosstalkSources_996974_52.stdout parsl.dataflow.dflow INFO: Standard error for task 453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/02988f69-6519-43a0-b3a9-f563965aee5c_isrForCrosstalkSources_996974_52.stderr parsl.dataflow.dflow DEBUG: Task 454 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 454 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 454 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 454 has memoization hash cebc0e9dd0064475d4e59c71e916a1ef parsl.dataflow.memoization INFO: Task 454 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 454 try 0 launched on executor multi with executor id 455 parsl.dataflow.dflow INFO: Standard output for task 454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8e8cb852-60d7-4d30-b355-4511345021cb_isrForCrosstalkSources_996976_61.stdout parsl.dataflow.dflow INFO: Standard error for task 454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8e8cb852-60d7-4d30-b355-4511345021cb_isrForCrosstalkSources_996976_61.stderr parsl.dataflow.dflow DEBUG: Task 455 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 455 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 455 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 455 has memoization hash 2ca25523502d7b346c1cf0118b3034f3 parsl.dataflow.memoization INFO: Task 455 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 455 try 0 launched on executor multi with executor id 456 parsl.dataflow.dflow INFO: Standard output for task 455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/31a1a6ef-47bf-4da2-a562-4f06aaae9e6d_isrForCrosstalkSources_996975_54.stdout parsl.dataflow.dflow INFO: Standard error for task 455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/31a1a6ef-47bf-4da2-a562-4f06aaae9e6d_isrForCrosstalkSources_996975_54.stderr parsl.dataflow.dflow DEBUG: Task 456 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 456 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 456 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 456 has memoization hash ed61091b39c346ca4c4883412038a22c parsl.dataflow.memoization INFO: Task 456 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803220c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 456 try 0 launched on executor multi with executor id 457 parsl.dataflow.dflow INFO: Standard output for task 456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ff4dfac3-906b-4064-8597-72d37eca882c_isrForCrosstalkSources_996959_42.stdout parsl.dataflow.dflow INFO: Standard error for task 456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ff4dfac3-906b-4064-8597-72d37eca882c_isrForCrosstalkSources_996959_42.stderr parsl.dataflow.dflow DEBUG: Task 457 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 457 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 457 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 457 has memoization hash 7513b3cf404804243afc1db8126c0f25 parsl.dataflow.memoization INFO: Task 457 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 457 try 0 launched on executor multi with executor id 458 parsl.dataflow.dflow INFO: Standard output for task 457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/40025d54-18ff-4297-aa32-b86f076876c4_isrForCrosstalkSources_996990_49.stdout parsl.dataflow.dflow INFO: Standard error for task 457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/40025d54-18ff-4297-aa32-b86f076876c4_isrForCrosstalkSources_996990_49.stderr parsl.dataflow.dflow DEBUG: Task 458 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 458 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 458 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 458 has memoization hash 53b8e84fd8a689c5880d0c13355f7a5e parsl.dataflow.memoization INFO: Task 458 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 458 try 0 launched on executor multi with executor id 459 parsl.dataflow.dflow INFO: Standard output for task 458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7757b19c-a127-4288-89ae-1ec480447739_isrForCrosstalkSources_997026_19.stdout parsl.dataflow.dflow INFO: Standard error for task 458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7757b19c-a127-4288-89ae-1ec480447739_isrForCrosstalkSources_997026_19.stderr parsl.dataflow.dflow DEBUG: Task 459 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 459 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 459 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 459 has memoization hash b83a07ef4a2a9e38caee0c9b8306b190 parsl.dataflow.memoization INFO: Task 459 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803222a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 459 try 0 launched on executor multi with executor id 460 parsl.dataflow.dflow INFO: Standard output for task 459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e21126aa-c55e-45f9-b528-ee494050db96_isrForCrosstalkSources_996961_1.stdout parsl.dataflow.dflow INFO: Standard error for task 459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e21126aa-c55e-45f9-b528-ee494050db96_isrForCrosstalkSources_996961_1.stderr parsl.dataflow.dflow DEBUG: Task 460 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 460 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 460 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 460 has memoization hash 422a6096b1a49274b80214da6a157e59 parsl.dataflow.memoization INFO: Task 460 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 460 try 0 launched on executor multi with executor id 461 parsl.dataflow.dflow INFO: Standard output for task 460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/be14dd3c-cd69-464d-9fa6-0cb3a23758ff_isrForCrosstalkSources_996973_30.stdout parsl.dataflow.dflow INFO: Standard error for task 460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/be14dd3c-cd69-464d-9fa6-0cb3a23758ff_isrForCrosstalkSources_996973_30.stderr parsl.dataflow.dflow DEBUG: Task 461 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 461 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 461 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 461 has memoization hash 4d7118453d804a0a3409c418976d65f3 parsl.dataflow.memoization INFO: Task 461 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803223e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 461 try 0 launched on executor multi with executor id 462 parsl.dataflow.dflow INFO: Standard output for task 461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f64e152e-3531-4463-b776-12622db2e852_isrForCrosstalkSources_997026_14.stdout parsl.dataflow.dflow INFO: Standard error for task 461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f64e152e-3531-4463-b776-12622db2e852_isrForCrosstalkSources_997026_14.stderr parsl.dataflow.dflow DEBUG: Task 462 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 462 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 462 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 462 has memoization hash 3eefa632ec9917b9b1ed19e03ae104ed parsl.dataflow.memoization INFO: Task 462 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 462 try 0 launched on executor multi with executor id 463 parsl.dataflow.dflow INFO: Standard output for task 462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/aecbb434-7315-4de4-8d48-b93d7fa61c64_isrForCrosstalkSources_997012_20.stdout parsl.dataflow.dflow INFO: Standard error for task 462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/aecbb434-7315-4de4-8d48-b93d7fa61c64_isrForCrosstalkSources_997012_20.stderr parsl.dataflow.dflow DEBUG: Task 463 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 463 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 463 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 463 has memoization hash 838789c700b7eab6c3cd6bef63ace59a parsl.dataflow.memoization INFO: Task 463 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 463 try 0 launched on executor multi with executor id 464 parsl.dataflow.dflow INFO: Standard output for task 463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/bb1c328a-fa0c-4335-9070-4e13408671e5_isrForCrosstalkSources_997019_28.stdout parsl.dataflow.dflow INFO: Standard error for task 463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/bb1c328a-fa0c-4335-9070-4e13408671e5_isrForCrosstalkSources_997019_28.stderr parsl.dataflow.dflow DEBUG: Task 464 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 464 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 464 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 464 has memoization hash 06fef5004bbce98db45cc9226144a6d5 parsl.dataflow.memoization INFO: Task 464 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 464 try 0 launched on executor multi with executor id 465 parsl.dataflow.dflow INFO: Standard output for task 464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/cbb77257-88c4-4740-b60f-3e59201d5191_isrForCrosstalkSources_996982_49.stdout parsl.dataflow.dflow INFO: Standard error for task 464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/cbb77257-88c4-4740-b60f-3e59201d5191_isrForCrosstalkSources_996982_49.stderr parsl.dataflow.dflow DEBUG: Task 465 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 465 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 465 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 465 has memoization hash 7d90c23785b5742bea3cc7beb0390487 parsl.dataflow.memoization INFO: Task 465 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 465 try 0 launched on executor multi with executor id 466 parsl.dataflow.dflow INFO: Standard output for task 465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c2e33e09-942d-4668-8c36-5717abf75860_isrForCrosstalkSources_997003_59.stdout parsl.dataflow.dflow INFO: Standard error for task 465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c2e33e09-942d-4668-8c36-5717abf75860_isrForCrosstalkSources_997003_59.stderr parsl.dataflow.dflow DEBUG: Task 466 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 466 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 466 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 466 has memoization hash 7dd001da6d4578030d91d2022f003a82 parsl.dataflow.memoization INFO: Task 466 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803225c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 466 try 0 launched on executor multi with executor id 467 parsl.dataflow.dflow INFO: Standard output for task 466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/05ab7ec8-9660-4f5e-85c0-8de773f410ef_isrForCrosstalkSources_996973_56.stdout parsl.dataflow.dflow INFO: Standard error for task 466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/05ab7ec8-9660-4f5e-85c0-8de773f410ef_isrForCrosstalkSources_996973_56.stderr parsl.dataflow.dflow DEBUG: Task 467 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 467 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 467 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 467 has memoization hash 96052534ddaaf17fbe8f979f790331ce parsl.dataflow.memoization INFO: Task 467 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 467 try 0 launched on executor multi with executor id 468 parsl.dataflow.dflow INFO: Standard output for task 467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f1b1afc3-c011-405d-97c8-e0bdc0b508c3_isrForCrosstalkSources_996989_9.stdout parsl.dataflow.dflow INFO: Standard error for task 467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f1b1afc3-c011-405d-97c8-e0bdc0b508c3_isrForCrosstalkSources_996989_9.stderr parsl.dataflow.dflow DEBUG: Task 468 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 468 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 468 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 468 has memoization hash a821a67eb847f65a60d0db172e157882 parsl.dataflow.memoization INFO: Task 468 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 468 try 0 launched on executor multi with executor id 469 parsl.dataflow.dflow INFO: Standard output for task 468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/92622fba-760c-4bf3-a059-99f6320aac34_isrForCrosstalkSources_996958_47.stdout parsl.dataflow.dflow INFO: Standard error for task 468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/92622fba-760c-4bf3-a059-99f6320aac34_isrForCrosstalkSources_996958_47.stderr parsl.dataflow.dflow DEBUG: Task 469 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 469 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 469 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 469 has memoization hash c9399cf39f34a9fe952c0c94d94e4fb6 parsl.dataflow.memoization INFO: Task 469 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 469 try 0 launched on executor multi with executor id 470 parsl.dataflow.dflow INFO: Standard output for task 469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/af5a69b8-069d-4174-aca1-916f54bb8025_isrForCrosstalkSources_996983_3.stdout parsl.dataflow.dflow INFO: Standard error for task 469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/af5a69b8-069d-4174-aca1-916f54bb8025_isrForCrosstalkSources_996983_3.stderr parsl.dataflow.dflow DEBUG: Task 470 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 470 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 470 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 470 has memoization hash d464d1e7d538b6b6a724f3ed9eaf03e7 parsl.dataflow.memoization INFO: Task 470 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803227a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 470 try 0 launched on executor multi with executor id 471 parsl.dataflow.dflow INFO: Standard output for task 470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/53b67780-93bc-46a0-b5f7-af7c83e456e4_isrForCrosstalkSources_997002_26.stdout parsl.dataflow.dflow INFO: Standard error for task 470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/53b67780-93bc-46a0-b5f7-af7c83e456e4_isrForCrosstalkSources_997002_26.stderr parsl.dataflow.dflow DEBUG: Task 471 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 471 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 471 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 471 has memoization hash 0a14d8e3f8534ec40f4a3fc56e30fb21 parsl.dataflow.memoization INFO: Task 471 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803228e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 471 try 0 launched on executor multi with executor id 472 parsl.dataflow.dflow INFO: Standard output for task 471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a464dc41-67ab-4047-bca1-54cbe3ed09b8_isrForCrosstalkSources_996972_41.stdout parsl.dataflow.dflow INFO: Standard error for task 471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a464dc41-67ab-4047-bca1-54cbe3ed09b8_isrForCrosstalkSources_996972_41.stderr parsl.dataflow.dflow DEBUG: Task 472 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 472 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 472 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 472 has memoization hash 7e2526c1445bc2be1e3ebfc342afd254 parsl.dataflow.memoization INFO: Task 472 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 472 try 0 launched on executor multi with executor id 473 parsl.dataflow.dflow INFO: Standard output for task 472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7bda589e-a469-499b-8439-e2ce651f3f2d_isrForCrosstalkSources_997013_7.stdout parsl.dataflow.dflow INFO: Standard error for task 472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7bda589e-a469-499b-8439-e2ce651f3f2d_isrForCrosstalkSources_997013_7.stderr parsl.dataflow.dflow DEBUG: Task 473 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 473 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 473 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 473 has memoization hash f4f30cd2a96f3492713edbf5d2e24e04 parsl.dataflow.memoization INFO: Task 473 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 473 try 0 launched on executor multi with executor id 474 parsl.dataflow.dflow INFO: Standard output for task 473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/c9c15df6-012b-44be-87b7-231f9ae7e60e_isrForCrosstalkSources_996987_10.stdout parsl.dataflow.dflow INFO: Standard error for task 473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/c9c15df6-012b-44be-87b7-231f9ae7e60e_isrForCrosstalkSources_996987_10.stderr parsl.dataflow.dflow DEBUG: Task 474 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 474 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 474 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 474 has memoization hash 2d8b69fc2441252a5ae7a8c057f55c45 parsl.dataflow.memoization INFO: Task 474 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 474 try 0 launched on executor multi with executor id 475 parsl.dataflow.dflow INFO: Standard output for task 474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/bb80437c-5930-4c0f-a10b-8ffdefba2b8c_isrForCrosstalkSources_997003_32.stdout parsl.dataflow.dflow INFO: Standard error for task 474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/bb80437c-5930-4c0f-a10b-8ffdefba2b8c_isrForCrosstalkSources_997003_32.stderr parsl.dataflow.dflow DEBUG: Task 475 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 475 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 475 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 475 has memoization hash 3ad34a98b99acbc09968189ddf88a766 parsl.dataflow.memoization INFO: Task 475 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 475 try 0 launched on executor multi with executor id 476 parsl.dataflow.dflow INFO: Standard output for task 475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f61b22b7-4cce-47f9-8bcf-da9d5d9ecc44_isrForCrosstalkSources_996985_59.stdout parsl.dataflow.dflow INFO: Standard error for task 475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f61b22b7-4cce-47f9-8bcf-da9d5d9ecc44_isrForCrosstalkSources_996985_59.stderr parsl.dataflow.dflow DEBUG: Task 476 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 476 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 476 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 476 has memoization hash ae7fdd7195f1beb4fb166ed9056f2f52 parsl.dataflow.memoization INFO: Task 476 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 476 try 0 launched on executor multi with executor id 477 parsl.dataflow.dflow INFO: Standard output for task 476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/44ff053c-9678-496b-bde5-6e541637c0e5_isrForCrosstalkSources_996984_34.stdout parsl.dataflow.dflow INFO: Standard error for task 476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/44ff053c-9678-496b-bde5-6e541637c0e5_isrForCrosstalkSources_996984_34.stderr parsl.dataflow.dflow DEBUG: Task 477 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 477 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 477 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 477 has memoization hash e4528ee1deb316d61855e1b799fde387 parsl.dataflow.memoization INFO: Task 477 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 477 try 0 launched on executor multi with executor id 478 parsl.dataflow.dflow INFO: Standard output for task 477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/0460b0c5-d728-4a72-9040-cd0752c6b8c2_isrForCrosstalkSources_997024_43.stdout parsl.dataflow.dflow INFO: Standard error for task 477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/0460b0c5-d728-4a72-9040-cd0752c6b8c2_isrForCrosstalkSources_997024_43.stderr parsl.dataflow.dflow DEBUG: Task 478 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 478 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 478 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 478 has memoization hash 324b8168d5cc3c61fb375a5ffd005291 parsl.dataflow.memoization INFO: Task 478 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 478 try 0 launched on executor multi with executor id 479 parsl.dataflow.dflow INFO: Standard output for task 478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5867e8fc-2655-4600-b665-ba9e0afe5c3d_isrForCrosstalkSources_996999_19.stdout parsl.dataflow.dflow INFO: Standard error for task 478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5867e8fc-2655-4600-b665-ba9e0afe5c3d_isrForCrosstalkSources_996999_19.stderr parsl.dataflow.dflow DEBUG: Task 479 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 479 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 479 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 479 has memoization hash 81ebf205e1f05265c2a2ba9738cb9b69 parsl.dataflow.memoization INFO: Task 479 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 479 try 0 launched on executor multi with executor id 480 parsl.dataflow.dflow INFO: Standard output for task 479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/24a783dc-0a93-4c8b-b630-fab9ee382804_isrForCrosstalkSources_996994_39.stdout parsl.dataflow.dflow INFO: Standard error for task 479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/24a783dc-0a93-4c8b-b630-fab9ee382804_isrForCrosstalkSources_996994_39.stderr parsl.dataflow.dflow DEBUG: Task 480 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 480 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 480 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 480 has memoization hash 734385025ac7ae0ab1fcc80ffe0920c8 parsl.dataflow.memoization INFO: Task 480 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 480 try 0 launched on executor multi with executor id 481 parsl.dataflow.dflow INFO: Standard output for task 480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/c9306167-ee3b-4b04-94d7-2b6f6b65b41a_isrForCrosstalkSources_997005_28.stdout parsl.dataflow.dflow INFO: Standard error for task 480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/c9306167-ee3b-4b04-94d7-2b6f6b65b41a_isrForCrosstalkSources_997005_28.stderr parsl.dataflow.dflow DEBUG: Task 481 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 481 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 481 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 481 has memoization hash d023ef54fcbdd578066cd919360cd9cc parsl.dataflow.memoization INFO: Task 481 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 481 try 0 launched on executor multi with executor id 482 parsl.dataflow.dflow INFO: Standard output for task 481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f901a167-79e9-41ad-b404-51b1d376fec8_isrForCrosstalkSources_996989_56.stdout parsl.dataflow.dflow INFO: Standard error for task 481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f901a167-79e9-41ad-b404-51b1d376fec8_isrForCrosstalkSources_996989_56.stderr parsl.dataflow.dflow DEBUG: Task 482 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 482 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 482 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 482 has memoization hash 971664c386c1e26a1b66e8145e6c2688 parsl.dataflow.memoization INFO: Task 482 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 482 try 0 launched on executor multi with executor id 483 parsl.dataflow.dflow INFO: Standard output for task 482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/3898d01d-c7a5-487e-9ab6-ad276719c87b_isrForCrosstalkSources_997024_41.stdout parsl.dataflow.dflow INFO: Standard error for task 482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/3898d01d-c7a5-487e-9ab6-ad276719c87b_isrForCrosstalkSources_997024_41.stderr parsl.dataflow.dflow DEBUG: Task 483 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 483 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 483 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 483 has memoization hash f19722b755bf71a3b65b9abcbbb10e04 parsl.dataflow.memoization INFO: Task 483 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780322f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 483 try 0 launched on executor multi with executor id 484 parsl.dataflow.dflow INFO: Standard output for task 483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/eb0863b4-f094-440c-b37c-08fd44d26be1_isrForCrosstalkSources_996980_21.stdout parsl.dataflow.dflow INFO: Standard error for task 483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/eb0863b4-f094-440c-b37c-08fd44d26be1_isrForCrosstalkSources_996980_21.stderr parsl.dataflow.dflow DEBUG: Task 484 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 484 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 484 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 484 has memoization hash 4a822a58ef95096bda6357518763146b parsl.dataflow.memoization INFO: Task 484 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803232e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 484 try 0 launched on executor multi with executor id 485 parsl.dataflow.dflow INFO: Standard output for task 484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/040c5088-c92c-4f97-bf35-0cb492250fcb_isrForCrosstalkSources_997011_54.stdout parsl.dataflow.dflow INFO: Standard error for task 484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/040c5088-c92c-4f97-bf35-0cb492250fcb_isrForCrosstalkSources_997011_54.stderr parsl.dataflow.dflow DEBUG: Task 485 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 485 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 485 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 485 has memoization hash a9c1a01ceeb78cfd45e0cbdbebee4dfe parsl.dataflow.memoization INFO: Task 485 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 485 try 0 launched on executor multi with executor id 486 parsl.dataflow.dflow INFO: Standard output for task 485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c8a82da2-c1bb-487d-bf21-c23768edc87d_isrForCrosstalkSources_997025_33.stdout parsl.dataflow.dflow INFO: Standard error for task 485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c8a82da2-c1bb-487d-bf21-c23768edc87d_isrForCrosstalkSources_997025_33.stderr parsl.dataflow.dflow DEBUG: Task 486 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 486 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 486 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 486 has memoization hash 5c96c94c80b7b2f164648dfe3d87f1a3 parsl.dataflow.memoization INFO: Task 486 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 486 try 0 launched on executor multi with executor id 487 parsl.dataflow.dflow INFO: Standard output for task 486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a55c83ea-b392-4bf1-80bc-3f7a8b322ced_isrForCrosstalkSources_996994_27.stdout parsl.dataflow.dflow INFO: Standard error for task 486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a55c83ea-b392-4bf1-80bc-3f7a8b322ced_isrForCrosstalkSources_996994_27.stderr parsl.dataflow.dflow DEBUG: Task 487 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 487 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 487 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 487 has memoization hash 73066490685a8787a6cc4f7790e909cb parsl.dataflow.memoization INFO: Task 487 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803231a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 487 try 0 launched on executor multi with executor id 488 parsl.dataflow.dflow INFO: Standard output for task 487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/7d12f4d0-e8b6-4e35-a081-becdfbe59ad7_isrForCrosstalkSources_996965_47.stdout parsl.dataflow.dflow INFO: Standard error for task 487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/7d12f4d0-e8b6-4e35-a081-becdfbe59ad7_isrForCrosstalkSources_996965_47.stderr parsl.dataflow.dflow DEBUG: Task 488 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 488 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 488 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 488 has memoization hash 8d4486059fa40ff57d74f1f11a1ef268 parsl.dataflow.memoization INFO: Task 488 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 488 try 0 launched on executor multi with executor id 489 parsl.dataflow.dflow INFO: Standard output for task 488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/9dd7ee75-e024-46a8-93e3-cea188335743_isrForCrosstalkSources_997028_30.stdout parsl.dataflow.dflow INFO: Standard error for task 488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/9dd7ee75-e024-46a8-93e3-cea188335743_isrForCrosstalkSources_997028_30.stderr parsl.dataflow.dflow DEBUG: Task 489 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 489 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 489 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 489 has memoization hash e729917e987463cba33889b18e293352 parsl.dataflow.memoization INFO: Task 489 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 489 try 0 launched on executor multi with executor id 490 parsl.dataflow.dflow INFO: Standard output for task 489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/78cf5ac9-dd04-44a5-8cc8-451b59596766_isrForCrosstalkSources_997015_37.stdout parsl.dataflow.dflow INFO: Standard error for task 489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/78cf5ac9-dd04-44a5-8cc8-451b59596766_isrForCrosstalkSources_997015_37.stderr parsl.dataflow.dflow DEBUG: Task 490 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 490 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 490 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 490 has memoization hash 70f3abf4e4f9e27078eafb45a6dd1679 parsl.dataflow.memoization INFO: Task 490 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803234c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 490 try 0 launched on executor multi with executor id 491 parsl.dataflow.dflow INFO: Standard output for task 490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/20d6a9eb-e9e6-4c14-8bd6-ca086e8e2c03_isrForCrosstalkSources_996955_49.stdout parsl.dataflow.dflow INFO: Standard error for task 490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/20d6a9eb-e9e6-4c14-8bd6-ca086e8e2c03_isrForCrosstalkSources_996955_49.stderr parsl.dataflow.dflow DEBUG: Task 491 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 491 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 491 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 491 has memoization hash b63fe5a6bb8d23798e65c910d3076cf5 parsl.dataflow.memoization INFO: Task 491 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 491 try 0 launched on executor multi with executor id 492 parsl.dataflow.dflow INFO: Standard output for task 491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/67a56488-d264-4e7c-a5aa-736b33d318b1_isrForCrosstalkSources_997011_62.stdout parsl.dataflow.dflow INFO: Standard error for task 491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/67a56488-d264-4e7c-a5aa-736b33d318b1_isrForCrosstalkSources_997011_62.stderr parsl.dataflow.dflow DEBUG: Task 492 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 492 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 492 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 492 has memoization hash 9a8de389ab704d5d2281548511bc3c0a parsl.dataflow.memoization INFO: Task 492 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 492 try 0 launched on executor multi with executor id 493 parsl.dataflow.dflow INFO: Standard output for task 492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/019b1046-a190-44bd-b865-73beebff66d2_isrForCrosstalkSources_996971_57.stdout parsl.dataflow.dflow INFO: Standard error for task 492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/019b1046-a190-44bd-b865-73beebff66d2_isrForCrosstalkSources_996971_57.stderr parsl.dataflow.dflow DEBUG: Task 493 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 493 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 493 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 493 has memoization hash 07a98a81ce644841114265aa662f8d1b parsl.dataflow.memoization INFO: Task 493 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 493 try 0 launched on executor multi with executor id 494 parsl.dataflow.dflow INFO: Standard output for task 493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/00695c9e-94b6-4f6e-b4da-1949cb179150_isrForCrosstalkSources_997018_33.stdout parsl.dataflow.dflow INFO: Standard error for task 493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/00695c9e-94b6-4f6e-b4da-1949cb179150_isrForCrosstalkSources_997018_33.stderr parsl.dataflow.dflow DEBUG: Task 494 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 494 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 494 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 494 has memoization hash 99f9a0e829af8d4c06fa8bd54f4454e5 parsl.dataflow.memoization INFO: Task 494 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 494 try 0 launched on executor multi with executor id 495 parsl.dataflow.dflow INFO: Standard output for task 494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/be56c2b4-4ffd-4167-8481-affb41ad8160_isrForCrosstalkSources_996990_48.stdout parsl.dataflow.dflow INFO: Standard error for task 494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/be56c2b4-4ffd-4167-8481-affb41ad8160_isrForCrosstalkSources_996990_48.stderr parsl.dataflow.dflow DEBUG: Task 495 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 495 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 495 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 495 has memoization hash a985184d2e7a6ea0ede46237d5efc85a parsl.dataflow.memoization INFO: Task 495 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 495 try 0 launched on executor multi with executor id 496 parsl.dataflow.dflow INFO: Standard output for task 495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d9d04fbf-fbfe-425f-bcfa-141d07faf9ac_isrForCrosstalkSources_996972_62.stdout parsl.dataflow.dflow INFO: Standard error for task 495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d9d04fbf-fbfe-425f-bcfa-141d07faf9ac_isrForCrosstalkSources_996972_62.stderr parsl.dataflow.dflow DEBUG: Task 496 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 496 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 496 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 496 has memoization hash d369be0734cf95ee8bd875616328aac8 parsl.dataflow.memoization INFO: Task 496 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 496 try 0 launched on executor multi with executor id 497 parsl.dataflow.dflow INFO: Standard output for task 496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/1355b93a-5850-428c-9912-c8cc273cf883_isrForCrosstalkSources_997029_47.stdout parsl.dataflow.dflow INFO: Standard error for task 496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/1355b93a-5850-428c-9912-c8cc273cf883_isrForCrosstalkSources_997029_47.stderr parsl.dataflow.dflow DEBUG: Task 497 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 497 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 497 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 497 has memoization hash f93e5de04383feec0068a2272c21a907 parsl.dataflow.memoization INFO: Task 497 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803239c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 497 try 0 launched on executor multi with executor id 498 parsl.dataflow.dflow INFO: Standard output for task 497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b021ec37-388f-407a-b296-2f4ec8c088ec_isrForCrosstalkSources_996969_50.stdout parsl.dataflow.dflow INFO: Standard error for task 497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b021ec37-388f-407a-b296-2f4ec8c088ec_isrForCrosstalkSources_996969_50.stderr parsl.dataflow.dflow DEBUG: Task 498 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 498 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 498 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 498 has memoization hash 86f9de6538c4cf3bb04f017da965cecf parsl.dataflow.memoization INFO: Task 498 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 498 try 0 launched on executor multi with executor id 499 parsl.dataflow.dflow INFO: Standard output for task 498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/dc5a53ce-b503-47a2-b017-0feb4e994fa3_isrForCrosstalkSources_996988_13.stdout parsl.dataflow.dflow INFO: Standard error for task 498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/dc5a53ce-b503-47a2-b017-0feb4e994fa3_isrForCrosstalkSources_996988_13.stderr parsl.dataflow.dflow DEBUG: Task 499 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 499 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 499 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 499 has memoization hash 75f2766c4d7c9ce55d23c3d083d16565 parsl.dataflow.memoization INFO: Task 499 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803237e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 499 try 0 launched on executor multi with executor id 500 parsl.dataflow.dflow INFO: Standard output for task 499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cb5ec149-45c4-4c3e-be3c-3b472a9e2acc_isrForCrosstalkSources_997015_39.stdout parsl.dataflow.dflow INFO: Standard error for task 499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cb5ec149-45c4-4c3e-be3c-3b472a9e2acc_isrForCrosstalkSources_997015_39.stderr parsl.dataflow.dflow DEBUG: Task 500 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 500 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 500 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 500 has memoization hash 56f8d018da60f10f758c9421a358173f parsl.dataflow.memoization INFO: Task 500 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 500 try 0 launched on executor multi with executor id 501 parsl.dataflow.dflow INFO: Standard output for task 500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/80ec2437-2afc-4cee-a284-433ec9b2a57b_isrForCrosstalkSources_996994_21.stdout parsl.dataflow.dflow INFO: Standard error for task 500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/80ec2437-2afc-4cee-a284-433ec9b2a57b_isrForCrosstalkSources_996994_21.stderr parsl.dataflow.dflow DEBUG: Task 501 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 501 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 501 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 501 has memoization hash e469d2ce1edc15cdad6103ddc67a1497 parsl.dataflow.memoization INFO: Task 501 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 501 try 0 launched on executor multi with executor id 502 parsl.dataflow.dflow INFO: Standard output for task 501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ba5cf8ea-755a-4805-b1eb-cedfc31f4cb9_isrForCrosstalkSources_996959_3.stdout parsl.dataflow.dflow INFO: Standard error for task 501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ba5cf8ea-755a-4805-b1eb-cedfc31f4cb9_isrForCrosstalkSources_996959_3.stderr parsl.dataflow.dflow DEBUG: Task 502 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 502 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 502 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 502 has memoization hash 20240316b674dd835ccfe205db5754b5 parsl.dataflow.memoization INFO: Task 502 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803236a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 502 try 0 launched on executor multi with executor id 503 parsl.dataflow.dflow INFO: Standard output for task 502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/662b3ce9-45fa-4333-8b9e-7d9e5da3f360_isrForCrosstalkSources_996978_47.stdout parsl.dataflow.dflow INFO: Standard error for task 502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/662b3ce9-45fa-4333-8b9e-7d9e5da3f360_isrForCrosstalkSources_996978_47.stderr parsl.dataflow.dflow DEBUG: Task 503 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 503 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 503 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 503 has memoization hash 6bfb97465ae568bb259395dc952b4367 parsl.dataflow.memoization INFO: Task 503 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780323560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 503 try 0 launched on executor multi with executor id 504 parsl.dataflow.dflow INFO: Standard output for task 503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/77eca3c1-8ace-46b8-baca-038255a25741_isrForCrosstalkSources_996979_60.stdout parsl.dataflow.dflow INFO: Standard error for task 503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/77eca3c1-8ace-46b8-baca-038255a25741_isrForCrosstalkSources_996979_60.stderr parsl.dataflow.dflow DEBUG: Task 504 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 504 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 504 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 504 has memoization hash 29f1928794b0a0b242bc1cb0f88f41f3 parsl.dataflow.memoization INFO: Task 504 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 504 try 0 launched on executor multi with executor id 505 parsl.dataflow.dflow INFO: Standard output for task 504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/3e036e08-c0dd-4e1e-99cc-c9baafda1c21_isrForCrosstalkSources_996966_5.stdout parsl.dataflow.dflow INFO: Standard error for task 504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/3e036e08-c0dd-4e1e-99cc-c9baafda1c21_isrForCrosstalkSources_996966_5.stderr parsl.dataflow.dflow DEBUG: Task 505 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 505 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 505 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 505 has memoization hash 3716f9279834d3def44496de333c3093 parsl.dataflow.memoization INFO: Task 505 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:27 proc_lsst.multi:170 [INFO] [multi] job_id 17074836 proc_lsst.multi INFO: [multi] job_id 17074836 2024-03-15 13:49:27 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 4 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 4 2024-03-15 13:49:27 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:27 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074836 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074836 parsl.executors.status_handling DEBUG: Launched block 4 on executor multi with job ID 17074836 parsl.executors.status_handling INFO: Allocated block ID 5 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-5 2024-03-15 13:49:27 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=5 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-5 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=5 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-5 2024-03-15 13:49:27 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=5 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=5 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:27 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=5 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=5 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 4 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 4 2024-03-15 13:49:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:27 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=5 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=5 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 505 try 0 launched on executor multi with executor id 506 parsl.dataflow.dflow INFO: Standard output for task 505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c66d5de3-28f8-4e10-9b65-24919f0c1c74_isrForCrosstalkSources_996993_60.stdout parsl.dataflow.dflow INFO: Standard error for task 505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c66d5de3-28f8-4e10-9b65-24919f0c1c74_isrForCrosstalkSources_996993_60.stderr parsl.dataflow.dflow DEBUG: Task 506 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 506 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 506 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 506 has memoization hash 2d1cf61b7668a020369404a042c4f24b parsl.dataflow.memoization INFO: Task 506 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f80e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 506 try 0 launched on executor multi with executor id 507 parsl.dataflow.dflow INFO: Standard output for task 506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/49c93bae-dc85-42af-aaf1-9fe6416d77a1_isrForCrosstalkSources_996997_62.stdout parsl.dataflow.dflow INFO: Standard error for task 506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/49c93bae-dc85-42af-aaf1-9fe6416d77a1_isrForCrosstalkSources_996997_62.stderr parsl.dataflow.dflow DEBUG: Task 507 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 507 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 507 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 507 has memoization hash c5662459870561f2d08985009901212b parsl.dataflow.memoization INFO: Task 507 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 507 try 0 launched on executor multi with executor id 508 parsl.dataflow.dflow INFO: Standard output for task 507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/462160e7-95a0-4a20-afdd-41f511e1bc1a_isrForCrosstalkSources_997023_61.stdout parsl.dataflow.dflow INFO: Standard error for task 507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/462160e7-95a0-4a20-afdd-41f511e1bc1a_isrForCrosstalkSources_997023_61.stderr parsl.dataflow.dflow DEBUG: Task 508 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 508 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 508 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 508 has memoization hash 0c8ec764f24b6bc852953f08ccb88c81 parsl.dataflow.memoization INFO: Task 508 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 508 try 0 launched on executor multi with executor id 509 parsl.dataflow.dflow INFO: Standard output for task 508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/4bf746d9-4fdb-46f8-8064-3001f965555f_isrForCrosstalkSources_996977_25.stdout parsl.dataflow.dflow INFO: Standard error for task 508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/4bf746d9-4fdb-46f8-8064-3001f965555f_isrForCrosstalkSources_996977_25.stderr parsl.dataflow.dflow DEBUG: Task 509 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 509 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 509 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 509 has memoization hash b7435ade850134c4952ce71c7b5304b0 parsl.dataflow.memoization INFO: Task 509 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 509 try 0 launched on executor multi with executor id 510 parsl.dataflow.dflow INFO: Standard output for task 509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/875dd3d6-02e5-4f82-a045-c5fa26781101_isrForCrosstalkSources_996965_59.stdout parsl.dataflow.dflow INFO: Standard error for task 509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/875dd3d6-02e5-4f82-a045-c5fa26781101_isrForCrosstalkSources_996965_59.stderr parsl.dataflow.dflow DEBUG: Task 510 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 510 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 510 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 510 has memoization hash 83f99fb640bf1b459549fa8f4cd89abe parsl.dataflow.memoization INFO: Task 510 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f84a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 510 try 0 launched on executor multi with executor id 511 parsl.dataflow.dflow INFO: Standard output for task 510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/d06d2740-de19-4c7a-860a-3d989a442933_isrForCrosstalkSources_997006_30.stdout parsl.dataflow.dflow INFO: Standard error for task 510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/d06d2740-de19-4c7a-860a-3d989a442933_isrForCrosstalkSources_997006_30.stderr parsl.dataflow.dflow DEBUG: Task 511 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 511 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 511 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 511 has memoization hash 0b79bf7de4d79e342c97014ac0a2f189 parsl.dataflow.memoization INFO: Task 511 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 511 try 0 launched on executor multi with executor id 512 parsl.dataflow.dflow INFO: Standard output for task 511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/073f2c7e-6fff-48ff-abca-f5b438dbc53c_isrForCrosstalkSources_996956_42.stdout parsl.dataflow.dflow INFO: Standard error for task 511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/073f2c7e-6fff-48ff-abca-f5b438dbc53c_isrForCrosstalkSources_996956_42.stderr parsl.dataflow.dflow DEBUG: Task 512 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 512 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 512 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 512 has memoization hash e3cda27b83b8ee9ff28d56ad70bb1cc4 parsl.dataflow.memoization INFO: Task 512 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f82c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 512 try 0 launched on executor multi with executor id 513 parsl.dataflow.dflow INFO: Standard output for task 512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/73801838-25ce-4e21-95fb-c38065359c16_isrForCrosstalkSources_997021_29.stdout parsl.dataflow.dflow INFO: Standard error for task 512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/73801838-25ce-4e21-95fb-c38065359c16_isrForCrosstalkSources_997021_29.stderr parsl.dataflow.dflow DEBUG: Task 513 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 513 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 513 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 513 has memoization hash a94a67289271124ae2a27ad7072acc30 parsl.dataflow.memoization INFO: Task 513 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 513 try 0 launched on executor multi with executor id 514 parsl.dataflow.dflow INFO: Standard output for task 513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/95c94cae-11d6-4a16-8509-61799c443c67_isrForCrosstalkSources_997024_19.stdout parsl.dataflow.dflow INFO: Standard error for task 513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/95c94cae-11d6-4a16-8509-61799c443c67_isrForCrosstalkSources_997024_19.stderr parsl.dataflow.dflow DEBUG: Task 514 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 514 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 514 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 514 has memoization hash 62b578f385c168edadd0de36772b4ff9 parsl.dataflow.memoization INFO: Task 514 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f85e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 514 try 0 launched on executor multi with executor id 515 parsl.dataflow.dflow INFO: Standard output for task 514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9657cfa1-17a6-4e10-b023-54df38e61c95_isrForCrosstalkSources_997000_12.stdout parsl.dataflow.dflow INFO: Standard error for task 514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9657cfa1-17a6-4e10-b023-54df38e61c95_isrForCrosstalkSources_997000_12.stderr parsl.dataflow.dflow DEBUG: Task 515 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 515 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 515 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 515 has memoization hash ed789d5570b00d5c951f64c0ce305b38 parsl.dataflow.memoization INFO: Task 515 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f87c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 515 try 0 launched on executor multi with executor id 516 parsl.dataflow.dflow INFO: Standard output for task 515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/74fc3d8b-f00e-4591-be65-e0c4a7b3e094_isrForCrosstalkSources_997004_6.stdout parsl.dataflow.dflow INFO: Standard error for task 515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/74fc3d8b-f00e-4591-be65-e0c4a7b3e094_isrForCrosstalkSources_997004_6.stderr parsl.dataflow.dflow DEBUG: Task 516 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 516 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 516 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 516 has memoization hash 6e4080f4cdd1deed6dc2f4a36ec6aa1a parsl.dataflow.memoization INFO: Task 516 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 516 try 0 launched on executor multi with executor id 517 parsl.dataflow.dflow INFO: Standard output for task 516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/bb71f467-6ca0-47cd-adb1-81419f974eab_isrForCrosstalkSources_997015_44.stdout parsl.dataflow.dflow INFO: Standard error for task 516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/bb71f467-6ca0-47cd-adb1-81419f974eab_isrForCrosstalkSources_997015_44.stderr parsl.dataflow.dflow DEBUG: Task 517 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 517 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 517 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 517 has memoization hash 849da78cd238eedbc1bc7ee4075c73e7 parsl.dataflow.memoization INFO: Task 517 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 517 try 0 launched on executor multi with executor id 518 parsl.dataflow.dflow INFO: Standard output for task 517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/390f7aaa-237e-49ed-912b-6113e7da3dd9_isrForCrosstalkSources_996992_23.stdout parsl.dataflow.dflow INFO: Standard error for task 517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/390f7aaa-237e-49ed-912b-6113e7da3dd9_isrForCrosstalkSources_996992_23.stderr parsl.dataflow.dflow DEBUG: Task 518 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 518 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 518 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 518 has memoization hash 91319e594445adde9aa69c46c16a98b9 parsl.dataflow.memoization INFO: Task 518 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 518 try 0 launched on executor multi with executor id 519 parsl.dataflow.dflow INFO: Standard output for task 518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/c550c26b-e42b-4a91-9244-ea8040753899_isrForCrosstalkSources_996996_42.stdout parsl.dataflow.dflow INFO: Standard error for task 518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/c550c26b-e42b-4a91-9244-ea8040753899_isrForCrosstalkSources_996996_42.stderr parsl.dataflow.dflow DEBUG: Task 519 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 519 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 519 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 519 has memoization hash e08af2aa882f28295cee1f558890af9a parsl.dataflow.memoization INFO: Task 519 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 519 try 0 launched on executor multi with executor id 520 parsl.dataflow.dflow INFO: Standard output for task 519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4d603963-953a-4cc7-a7e4-14dee912270b_isrForCrosstalkSources_996958_48.stdout parsl.dataflow.dflow INFO: Standard error for task 519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4d603963-953a-4cc7-a7e4-14dee912270b_isrForCrosstalkSources_996958_48.stderr parsl.dataflow.dflow DEBUG: Task 520 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 520 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 520 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 520 has memoization hash a23766e06c337f4438d2771518df665e parsl.dataflow.memoization INFO: Task 520 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 520 try 0 launched on executor multi with executor id 521 parsl.dataflow.dflow INFO: Standard output for task 520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e26b6584-787d-4f6e-9817-5bb88fb891af_isrForCrosstalkSources_996984_48.stdout parsl.dataflow.dflow INFO: Standard error for task 520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e26b6584-787d-4f6e-9817-5bb88fb891af_isrForCrosstalkSources_996984_48.stderr parsl.dataflow.dflow DEBUG: Task 521 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 521 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 521 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 521 has memoization hash 3cff5dd9059e740116f6f77f4afc3a70 parsl.dataflow.memoization INFO: Task 521 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f89a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 521 try 0 launched on executor multi with executor id 522 parsl.dataflow.dflow INFO: Standard output for task 521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e84e5307-afdf-4e41-a46b-fbe17bba9b1a_isrForCrosstalkSources_996954_25.stdout parsl.dataflow.dflow INFO: Standard error for task 521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e84e5307-afdf-4e41-a46b-fbe17bba9b1a_isrForCrosstalkSources_996954_25.stderr parsl.dataflow.dflow DEBUG: Task 522 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 522 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 522 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 522 has memoization hash 158cbe5dc2d26449b91302a2df3e634c parsl.dataflow.memoization INFO: Task 522 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 522 try 0 launched on executor multi with executor id 523 parsl.dataflow.dflow INFO: Standard output for task 522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/eb232e8c-9874-4faf-ab40-e36e240c8fa9_isrForCrosstalkSources_996971_19.stdout parsl.dataflow.dflow INFO: Standard error for task 522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/eb232e8c-9874-4faf-ab40-e36e240c8fa9_isrForCrosstalkSources_996971_19.stderr parsl.dataflow.dflow DEBUG: Task 523 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 523 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 523 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 523 has memoization hash 48b0e307dfee560a87e11984892d2a18 parsl.dataflow.memoization INFO: Task 523 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 523 try 0 launched on executor multi with executor id 524 parsl.dataflow.dflow INFO: Standard output for task 523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3be1543c-bd96-4a30-ab07-d388dc77bbdd_isrForCrosstalkSources_996963_28.stdout parsl.dataflow.dflow INFO: Standard error for task 523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3be1543c-bd96-4a30-ab07-d388dc77bbdd_isrForCrosstalkSources_996963_28.stderr parsl.dataflow.dflow DEBUG: Task 524 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 524 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 524 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 524 has memoization hash 89eb6e57d7d51a2ad8a58216860384df parsl.dataflow.memoization INFO: Task 524 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 524 try 0 launched on executor multi with executor id 525 parsl.dataflow.dflow INFO: Standard output for task 524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/432f9f79-1618-4f30-8516-3e6e863a93e3_isrForCrosstalkSources_996979_45.stdout parsl.dataflow.dflow INFO: Standard error for task 524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/432f9f79-1618-4f30-8516-3e6e863a93e3_isrForCrosstalkSources_996979_45.stderr parsl.dataflow.dflow DEBUG: Task 525 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 525 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 525 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 525 has memoization hash 99b24b48755eb3b8726882ed9e1e3d10 parsl.dataflow.memoization INFO: Task 525 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 525 try 0 launched on executor multi with executor id 526 parsl.dataflow.dflow INFO: Standard output for task 525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/35c68243-a1ca-4bdd-b7aa-f9f04ecc2e84_isrForCrosstalkSources_996988_45.stdout parsl.dataflow.dflow INFO: Standard error for task 525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/35c68243-a1ca-4bdd-b7aa-f9f04ecc2e84_isrForCrosstalkSources_996988_45.stderr parsl.dataflow.dflow DEBUG: Task 526 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 526 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 526 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 526 has memoization hash bd189571e5aca29b8eb9e35e4d314af5 parsl.dataflow.memoization INFO: Task 526 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 526 try 0 launched on executor multi with executor id 527 parsl.dataflow.dflow INFO: Standard output for task 526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ea180c96-c1e4-4722-838c-95be09a8d437_isrForCrosstalkSources_997029_19.stdout parsl.dataflow.dflow INFO: Standard error for task 526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ea180c96-c1e4-4722-838c-95be09a8d437_isrForCrosstalkSources_997029_19.stderr parsl.dataflow.dflow DEBUG: Task 527 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 527 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 527 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 527 has memoization hash a5e4dd19bfa538169c98a1235597753d parsl.dataflow.memoization INFO: Task 527 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 527 try 0 launched on executor multi with executor id 528 parsl.dataflow.dflow INFO: Standard output for task 527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/4d692560-61eb-4462-9c12-d049963fea68_isrForCrosstalkSources_996985_3.stdout parsl.dataflow.dflow INFO: Standard error for task 527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/4d692560-61eb-4462-9c12-d049963fea68_isrForCrosstalkSources_996985_3.stderr parsl.dataflow.dflow DEBUG: Task 528 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 528 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 528 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 528 has memoization hash ad7ab804fb39eaf82600c89abf0473c0 parsl.dataflow.memoization INFO: Task 528 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 528 try 0 launched on executor multi with executor id 529 parsl.dataflow.dflow INFO: Standard output for task 528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/979007e6-5ea3-4910-bdba-985158e344e6_isrForCrosstalkSources_997031_35.stdout parsl.dataflow.dflow INFO: Standard error for task 528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/979007e6-5ea3-4910-bdba-985158e344e6_isrForCrosstalkSources_997031_35.stderr parsl.dataflow.dflow DEBUG: Task 529 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 529 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 529 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 529 has memoization hash ea52587f2590c6ebfe57e994127a2133 parsl.dataflow.memoization INFO: Task 529 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 529 try 0 launched on executor multi with executor id 530 parsl.dataflow.dflow INFO: Standard output for task 529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/28cf44e2-9228-4699-99e5-d7e7c35758ba_isrForCrosstalkSources_996956_29.stdout parsl.dataflow.dflow INFO: Standard error for task 529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/28cf44e2-9228-4699-99e5-d7e7c35758ba_isrForCrosstalkSources_996956_29.stderr parsl.dataflow.dflow DEBUG: Task 530 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 530 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 530 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 530 has memoization hash 7ed85edc54c973c45bb16fb659c091e0 parsl.dataflow.memoization INFO: Task 530 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 530 try 0 launched on executor multi with executor id 531 parsl.dataflow.dflow INFO: Standard output for task 530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0c226aae-2ffc-4ea2-899d-73775c98187d_isrForCrosstalkSources_996997_14.stdout parsl.dataflow.dflow INFO: Standard error for task 530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0c226aae-2ffc-4ea2-899d-73775c98187d_isrForCrosstalkSources_996997_14.stderr parsl.dataflow.dflow DEBUG: Task 531 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 531 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 531 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 531 has memoization hash de5f6bb49c0320cb2e6eb2bef20ca90c parsl.dataflow.memoization INFO: Task 531 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 531 try 0 launched on executor multi with executor id 532 parsl.dataflow.dflow INFO: Standard output for task 531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7f2ca16a-c7d8-495e-ac5c-7cd019fb4285_isrForCrosstalkSources_997024_47.stdout parsl.dataflow.dflow INFO: Standard error for task 531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7f2ca16a-c7d8-495e-ac5c-7cd019fb4285_isrForCrosstalkSources_997024_47.stderr parsl.dataflow.dflow DEBUG: Task 532 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 532 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 532 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 532 has memoization hash bc6110aba7ed531d012375118ae456eb parsl.dataflow.memoization INFO: Task 532 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 532 try 0 launched on executor multi with executor id 533 parsl.dataflow.dflow INFO: Standard output for task 532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/21a99695-0273-4c89-a15c-69cda77fa959_isrForCrosstalkSources_997017_42.stdout parsl.dataflow.dflow INFO: Standard error for task 532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/21a99695-0273-4c89-a15c-69cda77fa959_isrForCrosstalkSources_997017_42.stderr parsl.dataflow.dflow DEBUG: Task 533 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 533 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 533 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 533 has memoization hash 22c45d77cec4c69e1a204fcc8d85c8ce parsl.dataflow.memoization INFO: Task 533 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f91c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 533 try 0 launched on executor multi with executor id 534 parsl.dataflow.dflow INFO: Standard output for task 533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d1a0c4bd-66c7-4357-b161-40cd7c2872d1_isrForCrosstalkSources_996964_61.stdout parsl.dataflow.dflow INFO: Standard error for task 533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d1a0c4bd-66c7-4357-b161-40cd7c2872d1_isrForCrosstalkSources_996964_61.stderr parsl.dataflow.dflow DEBUG: Task 534 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 534 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 534 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 534 has memoization hash 21b8f89995e21fd168095ea7910117ac parsl.dataflow.memoization INFO: Task 534 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 534 try 0 launched on executor multi with executor id 535 parsl.dataflow.dflow INFO: Standard output for task 534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/d5152f07-3fe1-4281-b795-0c073ff5f981_isrForCrosstalkSources_997014_38.stdout parsl.dataflow.dflow INFO: Standard error for task 534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/d5152f07-3fe1-4281-b795-0c073ff5f981_isrForCrosstalkSources_997014_38.stderr parsl.dataflow.dflow DEBUG: Task 535 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 535 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 535 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 535 has memoization hash 78e0026ad34b8177673c8ceadc2abe6a parsl.dataflow.memoization INFO: Task 535 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 535 try 0 launched on executor multi with executor id 536 parsl.dataflow.dflow INFO: Standard output for task 535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6c3670ab-b3b2-4b03-b781-a8f63f3cb58e_isrForCrosstalkSources_996952_60.stdout parsl.dataflow.dflow INFO: Standard error for task 535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6c3670ab-b3b2-4b03-b781-a8f63f3cb58e_isrForCrosstalkSources_996952_60.stderr parsl.dataflow.dflow DEBUG: Task 536 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 536 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 536 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 536 has memoization hash f5d6268c373881fbc6de1e05eb6a68fb parsl.dataflow.memoization INFO: Task 536 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 536 try 0 launched on executor multi with executor id 537 parsl.dataflow.dflow INFO: Standard output for task 536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bb561524-4366-4364-8315-387fa3a4f275_isrForCrosstalkSources_996959_20.stdout parsl.dataflow.dflow INFO: Standard error for task 536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bb561524-4366-4364-8315-387fa3a4f275_isrForCrosstalkSources_996959_20.stderr parsl.dataflow.dflow DEBUG: Task 537 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 537 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 537 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 537 has memoization hash 016d767a74271b21145ec804e490db46 parsl.dataflow.memoization INFO: Task 537 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f93a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 537 try 0 launched on executor multi with executor id 538 parsl.dataflow.dflow INFO: Standard output for task 537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d2235bd6-4b3a-4e8c-b602-fa092f96192e_isrForCrosstalkSources_997008_5.stdout parsl.dataflow.dflow INFO: Standard error for task 537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d2235bd6-4b3a-4e8c-b602-fa092f96192e_isrForCrosstalkSources_997008_5.stderr parsl.dataflow.dflow DEBUG: Task 538 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 538 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 538 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 538 has memoization hash 570d117b156384535e00ed761dc45c3a parsl.dataflow.memoization INFO: Task 538 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f94e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 538 try 0 launched on executor multi with executor id 539 parsl.dataflow.dflow INFO: Standard output for task 538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/93a69597-2253-465e-99f1-aa6fbc20dabe_isrForCrosstalkSources_996956_21.stdout parsl.dataflow.dflow INFO: Standard error for task 538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/93a69597-2253-465e-99f1-aa6fbc20dabe_isrForCrosstalkSources_996956_21.stderr parsl.dataflow.dflow DEBUG: Task 539 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 539 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 539 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 539 has memoization hash 31b8d4a1beaa7cc87fa5cab358585a72 parsl.dataflow.memoization INFO: Task 539 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 539 try 0 launched on executor multi with executor id 540 parsl.dataflow.dflow INFO: Standard output for task 539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e659f389-ad93-4fdf-be69-b02800800ead_isrForCrosstalkSources_996993_54.stdout parsl.dataflow.dflow INFO: Standard error for task 539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e659f389-ad93-4fdf-be69-b02800800ead_isrForCrosstalkSources_996993_54.stderr parsl.dataflow.dflow DEBUG: Task 540 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 540 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 540 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 540 has memoization hash da235e03d989673b8bcd1afdcefab589 parsl.dataflow.memoization INFO: Task 540 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 540 try 0 launched on executor multi with executor id 541 parsl.dataflow.dflow INFO: Standard output for task 540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/6c89c288-5942-452a-85fe-55749065e483_isrForCrosstalkSources_996968_33.stdout parsl.dataflow.dflow INFO: Standard error for task 540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/6c89c288-5942-452a-85fe-55749065e483_isrForCrosstalkSources_996968_33.stderr parsl.dataflow.dflow DEBUG: Task 541 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 541 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 541 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 541 has memoization hash 8b35e8f24f0c97d31fdd29c5e3c3f988 parsl.dataflow.memoization INFO: Task 541 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 541 try 0 launched on executor multi with executor id 542 parsl.dataflow.dflow INFO: Standard output for task 541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d6139823-10fc-4821-986e-a8abe1b1ccb1_isrForCrosstalkSources_997027_39.stdout parsl.dataflow.dflow INFO: Standard error for task 541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d6139823-10fc-4821-986e-a8abe1b1ccb1_isrForCrosstalkSources_997027_39.stderr parsl.dataflow.dflow DEBUG: Task 542 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 542 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 542 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 542 has memoization hash 7c002c633d818670615228a3c118a8a9 parsl.dataflow.memoization INFO: Task 542 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f96c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 542 try 0 launched on executor multi with executor id 543 parsl.dataflow.dflow INFO: Standard output for task 542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f8928018-f5c3-4436-bb34-3473ca9c6c01_isrForCrosstalkSources_996990_33.stdout parsl.dataflow.dflow INFO: Standard error for task 542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f8928018-f5c3-4436-bb34-3473ca9c6c01_isrForCrosstalkSources_996990_33.stderr parsl.dataflow.dflow DEBUG: Task 543 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 543 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 543 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 543 has memoization hash 65c69d101bfaf872c5199748fc02046f parsl.dataflow.memoization INFO: Task 543 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f98a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 543 try 0 launched on executor multi with executor id 544 parsl.dataflow.dflow INFO: Standard output for task 543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/58c021ce-2b10-43f4-97ab-911354956e40_isrForCrosstalkSources_997027_30.stdout parsl.dataflow.dflow INFO: Standard error for task 543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/58c021ce-2b10-43f4-97ab-911354956e40_isrForCrosstalkSources_997027_30.stderr parsl.dataflow.dflow DEBUG: Task 544 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 544 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 544 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 544 has memoization hash fd9c0bd9e0f79ea79c003b1d4abbdd09 parsl.dataflow.memoization INFO: Task 544 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 544 try 0 launched on executor multi with executor id 545 parsl.dataflow.dflow INFO: Standard output for task 544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/1abdd64a-6055-4838-8924-b7b944014c30_isrForCrosstalkSources_996955_48.stdout parsl.dataflow.dflow INFO: Standard error for task 544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/1abdd64a-6055-4838-8924-b7b944014c30_isrForCrosstalkSources_996955_48.stderr parsl.dataflow.dflow DEBUG: Task 545 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 545 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 545 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 545 has memoization hash a1adf9e94315929cf989d6264c1fad7b parsl.dataflow.memoization INFO: Task 545 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 545 try 0 launched on executor multi with executor id 546 parsl.dataflow.dflow INFO: Standard output for task 545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a6db1b68-de6f-4e28-b2a5-2a82f86c1c8d_isrForCrosstalkSources_997013_61.stdout parsl.dataflow.dflow INFO: Standard error for task 545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a6db1b68-de6f-4e28-b2a5-2a82f86c1c8d_isrForCrosstalkSources_997013_61.stderr parsl.dataflow.dflow DEBUG: Task 546 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 546 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 546 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 546 has memoization hash adbb97bd346ff9a4d725ebc764304a86 parsl.dataflow.memoization INFO: Task 546 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 546 try 0 launched on executor multi with executor id 547 parsl.dataflow.dflow INFO: Standard output for task 546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/2704a84f-08e3-4724-b0ae-a16c7a4d54cf_isrForCrosstalkSources_997023_29.stdout parsl.dataflow.dflow INFO: Standard error for task 546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/2704a84f-08e3-4724-b0ae-a16c7a4d54cf_isrForCrosstalkSources_997023_29.stderr parsl.dataflow.dflow DEBUG: Task 547 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 547 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 547 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 547 has memoization hash 4b2f31ee3480f5678c7f18d2ddf1b98b parsl.dataflow.memoization INFO: Task 547 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 547 try 0 launched on executor multi with executor id 548 parsl.dataflow.dflow INFO: Standard output for task 547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/9558db22-35dc-487d-ad84-17073c5e49ed_isrForCrosstalkSources_996989_12.stdout parsl.dataflow.dflow INFO: Standard error for task 547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/9558db22-35dc-487d-ad84-17073c5e49ed_isrForCrosstalkSources_996989_12.stderr parsl.dataflow.dflow DEBUG: Task 548 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 548 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 548 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 548 has memoization hash 9ebfaeb2ad0da464b25ee32eae85ba90 parsl.dataflow.memoization INFO: Task 548 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 548 try 0 launched on executor multi with executor id 549 parsl.dataflow.dflow INFO: Standard output for task 548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/282313ed-b4f0-4a12-b195-ae7c9d34bd61_isrForCrosstalkSources_996954_61.stdout parsl.dataflow.dflow INFO: Standard error for task 548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/282313ed-b4f0-4a12-b195-ae7c9d34bd61_isrForCrosstalkSources_996954_61.stderr parsl.dataflow.dflow DEBUG: Task 549 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 549 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 549 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 549 has memoization hash a4da8d7bb2db767c6861f87ee6b2a96a parsl.dataflow.memoization INFO: Task 549 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f99e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 549 try 0 launched on executor multi with executor id 550 parsl.dataflow.dflow INFO: Standard output for task 549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/380d4714-6a22-4189-b63a-91151a0601a7_isrForCrosstalkSources_997022_21.stdout parsl.dataflow.dflow INFO: Standard error for task 549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/380d4714-6a22-4189-b63a-91151a0601a7_isrForCrosstalkSources_997022_21.stderr parsl.dataflow.dflow DEBUG: Task 550 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 550 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 550 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 550 has memoization hash 89cc250c351e3f5e0ea77a0d702fba88 parsl.dataflow.memoization INFO: Task 550 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 550 try 0 launched on executor multi with executor id 551 parsl.dataflow.dflow INFO: Standard output for task 550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/30291cbf-6b74-4c61-a978-a3857fb12aef_isrForCrosstalkSources_997028_43.stdout parsl.dataflow.dflow INFO: Standard error for task 550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/30291cbf-6b74-4c61-a978-a3857fb12aef_isrForCrosstalkSources_997028_43.stderr parsl.dataflow.dflow DEBUG: Task 551 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 551 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 551 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 551 has memoization hash cb88391a6f1e4fd261d41da5e66f287d parsl.dataflow.memoization INFO: Task 551 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:27 proc_lsst.multi:170 [INFO] [multi] job_id 17074837 proc_lsst.multi INFO: [multi] job_id 17074837 2024-03-15 13:49:27 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:49:27 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:27 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074837 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074837 parsl.executors.status_handling DEBUG: Launched block 5 on executor multi with job ID 17074837 parsl.executors.status_handling INFO: Allocated block ID 6 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-6 2024-03-15 13:49:27 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=6 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-6 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=6 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-6 2024-03-15 13:49:27 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=6 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=6 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:27 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=6 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=6 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:49:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:27 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=6 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=6 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 551 try 0 launched on executor multi with executor id 552 parsl.dataflow.dflow INFO: Standard output for task 551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/746fe48c-80d8-4c9e-b4d1-6221668928f8_isrForCrosstalkSources_996966_22.stdout parsl.dataflow.dflow INFO: Standard error for task 551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/746fe48c-80d8-4c9e-b4d1-6221668928f8_isrForCrosstalkSources_996966_22.stderr parsl.dataflow.dflow DEBUG: Task 552 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 552 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 552 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 552 has memoization hash 6a727b1a3e96a9bf504b88e80ecb88ee parsl.dataflow.memoization INFO: Task 552 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 552 try 0 launched on executor multi with executor id 553 parsl.dataflow.dflow INFO: Standard output for task 552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a65c2fa8-8ec6-43ff-8b09-8f79991fedec_isrForCrosstalkSources_997002_2.stdout parsl.dataflow.dflow INFO: Standard error for task 552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a65c2fa8-8ec6-43ff-8b09-8f79991fedec_isrForCrosstalkSources_997002_2.stderr parsl.dataflow.dflow DEBUG: Task 553 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 553 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 553 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 553 has memoization hash 7fb6890947e0cebe342ddaac6864b2db parsl.dataflow.memoization INFO: Task 553 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 553 try 0 launched on executor multi with executor id 554 parsl.dataflow.dflow INFO: Standard output for task 553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/d089db6c-dd9b-4968-a681-7de617ae429e_isrForCrosstalkSources_997019_14.stdout parsl.dataflow.dflow INFO: Standard error for task 553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/d089db6c-dd9b-4968-a681-7de617ae429e_isrForCrosstalkSources_997019_14.stderr parsl.dataflow.dflow DEBUG: Task 554 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 554 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 554 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 554 has memoization hash 735fb01643a17504f779b229654b7f26 parsl.dataflow.memoization INFO: Task 554 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 554 try 0 launched on executor multi with executor id 555 parsl.dataflow.dflow INFO: Standard output for task 554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/7bfb47b0-e610-428d-a0ab-492ed70d798d_isrForCrosstalkSources_997002_54.stdout parsl.dataflow.dflow INFO: Standard error for task 554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/7bfb47b0-e610-428d-a0ab-492ed70d798d_isrForCrosstalkSources_997002_54.stderr parsl.dataflow.dflow DEBUG: Task 555 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 555 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 555 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 555 has memoization hash 04924206670c83a03d2ed26542fcd58f parsl.dataflow.memoization INFO: Task 555 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f9f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 555 try 0 launched on executor multi with executor id 556 parsl.dataflow.dflow INFO: Standard output for task 555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2009a79d-3d6f-470a-af2e-96a1374155be_isrForCrosstalkSources_996995_5.stdout parsl.dataflow.dflow INFO: Standard error for task 555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2009a79d-3d6f-470a-af2e-96a1374155be_isrForCrosstalkSources_996995_5.stderr parsl.dataflow.dflow DEBUG: Task 556 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 556 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 556 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 556 has memoization hash 4a9eb76452c4027d026b466859670598 parsl.dataflow.memoization INFO: Task 556 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fbec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 556 try 0 launched on executor multi with executor id 557 parsl.dataflow.dflow INFO: Standard output for task 556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/da55b1d4-4362-4571-b113-b884537fe25f_isrForCrosstalkSources_996958_57.stdout parsl.dataflow.dflow INFO: Standard error for task 556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/da55b1d4-4362-4571-b113-b884537fe25f_isrForCrosstalkSources_996958_57.stderr parsl.dataflow.dflow DEBUG: Task 557 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 557 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 557 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 557 has memoization hash 3056959c6fbf3fbe131f68813401b26e parsl.dataflow.memoization INFO: Task 557 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 557 try 0 launched on executor multi with executor id 558 parsl.dataflow.dflow INFO: Standard output for task 557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/f5f3b95b-ff68-48a8-b06c-5190b61848a1_isrForCrosstalkSources_996968_51.stdout parsl.dataflow.dflow INFO: Standard error for task 557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/f5f3b95b-ff68-48a8-b06c-5190b61848a1_isrForCrosstalkSources_996968_51.stderr parsl.dataflow.dflow DEBUG: Task 558 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 558 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 558 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 558 has memoization hash 680881e13b18637044a07b3be826bb89 parsl.dataflow.memoization INFO: Task 558 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 558 try 0 launched on executor multi with executor id 559 parsl.dataflow.dflow INFO: Standard output for task 558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/4821582d-f328-49d5-9ca7-925ebb9d3f68_isrForCrosstalkSources_997012_36.stdout parsl.dataflow.dflow INFO: Standard error for task 558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/4821582d-f328-49d5-9ca7-925ebb9d3f68_isrForCrosstalkSources_997012_36.stderr parsl.dataflow.dflow DEBUG: Task 559 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 559 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 559 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 559 has memoization hash 8095446c3af33ef588f31d8d1bcd4ffc parsl.dataflow.memoization INFO: Task 559 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 559 try 0 launched on executor multi with executor id 560 parsl.dataflow.dflow INFO: Standard output for task 559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/91cff648-37fc-424f-9940-97e44c107b86_isrForCrosstalkSources_997020_3.stdout parsl.dataflow.dflow INFO: Standard error for task 559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/91cff648-37fc-424f-9940-97e44c107b86_isrForCrosstalkSources_997020_3.stderr parsl.dataflow.dflow DEBUG: Task 560 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 560 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 560 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 560 has memoization hash d72044236427c5cc4ff95648635f5d95 parsl.dataflow.memoization INFO: Task 560 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803f8040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 560 try 0 launched on executor multi with executor id 561 parsl.dataflow.dflow INFO: Standard output for task 560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/3070675a-7c38-4fd7-bdd0-aff4ab9a8f60_isrForCrosstalkSources_997017_54.stdout parsl.dataflow.dflow INFO: Standard error for task 560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/3070675a-7c38-4fd7-bdd0-aff4ab9a8f60_isrForCrosstalkSources_997017_54.stderr parsl.dataflow.dflow DEBUG: Task 561 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 561 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 561 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 561 has memoization hash 0f2e3c11fafc27a0b73e399097ca17c4 parsl.dataflow.memoization INFO: Task 561 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 561 try 0 launched on executor multi with executor id 562 parsl.dataflow.dflow INFO: Standard output for task 561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/0f9d5b2c-b222-4d1d-9594-94b03b33a53b_isrForCrosstalkSources_996963_61.stdout parsl.dataflow.dflow INFO: Standard error for task 561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/0f9d5b2c-b222-4d1d-9594-94b03b33a53b_isrForCrosstalkSources_996963_61.stderr parsl.dataflow.dflow DEBUG: Task 562 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 562 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 562 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 562 has memoization hash b39ea0adee7a8f5f81dcbff1ac33365b parsl.dataflow.memoization INFO: Task 562 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 562 try 0 launched on executor multi with executor id 563 parsl.dataflow.dflow INFO: Standard output for task 562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/4241c2a3-8620-4e93-963c-627ce8a49987_isrForCrosstalkSources_997007_17.stdout parsl.dataflow.dflow INFO: Standard error for task 562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/4241c2a3-8620-4e93-963c-627ce8a49987_isrForCrosstalkSources_997007_17.stderr parsl.dataflow.dflow DEBUG: Task 563 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 563 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 563 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 563 has memoization hash dfada8d95b91d8337c863e68319ca7e2 parsl.dataflow.memoization INFO: Task 563 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 563 try 0 launched on executor multi with executor id 564 parsl.dataflow.dflow INFO: Standard output for task 563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/92166d78-b5ad-4fd0-bfcb-17a4c7ac0f8b_isrForCrosstalkSources_996986_42.stdout parsl.dataflow.dflow INFO: Standard error for task 563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/92166d78-b5ad-4fd0-bfcb-17a4c7ac0f8b_isrForCrosstalkSources_996986_42.stderr parsl.dataflow.dflow DEBUG: Task 564 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 564 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 564 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 564 has memoization hash 81daa58bd08baac5409db71625cc9c71 parsl.dataflow.memoization INFO: Task 564 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 564 try 0 launched on executor multi with executor id 565 parsl.dataflow.dflow INFO: Standard output for task 564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/7b4994e1-a2ff-4dfd-adf1-f979159e9e29_isrForCrosstalkSources_997018_16.stdout parsl.dataflow.dflow INFO: Standard error for task 564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/7b4994e1-a2ff-4dfd-adf1-f979159e9e29_isrForCrosstalkSources_997018_16.stderr parsl.dataflow.dflow DEBUG: Task 565 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 565 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 565 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 565 has memoization hash c937a785ce07f09dfcce459375727118 parsl.dataflow.memoization INFO: Task 565 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 565 try 0 launched on executor multi with executor id 566 parsl.dataflow.dflow INFO: Standard output for task 565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/81ee3942-0235-4656-891b-c3fbb18f5b68_isrForCrosstalkSources_996994_30.stdout parsl.dataflow.dflow INFO: Standard error for task 565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/81ee3942-0235-4656-891b-c3fbb18f5b68_isrForCrosstalkSources_996994_30.stderr parsl.dataflow.dflow DEBUG: Task 566 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 566 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 566 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 566 has memoization hash 5ba2d1aa6210d80d1e91c3166b707a85 parsl.dataflow.memoization INFO: Task 566 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 566 try 0 launched on executor multi with executor id 567 parsl.dataflow.dflow INFO: Standard output for task 566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/b71cd5eb-4f49-4be0-b464-bc1409da2f10_isrForCrosstalkSources_996961_52.stdout parsl.dataflow.dflow INFO: Standard error for task 566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/b71cd5eb-4f49-4be0-b464-bc1409da2f10_isrForCrosstalkSources_996961_52.stderr parsl.dataflow.dflow DEBUG: Task 567 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 567 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 567 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 567 has memoization hash 64ff9bf7aa2670b0353a8dc007d530d0 parsl.dataflow.memoization INFO: Task 567 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 567 try 0 launched on executor multi with executor id 568 parsl.dataflow.dflow INFO: Standard output for task 567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/21942f4b-d6a3-4257-b5b3-cd4efa287b85_isrForCrosstalkSources_997013_13.stdout parsl.dataflow.dflow INFO: Standard error for task 567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/21942f4b-d6a3-4257-b5b3-cd4efa287b85_isrForCrosstalkSources_997013_13.stderr parsl.dataflow.dflow DEBUG: Task 568 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 568 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 568 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 568 has memoization hash 2938d475be02f825d3bc70e146979c81 parsl.dataflow.memoization INFO: Task 568 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803faf20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 568 try 0 launched on executor multi with executor id 569 parsl.dataflow.dflow INFO: Standard output for task 568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/d4220be6-50b2-44d4-bce0-f98f1feee772_isrForCrosstalkSources_997002_38.stdout parsl.dataflow.dflow INFO: Standard error for task 568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/d4220be6-50b2-44d4-bce0-f98f1feee772_isrForCrosstalkSources_997002_38.stderr parsl.dataflow.dflow DEBUG: Task 569 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 569 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 569 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 569 has memoization hash 04e5c98defd8788fb3429c4461511f45 parsl.dataflow.memoization INFO: Task 569 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 569 try 0 launched on executor multi with executor id 570 parsl.dataflow.dflow INFO: Standard output for task 569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/853627a3-3d21-4cc6-a688-3bfd19ada7f9_isrForCrosstalkSources_996989_5.stdout parsl.dataflow.dflow INFO: Standard error for task 569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/853627a3-3d21-4cc6-a688-3bfd19ada7f9_isrForCrosstalkSources_996989_5.stderr parsl.dataflow.dflow DEBUG: Task 570 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 570 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 570 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 570 has memoization hash 00f30cfaafcf942dfe2efec5193c92e4 parsl.dataflow.memoization INFO: Task 570 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 570 try 0 launched on executor multi with executor id 571 parsl.dataflow.dflow INFO: Standard output for task 570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/d925ace7-2817-4005-859c-25b81e1a0e03_isrForCrosstalkSources_996969_52.stdout parsl.dataflow.dflow INFO: Standard error for task 570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/d925ace7-2817-4005-859c-25b81e1a0e03_isrForCrosstalkSources_996969_52.stderr parsl.dataflow.dflow DEBUG: Task 571 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 571 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 571 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 571 has memoization hash a16c046599e09a3cfbbc508c3eba3955 parsl.dataflow.memoization INFO: Task 571 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fafc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 571 try 0 launched on executor multi with executor id 572 parsl.dataflow.dflow INFO: Standard output for task 571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/195b6d9b-d264-4421-82bb-b3b91b49ed53_isrForCrosstalkSources_997026_53.stdout parsl.dataflow.dflow INFO: Standard error for task 571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/195b6d9b-d264-4421-82bb-b3b91b49ed53_isrForCrosstalkSources_997026_53.stderr parsl.dataflow.dflow DEBUG: Task 572 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 572 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 572 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 572 has memoization hash 91307d14425539429d9faff2fd6af21b parsl.dataflow.memoization INFO: Task 572 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 572 try 0 launched on executor multi with executor id 573 parsl.dataflow.dflow INFO: Standard output for task 572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/4fcc4eed-9826-4809-bcc2-5be73e0bce07_isrForCrosstalkSources_997000_6.stdout parsl.dataflow.dflow INFO: Standard error for task 572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/4fcc4eed-9826-4809-bcc2-5be73e0bce07_isrForCrosstalkSources_997000_6.stderr parsl.dataflow.dflow DEBUG: Task 573 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 573 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 573 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 573 has memoization hash e8abee0aeede9d5d7958063e3b772bdd parsl.dataflow.memoization INFO: Task 573 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803faca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 573 try 0 launched on executor multi with executor id 574 parsl.dataflow.dflow INFO: Standard output for task 573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/deb15880-50d2-4d8f-8add-936bfa8b21c2_isrForCrosstalkSources_996993_40.stdout parsl.dataflow.dflow INFO: Standard error for task 573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/deb15880-50d2-4d8f-8add-936bfa8b21c2_isrForCrosstalkSources_996993_40.stderr parsl.dataflow.dflow DEBUG: Task 574 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 574 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 574 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 574 has memoization hash a801ed85672d51987bd1efaa53f47378 parsl.dataflow.memoization INFO: Task 574 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 574 try 0 launched on executor multi with executor id 575 parsl.dataflow.dflow INFO: Standard output for task 574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/aee1c00b-2ab0-4ac9-8271-b303f2cf3803_isrForCrosstalkSources_997029_5.stdout parsl.dataflow.dflow INFO: Standard error for task 574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/aee1c00b-2ab0-4ac9-8271-b303f2cf3803_isrForCrosstalkSources_997029_5.stderr parsl.dataflow.dflow DEBUG: Task 575 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 575 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 575 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 575 has memoization hash 475239db976f6496db82d11417350270 parsl.dataflow.memoization INFO: Task 575 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803faa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 575 try 0 launched on executor multi with executor id 576 parsl.dataflow.dflow INFO: Standard output for task 575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/355bb696-9c28-4160-bd86-1eb2fd2f036b_isrForCrosstalkSources_996984_31.stdout parsl.dataflow.dflow INFO: Standard error for task 575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/355bb696-9c28-4160-bd86-1eb2fd2f036b_isrForCrosstalkSources_996984_31.stderr parsl.dataflow.dflow DEBUG: Task 576 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 576 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 576 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 576 has memoization hash 25f49186da55eb63e091b35ea509d4c4 parsl.dataflow.memoization INFO: Task 576 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803faac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 576 try 0 launched on executor multi with executor id 577 parsl.dataflow.dflow INFO: Standard output for task 576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7ae1b603-6e65-45e4-b2b9-b88c694cdf3a_isrForCrosstalkSources_997024_24.stdout parsl.dataflow.dflow INFO: Standard error for task 576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7ae1b603-6e65-45e4-b2b9-b88c694cdf3a_isrForCrosstalkSources_997024_24.stderr parsl.dataflow.dflow DEBUG: Task 577 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 577 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 577 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 577 has memoization hash 62e72055d7ef8d41fd6c5dd8a119cedd parsl.dataflow.memoization INFO: Task 577 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 577 try 0 launched on executor multi with executor id 578 parsl.dataflow.dflow INFO: Standard output for task 577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b9b6d412-ad60-4a35-8501-006f9f013ca6_isrForCrosstalkSources_997015_57.stdout parsl.dataflow.dflow INFO: Standard error for task 577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b9b6d412-ad60-4a35-8501-006f9f013ca6_isrForCrosstalkSources_997015_57.stderr parsl.dataflow.dflow DEBUG: Task 578 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 578 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 578 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 578 has memoization hash 123eacf08d68b8da6b362d5814d55cfc parsl.dataflow.memoization INFO: Task 578 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 578 try 0 launched on executor multi with executor id 579 parsl.dataflow.dflow INFO: Standard output for task 578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5af276a1-5cf7-474a-aada-a83787325421_isrForCrosstalkSources_996968_7.stdout parsl.dataflow.dflow INFO: Standard error for task 578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5af276a1-5cf7-474a-aada-a83787325421_isrForCrosstalkSources_996968_7.stderr parsl.dataflow.dflow DEBUG: Task 579 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 579 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 579 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 579 has memoization hash e1b0bd51ffd3620c3211c78e77878ff8 parsl.dataflow.memoization INFO: Task 579 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 579 try 0 launched on executor multi with executor id 580 parsl.dataflow.dflow INFO: Standard output for task 579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a824a98d-a972-4d01-a618-7b65db98a65b_isrForCrosstalkSources_997015_12.stdout parsl.dataflow.dflow INFO: Standard error for task 579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a824a98d-a972-4d01-a618-7b65db98a65b_isrForCrosstalkSources_997015_12.stderr parsl.dataflow.dflow DEBUG: Task 580 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 580 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 580 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 580 has memoization hash 543142a9a78de75897970847ff89f162 parsl.dataflow.memoization INFO: Task 580 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 580 try 0 launched on executor multi with executor id 581 parsl.dataflow.dflow INFO: Standard output for task 580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f1df6ac4-3db4-4b50-ac40-28696bccb3d9_isrForCrosstalkSources_996956_19.stdout parsl.dataflow.dflow INFO: Standard error for task 580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f1df6ac4-3db4-4b50-ac40-28696bccb3d9_isrForCrosstalkSources_996956_19.stderr parsl.dataflow.dflow DEBUG: Task 581 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 581 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 581 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 581 has memoization hash 8ca26f5e9f706871d2abe995ee19242e parsl.dataflow.memoization INFO: Task 581 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 581 try 0 launched on executor multi with executor id 582 parsl.dataflow.dflow INFO: Standard output for task 581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f735536d-e876-4960-af6e-8b5eb448c36b_isrForCrosstalkSources_996961_3.stdout parsl.dataflow.dflow INFO: Standard error for task 581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f735536d-e876-4960-af6e-8b5eb448c36b_isrForCrosstalkSources_996961_3.stderr parsl.dataflow.dflow DEBUG: Task 582 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 582 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 582 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 582 has memoization hash 923e5e57b3675fa0cdf5b333c8908832 parsl.dataflow.memoization INFO: Task 582 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 582 try 0 launched on executor multi with executor id 583 parsl.dataflow.dflow INFO: Standard output for task 582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4660ce9b-d261-4a87-a033-08f9977a3a67_isrForCrosstalkSources_996972_5.stdout parsl.dataflow.dflow INFO: Standard error for task 582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4660ce9b-d261-4a87-a033-08f9977a3a67_isrForCrosstalkSources_996972_5.stderr parsl.dataflow.dflow DEBUG: Task 583 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 583 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 583 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 583 has memoization hash 3805655fdac5a676513b91febfe2aa62 parsl.dataflow.memoization INFO: Task 583 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 583 try 0 launched on executor multi with executor id 584 parsl.dataflow.dflow INFO: Standard output for task 583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/33b59da4-8b09-4d6c-a8af-4bf3ae4b2351_isrForCrosstalkSources_997003_60.stdout parsl.dataflow.dflow INFO: Standard error for task 583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/33b59da4-8b09-4d6c-a8af-4bf3ae4b2351_isrForCrosstalkSources_997003_60.stderr parsl.dataflow.dflow DEBUG: Task 584 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 584 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 584 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 584 has memoization hash d7c8f2c95e125c48966764f4700a8d74 parsl.dataflow.memoization INFO: Task 584 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 584 try 0 launched on executor multi with executor id 585 parsl.dataflow.dflow INFO: Standard output for task 584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/82d9b2e5-6d66-47da-823b-b6aa7769cfd0_isrForCrosstalkSources_997009_25.stdout parsl.dataflow.dflow INFO: Standard error for task 584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/82d9b2e5-6d66-47da-823b-b6aa7769cfd0_isrForCrosstalkSources_997009_25.stderr parsl.dataflow.dflow DEBUG: Task 585 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 585 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 585 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 585 has memoization hash e618f0c1a7a589b4099494b7c5135d8e parsl.dataflow.memoization INFO: Task 585 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 585 try 0 launched on executor multi with executor id 586 parsl.dataflow.dflow INFO: Standard output for task 585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b8babf2c-b147-4532-82c3-8742c4886503_isrForCrosstalkSources_997025_34.stdout parsl.dataflow.dflow INFO: Standard error for task 585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b8babf2c-b147-4532-82c3-8742c4886503_isrForCrosstalkSources_997025_34.stderr parsl.dataflow.dflow DEBUG: Task 586 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 586 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 586 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 586 has memoization hash 9112904ffa1bac6458e7b243bbaf06d7 parsl.dataflow.memoization INFO: Task 586 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 586 try 0 launched on executor multi with executor id 587 parsl.dataflow.dflow INFO: Standard output for task 586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0ea418c6-a986-47ef-bf26-b5274a164f48_isrForCrosstalkSources_997021_43.stdout parsl.dataflow.dflow INFO: Standard error for task 586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0ea418c6-a986-47ef-bf26-b5274a164f48_isrForCrosstalkSources_997021_43.stderr parsl.dataflow.dflow DEBUG: Task 587 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 587 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 587 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 587 has memoization hash c3a9a15e9ecbb4530a3ca82f4030bc73 parsl.dataflow.memoization INFO: Task 587 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 587 try 0 launched on executor multi with executor id 588 parsl.dataflow.dflow INFO: Standard output for task 587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/287f4451-1425-48c6-8c4d-31a598ea488e_isrForCrosstalkSources_997005_44.stdout parsl.dataflow.dflow INFO: Standard error for task 587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/287f4451-1425-48c6-8c4d-31a598ea488e_isrForCrosstalkSources_997005_44.stderr parsl.dataflow.dflow DEBUG: Task 588 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 588 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 588 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 588 has memoization hash 36a22970544b08bb4939e85a0a905b29 parsl.dataflow.memoization INFO: Task 588 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 588 try 0 launched on executor multi with executor id 589 parsl.dataflow.dflow INFO: Standard output for task 588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/64138df4-d433-45b3-9c28-6da31e4ee11b_isrForCrosstalkSources_996970_30.stdout parsl.dataflow.dflow INFO: Standard error for task 588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/64138df4-d433-45b3-9c28-6da31e4ee11b_isrForCrosstalkSources_996970_30.stderr parsl.dataflow.dflow DEBUG: Task 589 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 589 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 589 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 589 has memoization hash 22ae72f1fb8de00c38e7dab94108a152 parsl.dataflow.memoization INFO: Task 589 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 589 try 0 launched on executor multi with executor id 590 parsl.dataflow.dflow INFO: Standard output for task 589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/068c6bb9-bc64-4ed3-9608-a0891c521836_isrForCrosstalkSources_997013_22.stdout parsl.dataflow.dflow INFO: Standard error for task 589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/068c6bb9-bc64-4ed3-9608-a0891c521836_isrForCrosstalkSources_997013_22.stderr parsl.dataflow.dflow DEBUG: Task 590 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 590 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 590 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 590 has memoization hash 2e593ab4fb9dab4352efabd8072073b9 parsl.dataflow.memoization INFO: Task 590 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 590 try 0 launched on executor multi with executor id 591 parsl.dataflow.dflow INFO: Standard output for task 590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d0130a21-235d-42f0-b2c9-8fa99782771c_isrForCrosstalkSources_996958_46.stdout parsl.dataflow.dflow INFO: Standard error for task 590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d0130a21-235d-42f0-b2c9-8fa99782771c_isrForCrosstalkSources_996958_46.stderr parsl.dataflow.dflow DEBUG: Task 591 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 591 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 591 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 591 has memoization hash 7910cafe5c3f2e1ed48048aed9667f8a parsl.dataflow.memoization INFO: Task 591 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 591 try 0 launched on executor multi with executor id 592 parsl.dataflow.dflow INFO: Standard output for task 591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/764db181-1bac-4bde-a5da-fae6523d2fc9_isrForCrosstalkSources_997014_54.stdout parsl.dataflow.dflow INFO: Standard error for task 591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/764db181-1bac-4bde-a5da-fae6523d2fc9_isrForCrosstalkSources_997014_54.stderr parsl.dataflow.dflow DEBUG: Task 592 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 592 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 592 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 592 has memoization hash 257fb7187d640fd7c42ea40a2a3b786e parsl.dataflow.memoization INFO: Task 592 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 592 try 0 launched on executor multi with executor id 593 parsl.dataflow.dflow INFO: Standard output for task 592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/27c016e3-bb8a-4391-8fb7-6976a7267245_isrForCrosstalkSources_996981_16.stdout parsl.dataflow.dflow INFO: Standard error for task 592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/27c016e3-bb8a-4391-8fb7-6976a7267245_isrForCrosstalkSources_996981_16.stderr parsl.dataflow.dflow DEBUG: Task 593 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 593 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 593 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 593 has memoization hash 2ecb6c9fad05eaa1ab82fd87aed49333 parsl.dataflow.memoization INFO: Task 593 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 593 try 0 launched on executor multi with executor id 594 parsl.dataflow.dflow INFO: Standard output for task 593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6f353c39-5b14-408c-aeff-282843ef8a5c_isrForCrosstalkSources_996991_13.stdout parsl.dataflow.dflow INFO: Standard error for task 593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6f353c39-5b14-408c-aeff-282843ef8a5c_isrForCrosstalkSources_996991_13.stderr parsl.dataflow.dflow DEBUG: Task 594 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 594 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 594 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 594 has memoization hash da45ad4c455d1b88b39f07bd459cb9c8 parsl.dataflow.memoization INFO: Task 594 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 594 try 0 launched on executor multi with executor id 595 parsl.dataflow.dflow INFO: Standard output for task 594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/06d34a27-26b0-4faf-bf1e-a478f0db1161_isrForCrosstalkSources_997029_43.stdout parsl.dataflow.dflow INFO: Standard error for task 594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/06d34a27-26b0-4faf-bf1e-a478f0db1161_isrForCrosstalkSources_997029_43.stderr parsl.dataflow.dflow DEBUG: Task 595 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 595 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 595 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 595 has memoization hash 5b05ad3b70086ddefd2c18088a12ff56 parsl.dataflow.memoization INFO: Task 595 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 595 try 0 launched on executor multi with executor id 596 parsl.dataflow.dflow INFO: Standard output for task 595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e03298d3-3f2b-4f6f-bfb9-2dc0289b1331_isrForCrosstalkSources_996992_43.stdout parsl.dataflow.dflow INFO: Standard error for task 595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e03298d3-3f2b-4f6f-bfb9-2dc0289b1331_isrForCrosstalkSources_996992_43.stderr parsl.dataflow.dflow DEBUG: Task 596 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 596 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 596 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 596 has memoization hash 3cfd8701575cbb6815adab12064ca8be parsl.dataflow.memoization INFO: Task 596 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 596 try 0 launched on executor multi with executor id 597 parsl.dataflow.dflow INFO: Standard output for task 596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/cb27cffe-83fe-46ac-811b-eb23a9e3d951_isrForCrosstalkSources_997011_13.stdout parsl.dataflow.dflow INFO: Standard error for task 596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/cb27cffe-83fe-46ac-811b-eb23a9e3d951_isrForCrosstalkSources_997011_13.stderr parsl.dataflow.dflow DEBUG: Task 597 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 597 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 597 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 597 has memoization hash 887f4c4e115af61681867c5456aed996 parsl.dataflow.memoization INFO: Task 597 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 597 try 0 launched on executor multi with executor id 598 parsl.dataflow.dflow INFO: Standard output for task 597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9904f1bf-0a72-411d-acac-f658456e85d6_isrForCrosstalkSources_996958_52.stdout parsl.dataflow.dflow INFO: Standard error for task 597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9904f1bf-0a72-411d-acac-f658456e85d6_isrForCrosstalkSources_996958_52.stderr parsl.dataflow.dflow DEBUG: Task 598 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 598 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 598 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 598 has memoization hash fd9c56ff417c098dd21fd712e027c111 parsl.dataflow.memoization INFO: Task 598 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 598 try 0 launched on executor multi with executor id 599 parsl.dataflow.dflow INFO: Standard output for task 598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/07e65e51-4cef-4b82-8ed4-7dceb76532df_isrForCrosstalkSources_996994_23.stdout parsl.dataflow.dflow INFO: Standard error for task 598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/07e65e51-4cef-4b82-8ed4-7dceb76532df_isrForCrosstalkSources_996994_23.stderr parsl.dataflow.dflow DEBUG: Task 599 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 599 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 599 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 599 has memoization hash 73aa30cabb2c3112656c087ce49d6687 parsl.dataflow.memoization INFO: Task 599 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 599 try 0 launched on executor multi with executor id 600 parsl.dataflow.dflow INFO: Standard output for task 599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/2f5e9fb7-7982-41cc-adb4-31a97019808a_isrForCrosstalkSources_997010_43.stdout parsl.dataflow.dflow INFO: Standard error for task 599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/2f5e9fb7-7982-41cc-adb4-31a97019808a_isrForCrosstalkSources_997010_43.stderr parsl.dataflow.dflow DEBUG: Task 600 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 600 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 600 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 600 has memoization hash 0b59c8c61cf7b3ab12f9172e38d95c3c parsl.dataflow.memoization INFO: Task 600 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fb880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 600 try 0 launched on executor multi with executor id 601 parsl.dataflow.dflow INFO: Standard output for task 600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/4f65d7e4-69bc-42e1-a1d3-f4cf0d07f7dd_isrForCrosstalkSources_997001_54.stdout parsl.dataflow.dflow INFO: Standard error for task 600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/4f65d7e4-69bc-42e1-a1d3-f4cf0d07f7dd_isrForCrosstalkSources_997001_54.stderr parsl.dataflow.dflow DEBUG: Task 601 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 601 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 601 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 601 has memoization hash 4be9adb0b7f14127624b35b3ee21219a parsl.dataflow.memoization INFO: Task 601 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 601 try 0 launched on executor multi with executor id 602 parsl.dataflow.dflow INFO: Standard output for task 601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7c124250-e9e1-4e7a-9b65-4b45af6e9b18_isrForCrosstalkSources_997003_36.stdout parsl.dataflow.dflow INFO: Standard error for task 601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7c124250-e9e1-4e7a-9b65-4b45af6e9b18_isrForCrosstalkSources_997003_36.stderr parsl.dataflow.dflow DEBUG: Task 602 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 602 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 602 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 602 has memoization hash 2b5fe67bc54ec108c668b4fe01f276eb parsl.dataflow.memoization INFO: Task 602 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fbba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 602 try 0 launched on executor multi with executor id 603 parsl.dataflow.dflow INFO: Standard output for task 602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d7f21927-6118-4504-871d-b9efb09366b2_isrForCrosstalkSources_997027_52.stdout parsl.dataflow.dflow INFO: Standard error for task 602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d7f21927-6118-4504-871d-b9efb09366b2_isrForCrosstalkSources_997027_52.stderr parsl.dataflow.dflow DEBUG: Task 603 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 603 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 603 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 603 has memoization hash 288530e68aa25cb005fdfaaa90f2f26d parsl.dataflow.memoization INFO: Task 603 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fbc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 603 try 0 launched on executor multi with executor id 604 parsl.dataflow.dflow INFO: Standard output for task 603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d4b009d7-0672-40e9-99c6-4a92c6af878f_isrForCrosstalkSources_996958_53.stdout parsl.dataflow.dflow INFO: Standard error for task 603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d4b009d7-0672-40e9-99c6-4a92c6af878f_isrForCrosstalkSources_996958_53.stderr parsl.dataflow.dflow DEBUG: Task 604 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 604 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 604 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 604 has memoization hash 0691244c51e8cbafb3db4c92d1f8809a parsl.dataflow.memoization INFO: Task 604 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fbce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 604 try 0 launched on executor multi with executor id 605 parsl.dataflow.dflow INFO: Standard output for task 604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7a1964e5-b3f6-43ef-ad4b-2f294f5f6f78_isrForCrosstalkSources_996993_3.stdout parsl.dataflow.dflow INFO: Standard error for task 604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7a1964e5-b3f6-43ef-ad4b-2f294f5f6f78_isrForCrosstalkSources_996993_3.stderr parsl.dataflow.dflow DEBUG: Task 605 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 605 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 605 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 605 has memoization hash 1d7ee43f256568934eeb11b7fff5d8ba parsl.dataflow.memoization INFO: Task 605 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fbd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 605 try 0 launched on executor multi with executor id 606 parsl.dataflow.dflow INFO: Standard output for task 605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/4476cd38-7168-4ddf-b5ca-91554ae02007_isrForCrosstalkSources_997019_17.stdout parsl.dataflow.dflow INFO: Standard error for task 605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/4476cd38-7168-4ddf-b5ca-91554ae02007_isrForCrosstalkSources_997019_17.stderr parsl.dataflow.dflow DEBUG: Task 606 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 606 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 606 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 606 has memoization hash e6d411e91e0afcf0dbb4172c97f8a91b parsl.dataflow.memoization INFO: Task 606 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fbb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 606 try 0 launched on executor multi with executor id 607 parsl.dataflow.dflow INFO: Standard output for task 606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/2ccd3756-9005-4145-a9a1-78faaf6904e5_isrForCrosstalkSources_996979_40.stdout parsl.dataflow.dflow INFO: Standard error for task 606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/2ccd3756-9005-4145-a9a1-78faaf6904e5_isrForCrosstalkSources_996979_40.stderr parsl.dataflow.dflow DEBUG: Task 607 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 607 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 607 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 607 has memoization hash 4fdc95a3b7893cc57a55366e29bae2c2 parsl.dataflow.memoization INFO: Task 607 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fbe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 607 try 0 launched on executor multi with executor id 608 parsl.dataflow.dflow INFO: Standard output for task 607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/de248182-8559-460d-aa1e-d3a8bbff5e82_isrForCrosstalkSources_997032_17.stdout parsl.dataflow.dflow INFO: Standard error for task 607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/de248182-8559-460d-aa1e-d3a8bbff5e82_isrForCrosstalkSources_997032_17.stderr parsl.dataflow.dflow DEBUG: Task 608 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 608 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 608 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 608 has memoization hash 2a2819be5bea3314b816e8cc9cb9d8d8 parsl.dataflow.memoization INFO: Task 608 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fa0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 608 try 0 launched on executor multi with executor id 609 parsl.dataflow.dflow INFO: Standard output for task 608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c97cebe8-ae08-4309-b6c7-b63a545af6a6_isrForCrosstalkSources_996965_55.stdout parsl.dataflow.dflow INFO: Standard error for task 608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c97cebe8-ae08-4309-b6c7-b63a545af6a6_isrForCrosstalkSources_996965_55.stderr parsl.dataflow.dflow DEBUG: Task 609 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 609 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 609 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 609 has memoization hash 6252e36bf5b68102384068cc1572c42b parsl.dataflow.memoization INFO: Task 609 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b93a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 609 try 0 launched on executor multi with executor id 610 parsl.dataflow.dflow INFO: Standard output for task 609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/92a78bbb-1057-4a22-816b-c6916ae03e01_isrForCrosstalkSources_996973_62.stdout parsl.dataflow.dflow INFO: Standard error for task 609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/92a78bbb-1057-4a22-816b-c6916ae03e01_isrForCrosstalkSources_996973_62.stderr parsl.dataflow.dflow DEBUG: Task 610 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 610 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 610 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 610 has memoization hash d15fe4e00dacafef5ded14e82243dc98 parsl.dataflow.memoization INFO: Task 610 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 610 try 0 launched on executor multi with executor id 611 parsl.dataflow.dflow INFO: Standard output for task 610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/3db517de-4d1d-4260-ae5e-e85c13e87c27_isrForCrosstalkSources_997026_11.stdout parsl.dataflow.dflow INFO: Standard error for task 610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/3db517de-4d1d-4260-ae5e-e85c13e87c27_isrForCrosstalkSources_997026_11.stderr parsl.dataflow.dflow DEBUG: Task 611 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 611 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 611 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 611 has memoization hash bceed166e2e7cb0a560f1d056f20df0a parsl.dataflow.memoization INFO: Task 611 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b91c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 611 try 0 launched on executor multi with executor id 612 parsl.dataflow.dflow INFO: Standard output for task 611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/9614842d-eb11-48ec-8b4f-dbeeef5fc7b3_isrForCrosstalkSources_996967_14.stdout parsl.dataflow.dflow INFO: Standard error for task 611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/9614842d-eb11-48ec-8b4f-dbeeef5fc7b3_isrForCrosstalkSources_996967_14.stderr parsl.dataflow.dflow DEBUG: Task 612 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 612 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 612 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 612 has memoization hash d55b612f5e127fae0378784287db20fb parsl.dataflow.memoization INFO: Task 612 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 612 try 0 launched on executor multi with executor id 613 parsl.dataflow.dflow INFO: Standard output for task 612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8bc66dec-96f2-43a0-a246-f9cad3777214_isrForCrosstalkSources_996952_45.stdout parsl.dataflow.dflow INFO: Standard error for task 612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8bc66dec-96f2-43a0-a246-f9cad3777214_isrForCrosstalkSources_996952_45.stderr parsl.dataflow.dflow DEBUG: Task 613 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 613 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 613 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 613 has memoization hash f35d57039026e6c58815968f211e8138 parsl.dataflow.memoization INFO: Task 613 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 613 try 0 launched on executor multi with executor id 614 parsl.dataflow.dflow INFO: Standard output for task 613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6a685536-63a1-4950-a1f0-c86493fdc49f_isrForCrosstalkSources_996971_4.stdout parsl.dataflow.dflow INFO: Standard error for task 613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6a685536-63a1-4950-a1f0-c86493fdc49f_isrForCrosstalkSources_996971_4.stderr parsl.dataflow.dflow DEBUG: Task 614 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 614 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 614 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 614 has memoization hash 4a4ba17fd0559e0f98aaa805443be554 parsl.dataflow.memoization INFO: Task 614 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 614 try 0 launched on executor multi with executor id 615 2024-03-15 13:49:28 proc_lsst.multi:170 [INFO] [multi] job_id 17074838 proc_lsst.multi INFO: [multi] job_id 17074838 2024-03-15 13:49:28 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 6 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 6 2024-03-15 13:49:28 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:28 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074838 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074838 parsl.dataflow.dflow INFO: Standard output for task 614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/aafe6966-14a9-4bf2-88d7-c326a36a2f88_isrForCrosstalkSources_997009_8.stdout parsl.dataflow.dflow INFO: Standard error for task 614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/aafe6966-14a9-4bf2-88d7-c326a36a2f88_isrForCrosstalkSources_997009_8.stderr parsl.dataflow.dflow DEBUG: Task 615 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 615 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 615 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 615 has memoization hash 19b33d10e40122270249a0b60ed3ceb7 parsl.dataflow.memoization INFO: Task 615 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f832020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.executors.status_handling DEBUG: Launched block 6 on executor multi with job ID 17074838 parsl.executors.status_handling INFO: Allocated block ID 7 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-7 2024-03-15 13:49:28 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=7 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-7 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=7 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-7 2024-03-15 13:49:28 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=7 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=7 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:28 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=7 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=7 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 6 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 6 2024-03-15 13:49:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:28 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=7 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=7 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 615 try 0 launched on executor multi with executor id 616 parsl.dataflow.dflow INFO: Standard output for task 615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d3190d70-9bd3-4db6-a3e2-27e569e12170_isrForCrosstalkSources_997027_27.stdout parsl.dataflow.dflow INFO: Standard error for task 615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d3190d70-9bd3-4db6-a3e2-27e569e12170_isrForCrosstalkSources_997027_27.stderr parsl.dataflow.dflow DEBUG: Task 616 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 616 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 616 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 616 has memoization hash a8025965d9c8375ee560b2f94ee5bd17 parsl.dataflow.memoization INFO: Task 616 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77f831ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 616 try 0 launched on executor multi with executor id 617 parsl.dataflow.dflow INFO: Standard output for task 616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/494d4d83-224b-4f31-8abb-00b79c7115c7_isrForCrosstalkSources_996969_27.stdout parsl.dataflow.dflow INFO: Standard error for task 616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/494d4d83-224b-4f31-8abb-00b79c7115c7_isrForCrosstalkSources_996969_27.stderr parsl.dataflow.dflow DEBUG: Task 617 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 617 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 617 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 617 has memoization hash 9906211041894db959757489654ec683 parsl.dataflow.memoization INFO: Task 617 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee5120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 617 try 0 launched on executor multi with executor id 618 parsl.dataflow.dflow INFO: Standard output for task 617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/26cdff44-2d85-46c4-b115-d788e3d4d93a_isrForCrosstalkSources_996990_61.stdout parsl.dataflow.dflow INFO: Standard error for task 617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/26cdff44-2d85-46c4-b115-d788e3d4d93a_isrForCrosstalkSources_996990_61.stderr parsl.dataflow.dflow DEBUG: Task 618 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 618 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 618 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 618 has memoization hash 5e29b406136fe8b8e682763890f543f7 parsl.dataflow.memoization INFO: Task 618 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 618 try 0 launched on executor multi with executor id 619 parsl.dataflow.dflow INFO: Standard output for task 618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4f64564f-a322-4095-ae4f-63c2975d0ab0_isrForCrosstalkSources_997005_60.stdout parsl.dataflow.dflow INFO: Standard error for task 618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4f64564f-a322-4095-ae4f-63c2975d0ab0_isrForCrosstalkSources_997005_60.stderr parsl.dataflow.dflow DEBUG: Task 619 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 619 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 619 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 619 has memoization hash 89791decc03f63f74d370def5e7d5f95 parsl.dataflow.memoization INFO: Task 619 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 619 try 0 launched on executor multi with executor id 620 parsl.dataflow.dflow INFO: Standard output for task 619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1e5c7299-ea5a-47e6-8b1c-73837f18d03a_isrForCrosstalkSources_997014_24.stdout parsl.dataflow.dflow INFO: Standard error for task 619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1e5c7299-ea5a-47e6-8b1c-73837f18d03a_isrForCrosstalkSources_997014_24.stderr parsl.dataflow.dflow DEBUG: Task 620 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 620 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 620 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 620 has memoization hash 3c7a3ffbc967b0996ec4f251372ab791 parsl.dataflow.memoization INFO: Task 620 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bbce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 620 try 0 launched on executor multi with executor id 621 parsl.dataflow.dflow INFO: Standard output for task 620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3ce02e0b-a885-4de4-85e7-71f768e02043_isrForCrosstalkSources_997031_45.stdout parsl.dataflow.dflow INFO: Standard error for task 620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3ce02e0b-a885-4de4-85e7-71f768e02043_isrForCrosstalkSources_997031_45.stderr parsl.dataflow.dflow DEBUG: Task 621 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 621 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 621 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 621 has memoization hash 5fa9cb50720869aaaad03717d8ed13b2 parsl.dataflow.memoization INFO: Task 621 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 621 try 0 launched on executor multi with executor id 622 parsl.dataflow.dflow INFO: Standard output for task 621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b85389b9-8593-4db5-b767-b212a214a19e_isrForCrosstalkSources_996979_9.stdout parsl.dataflow.dflow INFO: Standard error for task 621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b85389b9-8593-4db5-b767-b212a214a19e_isrForCrosstalkSources_996979_9.stderr parsl.dataflow.dflow DEBUG: Task 622 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 622 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 622 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 622 has memoization hash 9a78e1275f198cee109e9a9e2b6493d2 parsl.dataflow.memoization INFO: Task 622 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bbc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 622 try 0 launched on executor multi with executor id 623 parsl.dataflow.dflow INFO: Standard output for task 622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f5de12e8-cc6b-4334-b2a2-6a3a2993f597_isrForCrosstalkSources_996967_22.stdout parsl.dataflow.dflow INFO: Standard error for task 622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f5de12e8-cc6b-4334-b2a2-6a3a2993f597_isrForCrosstalkSources_996967_22.stderr parsl.dataflow.dflow DEBUG: Task 623 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 623 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 623 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 623 has memoization hash 19e5268f4bcc08491507b56c18430beb parsl.dataflow.memoization INFO: Task 623 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bbb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 623 try 0 launched on executor multi with executor id 624 parsl.dataflow.dflow INFO: Standard output for task 623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/dc215cf0-4845-4d56-b557-c49445e32497_isrForCrosstalkSources_996973_57.stdout parsl.dataflow.dflow INFO: Standard error for task 623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/dc215cf0-4845-4d56-b557-c49445e32497_isrForCrosstalkSources_996973_57.stderr parsl.dataflow.dflow DEBUG: Task 624 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 624 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 624 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 624 has memoization hash 7d8ccad5648e381315937ad1d0a19cdb parsl.dataflow.memoization INFO: Task 624 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 624 try 0 launched on executor multi with executor id 625 parsl.dataflow.dflow INFO: Standard output for task 624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/9f5974ba-8178-477d-9fd5-827f712a5371_isrForCrosstalkSources_997002_48.stdout parsl.dataflow.dflow INFO: Standard error for task 624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/9f5974ba-8178-477d-9fd5-827f712a5371_isrForCrosstalkSources_997002_48.stderr parsl.dataflow.dflow DEBUG: Task 625 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 625 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 625 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 625 has memoization hash 6da9a0ff8fa2d2a9725050d476f0cb80 parsl.dataflow.memoization INFO: Task 625 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bbba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 625 try 0 launched on executor multi with executor id 626 parsl.dataflow.dflow INFO: Standard output for task 625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/313fde2d-b413-4f2c-97e7-2daad3c2961f_isrForCrosstalkSources_997006_22.stdout parsl.dataflow.dflow INFO: Standard error for task 625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/313fde2d-b413-4f2c-97e7-2daad3c2961f_isrForCrosstalkSources_997006_22.stderr parsl.dataflow.dflow DEBUG: Task 626 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 626 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 626 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 626 has memoization hash f116f8cff250542e9ef5ce09dd112da2 parsl.dataflow.memoization INFO: Task 626 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b89a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 626 try 0 launched on executor multi with executor id 627 parsl.dataflow.dflow INFO: Standard output for task 626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/183483ab-232a-4776-ac13-6f58774453d9_isrForCrosstalkSources_996968_40.stdout parsl.dataflow.dflow INFO: Standard error for task 626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/183483ab-232a-4776-ac13-6f58774453d9_isrForCrosstalkSources_996968_40.stderr parsl.dataflow.dflow DEBUG: Task 627 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 627 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 627 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 627 has memoization hash 5256f40c993bab4da6a23ad6f5ba1d10 parsl.dataflow.memoization INFO: Task 627 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 627 try 0 launched on executor multi with executor id 628 parsl.dataflow.dflow INFO: Standard output for task 627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/13c4f091-244f-4856-9eda-07bee3914cdf_isrForCrosstalkSources_997020_42.stdout parsl.dataflow.dflow INFO: Standard error for task 627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/13c4f091-244f-4856-9eda-07bee3914cdf_isrForCrosstalkSources_997020_42.stderr parsl.dataflow.dflow DEBUG: Task 628 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 628 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 628 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 628 has memoization hash d23d5c53b6c7ca007bb15991c92ca265 parsl.dataflow.memoization INFO: Task 628 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 628 try 0 launched on executor multi with executor id 629 parsl.dataflow.dflow INFO: Standard output for task 628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ba028b4a-40d6-4e2f-9a95-553106e31df5_isrForCrosstalkSources_996988_9.stdout parsl.dataflow.dflow INFO: Standard error for task 628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ba028b4a-40d6-4e2f-9a95-553106e31df5_isrForCrosstalkSources_996988_9.stderr parsl.dataflow.dflow DEBUG: Task 629 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 629 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 629 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 629 has memoization hash b1a96400962371c252beb63c331327bd parsl.dataflow.memoization INFO: Task 629 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 629 try 0 launched on executor multi with executor id 630 parsl.dataflow.dflow INFO: Standard output for task 629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/28d8777b-2392-48a8-81f7-317c5079ca54_isrForCrosstalkSources_997016_33.stdout parsl.dataflow.dflow INFO: Standard error for task 629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/28d8777b-2392-48a8-81f7-317c5079ca54_isrForCrosstalkSources_997016_33.stderr parsl.dataflow.dflow DEBUG: Task 630 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 630 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 630 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 630 has memoization hash 90bf3f06518baa4b66501a02d2621306 parsl.dataflow.memoization INFO: Task 630 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b87c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 630 try 0 launched on executor multi with executor id 631 parsl.dataflow.dflow INFO: Standard output for task 630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e7abe875-9172-4daa-8e16-20ba83742217_isrForCrosstalkSources_996995_51.stdout parsl.dataflow.dflow INFO: Standard error for task 630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e7abe875-9172-4daa-8e16-20ba83742217_isrForCrosstalkSources_996995_51.stderr parsl.dataflow.dflow DEBUG: Task 631 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 631 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 631 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 631 has memoization hash 239f89c604ffc68ef8f24236e4e37635 parsl.dataflow.memoization INFO: Task 631 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 631 try 0 launched on executor multi with executor id 632 parsl.dataflow.dflow INFO: Standard output for task 631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/16257e6e-f69b-4d1b-af32-4cc5351926ca_isrForCrosstalkSources_996987_51.stdout parsl.dataflow.dflow INFO: Standard error for task 631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/16257e6e-f69b-4d1b-af32-4cc5351926ca_isrForCrosstalkSources_996987_51.stderr parsl.dataflow.dflow DEBUG: Task 632 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 632 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 632 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 632 has memoization hash c0ddd2cdb7b8b47ea23424abe3207d42 parsl.dataflow.memoization INFO: Task 632 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 632 try 0 launched on executor multi with executor id 633 parsl.dataflow.dflow INFO: Standard output for task 632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/3188d077-cd2e-41e7-972a-0d912d1f8066_isrForCrosstalkSources_997003_40.stdout parsl.dataflow.dflow INFO: Standard error for task 632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/3188d077-cd2e-41e7-972a-0d912d1f8066_isrForCrosstalkSources_997003_40.stderr parsl.dataflow.dflow DEBUG: Task 633 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 633 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 633 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 633 has memoization hash df3cf6565b9655060768d701e9b1844a parsl.dataflow.memoization INFO: Task 633 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b85e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 633 try 0 launched on executor multi with executor id 634 parsl.dataflow.dflow INFO: Standard output for task 633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/91e45c7e-58d9-4037-83c1-218e17130dd1_isrForCrosstalkSources_996996_28.stdout parsl.dataflow.dflow INFO: Standard error for task 633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/91e45c7e-58d9-4037-83c1-218e17130dd1_isrForCrosstalkSources_996996_28.stderr parsl.dataflow.dflow DEBUG: Task 634 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 634 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 634 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 634 has memoization hash 197c879eff5e0bd3241b3660d0c6fd4c parsl.dataflow.memoization INFO: Task 634 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 634 try 0 launched on executor multi with executor id 635 parsl.dataflow.dflow INFO: Standard output for task 634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/377c2b77-8177-403d-a99d-db8e9ad18dbc_isrForCrosstalkSources_996997_48.stdout parsl.dataflow.dflow INFO: Standard error for task 634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/377c2b77-8177-403d-a99d-db8e9ad18dbc_isrForCrosstalkSources_996997_48.stderr parsl.dataflow.dflow DEBUG: Task 635 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 635 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 635 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 635 has memoization hash 346451b65f05439980a214abcc55abb1 parsl.dataflow.memoization INFO: Task 635 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 635 try 0 launched on executor multi with executor id 636 parsl.dataflow.dflow INFO: Standard output for task 635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b941c94c-9fb6-459d-9a86-12ead70062ad_isrForCrosstalkSources_996996_38.stdout parsl.dataflow.dflow INFO: Standard error for task 635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b941c94c-9fb6-459d-9a86-12ead70062ad_isrForCrosstalkSources_996996_38.stderr parsl.dataflow.dflow DEBUG: Task 636 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 636 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 636 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 636 has memoization hash 1f105b2cc0f3cd537dddce9e0b60b028 parsl.dataflow.memoization INFO: Task 636 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 636 try 0 launched on executor multi with executor id 637 parsl.dataflow.dflow INFO: Standard output for task 636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7eb9faac-674a-4616-b7b0-eaa1a67edef5_isrForCrosstalkSources_997011_9.stdout parsl.dataflow.dflow INFO: Standard error for task 636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7eb9faac-674a-4616-b7b0-eaa1a67edef5_isrForCrosstalkSources_997011_9.stderr parsl.dataflow.dflow DEBUG: Task 637 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 637 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 637 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 637 has memoization hash 02c835afacc492f48e30e16f6812dd11 parsl.dataflow.memoization INFO: Task 637 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b84a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 637 try 0 launched on executor multi with executor id 638 parsl.dataflow.dflow INFO: Standard output for task 637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5c4b0430-fe37-4a00-b579-9b03cdbbbfb9_isrForCrosstalkSources_997012_21.stdout parsl.dataflow.dflow INFO: Standard error for task 637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5c4b0430-fe37-4a00-b579-9b03cdbbbfb9_isrForCrosstalkSources_997012_21.stderr parsl.dataflow.dflow DEBUG: Task 638 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 638 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 638 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 638 has memoization hash 0e4e76d69b15998f672375c02370512c parsl.dataflow.memoization INFO: Task 638 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b82c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 638 try 0 launched on executor multi with executor id 639 parsl.dataflow.dflow INFO: Standard output for task 638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/f1f8482b-40b4-4453-872c-7383ae60ed04_isrForCrosstalkSources_997033_56.stdout parsl.dataflow.dflow INFO: Standard error for task 638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/f1f8482b-40b4-4453-872c-7383ae60ed04_isrForCrosstalkSources_997033_56.stderr parsl.dataflow.dflow DEBUG: Task 639 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 639 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 639 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 639 has memoization hash b15a3419641a71cffd951968909ce38a parsl.dataflow.memoization INFO: Task 639 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 639 try 0 launched on executor multi with executor id 640 parsl.dataflow.dflow INFO: Standard output for task 639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/ef6d33f5-4002-4cd0-a2bd-e850f7f5291c_isrForCrosstalkSources_996983_27.stdout parsl.dataflow.dflow INFO: Standard error for task 639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/ef6d33f5-4002-4cd0-a2bd-e850f7f5291c_isrForCrosstalkSources_996983_27.stderr parsl.dataflow.dflow DEBUG: Task 640 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 640 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 640 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 640 has memoization hash 85896d41a086835458c8581e0107e276 parsl.dataflow.memoization INFO: Task 640 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 640 try 0 launched on executor multi with executor id 641 parsl.dataflow.dflow INFO: Standard output for task 640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8612bf38-2d88-4d92-9612-96d779132327_isrForCrosstalkSources_996967_54.stdout parsl.dataflow.dflow INFO: Standard error for task 640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8612bf38-2d88-4d92-9612-96d779132327_isrForCrosstalkSources_996967_54.stderr parsl.dataflow.dflow DEBUG: Task 641 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 641 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 641 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 641 has memoization hash b47410fcae86510fffa3c68062d280c9 parsl.dataflow.memoization INFO: Task 641 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 641 try 0 launched on executor multi with executor id 642 parsl.dataflow.dflow INFO: Standard output for task 641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6b18b9ad-599a-41ab-a4c5-0ce252568268_isrForCrosstalkSources_997006_45.stdout parsl.dataflow.dflow INFO: Standard error for task 641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6b18b9ad-599a-41ab-a4c5-0ce252568268_isrForCrosstalkSources_997006_45.stderr parsl.dataflow.dflow DEBUG: Task 642 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 642 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 642 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 642 has memoization hash 1d616f6997ab5e08d502fd0a548ceff7 parsl.dataflow.memoization INFO: Task 642 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 642 try 0 launched on executor multi with executor id 643 parsl.dataflow.dflow INFO: Standard output for task 642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/846bd7b3-69d1-4dbf-8887-2b95a0b221c3_isrForCrosstalkSources_996999_5.stdout parsl.dataflow.dflow INFO: Standard error for task 642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/846bd7b3-69d1-4dbf-8887-2b95a0b221c3_isrForCrosstalkSources_996999_5.stderr parsl.dataflow.dflow DEBUG: Task 643 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 643 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 643 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 643 has memoization hash 336df94ab49025f5e7c1e19dd5c3a861 parsl.dataflow.memoization INFO: Task 643 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b94e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 643 try 0 launched on executor multi with executor id 644 parsl.dataflow.dflow INFO: Standard output for task 643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/6dcba814-aa6b-4d5e-a507-5a302056a4d2_isrForCrosstalkSources_996984_16.stdout parsl.dataflow.dflow INFO: Standard error for task 643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/6dcba814-aa6b-4d5e-a507-5a302056a4d2_isrForCrosstalkSources_996984_16.stderr parsl.dataflow.dflow DEBUG: Task 644 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 644 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 644 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 644 has memoization hash 025135034275e904ffba3219dddc05b4 parsl.dataflow.memoization INFO: Task 644 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 644 try 0 launched on executor multi with executor id 645 parsl.dataflow.dflow INFO: Standard output for task 644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/5781801d-8d9e-4af4-9762-4a81937f3835_isrForCrosstalkSources_996995_62.stdout parsl.dataflow.dflow INFO: Standard error for task 644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/5781801d-8d9e-4af4-9762-4a81937f3835_isrForCrosstalkSources_996995_62.stderr parsl.dataflow.dflow DEBUG: Task 645 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 645 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 645 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 645 has memoization hash ac71fbf6e67e9df755ca5336aed49a0e parsl.dataflow.memoization INFO: Task 645 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b80e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 645 try 0 launched on executor multi with executor id 646 parsl.dataflow.dflow INFO: Standard output for task 645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/61495e3b-748d-4f77-ad13-177073a2941d_isrForCrosstalkSources_996967_12.stdout parsl.dataflow.dflow INFO: Standard error for task 645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/61495e3b-748d-4f77-ad13-177073a2941d_isrForCrosstalkSources_996967_12.stderr parsl.dataflow.dflow DEBUG: Task 646 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 646 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 646 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 646 has memoization hash f46bfd5ac87afcdc960575dc5a2d61ff parsl.dataflow.memoization INFO: Task 646 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b96c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 646 try 0 launched on executor multi with executor id 647 parsl.dataflow.dflow INFO: Standard output for task 646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/b002214a-327e-4187-ae0d-758c3697038b_isrForCrosstalkSources_996970_42.stdout parsl.dataflow.dflow INFO: Standard error for task 646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/b002214a-327e-4187-ae0d-758c3697038b_isrForCrosstalkSources_996970_42.stderr parsl.dataflow.dflow DEBUG: Task 647 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 647 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 647 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 647 has memoization hash 0bc79f41083eb6d980f61ab4a1fdf7ff parsl.dataflow.memoization INFO: Task 647 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 647 try 0 launched on executor multi with executor id 648 parsl.dataflow.dflow INFO: Standard output for task 647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ae752355-8191-4c55-b084-2c0f86802d20_isrForCrosstalkSources_996972_28.stdout parsl.dataflow.dflow INFO: Standard error for task 647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ae752355-8191-4c55-b084-2c0f86802d20_isrForCrosstalkSources_996972_28.stderr parsl.dataflow.dflow DEBUG: Task 648 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 648 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 648 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 648 has memoization hash 12fdb1f2a207c51be279a5d402216ab8 parsl.dataflow.memoization INFO: Task 648 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 648 try 0 launched on executor multi with executor id 649 parsl.dataflow.dflow INFO: Standard output for task 648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a372db81-8893-4c7e-991d-8cd90f2f2be2_isrForCrosstalkSources_996993_62.stdout parsl.dataflow.dflow INFO: Standard error for task 648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a372db81-8893-4c7e-991d-8cd90f2f2be2_isrForCrosstalkSources_996993_62.stderr parsl.dataflow.dflow DEBUG: Task 649 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 649 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 649 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 649 has memoization hash 6eceac8f4a141e2a9b642bb3e48f80f6 parsl.dataflow.memoization INFO: Task 649 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 649 try 0 launched on executor multi with executor id 650 parsl.dataflow.dflow INFO: Standard output for task 649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f02c1d17-412b-4f93-94de-a34e55ba132a_isrForCrosstalkSources_997031_12.stdout parsl.dataflow.dflow INFO: Standard error for task 649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f02c1d17-412b-4f93-94de-a34e55ba132a_isrForCrosstalkSources_997031_12.stderr parsl.dataflow.dflow DEBUG: Task 650 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 650 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 650 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 650 has memoization hash c0baf43f2812aa3d3977d77ad257af6f parsl.dataflow.memoization INFO: Task 650 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 650 try 0 launched on executor multi with executor id 651 parsl.dataflow.dflow INFO: Standard output for task 650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9adeae47-579b-4b9f-9853-5be5fa0a32e0_isrForCrosstalkSources_996984_25.stdout parsl.dataflow.dflow INFO: Standard error for task 650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9adeae47-579b-4b9f-9853-5be5fa0a32e0_isrForCrosstalkSources_996984_25.stderr parsl.dataflow.dflow DEBUG: Task 651 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 651 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 651 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 651 has memoization hash 9ac6b32fc5e64b01bf635ce70d84366a parsl.dataflow.memoization INFO: Task 651 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b98a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 651 try 0 launched on executor multi with executor id 652 parsl.dataflow.dflow INFO: Standard output for task 651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1ea7cc0f-904d-42a1-ad71-bc1f04907aed_isrForCrosstalkSources_997027_44.stdout parsl.dataflow.dflow INFO: Standard error for task 651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1ea7cc0f-904d-42a1-ad71-bc1f04907aed_isrForCrosstalkSources_997027_44.stderr parsl.dataflow.dflow DEBUG: Task 652 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 652 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 652 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 652 has memoization hash 8d39b5641193c7830c9c5df82f9d1ce4 parsl.dataflow.memoization INFO: Task 652 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b99e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 652 try 0 launched on executor multi with executor id 653 parsl.dataflow.dflow INFO: Standard output for task 652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6435b05f-39e2-424c-9651-7e4bd1687d8b_isrForCrosstalkSources_997005_50.stdout parsl.dataflow.dflow INFO: Standard error for task 652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6435b05f-39e2-424c-9651-7e4bd1687d8b_isrForCrosstalkSources_997005_50.stderr parsl.dataflow.dflow DEBUG: Task 653 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 653 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 653 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 653 has memoization hash 8e98204978d27bbd67d09cb5554e4b83 parsl.dataflow.memoization INFO: Task 653 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 653 try 0 launched on executor multi with executor id 654 parsl.dataflow.dflow INFO: Standard output for task 653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ef058422-086a-4ce2-bf5d-9eabe27128bb_isrForCrosstalkSources_996980_56.stdout parsl.dataflow.dflow INFO: Standard error for task 653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ef058422-086a-4ce2-bf5d-9eabe27128bb_isrForCrosstalkSources_996980_56.stderr parsl.dataflow.dflow DEBUG: Task 654 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 654 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 654 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 654 has memoization hash fa24fd5b891dbe63db8fbae2f05aeb6b parsl.dataflow.memoization INFO: Task 654 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 654 try 0 launched on executor multi with executor id 655 parsl.dataflow.dflow INFO: Standard output for task 654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/32f03f31-8e8a-4c24-a07c-1c90ed882ee6_isrForCrosstalkSources_996964_56.stdout parsl.dataflow.dflow INFO: Standard error for task 654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/32f03f31-8e8a-4c24-a07c-1c90ed882ee6_isrForCrosstalkSources_996964_56.stderr parsl.dataflow.dflow DEBUG: Task 655 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 655 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 655 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 655 has memoization hash 1d0d9e7e908c060b8b65e4b1872955a5 parsl.dataflow.memoization INFO: Task 655 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 655 try 0 launched on executor multi with executor id 656 parsl.dataflow.dflow INFO: Standard output for task 655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/5c6dbff9-9042-4229-a66d-7c7c2748fecb_isrForCrosstalkSources_996963_14.stdout parsl.dataflow.dflow INFO: Standard error for task 655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/5c6dbff9-9042-4229-a66d-7c7c2748fecb_isrForCrosstalkSources_996963_14.stderr parsl.dataflow.dflow DEBUG: Task 656 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 656 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 656 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 656 has memoization hash 83884a378bdeead44e0a7dab89f9c011 parsl.dataflow.memoization INFO: Task 656 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 656 try 0 launched on executor multi with executor id 657 parsl.dataflow.dflow INFO: Standard output for task 656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/9880ec23-1dd6-48de-84c1-b94bd324c917_isrForCrosstalkSources_996969_7.stdout parsl.dataflow.dflow INFO: Standard error for task 656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/9880ec23-1dd6-48de-84c1-b94bd324c917_isrForCrosstalkSources_996969_7.stderr parsl.dataflow.dflow DEBUG: Task 657 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 657 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 657 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 657 has memoization hash e65566a4837094ac1441a797b5c0b7d7 parsl.dataflow.memoization INFO: Task 657 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 657 try 0 launched on executor multi with executor id 658 parsl.dataflow.dflow INFO: Standard output for task 657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/bce97935-7ad9-4808-b21c-5bc3d74e4fc1_isrForCrosstalkSources_997003_29.stdout parsl.dataflow.dflow INFO: Standard error for task 657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/bce97935-7ad9-4808-b21c-5bc3d74e4fc1_isrForCrosstalkSources_997003_29.stderr parsl.dataflow.dflow DEBUG: Task 658 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 658 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 658 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 658 has memoization hash 84b8abe0cd214f9b702479f00358e717 parsl.dataflow.memoization INFO: Task 658 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 658 try 0 launched on executor multi with executor id 659 parsl.dataflow.dflow INFO: Standard output for task 658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/5f046451-1242-4eb6-ab2f-36c95f601a78_isrForCrosstalkSources_997026_58.stdout parsl.dataflow.dflow INFO: Standard error for task 658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/5f046451-1242-4eb6-ab2f-36c95f601a78_isrForCrosstalkSources_997026_58.stderr parsl.dataflow.dflow DEBUG: Task 659 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 659 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 659 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 659 has memoization hash 7afeb32312bd1bc4add96ae98dfbce8d parsl.dataflow.memoization INFO: Task 659 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 659 try 0 launched on executor multi with executor id 660 parsl.dataflow.dflow INFO: Standard output for task 659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/97d1eef5-d34e-42fb-8a03-6647f112f847_isrForCrosstalkSources_996981_13.stdout parsl.dataflow.dflow INFO: Standard error for task 659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/97d1eef5-d34e-42fb-8a03-6647f112f847_isrForCrosstalkSources_996981_13.stderr parsl.dataflow.dflow DEBUG: Task 660 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 660 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 660 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 660 has memoization hash 2cbc067d234701a6e3bd9b2f9bbc28e6 parsl.dataflow.memoization INFO: Task 660 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 660 try 0 launched on executor multi with executor id 661 parsl.dataflow.dflow INFO: Standard output for task 660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/00ad1a39-6428-44ac-a3da-9bf371beeae6_isrForCrosstalkSources_997021_62.stdout parsl.dataflow.dflow INFO: Standard error for task 660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/00ad1a39-6428-44ac-a3da-9bf371beeae6_isrForCrosstalkSources_997021_62.stderr parsl.dataflow.dflow DEBUG: Task 661 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 661 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 661 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 661 has memoization hash 91512e256b0b9f3a98ab2036985e67db parsl.dataflow.memoization INFO: Task 661 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 661 try 0 launched on executor multi with executor id 662 parsl.dataflow.dflow INFO: Standard output for task 661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/4b92a6a6-cb85-47d7-b09b-1f807156d17f_isrForCrosstalkSources_997016_5.stdout parsl.dataflow.dflow INFO: Standard error for task 661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/4b92a6a6-cb85-47d7-b09b-1f807156d17f_isrForCrosstalkSources_997016_5.stderr parsl.dataflow.dflow DEBUG: Task 662 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 662 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 662 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 662 has memoization hash d588a9082f5cc71fe07e9ccd2fc45947 parsl.dataflow.memoization INFO: Task 662 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 662 try 0 launched on executor multi with executor id 663 parsl.dataflow.dflow INFO: Standard output for task 662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/af36ff8e-6438-4994-8814-e8bb87c34380_isrForCrosstalkSources_996957_60.stdout parsl.dataflow.dflow INFO: Standard error for task 662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/af36ff8e-6438-4994-8814-e8bb87c34380_isrForCrosstalkSources_996957_60.stderr parsl.dataflow.dflow DEBUG: Task 663 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 663 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 663 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 663 has memoization hash 32e6c66dfde3cf5cf8223c7efe6f3cd6 parsl.dataflow.memoization INFO: Task 663 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 663 try 0 launched on executor multi with executor id 664 parsl.dataflow.dflow INFO: Standard output for task 663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/2c8cff96-b806-47ac-8fc3-04596c6152e6_isrForCrosstalkSources_996983_25.stdout parsl.dataflow.dflow INFO: Standard error for task 663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/2c8cff96-b806-47ac-8fc3-04596c6152e6_isrForCrosstalkSources_996983_25.stderr parsl.dataflow.dflow DEBUG: Task 664 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 664 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 664 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 664 has memoization hash 7fd21e11aa08c5214bad6d62ae602da7 parsl.dataflow.memoization INFO: Task 664 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 664 try 0 launched on executor multi with executor id 665 parsl.dataflow.dflow INFO: Standard output for task 664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2d2971f0-5357-427c-bfff-62a71e267c6f_isrForCrosstalkSources_996981_4.stdout parsl.dataflow.dflow INFO: Standard error for task 664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2d2971f0-5357-427c-bfff-62a71e267c6f_isrForCrosstalkSources_996981_4.stderr parsl.dataflow.dflow DEBUG: Task 665 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 665 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 665 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 665 has memoization hash 89e9e44f5b55f9d2dea80c3c11e67800 parsl.dataflow.memoization INFO: Task 665 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 665 try 0 launched on executor multi with executor id 666 parsl.dataflow.dflow INFO: Standard output for task 665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/b60f1914-3796-4374-beb3-21f087627491_isrForCrosstalkSources_996995_60.stdout parsl.dataflow.dflow INFO: Standard error for task 665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/b60f1914-3796-4374-beb3-21f087627491_isrForCrosstalkSources_996995_60.stderr parsl.dataflow.dflow DEBUG: Task 666 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 666 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 666 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 666 has memoization hash b5ceadf29437a9f0f10905ab79db4920 parsl.dataflow.memoization INFO: Task 666 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 666 try 0 launched on executor multi with executor id 667 parsl.dataflow.dflow INFO: Standard output for task 666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/a42947e2-9441-4af0-b45a-4201636ab8c1_isrForCrosstalkSources_997030_20.stdout parsl.dataflow.dflow INFO: Standard error for task 666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/a42947e2-9441-4af0-b45a-4201636ab8c1_isrForCrosstalkSources_997030_20.stderr parsl.dataflow.dflow DEBUG: Task 667 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 667 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 667 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 667 has memoization hash e4ea86205caaa4e46e5e75248e49c7c8 parsl.dataflow.memoization INFO: Task 667 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 667 try 0 launched on executor multi with executor id 668 parsl.dataflow.dflow INFO: Standard output for task 667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/98b8409d-5955-42f4-a13e-69f7192d78a6_isrForCrosstalkSources_997018_17.stdout parsl.dataflow.dflow INFO: Standard error for task 667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/98b8409d-5955-42f4-a13e-69f7192d78a6_isrForCrosstalkSources_997018_17.stderr parsl.dataflow.dflow DEBUG: Task 668 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 668 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 668 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 668 has memoization hash 1f58d780b385420b4f64d0661c68cee1 parsl.dataflow.memoization INFO: Task 668 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 668 try 0 launched on executor multi with executor id 669 parsl.dataflow.dflow INFO: Standard output for task 668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3d560042-bfcb-401c-942c-754db5b50f82_isrForCrosstalkSources_996984_13.stdout parsl.dataflow.dflow INFO: Standard error for task 668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3d560042-bfcb-401c-942c-754db5b50f82_isrForCrosstalkSources_996984_13.stderr parsl.dataflow.dflow DEBUG: Task 669 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 669 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 669 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 669 has memoization hash ce3b4657cd32688440c55f240d699fc6 parsl.dataflow.memoization INFO: Task 669 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 669 try 0 launched on executor multi with executor id 670 parsl.dataflow.dflow INFO: Standard output for task 669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2d4da697-f669-47c6-b566-c4992377d23f_isrForCrosstalkSources_997007_38.stdout parsl.dataflow.dflow INFO: Standard error for task 669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2d4da697-f669-47c6-b566-c4992377d23f_isrForCrosstalkSources_997007_38.stderr parsl.dataflow.dflow DEBUG: Task 670 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 670 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 670 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 670 has memoization hash 3fe828176a1d941603dbcde347072599 parsl.dataflow.memoization INFO: Task 670 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 670 try 0 launched on executor multi with executor id 671 parsl.dataflow.dflow INFO: Standard output for task 670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7bae6981-4dd1-46b2-ae8f-f140a500dad8_isrForCrosstalkSources_996989_59.stdout parsl.dataflow.dflow INFO: Standard error for task 670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7bae6981-4dd1-46b2-ae8f-f140a500dad8_isrForCrosstalkSources_996989_59.stderr parsl.dataflow.dflow DEBUG: Task 671 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 671 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 671 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 671 has memoization hash 952ecc04faeaa7b8d26d5a2c57321ab9 parsl.dataflow.memoization INFO: Task 671 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 671 try 0 launched on executor multi with executor id 672 parsl.dataflow.dflow INFO: Standard output for task 671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d9ab12b3-225f-48ab-82dc-795bab87cb07_isrForCrosstalkSources_997003_62.stdout parsl.dataflow.dflow INFO: Standard error for task 671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d9ab12b3-225f-48ab-82dc-795bab87cb07_isrForCrosstalkSources_997003_62.stderr parsl.dataflow.dflow DEBUG: Task 672 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 672 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 672 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 672 has memoization hash e5dadbd677556af4350d654ee85a9766 parsl.dataflow.memoization INFO: Task 672 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 672 try 0 launched on executor multi with executor id 673 parsl.dataflow.dflow INFO: Standard output for task 672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/aebd6608-5238-4177-85b1-985460e884b6_isrForCrosstalkSources_996963_48.stdout parsl.dataflow.dflow INFO: Standard error for task 672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/aebd6608-5238-4177-85b1-985460e884b6_isrForCrosstalkSources_996963_48.stderr parsl.dataflow.dflow DEBUG: Task 673 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 673 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 673 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 673 has memoization hash e9afbe21cdac247b803c2b7cd163f8ea parsl.dataflow.memoization INFO: Task 673 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 673 try 0 launched on executor multi with executor id 674 parsl.dataflow.dflow INFO: Standard output for task 673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/bcc5f387-e13c-4c42-a62b-78ce755cae5c_isrForCrosstalkSources_996975_21.stdout parsl.dataflow.dflow INFO: Standard error for task 673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/bcc5f387-e13c-4c42-a62b-78ce755cae5c_isrForCrosstalkSources_996975_21.stderr parsl.dataflow.dflow DEBUG: Task 674 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 674 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 674 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 674 has memoization hash 65369756c46d644eafbacbf0d498770d parsl.dataflow.memoization INFO: Task 674 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 674 try 0 launched on executor multi with executor id 675 parsl.dataflow.dflow INFO: Standard output for task 674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/e82e3242-8816-4939-a078-88a670954699_isrForCrosstalkSources_997004_13.stdout parsl.dataflow.dflow INFO: Standard error for task 674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/e82e3242-8816-4939-a078-88a670954699_isrForCrosstalkSources_997004_13.stderr parsl.dataflow.dflow DEBUG: Task 675 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 675 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 675 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 675 has memoization hash 67bf639537160db410a4a762fec6ee9b parsl.dataflow.memoization INFO: Task 675 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 675 try 0 launched on executor multi with executor id 676 parsl.dataflow.dflow INFO: Standard output for task 675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d8a601f9-8c34-4515-87fa-1769fc1ee7c6_isrForCrosstalkSources_997000_5.stdout parsl.dataflow.dflow INFO: Standard error for task 675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d8a601f9-8c34-4515-87fa-1769fc1ee7c6_isrForCrosstalkSources_997000_5.stderr parsl.dataflow.dflow DEBUG: Task 676 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 676 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 676 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 676 has memoization hash 4e3148023ba867422c347c42eb442c04 parsl.dataflow.memoization INFO: Task 676 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 676 try 0 launched on executor multi with executor id 677 parsl.dataflow.dflow INFO: Standard output for task 676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/8337531e-fcc7-4703-9f89-71838b0f5877_isrForCrosstalkSources_996959_48.stdout parsl.dataflow.dflow INFO: Standard error for task 676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/8337531e-fcc7-4703-9f89-71838b0f5877_isrForCrosstalkSources_996959_48.stderr parsl.dataflow.dflow DEBUG: Task 677 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 677 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 677 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 677 has memoization hash 7a1070ca590f816aacde4adec7748fb3 parsl.dataflow.memoization INFO: Task 677 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 677 try 0 launched on executor multi with executor id 678 parsl.dataflow.dflow INFO: Standard output for task 677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ee3fd40f-a58c-4c0c-8c21-1a5fbe9b301c_isrForCrosstalkSources_996991_48.stdout parsl.dataflow.dflow INFO: Standard error for task 677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ee3fd40f-a58c-4c0c-8c21-1a5fbe9b301c_isrForCrosstalkSources_996991_48.stderr parsl.dataflow.dflow DEBUG: Task 678 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 678 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 678 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 678 has memoization hash 2f7e51198d398e39548649ec1c42b90b parsl.dataflow.memoization INFO: Task 678 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 678 try 0 launched on executor multi with executor id 679 parsl.dataflow.dflow INFO: Standard output for task 678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ee724f90-b04f-48d9-90bb-607a055649be_isrForCrosstalkSources_997002_35.stdout parsl.dataflow.dflow INFO: Standard error for task 678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ee724f90-b04f-48d9-90bb-607a055649be_isrForCrosstalkSources_997002_35.stderr parsl.dataflow.dflow DEBUG: Task 679 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 679 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 679 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 679 has memoization hash 311bec8e2a42e6be1a0d25ff3856c0a1 parsl.dataflow.memoization INFO: Task 679 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 679 try 0 launched on executor multi with executor id 680 parsl.dataflow.dflow INFO: Standard output for task 679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7a8cac0a-8d43-4a9f-b4d2-0c16e8d03bad_isrForCrosstalkSources_996994_18.stdout parsl.dataflow.dflow INFO: Standard error for task 679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7a8cac0a-8d43-4a9f-b4d2-0c16e8d03bad_isrForCrosstalkSources_996994_18.stderr parsl.dataflow.dflow DEBUG: Task 680 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 680 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 680 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 680 has memoization hash 473c23e47551f26356a30b5e800e7365 parsl.dataflow.memoization INFO: Task 680 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 680 try 0 launched on executor multi with executor id 681 parsl.dataflow.dflow INFO: Standard output for task 680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/46d16f4a-e765-49c7-b273-0f4de994c281_isrForCrosstalkSources_996975_61.stdout parsl.dataflow.dflow INFO: Standard error for task 680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/46d16f4a-e765-49c7-b273-0f4de994c281_isrForCrosstalkSources_996975_61.stderr parsl.dataflow.dflow DEBUG: Task 681 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 681 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 681 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 681 has memoization hash a8b7afb1da7b938ae39055dda088c509 parsl.dataflow.memoization INFO: Task 681 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 681 try 0 launched on executor multi with executor id 682 parsl.dataflow.dflow INFO: Standard output for task 681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/91b49bd0-1c1a-4688-9cb4-e3c1ac025c74_isrForCrosstalkSources_996969_35.stdout parsl.dataflow.dflow INFO: Standard error for task 681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/91b49bd0-1c1a-4688-9cb4-e3c1ac025c74_isrForCrosstalkSources_996969_35.stderr parsl.dataflow.dflow DEBUG: Task 682 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 682 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 682 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 682 has memoization hash 193373f5e4d3975e4334c889296585dc parsl.dataflow.memoization INFO: Task 682 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 682 try 0 launched on executor multi with executor id 683 parsl.dataflow.dflow INFO: Standard output for task 682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c7d2c6cd-c324-44dc-bebc-4844f7a669ca_isrForCrosstalkSources_997000_36.stdout parsl.dataflow.dflow INFO: Standard error for task 682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c7d2c6cd-c324-44dc-bebc-4844f7a669ca_isrForCrosstalkSources_997000_36.stderr parsl.dataflow.dflow DEBUG: Task 683 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 683 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 683 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 683 has memoization hash c916bf80bc92f9740a1e8aff6a55e8c3 parsl.dataflow.memoization INFO: Task 683 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 683 try 0 launched on executor multi with executor id 684 parsl.dataflow.dflow INFO: Standard output for task 683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/e076199e-7953-4027-aa42-284101a28f71_isrForCrosstalkSources_996980_47.stdout parsl.dataflow.dflow INFO: Standard error for task 683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/e076199e-7953-4027-aa42-284101a28f71_isrForCrosstalkSources_996980_47.stderr parsl.dataflow.dflow DEBUG: Task 684 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 684 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 684 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 684 has memoization hash abb51fad989b4c89e64aa8e7decefc89 parsl.dataflow.memoization INFO: Task 684 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 684 try 0 launched on executor multi with executor id 685 parsl.dataflow.dflow INFO: Standard output for task 684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/301f955c-1ad0-4452-8452-8429c2bf6e48_isrForCrosstalkSources_996996_53.stdout parsl.dataflow.dflow INFO: Standard error for task 684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/301f955c-1ad0-4452-8452-8429c2bf6e48_isrForCrosstalkSources_996996_53.stderr parsl.dataflow.dflow DEBUG: Task 685 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 685 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 685 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 685 has memoization hash 4d80d18797a1bf92951d78fc59af4a01 parsl.dataflow.memoization INFO: Task 685 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 685 try 0 launched on executor multi with executor id 686 parsl.dataflow.dflow INFO: Standard output for task 685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1910f109-f449-430c-8708-256f8583f883_isrForCrosstalkSources_997014_53.stdout parsl.dataflow.dflow INFO: Standard error for task 685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1910f109-f449-430c-8708-256f8583f883_isrForCrosstalkSources_997014_53.stderr parsl.dataflow.dflow DEBUG: Task 686 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 686 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 686 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 686 has memoization hash 39207d59dcd0a2a209a3e73fc2ab4069 parsl.dataflow.memoization INFO: Task 686 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 686 try 0 launched on executor multi with executor id 687 parsl.dataflow.dflow INFO: Standard output for task 686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/44f2089c-212b-4c0c-adc7-e08f2a2db08c_isrForCrosstalkSources_997029_18.stdout parsl.dataflow.dflow INFO: Standard error for task 686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/44f2089c-212b-4c0c-adc7-e08f2a2db08c_isrForCrosstalkSources_997029_18.stderr parsl.dataflow.dflow DEBUG: Task 687 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 687 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 687 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 687 has memoization hash ed8085fa4b981dfa53a5fb1ae2cdb727 parsl.dataflow.memoization INFO: Task 687 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 687 try 0 launched on executor multi with executor id 688 parsl.dataflow.dflow INFO: Standard output for task 687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c9ec783e-9c1d-4923-8149-72b0afe0c7b4_isrForCrosstalkSources_997007_26.stdout parsl.dataflow.dflow INFO: Standard error for task 687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c9ec783e-9c1d-4923-8149-72b0afe0c7b4_isrForCrosstalkSources_997007_26.stderr parsl.dataflow.dflow DEBUG: Task 688 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 688 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 688 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 688 has memoization hash cae26caf78b3386b452c95991b913e50 parsl.dataflow.memoization INFO: Task 688 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:29 proc_lsst.multi:170 [INFO] [multi] job_id 17074839 proc_lsst.multi INFO: [multi] job_id 17074839 2024-03-15 13:49:29 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 7 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 7 2024-03-15 13:49:29 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:29 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074839 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074839 parsl.executors.status_handling DEBUG: Launched block 7 on executor multi with job ID 17074839 parsl.executors.status_handling INFO: Allocated block ID 8 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-8 2024-03-15 13:49:29 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=8 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-8 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=8 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-8 2024-03-15 13:49:29 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=8 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=8 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:29 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=8 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=8 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 7 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 7 2024-03-15 13:49:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:29 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=8 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=8 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 688 try 0 launched on executor multi with executor id 689 parsl.dataflow.dflow INFO: Standard output for task 688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/948df5a6-0cca-4e69-aa99-633e928a1da2_isrForCrosstalkSources_997031_26.stdout parsl.dataflow.dflow INFO: Standard error for task 688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/948df5a6-0cca-4e69-aa99-633e928a1da2_isrForCrosstalkSources_997031_26.stderr parsl.dataflow.dflow DEBUG: Task 689 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 689 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 689 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 689 has memoization hash 636ae4ca63bfa6b93286a6b4e43e5a13 parsl.dataflow.memoization INFO: Task 689 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 689 try 0 launched on executor multi with executor id 690 parsl.dataflow.dflow INFO: Standard output for task 689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/06cb429b-3ebd-4ec0-b38d-e3deca191b0f_isrForCrosstalkSources_996958_29.stdout parsl.dataflow.dflow INFO: Standard error for task 689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/06cb429b-3ebd-4ec0-b38d-e3deca191b0f_isrForCrosstalkSources_996958_29.stderr parsl.dataflow.dflow DEBUG: Task 690 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 690 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 690 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 690 has memoization hash 255b5f1254b5c872e678b31d91999285 parsl.dataflow.memoization INFO: Task 690 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 690 try 0 launched on executor multi with executor id 691 parsl.dataflow.dflow INFO: Standard output for task 690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/3baab248-11b4-4c04-8cfe-97d639c9b0e7_isrForCrosstalkSources_997026_61.stdout parsl.dataflow.dflow INFO: Standard error for task 690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/3baab248-11b4-4c04-8cfe-97d639c9b0e7_isrForCrosstalkSources_997026_61.stderr parsl.dataflow.dflow DEBUG: Task 691 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 691 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 691 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 691 has memoization hash 3f09c13fe2d87f0dd5a5129eee578079 parsl.dataflow.memoization INFO: Task 691 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bb060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 691 try 0 launched on executor multi with executor id 692 parsl.dataflow.dflow INFO: Standard output for task 691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/34f7622f-9021-4910-8bfd-60bdfc40c88d_isrForCrosstalkSources_997000_15.stdout parsl.dataflow.dflow INFO: Standard error for task 691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/34f7622f-9021-4910-8bfd-60bdfc40c88d_isrForCrosstalkSources_997000_15.stderr parsl.dataflow.dflow DEBUG: Task 692 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 692 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 692 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 692 has memoization hash 6c1e257b4fb98b34eb021d7cf0d9fa65 parsl.dataflow.memoization INFO: Task 692 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b9080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 692 try 0 launched on executor multi with executor id 693 parsl.dataflow.dflow INFO: Standard output for task 692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4e47a46d-9d31-455c-bc5c-ea330a06e626_isrForCrosstalkSources_997023_57.stdout parsl.dataflow.dflow INFO: Standard error for task 692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4e47a46d-9d31-455c-bc5c-ea330a06e626_isrForCrosstalkSources_997023_57.stderr parsl.dataflow.dflow DEBUG: Task 693 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 693 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 693 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 693 has memoization hash e2e501b045646e1e32b667877796036b parsl.dataflow.memoization INFO: Task 693 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 693 try 0 launched on executor multi with executor id 694 parsl.dataflow.dflow INFO: Standard output for task 693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/a6a87cc7-7487-4226-8326-9740b0777fc4_isrForCrosstalkSources_996998_21.stdout parsl.dataflow.dflow INFO: Standard error for task 693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/a6a87cc7-7487-4226-8326-9740b0777fc4_isrForCrosstalkSources_996998_21.stderr parsl.dataflow.dflow DEBUG: Task 694 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 694 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 694 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 694 has memoization hash 5a116205c97be8bfab1626660ca5a5d7 parsl.dataflow.memoization INFO: Task 694 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 694 try 0 launched on executor multi with executor id 695 parsl.dataflow.dflow INFO: Standard output for task 694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/bf79eb25-1944-4d8e-867e-d6b1e6862488_isrForCrosstalkSources_996970_13.stdout parsl.dataflow.dflow INFO: Standard error for task 694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/bf79eb25-1944-4d8e-867e-d6b1e6862488_isrForCrosstalkSources_996970_13.stderr parsl.dataflow.dflow DEBUG: Task 695 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 695 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 695 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 695 has memoization hash ddc89f6d8a3f6b03933231fa378053b7 parsl.dataflow.memoization INFO: Task 695 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 695 try 0 launched on executor multi with executor id 696 parsl.dataflow.dflow INFO: Standard output for task 695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/8778cd7c-c41b-42ec-a626-e4da331bf9ab_isrForCrosstalkSources_997028_19.stdout parsl.dataflow.dflow INFO: Standard error for task 695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/8778cd7c-c41b-42ec-a626-e4da331bf9ab_isrForCrosstalkSources_997028_19.stderr parsl.dataflow.dflow DEBUG: Task 696 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 696 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 696 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 696 has memoization hash f858753a92ec661f0e0c2edf0ac1b208 parsl.dataflow.memoization INFO: Task 696 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 696 try 0 launched on executor multi with executor id 697 parsl.dataflow.dflow INFO: Standard output for task 696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/af962c92-5307-4ad8-a3e8-be6425543a47_isrForCrosstalkSources_997005_48.stdout parsl.dataflow.dflow INFO: Standard error for task 696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/af962c92-5307-4ad8-a3e8-be6425543a47_isrForCrosstalkSources_997005_48.stderr parsl.dataflow.dflow DEBUG: Task 697 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 697 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 697 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 697 has memoization hash 6ba4216315fd71e93c773bc9752affde parsl.dataflow.memoization INFO: Task 697 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 697 try 0 launched on executor multi with executor id 698 parsl.dataflow.dflow INFO: Standard output for task 697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/8749896f-1486-456f-b290-f20bfdd265a1_isrForCrosstalkSources_996958_2.stdout parsl.dataflow.dflow INFO: Standard error for task 697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/8749896f-1486-456f-b290-f20bfdd265a1_isrForCrosstalkSources_996958_2.stderr parsl.dataflow.dflow DEBUG: Task 698 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 698 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 698 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 698 has memoization hash 51fb4ef1b1bae807384c04cc6f5f192d parsl.dataflow.memoization INFO: Task 698 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806baac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 698 try 0 launched on executor multi with executor id 699 parsl.dataflow.dflow INFO: Standard output for task 698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f43cd18-1e97-4e33-86bc-847f52f59f85_isrForCrosstalkSources_996994_13.stdout parsl.dataflow.dflow INFO: Standard error for task 698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f43cd18-1e97-4e33-86bc-847f52f59f85_isrForCrosstalkSources_996994_13.stderr parsl.dataflow.dflow DEBUG: Task 699 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 699 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 699 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 699 has memoization hash a6fa4c1a250c443b3fd2bc754cbf2540 parsl.dataflow.memoization INFO: Task 699 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806baa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 699 try 0 launched on executor multi with executor id 700 parsl.dataflow.dflow INFO: Standard output for task 699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7beca4d5-7630-4f98-8ba7-51f910f11463_isrForCrosstalkSources_997003_53.stdout parsl.dataflow.dflow INFO: Standard error for task 699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7beca4d5-7630-4f98-8ba7-51f910f11463_isrForCrosstalkSources_997003_53.stderr parsl.dataflow.dflow DEBUG: Task 700 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 700 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 700 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 700 has memoization hash 0bbf2f6450305eff7d9460338d6e9b58 parsl.dataflow.memoization INFO: Task 700 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806baca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 700 try 0 launched on executor multi with executor id 701 parsl.dataflow.dflow INFO: Standard output for task 700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/56e10972-8a3f-4942-8e31-915507104720_isrForCrosstalkSources_996961_34.stdout parsl.dataflow.dflow INFO: Standard error for task 700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/56e10972-8a3f-4942-8e31-915507104720_isrForCrosstalkSources_996961_34.stderr parsl.dataflow.dflow DEBUG: Task 701 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 701 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 701 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 701 has memoization hash 413fb8d6ac5c30711be30278d26b5688 parsl.dataflow.memoization INFO: Task 701 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 701 try 0 launched on executor multi with executor id 702 parsl.dataflow.dflow INFO: Standard output for task 701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c988af9d-09d5-40dc-b511-3d942e320a37_isrForCrosstalkSources_996960_13.stdout parsl.dataflow.dflow INFO: Standard error for task 701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c988af9d-09d5-40dc-b511-3d942e320a37_isrForCrosstalkSources_996960_13.stderr parsl.dataflow.dflow DEBUG: Task 702 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 702 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 702 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 702 has memoization hash ba82a2c1ce0c3f1191f9fe3691d19318 parsl.dataflow.memoization INFO: Task 702 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806ba840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 702 try 0 launched on executor multi with executor id 703 parsl.dataflow.dflow INFO: Standard output for task 702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/20b77816-2e70-479c-ab74-170471c55479_isrForCrosstalkSources_996985_49.stdout parsl.dataflow.dflow INFO: Standard error for task 702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/20b77816-2e70-479c-ab74-170471c55479_isrForCrosstalkSources_996985_49.stderr parsl.dataflow.dflow DEBUG: Task 703 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 703 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 703 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 703 has memoization hash 729cbb7f4007ad4f97868003e790b442 parsl.dataflow.memoization INFO: Task 703 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bbe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 703 try 0 launched on executor multi with executor id 704 parsl.dataflow.dflow INFO: Standard output for task 703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c80a20e5-e462-4054-85b7-bf5a6cf48e93_isrForCrosstalkSources_997021_14.stdout parsl.dataflow.dflow INFO: Standard error for task 703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c80a20e5-e462-4054-85b7-bf5a6cf48e93_isrForCrosstalkSources_997021_14.stderr parsl.dataflow.dflow DEBUG: Task 704 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 704 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 704 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 704 has memoization hash 4c32e99e0305773795959fab196cd94d parsl.dataflow.memoization INFO: Task 704 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bbd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 704 try 0 launched on executor multi with executor id 705 parsl.dataflow.dflow INFO: Standard output for task 704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1c9296de-c7cb-46d9-addf-d5c6eb35dab0_isrForCrosstalkSources_996998_14.stdout parsl.dataflow.dflow INFO: Standard error for task 704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1c9296de-c7cb-46d9-addf-d5c6eb35dab0_isrForCrosstalkSources_996998_14.stderr parsl.dataflow.dflow DEBUG: Task 705 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 705 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 705 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 705 has memoization hash 6d9c73c05e0985809a2d54c3dca3c154 parsl.dataflow.memoization INFO: Task 705 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bbec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 705 try 0 launched on executor multi with executor id 706 parsl.dataflow.dflow INFO: Standard output for task 705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b0ca83a8-b352-4c00-a177-c1d69eb2315b_isrForCrosstalkSources_997031_34.stdout parsl.dataflow.dflow INFO: Standard error for task 705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b0ca83a8-b352-4c00-a177-c1d69eb2315b_isrForCrosstalkSources_997031_34.stderr parsl.dataflow.dflow DEBUG: Task 706 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 706 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 706 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 706 has memoization hash 65017617e42b4e115ebdf3dd9662b662 parsl.dataflow.memoization INFO: Task 706 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bbf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 706 try 0 launched on executor multi with executor id 707 parsl.dataflow.dflow INFO: Standard output for task 706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/446a01de-1032-4867-93f9-915b4a804e41_isrForCrosstalkSources_997001_62.stdout parsl.dataflow.dflow INFO: Standard error for task 706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/446a01de-1032-4867-93f9-915b4a804e41_isrForCrosstalkSources_997001_62.stderr parsl.dataflow.dflow DEBUG: Task 707 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 707 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 707 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 707 has memoization hash b9cd963c6f2aea8bf4b04623d1ad6e0d parsl.dataflow.memoization INFO: Task 707 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bafc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 707 try 0 launched on executor multi with executor id 708 parsl.dataflow.dflow INFO: Standard output for task 707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/366d183b-c079-47e9-9d03-a0646e91523e_isrForCrosstalkSources_996953_56.stdout parsl.dataflow.dflow INFO: Standard error for task 707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/366d183b-c079-47e9-9d03-a0646e91523e_isrForCrosstalkSources_996953_56.stderr parsl.dataflow.dflow DEBUG: Task 708 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 708 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 708 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 708 has memoization hash fba6e8e7562bc7022a28d2024830575d parsl.dataflow.memoization INFO: Task 708 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 708 try 0 launched on executor multi with executor id 709 parsl.dataflow.dflow INFO: Standard output for task 708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/179bce96-e714-4bde-aebd-05871fde32a2_isrForCrosstalkSources_996959_41.stdout parsl.dataflow.dflow INFO: Standard error for task 708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/179bce96-e714-4bde-aebd-05871fde32a2_isrForCrosstalkSources_996959_41.stderr parsl.dataflow.dflow DEBUG: Task 709 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 709 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 709 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 709 has memoization hash f8861b5f6dc07ee4f862b49176526611 parsl.dataflow.memoization INFO: Task 709 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 709 try 0 launched on executor multi with executor id 710 parsl.dataflow.dflow INFO: Standard output for task 709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7665e6fe-9f69-491f-a215-99f4aa00c887_isrForCrosstalkSources_996989_48.stdout parsl.dataflow.dflow INFO: Standard error for task 709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7665e6fe-9f69-491f-a215-99f4aa00c887_isrForCrosstalkSources_996989_48.stderr parsl.dataflow.dflow DEBUG: Task 710 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 710 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 710 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 710 has memoization hash dfb4b01563a3208a222150c44b53a715 parsl.dataflow.memoization INFO: Task 710 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808300e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 710 try 0 launched on executor multi with executor id 711 parsl.dataflow.dflow INFO: Standard output for task 710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/485ff063-438d-4298-b666-a31276a97ce6_isrForCrosstalkSources_997010_20.stdout parsl.dataflow.dflow INFO: Standard error for task 710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/485ff063-438d-4298-b666-a31276a97ce6_isrForCrosstalkSources_997010_20.stderr parsl.dataflow.dflow DEBUG: Task 711 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 711 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 711 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 711 has memoization hash 7580fdc2b9fd7450001d408a81fe7e79 parsl.dataflow.memoization INFO: Task 711 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 711 try 0 launched on executor multi with executor id 712 parsl.dataflow.dflow INFO: Standard output for task 711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f1119573-4f98-40c0-89c1-8b337fdc0b8d_isrForCrosstalkSources_997032_37.stdout parsl.dataflow.dflow INFO: Standard error for task 711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f1119573-4f98-40c0-89c1-8b337fdc0b8d_isrForCrosstalkSources_997032_37.stderr parsl.dataflow.dflow DEBUG: Task 712 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 712 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 712 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 712 has memoization hash fe0ecce70094d9b7ec24559b2fbaf977 parsl.dataflow.memoization INFO: Task 712 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808302c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 712 try 0 launched on executor multi with executor id 713 parsl.dataflow.dflow INFO: Standard output for task 712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/3f436e99-cc32-45f3-9419-113603cc6227_isrForCrosstalkSources_996964_52.stdout parsl.dataflow.dflow INFO: Standard error for task 712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/3f436e99-cc32-45f3-9419-113603cc6227_isrForCrosstalkSources_996964_52.stderr parsl.dataflow.dflow DEBUG: Task 713 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 713 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 713 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 713 has memoization hash 2cc78bf8ee0bd148a4614aa5772a2516 parsl.dataflow.memoization INFO: Task 713 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 713 try 0 launched on executor multi with executor id 714 parsl.dataflow.dflow INFO: Standard output for task 713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7fa03746-907b-40fa-b67e-7c6618b809ee_isrForCrosstalkSources_997021_10.stdout parsl.dataflow.dflow INFO: Standard error for task 713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7fa03746-907b-40fa-b67e-7c6618b809ee_isrForCrosstalkSources_997021_10.stderr parsl.dataflow.dflow DEBUG: Task 714 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 714 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 714 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 714 has memoization hash ac661ad56c269fa6ac759a05723a6e24 parsl.dataflow.memoization INFO: Task 714 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808304a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 714 try 0 launched on executor multi with executor id 715 parsl.dataflow.dflow INFO: Standard output for task 714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/19e1b954-1026-4ba2-89a5-c0931ca434cf_isrForCrosstalkSources_997012_54.stdout parsl.dataflow.dflow INFO: Standard error for task 714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/19e1b954-1026-4ba2-89a5-c0931ca434cf_isrForCrosstalkSources_997012_54.stderr parsl.dataflow.dflow DEBUG: Task 715 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 715 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 715 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 715 has memoization hash 133b89e2ea80e2aa4107720cbe59d8a7 parsl.dataflow.memoization INFO: Task 715 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 715 try 0 launched on executor multi with executor id 716 parsl.dataflow.dflow INFO: Standard output for task 715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7dae67b8-eac3-4394-bd80-13227a1f7526_isrForCrosstalkSources_996993_36.stdout parsl.dataflow.dflow INFO: Standard error for task 715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7dae67b8-eac3-4394-bd80-13227a1f7526_isrForCrosstalkSources_996993_36.stderr parsl.dataflow.dflow DEBUG: Task 716 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 716 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 716 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 716 has memoization hash 8d894a61eb609554949179b0c5981ad6 parsl.dataflow.memoization INFO: Task 716 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 716 try 0 launched on executor multi with executor id 717 parsl.dataflow.dflow INFO: Standard output for task 716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/15bae4ec-4f64-4f05-850e-37b0370e99cf_isrForCrosstalkSources_996954_21.stdout parsl.dataflow.dflow INFO: Standard error for task 716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/15bae4ec-4f64-4f05-850e-37b0370e99cf_isrForCrosstalkSources_996954_21.stderr parsl.dataflow.dflow DEBUG: Task 717 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 717 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 717 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 717 has memoization hash bebeb8625a066ed3e3e76cc09bc1a1fa parsl.dataflow.memoization INFO: Task 717 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808305e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 717 try 0 launched on executor multi with executor id 718 parsl.dataflow.dflow INFO: Standard output for task 717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2d15abdd-0034-4c2f-a38b-a03a5c65b083_isrForCrosstalkSources_997006_51.stdout parsl.dataflow.dflow INFO: Standard error for task 717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2d15abdd-0034-4c2f-a38b-a03a5c65b083_isrForCrosstalkSources_997006_51.stderr parsl.dataflow.dflow DEBUG: Task 718 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 718 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 718 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 718 has memoization hash b06bc667de5ce66853ae1ea6707c5e0f parsl.dataflow.memoization INFO: Task 718 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 718 try 0 launched on executor multi with executor id 719 parsl.dataflow.dflow INFO: Standard output for task 718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/8bf7653a-b6e5-4ebc-91bc-3b4a71629c1e_isrForCrosstalkSources_997010_44.stdout parsl.dataflow.dflow INFO: Standard error for task 718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/8bf7653a-b6e5-4ebc-91bc-3b4a71629c1e_isrForCrosstalkSources_997010_44.stderr parsl.dataflow.dflow DEBUG: Task 719 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 719 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 719 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 719 has memoization hash 599b249eac47d6cb0879004ad577d711 parsl.dataflow.memoization INFO: Task 719 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808307c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 719 try 0 launched on executor multi with executor id 720 parsl.dataflow.dflow INFO: Standard output for task 719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b227bab8-5490-44de-a88f-54601b12eb04_isrForCrosstalkSources_997016_16.stdout parsl.dataflow.dflow INFO: Standard error for task 719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b227bab8-5490-44de-a88f-54601b12eb04_isrForCrosstalkSources_997016_16.stderr parsl.dataflow.dflow DEBUG: Task 720 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 720 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 720 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 720 has memoization hash 53d19db1e1c87a3adeb4c0d2146f903a parsl.dataflow.memoization INFO: Task 720 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 720 try 0 launched on executor multi with executor id 721 parsl.dataflow.dflow INFO: Standard output for task 720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/29679dab-e3a6-499c-aace-b1226520c540_isrForCrosstalkSources_997030_35.stdout parsl.dataflow.dflow INFO: Standard error for task 720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/29679dab-e3a6-499c-aace-b1226520c540_isrForCrosstalkSources_997030_35.stderr parsl.dataflow.dflow DEBUG: Task 721 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 721 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 721 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 721 has memoization hash 8a8927363430d74e86387edf26b40273 parsl.dataflow.memoization INFO: Task 721 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 721 try 0 launched on executor multi with executor id 722 parsl.dataflow.dflow INFO: Standard output for task 721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d1250c3f-a0ee-427f-8267-ad545d9f7e00_isrForCrosstalkSources_996980_26.stdout parsl.dataflow.dflow INFO: Standard error for task 721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d1250c3f-a0ee-427f-8267-ad545d9f7e00_isrForCrosstalkSources_996980_26.stderr parsl.dataflow.dflow DEBUG: Task 722 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 722 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 722 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 722 has memoization hash 7c59df0c2479d4d6c7fbf0627b89e2a4 parsl.dataflow.memoization INFO: Task 722 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 722 try 0 launched on executor multi with executor id 723 parsl.dataflow.dflow INFO: Standard output for task 722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/775a802d-a535-4ace-89ef-777968dabb9e_isrForCrosstalkSources_997007_22.stdout parsl.dataflow.dflow INFO: Standard error for task 722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/775a802d-a535-4ace-89ef-777968dabb9e_isrForCrosstalkSources_997007_22.stderr parsl.dataflow.dflow DEBUG: Task 723 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 723 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 723 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 723 has memoization hash 80fbe28f4343667c5369b5c1f617393a parsl.dataflow.memoization INFO: Task 723 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808309a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 723 try 0 launched on executor multi with executor id 724 parsl.dataflow.dflow INFO: Standard output for task 723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/cd911220-a76b-48ce-b002-d20e91cb4b8c_isrForCrosstalkSources_996972_24.stdout parsl.dataflow.dflow INFO: Standard error for task 723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/cd911220-a76b-48ce-b002-d20e91cb4b8c_isrForCrosstalkSources_996972_24.stderr parsl.dataflow.dflow DEBUG: Task 724 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 724 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 724 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 724 has memoization hash 4ccd37ea0643cd81ded0574039c045b5 parsl.dataflow.memoization INFO: Task 724 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 724 try 0 launched on executor multi with executor id 725 parsl.dataflow.dflow INFO: Standard output for task 724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cddaddde-7eb0-400e-a7b5-5d0a8972918c_isrForCrosstalkSources_996957_45.stdout parsl.dataflow.dflow INFO: Standard error for task 724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cddaddde-7eb0-400e-a7b5-5d0a8972918c_isrForCrosstalkSources_996957_45.stderr parsl.dataflow.dflow DEBUG: Task 725 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 725 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 725 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 725 has memoization hash b7747b46c8932aff0cf63f027ff5fbda parsl.dataflow.memoization INFO: Task 725 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 725 try 0 launched on executor multi with executor id 726 parsl.dataflow.dflow INFO: Standard output for task 725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/558e0422-ac27-459d-8453-11807c82e534_isrForCrosstalkSources_996987_48.stdout parsl.dataflow.dflow INFO: Standard error for task 725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/558e0422-ac27-459d-8453-11807c82e534_isrForCrosstalkSources_996987_48.stderr parsl.dataflow.dflow DEBUG: Task 726 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 726 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 726 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 726 has memoization hash 9490b7656e6406cba3e11e637c09d460 parsl.dataflow.memoization INFO: Task 726 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 726 try 0 launched on executor multi with executor id 727 parsl.dataflow.dflow INFO: Standard output for task 726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/38a91b94-9359-4b47-9379-edfc1b53b7f7_isrForCrosstalkSources_996976_30.stdout parsl.dataflow.dflow INFO: Standard error for task 726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/38a91b94-9359-4b47-9379-edfc1b53b7f7_isrForCrosstalkSources_996976_30.stderr parsl.dataflow.dflow DEBUG: Task 727 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 727 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 727 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 727 has memoization hash fbb249bfb1f7eb085a01215864e31b0d parsl.dataflow.memoization INFO: Task 727 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 727 try 0 launched on executor multi with executor id 728 parsl.dataflow.dflow INFO: Standard output for task 727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d6c5af40-86e5-48d4-a43b-44de9337cb8c_isrForCrosstalkSources_996990_55.stdout parsl.dataflow.dflow INFO: Standard error for task 727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d6c5af40-86e5-48d4-a43b-44de9337cb8c_isrForCrosstalkSources_996990_55.stderr parsl.dataflow.dflow DEBUG: Task 728 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 728 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 728 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 728 has memoization hash 96a06b8967f68bee6d4b8155b7d74225 parsl.dataflow.memoization INFO: Task 728 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:30 proc_lsst.multi:170 [INFO] [multi] job_id 17074841 proc_lsst.multi INFO: [multi] job_id 17074841 2024-03-15 13:49:30 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 8 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 8 2024-03-15 13:49:30 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:30 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074841 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074841 parsl.executors.status_handling DEBUG: Launched block 8 on executor multi with job ID 17074841 parsl.executors.status_handling INFO: Allocated block ID 9 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-9 2024-03-15 13:49:30 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=9 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-9 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=9 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-9 2024-03-15 13:49:30 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=9 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=9 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:30 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=9 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=9 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 8 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 8 2024-03-15 13:49:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:30 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=9 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=9 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 728 try 0 launched on executor multi with executor id 729 parsl.dataflow.dflow INFO: Standard output for task 728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e14f11fa-5694-42e0-807a-7ad4a7047b2f_isrForCrosstalkSources_996968_53.stdout parsl.dataflow.dflow INFO: Standard error for task 728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e14f11fa-5694-42e0-807a-7ad4a7047b2f_isrForCrosstalkSources_996968_53.stderr parsl.dataflow.dflow DEBUG: Task 729 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 729 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 729 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 729 has memoization hash e0805b0fadba4cd19315d64f382c6bec parsl.dataflow.memoization INFO: Task 729 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 729 try 0 launched on executor multi with executor id 730 parsl.dataflow.dflow INFO: Standard output for task 729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0fda6158-4a83-46ac-a7e6-149dad795d06_isrForCrosstalkSources_996958_39.stdout parsl.dataflow.dflow INFO: Standard error for task 729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0fda6158-4a83-46ac-a7e6-149dad795d06_isrForCrosstalkSources_996958_39.stderr parsl.dataflow.dflow DEBUG: Task 730 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 730 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 730 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 730 has memoization hash dee687fe481e088c58e0e7e9637decb7 parsl.dataflow.memoization INFO: Task 730 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 730 try 0 launched on executor multi with executor id 731 parsl.dataflow.dflow INFO: Standard output for task 730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/a58016b5-65fd-4938-be47-3872357cc888_isrForCrosstalkSources_996966_17.stdout parsl.dataflow.dflow INFO: Standard error for task 730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/a58016b5-65fd-4938-be47-3872357cc888_isrForCrosstalkSources_996966_17.stderr parsl.dataflow.dflow DEBUG: Task 731 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 731 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 731 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 731 has memoization hash 214a9bddb05b121f98202555a695cc28 parsl.dataflow.memoization INFO: Task 731 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 731 try 0 launched on executor multi with executor id 732 parsl.dataflow.dflow INFO: Standard output for task 731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/2c305899-5c07-42ea-a995-7361ef1bfb20_isrForCrosstalkSources_996986_36.stdout parsl.dataflow.dflow INFO: Standard error for task 731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/2c305899-5c07-42ea-a995-7361ef1bfb20_isrForCrosstalkSources_996986_36.stderr parsl.dataflow.dflow DEBUG: Task 732 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 732 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 732 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 732 has memoization hash 1bcd6274a0f94fd105ab7aa44745b008 parsl.dataflow.memoization INFO: Task 732 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 732 try 0 launched on executor multi with executor id 733 parsl.dataflow.dflow INFO: Standard output for task 732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/96d318c1-a8b3-41e0-825a-44e9e7a7835d_isrForCrosstalkSources_997025_54.stdout parsl.dataflow.dflow INFO: Standard error for task 732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/96d318c1-a8b3-41e0-825a-44e9e7a7835d_isrForCrosstalkSources_997025_54.stderr parsl.dataflow.dflow DEBUG: Task 733 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 733 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 733 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 733 has memoization hash 81e3d94b82a19b8b7d04cd0eb43a898b parsl.dataflow.memoization INFO: Task 733 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 733 try 0 launched on executor multi with executor id 734 parsl.dataflow.dflow INFO: Standard output for task 733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/aebd9bac-c45d-4f37-9c0c-0b23144198a0_isrForCrosstalkSources_996966_43.stdout parsl.dataflow.dflow INFO: Standard error for task 733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/aebd9bac-c45d-4f37-9c0c-0b23144198a0_isrForCrosstalkSources_996966_43.stderr parsl.dataflow.dflow DEBUG: Task 734 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 734 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 734 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 734 has memoization hash 4231f9b013438081de8f13a3e68e12bb parsl.dataflow.memoization INFO: Task 734 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 734 try 0 launched on executor multi with executor id 735 parsl.dataflow.dflow INFO: Standard output for task 734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/ac360468-9c08-4fd6-9782-9fbbd6c213ba_isrForCrosstalkSources_997004_12.stdout parsl.dataflow.dflow INFO: Standard error for task 734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/ac360468-9c08-4fd6-9782-9fbbd6c213ba_isrForCrosstalkSources_997004_12.stderr parsl.dataflow.dflow DEBUG: Task 735 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 735 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 735 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 735 has memoization hash 86b079fd2e413411a400b45c534fd9ae parsl.dataflow.memoization INFO: Task 735 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 735 try 0 launched on executor multi with executor id 736 parsl.dataflow.dflow INFO: Standard output for task 735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/f571cc35-6735-49a6-9cb1-8a1bf9b54533_isrForCrosstalkSources_997010_62.stdout parsl.dataflow.dflow INFO: Standard error for task 735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/f571cc35-6735-49a6-9cb1-8a1bf9b54533_isrForCrosstalkSources_997010_62.stderr parsl.dataflow.dflow DEBUG: Task 736 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 736 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 736 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 736 has memoization hash 6a40757c23f84f3d1fa425c4ce099293 parsl.dataflow.memoization INFO: Task 736 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 736 try 0 launched on executor multi with executor id 737 parsl.dataflow.dflow INFO: Standard output for task 736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/fdf5b9fa-fd81-48ff-b021-38dbeeafca75_isrForCrosstalkSources_996985_38.stdout parsl.dataflow.dflow INFO: Standard error for task 736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/fdf5b9fa-fd81-48ff-b021-38dbeeafca75_isrForCrosstalkSources_996985_38.stderr parsl.dataflow.dflow DEBUG: Task 737 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 737 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 737 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 737 has memoization hash d16b0978efca7854548dfdd7de19c41c parsl.dataflow.memoization INFO: Task 737 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 737 try 0 launched on executor multi with executor id 738 parsl.dataflow.dflow INFO: Standard output for task 737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f52d235d-8fdd-4833-90e2-24b6844950dd_isrForCrosstalkSources_996960_26.stdout parsl.dataflow.dflow INFO: Standard error for task 737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f52d235d-8fdd-4833-90e2-24b6844950dd_isrForCrosstalkSources_996960_26.stderr parsl.dataflow.dflow DEBUG: Task 738 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 738 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 738 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 738 has memoization hash 63dceabf2f84787d865a51ac7c71b90b parsl.dataflow.memoization INFO: Task 738 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 738 try 0 launched on executor multi with executor id 739 parsl.dataflow.dflow INFO: Standard output for task 738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5cbcca52-ceed-481b-9518-59d3f2a7d1cb_isrForCrosstalkSources_997003_52.stdout parsl.dataflow.dflow INFO: Standard error for task 738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5cbcca52-ceed-481b-9518-59d3f2a7d1cb_isrForCrosstalkSources_997003_52.stderr parsl.dataflow.dflow DEBUG: Task 739 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 739 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 739 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 739 has memoization hash 82711a38d8624030a82f707f0dd61eed parsl.dataflow.memoization INFO: Task 739 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780830f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 739 try 0 launched on executor multi with executor id 740 parsl.dataflow.dflow INFO: Standard output for task 739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/71503f6d-5c98-4f17-a16e-0124f2b499e9_isrForCrosstalkSources_996986_6.stdout parsl.dataflow.dflow INFO: Standard error for task 739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/71503f6d-5c98-4f17-a16e-0124f2b499e9_isrForCrosstalkSources_996986_6.stderr parsl.dataflow.dflow DEBUG: Task 740 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 740 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 740 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 740 has memoization hash 293fb803235e2a298b3e6b2854e95595 parsl.dataflow.memoization INFO: Task 740 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 740 try 0 launched on executor multi with executor id 741 parsl.dataflow.dflow INFO: Standard output for task 740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4060b6cb-eb16-4c7e-851f-048a98dc2ec1_isrForCrosstalkSources_996957_4.stdout parsl.dataflow.dflow INFO: Standard error for task 740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4060b6cb-eb16-4c7e-851f-048a98dc2ec1_isrForCrosstalkSources_996957_4.stderr parsl.dataflow.dflow DEBUG: Task 741 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 741 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 741 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 741 has memoization hash 2b36d1d04669f81fbfcabd1d66e72283 parsl.dataflow.memoization INFO: Task 741 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808311c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 741 try 0 launched on executor multi with executor id 742 parsl.dataflow.dflow INFO: Standard output for task 741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/d0ca6ff1-adb2-4818-a641-6c975b13d1e3_isrForCrosstalkSources_997021_26.stdout parsl.dataflow.dflow INFO: Standard error for task 741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/d0ca6ff1-adb2-4818-a641-6c975b13d1e3_isrForCrosstalkSources_997021_26.stderr parsl.dataflow.dflow DEBUG: Task 742 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 742 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 742 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 742 has memoization hash 7a10241ba52dffe645b701433836ce7a parsl.dataflow.memoization INFO: Task 742 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 742 try 0 launched on executor multi with executor id 743 parsl.dataflow.dflow INFO: Standard output for task 742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2bf62534-e40a-4a13-be76-a6935a3e4234_isrForCrosstalkSources_997003_61.stdout parsl.dataflow.dflow INFO: Standard error for task 742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2bf62534-e40a-4a13-be76-a6935a3e4234_isrForCrosstalkSources_997003_61.stderr parsl.dataflow.dflow DEBUG: Task 743 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 743 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 743 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 743 has memoization hash a3a7feef812b69afe9f5277a84dba53a parsl.dataflow.memoization INFO: Task 743 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 743 try 0 launched on executor multi with executor id 744 parsl.dataflow.dflow INFO: Standard output for task 743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f3cf5636-4535-48a0-95c8-fa24184d4d9d_isrForCrosstalkSources_997032_29.stdout parsl.dataflow.dflow INFO: Standard error for task 743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f3cf5636-4535-48a0-95c8-fa24184d4d9d_isrForCrosstalkSources_997032_29.stderr parsl.dataflow.dflow DEBUG: Task 744 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 744 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 744 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 744 has memoization hash 9bee68cc0feb3b17383800e58374cdc2 parsl.dataflow.memoization INFO: Task 744 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808314e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 744 try 0 launched on executor multi with executor id 745 parsl.dataflow.dflow INFO: Standard output for task 744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b3e85931-9c94-4708-8fbc-20b136d4da70_isrForCrosstalkSources_996972_50.stdout parsl.dataflow.dflow INFO: Standard error for task 744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b3e85931-9c94-4708-8fbc-20b136d4da70_isrForCrosstalkSources_996972_50.stderr parsl.dataflow.dflow DEBUG: Task 745 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 745 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 745 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 745 has memoization hash 771b0089ce83b0678771e5dd2daf2b9c parsl.dataflow.memoization INFO: Task 745 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 745 try 0 launched on executor multi with executor id 746 parsl.dataflow.dflow INFO: Standard output for task 745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/d2e3b29f-0ba3-43e9-8169-72ff63dcf368_isrForCrosstalkSources_997018_36.stdout parsl.dataflow.dflow INFO: Standard error for task 745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/d2e3b29f-0ba3-43e9-8169-72ff63dcf368_isrForCrosstalkSources_997018_36.stderr parsl.dataflow.dflow DEBUG: Task 746 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 746 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 746 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 746 has memoization hash 0e5d66479190f63d9ff0da46c284d73f parsl.dataflow.memoization INFO: Task 746 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808313a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 746 try 0 launched on executor multi with executor id 747 parsl.dataflow.dflow INFO: Standard output for task 746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/10af904c-7c2b-47b4-80e5-c3237d265540_isrForCrosstalkSources_996964_30.stdout parsl.dataflow.dflow INFO: Standard error for task 746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/10af904c-7c2b-47b4-80e5-c3237d265540_isrForCrosstalkSources_996964_30.stderr parsl.dataflow.dflow DEBUG: Task 747 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 747 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 747 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 747 has memoization hash 8c16d8caebbe903703396d56ef567369 parsl.dataflow.memoization INFO: Task 747 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 747 try 0 launched on executor multi with executor id 748 parsl.dataflow.dflow INFO: Standard output for task 747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/84dbcd9a-094d-4594-a4c7-2301a8c76629_isrForCrosstalkSources_996969_29.stdout parsl.dataflow.dflow INFO: Standard error for task 747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/84dbcd9a-094d-4594-a4c7-2301a8c76629_isrForCrosstalkSources_996969_29.stderr parsl.dataflow.dflow DEBUG: Task 748 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 748 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 748 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 748 has memoization hash 1352c14c28ab6ee2d2dd868745a3a661 parsl.dataflow.memoization INFO: Task 748 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 748 try 0 launched on executor multi with executor id 749 parsl.dataflow.dflow INFO: Standard output for task 748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1139286-4e58-4369-978d-d4d2137d777c_isrForCrosstalkSources_996975_5.stdout parsl.dataflow.dflow INFO: Standard error for task 748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1139286-4e58-4369-978d-d4d2137d777c_isrForCrosstalkSources_996975_5.stderr parsl.dataflow.dflow DEBUG: Task 749 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 749 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 749 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 749 has memoization hash e19aa3896229fc86790115a05518a020 parsl.dataflow.memoization INFO: Task 749 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 749 try 0 launched on executor multi with executor id 750 parsl.dataflow.dflow INFO: Standard output for task 749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/3ffea72a-ca29-445e-abb6-68a652e7efff_isrForCrosstalkSources_996974_34.stdout parsl.dataflow.dflow INFO: Standard error for task 749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/3ffea72a-ca29-445e-abb6-68a652e7efff_isrForCrosstalkSources_996974_34.stderr parsl.dataflow.dflow DEBUG: Task 750 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 750 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 750 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 750 has memoization hash bdfdfa9063da13b3873890af182417d5 parsl.dataflow.memoization INFO: Task 750 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808316c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 750 try 0 launched on executor multi with executor id 751 parsl.dataflow.dflow INFO: Standard output for task 750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/594f4542-c809-4a14-a17a-b1a6b13a424a_isrForCrosstalkSources_997033_48.stdout parsl.dataflow.dflow INFO: Standard error for task 750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/594f4542-c809-4a14-a17a-b1a6b13a424a_isrForCrosstalkSources_997033_48.stderr parsl.dataflow.dflow DEBUG: Task 751 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 751 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 751 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 751 has memoization hash e7b109b71db9c897464c39fb18161e9e parsl.dataflow.memoization INFO: Task 751 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808318a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 751 try 0 launched on executor multi with executor id 752 parsl.dataflow.dflow INFO: Standard output for task 751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/8559b384-612b-4f70-a527-95d8c6d8976b_isrForCrosstalkSources_997012_37.stdout parsl.dataflow.dflow INFO: Standard error for task 751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/8559b384-612b-4f70-a527-95d8c6d8976b_isrForCrosstalkSources_997012_37.stderr parsl.dataflow.dflow DEBUG: Task 752 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 752 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 752 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 752 has memoization hash a17f7919c5bcbd40a8a53ea0eb2cd087 parsl.dataflow.memoization INFO: Task 752 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 752 try 0 launched on executor multi with executor id 753 parsl.dataflow.dflow INFO: Standard output for task 752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e22e2eba-f3be-4ec1-8158-705cafa11ccd_isrForCrosstalkSources_997016_18.stdout parsl.dataflow.dflow INFO: Standard error for task 752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e22e2eba-f3be-4ec1-8158-705cafa11ccd_isrForCrosstalkSources_997016_18.stderr parsl.dataflow.dflow DEBUG: Task 753 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 753 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 753 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 753 has memoization hash 36a7e80620d11b410e3423fceb604db9 parsl.dataflow.memoization INFO: Task 753 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 753 try 0 launched on executor multi with executor id 754 parsl.dataflow.dflow INFO: Standard output for task 753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b4d32665-a611-45c9-9ee7-dc9a42936497_isrForCrosstalkSources_996966_16.stdout parsl.dataflow.dflow INFO: Standard error for task 753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b4d32665-a611-45c9-9ee7-dc9a42936497_isrForCrosstalkSources_996966_16.stderr parsl.dataflow.dflow DEBUG: Task 754 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 754 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 754 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 754 has memoization hash baf8378d6cc91ea66d80b6a14062343a parsl.dataflow.memoization INFO: Task 754 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 754 try 0 launched on executor multi with executor id 755 parsl.dataflow.dflow INFO: Standard output for task 754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/252c5470-d1e5-4e8f-a9db-80ea1b7f9e93_isrForCrosstalkSources_996964_15.stdout parsl.dataflow.dflow INFO: Standard error for task 754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/252c5470-d1e5-4e8f-a9db-80ea1b7f9e93_isrForCrosstalkSources_996964_15.stderr parsl.dataflow.dflow DEBUG: Task 755 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 755 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 755 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 755 has memoization hash d31778d9ffc4e30ae7dd9f7845351afb parsl.dataflow.memoization INFO: Task 755 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808319e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 755 try 0 launched on executor multi with executor id 756 parsl.dataflow.dflow INFO: Standard output for task 755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/8f54438f-9418-48a1-95ac-b693782fcde9_isrForCrosstalkSources_997020_16.stdout parsl.dataflow.dflow INFO: Standard error for task 755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/8f54438f-9418-48a1-95ac-b693782fcde9_isrForCrosstalkSources_997020_16.stderr parsl.dataflow.dflow DEBUG: Task 756 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 756 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 756 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 756 has memoization hash 09df370561300dc719dd5ea3a0d1a264 parsl.dataflow.memoization INFO: Task 756 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 756 try 0 launched on executor multi with executor id 757 parsl.dataflow.dflow INFO: Standard output for task 756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/6cbf75e4-74c1-4d02-abcd-67336408aaef_isrForCrosstalkSources_996978_23.stdout parsl.dataflow.dflow INFO: Standard error for task 756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/6cbf75e4-74c1-4d02-abcd-67336408aaef_isrForCrosstalkSources_996978_23.stderr parsl.dataflow.dflow DEBUG: Task 757 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 757 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 757 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 757 has memoization hash 370dc73d017e3c0cfe52536428b85c0d parsl.dataflow.memoization INFO: Task 757 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 757 try 0 launched on executor multi with executor id 758 parsl.dataflow.dflow INFO: Standard output for task 757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9434eb78-b555-4660-b271-6977c4f16001_isrForCrosstalkSources_996968_56.stdout parsl.dataflow.dflow INFO: Standard error for task 757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9434eb78-b555-4660-b271-6977c4f16001_isrForCrosstalkSources_996968_56.stderr parsl.dataflow.dflow DEBUG: Task 758 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 758 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 758 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 758 has memoization hash 3a65731536648e6163b97b60ee37ecc4 parsl.dataflow.memoization INFO: Task 758 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 758 try 0 launched on executor multi with executor id 759 parsl.dataflow.dflow INFO: Standard output for task 758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/bf898f6c-75b5-47dc-b836-73cc3adf541a_isrForCrosstalkSources_996985_53.stdout parsl.dataflow.dflow INFO: Standard error for task 758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/bf898f6c-75b5-47dc-b836-73cc3adf541a_isrForCrosstalkSources_996985_53.stderr parsl.dataflow.dflow DEBUG: Task 759 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 759 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 759 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 759 has memoization hash e82c38920000adef0746513b25bb2b4e parsl.dataflow.memoization INFO: Task 759 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 759 try 0 launched on executor multi with executor id 760 parsl.dataflow.dflow INFO: Standard output for task 759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0b908e9f-fe75-4820-87e9-fd356e1cd22f_isrForCrosstalkSources_996997_9.stdout parsl.dataflow.dflow INFO: Standard error for task 759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0b908e9f-fe75-4820-87e9-fd356e1cd22f_isrForCrosstalkSources_996997_9.stderr parsl.dataflow.dflow DEBUG: Task 760 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 760 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 760 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 760 has memoization hash 995978100c0cdc4e5587f1764931beb2 parsl.dataflow.memoization INFO: Task 760 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 760 try 0 launched on executor multi with executor id 761 parsl.dataflow.dflow INFO: Standard output for task 760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/fce7f8a5-da10-4148-b7b3-da1d861588b4_isrForCrosstalkSources_996982_4.stdout parsl.dataflow.dflow INFO: Standard error for task 760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/fce7f8a5-da10-4148-b7b3-da1d861588b4_isrForCrosstalkSources_996982_4.stderr parsl.dataflow.dflow DEBUG: Task 761 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 761 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 761 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 761 has memoization hash 6acc6e764ce8128074d3a49c2be6b52d parsl.dataflow.memoization INFO: Task 761 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 761 try 0 launched on executor multi with executor id 762 parsl.dataflow.dflow INFO: Standard output for task 761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2fd96589-64e5-40ee-9b61-742a62dd8bbe_isrForCrosstalkSources_996989_33.stdout parsl.dataflow.dflow INFO: Standard error for task 761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2fd96589-64e5-40ee-9b61-742a62dd8bbe_isrForCrosstalkSources_996989_33.stderr parsl.dataflow.dflow DEBUG: Task 762 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 762 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 762 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 762 has memoization hash db2278a08db7cd1fd848eb8f63c3864a parsl.dataflow.memoization INFO: Task 762 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 762 try 0 launched on executor multi with executor id 763 parsl.dataflow.dflow INFO: Standard output for task 762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/828843b4-dab3-4146-bfe7-3a63447088ec_isrForCrosstalkSources_997003_6.stdout parsl.dataflow.dflow INFO: Standard error for task 762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/828843b4-dab3-4146-bfe7-3a63447088ec_isrForCrosstalkSources_997003_6.stderr parsl.dataflow.dflow DEBUG: Task 763 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 763 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 763 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 763 has memoization hash 0a0d7909e1fd81c656c816f859e07575 parsl.dataflow.memoization INFO: Task 763 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780831ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 763 try 0 launched on executor multi with executor id 764 parsl.dataflow.dflow INFO: Standard output for task 763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a4b86c7b-54af-41b6-a1a0-acea979d056e_isrForCrosstalkSources_996964_8.stdout parsl.dataflow.dflow INFO: Standard error for task 763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a4b86c7b-54af-41b6-a1a0-acea979d056e_isrForCrosstalkSources_996964_8.stderr parsl.dataflow.dflow DEBUG: Task 764 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 764 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 764 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 764 has memoization hash d60f984655bac47b845b2f56dd1e99a0 parsl.dataflow.memoization INFO: Task 764 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 764 try 0 launched on executor multi with executor id 765 parsl.dataflow.dflow INFO: Standard output for task 764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/4276a8f6-5e75-4b12-b29a-1d60776c3ab2_isrForCrosstalkSources_997003_28.stdout parsl.dataflow.dflow INFO: Standard error for task 764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/4276a8f6-5e75-4b12-b29a-1d60776c3ab2_isrForCrosstalkSources_997003_28.stderr parsl.dataflow.dflow DEBUG: Task 765 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 765 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 765 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 765 has memoization hash d4c053c121ad483479e4e3c63ee0cc90 parsl.dataflow.memoization INFO: Task 765 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808320c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 765 try 0 launched on executor multi with executor id 766 parsl.dataflow.dflow INFO: Standard output for task 765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/32a42997-e2bf-41a7-ab7c-ebc48f43061c_isrForCrosstalkSources_996958_5.stdout parsl.dataflow.dflow INFO: Standard error for task 765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/32a42997-e2bf-41a7-ab7c-ebc48f43061c_isrForCrosstalkSources_996958_5.stderr parsl.dataflow.dflow DEBUG: Task 766 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 766 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 766 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 766 has memoization hash 8a572cad1174dfd529f5a1025555bd16 parsl.dataflow.memoization INFO: Task 766 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 766 try 0 launched on executor multi with executor id 767 parsl.dataflow.dflow INFO: Standard output for task 766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/45166a13-267f-4448-8f5a-7e33794df11c_isrForCrosstalkSources_996972_36.stdout parsl.dataflow.dflow INFO: Standard error for task 766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/45166a13-267f-4448-8f5a-7e33794df11c_isrForCrosstalkSources_996972_36.stderr parsl.dataflow.dflow DEBUG: Task 767 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 767 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 767 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 767 has memoization hash 8bdc6f828981ad07504acbc3e0c78d15 parsl.dataflow.memoization INFO: Task 767 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808322a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 767 try 0 launched on executor multi with executor id 768 parsl.dataflow.dflow INFO: Standard output for task 767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/202f49a2-3b3f-49ad-8654-5cb95e6ac5ff_isrForCrosstalkSources_997009_28.stdout parsl.dataflow.dflow INFO: Standard error for task 767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/202f49a2-3b3f-49ad-8654-5cb95e6ac5ff_isrForCrosstalkSources_997009_28.stderr parsl.dataflow.dflow DEBUG: Task 768 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 768 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 768 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 768 has memoization hash 436b8287878fb8c9ac1f4429c1698457 parsl.dataflow.memoization INFO: Task 768 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 768 try 0 launched on executor multi with executor id 769 parsl.dataflow.dflow INFO: Standard output for task 768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2e1718f5-4cfc-4944-aa03-c8dce4e0494b_isrForCrosstalkSources_996956_9.stdout parsl.dataflow.dflow INFO: Standard error for task 768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2e1718f5-4cfc-4944-aa03-c8dce4e0494b_isrForCrosstalkSources_996956_9.stderr parsl.dataflow.dflow DEBUG: Task 769 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 769 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 769 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 769 has memoization hash 057fc24d34bb6696654712c3dbeefc48 2024-03-15 13:49:31 proc_lsst.multi:170 [INFO] [multi] job_id 17074842 parsl.dataflow.memoization INFO: Task 769 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) proc_lsst.multi INFO: [multi] job_id 17074842 2024-03-15 13:49:31 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 9 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 9 2024-03-15 13:49:31 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:31 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074842 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074842 parsl.executors.status_handling DEBUG: Launched block 9 on executor multi with job ID 17074842 parsl.executors.status_handling INFO: Allocated block ID 10 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-10 2024-03-15 13:49:31 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=10 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-10 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=10 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-10 2024-03-15 13:49:31 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=10 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=10 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:31 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=10 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=10 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 9 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 9 2024-03-15 13:49:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:31 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=10 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=10 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 769 try 0 launched on executor multi with executor id 770 parsl.dataflow.dflow INFO: Standard output for task 769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c53edd47-42cb-44eb-b02a-9e7381b06bed_isrForCrosstalkSources_996968_5.stdout parsl.dataflow.dflow INFO: Standard error for task 769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c53edd47-42cb-44eb-b02a-9e7381b06bed_isrForCrosstalkSources_996968_5.stderr parsl.dataflow.dflow DEBUG: Task 770 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 770 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 770 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 770 has memoization hash 716d5d912c3f84e0700aa3e79106ce1c parsl.dataflow.memoization INFO: Task 770 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7803fbf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 770 try 0 launched on executor multi with executor id 771 parsl.dataflow.dflow INFO: Standard output for task 770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/71766269-67a9-4c01-8495-3e818c8ab35d_isrForCrosstalkSources_997025_43.stdout parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Standard error for task 770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/71766269-67a9-4c01-8495-3e818c8ab35d_isrForCrosstalkSources_997025_43.stderr parsl.dataflow.dflow DEBUG: Task 771 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 771 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 771 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 771 has memoization hash 80c4af5b5e98dd4cf14b16265d39c03f parsl.dataflow.memoization INFO: Task 771 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806baf20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 771 try 0 launched on executor multi with executor id 772 parsl.dataflow.dflow INFO: Standard output for task 771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/674a2622-e384-4125-afbb-460bc97c0a3f_isrForCrosstalkSources_996970_33.stdout parsl.dataflow.dflow INFO: Standard error for task 771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/674a2622-e384-4125-afbb-460bc97c0a3f_isrForCrosstalkSources_996970_33.stderr parsl.dataflow.dflow DEBUG: Task 772 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 772 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 772 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 772 has memoization hash c7b30d702e808b1e09d0bbdc79fb9176 parsl.dataflow.memoization INFO: Task 772 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 772 try 0 launched on executor multi with executor id 773 parsl.dataflow.dflow INFO: Standard output for task 772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/5f3645c8-f2f8-4555-b3ff-fd58aad9e8c4_isrForCrosstalkSources_996970_61.stdout parsl.dataflow.dflow INFO: Standard error for task 772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/5f3645c8-f2f8-4555-b3ff-fd58aad9e8c4_isrForCrosstalkSources_996970_61.stderr parsl.dataflow.dflow DEBUG: Task 773 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 773 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 773 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 773 has memoization hash a2ddcddacade1351b9c3b3f29647a05d parsl.dataflow.memoization INFO: Task 773 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 773 try 0 launched on executor multi with executor id 774 parsl.dataflow.dflow INFO: Standard output for task 773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/87b4c794-4d93-45a7-8678-753d50bae62e_isrForCrosstalkSources_996962_6.stdout parsl.dataflow.dflow INFO: Standard error for task 773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/87b4c794-4d93-45a7-8678-753d50bae62e_isrForCrosstalkSources_996962_6.stderr parsl.dataflow.dflow DEBUG: Task 774 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 774 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 774 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 774 has memoization hash bbefd56808831a3954dcac778d929391 parsl.dataflow.memoization INFO: Task 774 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 774 try 0 launched on executor multi with executor id 775 parsl.dataflow.dflow INFO: Standard output for task 774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/badde207-b509-4d67-88e3-5250e1242828_isrForCrosstalkSources_997014_57.stdout parsl.dataflow.dflow INFO: Standard error for task 774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/badde207-b509-4d67-88e3-5250e1242828_isrForCrosstalkSources_997014_57.stderr parsl.dataflow.dflow DEBUG: Task 775 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 775 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 775 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 775 has memoization hash 28ed076030bd56d6a8505e7b1bebdde1 parsl.dataflow.memoization INFO: Task 775 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 775 try 0 launched on executor multi with executor id 776 parsl.dataflow.dflow INFO: Standard output for task 775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/debf0699-64a4-43cf-accd-e49fe07a74e0_isrForCrosstalkSources_996977_15.stdout parsl.dataflow.dflow INFO: Standard error for task 775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/debf0699-64a4-43cf-accd-e49fe07a74e0_isrForCrosstalkSources_996977_15.stderr parsl.dataflow.dflow DEBUG: Task 776 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 776 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 776 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 776 has memoization hash 335ce041791dd596f3cb78e4464d954e parsl.dataflow.memoization INFO: Task 776 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 776 try 0 launched on executor multi with executor id 777 parsl.dataflow.dflow INFO: Standard output for task 776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/dde3d9f7-347b-4551-9538-33cb702568de_isrForCrosstalkSources_996988_53.stdout parsl.dataflow.dflow INFO: Standard error for task 776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/dde3d9f7-347b-4551-9538-33cb702568de_isrForCrosstalkSources_996988_53.stderr parsl.dataflow.dflow DEBUG: Task 777 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 777 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 777 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 777 has memoization hash ef5f19f7f87a03aff29670bc5876acd6 parsl.dataflow.memoization INFO: Task 777 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 777 try 0 launched on executor multi with executor id 778 parsl.dataflow.dflow INFO: Standard output for task 777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/e8bf53e6-3a2a-49f6-a57b-eb04ae6443b6_isrForCrosstalkSources_997014_2.stdout parsl.dataflow.dflow INFO: Standard error for task 777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/e8bf53e6-3a2a-49f6-a57b-eb04ae6443b6_isrForCrosstalkSources_997014_2.stderr parsl.dataflow.dflow DEBUG: Task 778 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 778 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 778 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 778 has memoization hash 40d92487d4e79b0127b8d261d81602ee parsl.dataflow.memoization INFO: Task 778 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 778 try 0 launched on executor multi with executor id 779 parsl.dataflow.dflow INFO: Standard output for task 778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/16f44b68-10f3-4358-a356-dae81d8cf8dc_isrForCrosstalkSources_997000_58.stdout parsl.dataflow.dflow INFO: Standard error for task 778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/16f44b68-10f3-4358-a356-dae81d8cf8dc_isrForCrosstalkSources_997000_58.stderr parsl.dataflow.dflow DEBUG: Task 779 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 779 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 779 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 779 has memoization hash 508f10ce245bb447d656b76e4c085b7f parsl.dataflow.memoization INFO: Task 779 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 779 try 0 launched on executor multi with executor id 780 parsl.dataflow.dflow INFO: Standard output for task 779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c1bb0696-daae-4774-b325-e0be466157d0_isrForCrosstalkSources_996977_39.stdout parsl.dataflow.dflow INFO: Standard error for task 779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c1bb0696-daae-4774-b325-e0be466157d0_isrForCrosstalkSources_996977_39.stderr parsl.dataflow.dflow DEBUG: Task 780 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 780 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 780 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 780 has memoization hash d451d93c5901754e84b24018d34a4742 parsl.dataflow.memoization INFO: Task 780 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 780 try 0 launched on executor multi with executor id 781 parsl.dataflow.dflow INFO: Standard output for task 780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/545db4f5-47dc-4582-94ee-625ac165677f_isrForCrosstalkSources_996990_10.stdout parsl.dataflow.dflow INFO: Standard error for task 780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/545db4f5-47dc-4582-94ee-625ac165677f_isrForCrosstalkSources_996990_10.stderr parsl.dataflow.dflow DEBUG: Task 781 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 781 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 781 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 781 has memoization hash 5cc3fd357893129b4091bb9cb9ee5090 parsl.dataflow.memoization INFO: Task 781 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808328e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 781 try 0 launched on executor multi with executor id 782 parsl.dataflow.dflow INFO: Standard output for task 781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e8e831cd-ff0c-43b1-9273-7e3b666cf5e4_isrForCrosstalkSources_996993_16.stdout parsl.dataflow.dflow INFO: Standard error for task 781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e8e831cd-ff0c-43b1-9273-7e3b666cf5e4_isrForCrosstalkSources_996993_16.stderr parsl.dataflow.dflow DEBUG: Task 782 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 782 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 782 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 782 has memoization hash 306cf27890a22d4248b9863e8163ef9a parsl.dataflow.memoization INFO: Task 782 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 782 try 0 launched on executor multi with executor id 783 parsl.dataflow.dflow INFO: Standard output for task 782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/b780aa8b-3d10-4c66-918f-662798d9304b_isrForCrosstalkSources_996967_37.stdout parsl.dataflow.dflow INFO: Standard error for task 782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/b780aa8b-3d10-4c66-918f-662798d9304b_isrForCrosstalkSources_996967_37.stderr parsl.dataflow.dflow DEBUG: Task 783 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 783 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 783 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 783 has memoization hash 697e0bb20faf4ea5ad604283c5df10be parsl.dataflow.memoization INFO: Task 783 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 783 try 0 launched on executor multi with executor id 784 parsl.dataflow.dflow INFO: Standard output for task 783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d753e2e6-c4dd-4ef3-94c3-3c60bca29be7_isrForCrosstalkSources_996985_15.stdout parsl.dataflow.dflow INFO: Standard error for task 783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d753e2e6-c4dd-4ef3-94c3-3c60bca29be7_isrForCrosstalkSources_996985_15.stderr parsl.dataflow.dflow DEBUG: Task 784 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 784 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 784 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 784 has memoization hash 05cb7c50dbb823996e5a4af3a6000568 parsl.dataflow.memoization INFO: Task 784 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 784 try 0 launched on executor multi with executor id 785 parsl.dataflow.dflow INFO: Standard output for task 784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/c6da1623-7bce-4fdf-83c7-ad16f107948e_isrForCrosstalkSources_996952_33.stdout parsl.dataflow.dflow INFO: Standard error for task 784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/c6da1623-7bce-4fdf-83c7-ad16f107948e_isrForCrosstalkSources_996952_33.stderr parsl.dataflow.dflow DEBUG: Task 785 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 785 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 785 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 785 has memoization hash 3aad201192270bdf434d7c25bb42180e parsl.dataflow.memoization INFO: Task 785 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 785 try 0 launched on executor multi with executor id 786 parsl.dataflow.dflow INFO: Standard output for task 785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/af74facd-f12e-4dea-be9c-419dae812455_isrForCrosstalkSources_996997_42.stdout parsl.dataflow.dflow INFO: Standard error for task 785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/af74facd-f12e-4dea-be9c-419dae812455_isrForCrosstalkSources_996997_42.stderr parsl.dataflow.dflow DEBUG: Task 786 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 786 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 786 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 786 has memoization hash d4e56b5966c13a730ca0260f87960209 parsl.dataflow.memoization INFO: Task 786 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 786 try 0 launched on executor multi with executor id 787 parsl.dataflow.dflow INFO: Standard output for task 786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4f5e74bb-c5b6-4f3c-bfc6-8484ff1e99dd_isrForCrosstalkSources_997014_43.stdout parsl.dataflow.dflow INFO: Standard error for task 786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4f5e74bb-c5b6-4f3c-bfc6-8484ff1e99dd_isrForCrosstalkSources_997014_43.stderr parsl.dataflow.dflow DEBUG: Task 787 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 787 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 787 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 787 has memoization hash 6ae083fb263f8523139448364ce471bc parsl.dataflow.memoization INFO: Task 787 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 787 try 0 launched on executor multi with executor id 788 parsl.dataflow.dflow INFO: Standard output for task 787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/4ddf1c30-7b66-4be6-970f-a63f64223494_isrForCrosstalkSources_997008_42.stdout parsl.dataflow.dflow INFO: Standard error for task 787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/4ddf1c30-7b66-4be6-970f-a63f64223494_isrForCrosstalkSources_997008_42.stderr parsl.dataflow.dflow DEBUG: Task 788 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 788 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 788 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 788 has memoization hash a52589be9c753cfc47910f5b7c5885e4 parsl.dataflow.memoization INFO: Task 788 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808339c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 788 try 0 launched on executor multi with executor id 789 parsl.dataflow.dflow INFO: Standard output for task 788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/cae5e6d9-3f54-4ad0-8c5d-fd8a2d713d26_isrForCrosstalkSources_996991_20.stdout parsl.dataflow.dflow INFO: Standard error for task 788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/cae5e6d9-3f54-4ad0-8c5d-fd8a2d713d26_isrForCrosstalkSources_996991_20.stderr parsl.dataflow.dflow DEBUG: Task 789 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 789 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 789 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 789 has memoization hash 7bc523e89ad68f8edfb7e4f5f074d9b7 parsl.dataflow.memoization INFO: Task 789 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 789 try 0 launched on executor multi with executor id 790 parsl.dataflow.dflow INFO: Standard output for task 789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e8c0ad20-4cf3-4860-b019-7868679050a3_isrForCrosstalkSources_996978_12.stdout parsl.dataflow.dflow INFO: Standard error for task 789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e8c0ad20-4cf3-4860-b019-7868679050a3_isrForCrosstalkSources_996978_12.stderr parsl.dataflow.dflow DEBUG: Task 790 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 790 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 790 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 790 has memoization hash faffe2ea6f23bd834643841fb67525d8 parsl.dataflow.memoization INFO: Task 790 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808337e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 790 try 0 launched on executor multi with executor id 791 parsl.dataflow.dflow INFO: Standard output for task 790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/700cd715-a166-4943-b595-8e6c847049ab_isrForCrosstalkSources_996992_27.stdout parsl.dataflow.dflow INFO: Standard error for task 790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/700cd715-a166-4943-b595-8e6c847049ab_isrForCrosstalkSources_996992_27.stderr parsl.dataflow.dflow DEBUG: Task 791 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 791 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 791 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 791 has memoization hash 951d17adba18ca9bdc09cb31aaf51546 parsl.dataflow.memoization INFO: Task 791 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 791 try 0 launched on executor multi with executor id 792 parsl.dataflow.dflow INFO: Standard output for task 791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b6562ccc-e5ab-492b-8f72-061b0f564a5f_isrForCrosstalkSources_997025_59.stdout parsl.dataflow.dflow INFO: Standard error for task 791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b6562ccc-e5ab-492b-8f72-061b0f564a5f_isrForCrosstalkSources_997025_59.stderr parsl.dataflow.dflow DEBUG: Task 792 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 792 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 792 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 792 has memoization hash 2d4694ec1bab359bec9bcc3e587f4bb2 parsl.dataflow.memoization INFO: Task 792 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 792 try 0 launched on executor multi with executor id 793 parsl.dataflow.dflow INFO: Standard output for task 792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/672cc3a2-77d3-4b25-925f-6eb907483ab1_isrForCrosstalkSources_996984_53.stdout parsl.dataflow.dflow INFO: Standard error for task 792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/672cc3a2-77d3-4b25-925f-6eb907483ab1_isrForCrosstalkSources_996984_53.stderr parsl.dataflow.dflow DEBUG: Task 793 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 793 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 793 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 793 has memoization hash 70e551464c81756abde88f3645f9dac2 parsl.dataflow.memoization INFO: Task 793 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808336a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 793 try 0 launched on executor multi with executor id 794 parsl.dataflow.dflow INFO: Standard output for task 793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/44d479f7-29a5-4c88-b070-f6c9cfbcafac_isrForCrosstalkSources_996982_52.stdout parsl.dataflow.dflow INFO: Standard error for task 793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/44d479f7-29a5-4c88-b070-f6c9cfbcafac_isrForCrosstalkSources_996982_52.stderr parsl.dataflow.dflow DEBUG: Task 794 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 794 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 794 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 794 has memoization hash 86c3e9a7501f8dc0579b115664632991 parsl.dataflow.memoization INFO: Task 794 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 794 try 0 launched on executor multi with executor id 795 parsl.dataflow.dflow INFO: Standard output for task 794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8ea1eed6-b1b2-4d9e-85ab-33af451c26f3_isrForCrosstalkSources_997013_34.stdout parsl.dataflow.dflow INFO: Standard error for task 794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8ea1eed6-b1b2-4d9e-85ab-33af451c26f3_isrForCrosstalkSources_997013_34.stderr parsl.dataflow.dflow DEBUG: Task 795 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 795 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 795 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 795 has memoization hash cc300fc44b6ed377655c32bbc22fce0b parsl.dataflow.memoization INFO: Task 795 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 795 try 0 launched on executor multi with executor id 796 parsl.dataflow.dflow INFO: Standard output for task 795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/58d0c6e7-d862-4666-bc02-d4310f44e0c7_isrForCrosstalkSources_996977_24.stdout parsl.dataflow.dflow INFO: Standard error for task 795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/58d0c6e7-d862-4666-bc02-d4310f44e0c7_isrForCrosstalkSources_996977_24.stderr parsl.dataflow.dflow DEBUG: Task 796 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 796 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 796 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 796 has memoization hash 59c0804df24b6ed08164ab02f555f8a9 parsl.dataflow.memoization INFO: Task 796 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 796 try 0 launched on executor multi with executor id 797 parsl.dataflow.dflow INFO: Standard output for task 796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/598cbe5a-1c50-4684-8d5c-2b48860ddfdb_isrForCrosstalkSources_997025_58.stdout parsl.dataflow.dflow INFO: Standard error for task 796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/598cbe5a-1c50-4684-8d5c-2b48860ddfdb_isrForCrosstalkSources_997025_58.stderr parsl.dataflow.dflow DEBUG: Task 797 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 797 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 797 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 797 has memoization hash afd01e6b4f1c787999be7aee04e92768 parsl.dataflow.memoization INFO: Task 797 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808334c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 797 try 0 launched on executor multi with executor id 798 parsl.dataflow.dflow INFO: Standard output for task 797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c2b0b837-351f-4e7f-bfd6-aaa309679fea_isrForCrosstalkSources_996961_6.stdout parsl.dataflow.dflow INFO: Standard error for task 797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c2b0b837-351f-4e7f-bfd6-aaa309679fea_isrForCrosstalkSources_996961_6.stderr parsl.dataflow.dflow DEBUG: Task 798 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 798 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 798 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 798 has memoization hash d55e5c2cc130f539d8085380088f88b4 parsl.dataflow.memoization INFO: Task 798 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 798 try 0 launched on executor multi with executor id 799 parsl.dataflow.dflow INFO: Standard output for task 798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/0a4d0421-3c81-4309-91ea-8f9123077bd0_isrForCrosstalkSources_996987_47.stdout parsl.dataflow.dflow INFO: Standard error for task 798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/0a4d0421-3c81-4309-91ea-8f9123077bd0_isrForCrosstalkSources_996987_47.stderr parsl.dataflow.dflow DEBUG: Task 799 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 799 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 799 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 799 has memoization hash 76217ff52a35847dc94a4ef3073ed6cc parsl.dataflow.memoization INFO: Task 799 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 799 try 0 launched on executor multi with executor id 800 parsl.dataflow.dflow INFO: Standard output for task 799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/8ecfeb72-a41a-4dfc-8584-a94b5692a41b_isrForCrosstalkSources_996962_4.stdout parsl.dataflow.dflow INFO: Standard error for task 799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/8ecfeb72-a41a-4dfc-8584-a94b5692a41b_isrForCrosstalkSources_996962_4.stderr parsl.dataflow.dflow DEBUG: Task 800 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 800 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 800 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 800 has memoization hash 34376c9583d4c0016f378b81c78882fc parsl.dataflow.memoization INFO: Task 800 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808332e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 800 try 0 launched on executor multi with executor id 801 parsl.dataflow.dflow INFO: Standard output for task 800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/b1852634-80ec-474b-8338-7e3e3c37d9a0_isrForCrosstalkSources_996961_5.stdout parsl.dataflow.dflow INFO: Standard error for task 800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/b1852634-80ec-474b-8338-7e3e3c37d9a0_isrForCrosstalkSources_996961_5.stderr parsl.dataflow.dflow DEBUG: Task 801 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 801 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 801 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 801 has memoization hash b650adf849e98f10488a520f5aa01b06 parsl.dataflow.memoization INFO: Task 801 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 801 try 0 launched on executor multi with executor id 802 parsl.dataflow.dflow INFO: Standard output for task 801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/006c502f-e828-4527-8d71-31812f33eaef_isrForCrosstalkSources_996958_15.stdout parsl.dataflow.dflow INFO: Standard error for task 801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/006c502f-e828-4527-8d71-31812f33eaef_isrForCrosstalkSources_996958_15.stderr parsl.dataflow.dflow DEBUG: Task 802 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 802 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 802 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 802 has memoization hash af730f98d7fe3b7a5c28b00f3531b190 parsl.dataflow.memoization INFO: Task 802 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808331a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 802 try 0 launched on executor multi with executor id 803 parsl.dataflow.dflow INFO: Standard output for task 802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/94479306-858d-445d-8219-78a0805f7120_isrForCrosstalkSources_997007_16.stdout parsl.dataflow.dflow INFO: Standard error for task 802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/94479306-858d-445d-8219-78a0805f7120_isrForCrosstalkSources_997007_16.stderr parsl.dataflow.dflow DEBUG: Task 803 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 803 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 803 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 803 has memoization hash 0ea8d29d8cb361bc9a1b5c028f0f8817 parsl.dataflow.memoization INFO: Task 803 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 803 try 0 launched on executor multi with executor id 804 parsl.dataflow.dflow INFO: Standard output for task 803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/b6482575-edb6-4d6f-b639-ab7a755da88e_isrForCrosstalkSources_997011_48.stdout parsl.dataflow.dflow INFO: Standard error for task 803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/b6482575-edb6-4d6f-b639-ab7a755da88e_isrForCrosstalkSources_997011_48.stderr parsl.dataflow.dflow DEBUG: Task 804 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 804 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 804 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 804 has memoization hash 5f6a03c8b02e450aaa90625cc06061c1 parsl.dataflow.memoization INFO: Task 804 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780833100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 804 try 0 launched on executor multi with executor id 805 parsl.dataflow.dflow INFO: Standard output for task 804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/d083fdf7-aab9-4caf-8821-fd25b6ce2e8b_isrForCrosstalkSources_997011_11.stdout parsl.dataflow.dflow INFO: Standard error for task 804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/d083fdf7-aab9-4caf-8821-fd25b6ce2e8b_isrForCrosstalkSources_997011_11.stderr parsl.dataflow.dflow DEBUG: Task 805 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 805 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 805 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 805 has memoization hash 0c97ed97a1aa2ae23ff1bd910813183c parsl.dataflow.memoization INFO: Task 805 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 805 try 0 launched on executor multi with executor id 806 parsl.dataflow.dflow INFO: Standard output for task 805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/a38b8586-5a0d-454b-bf40-a1668e1869c0_isrForCrosstalkSources_996990_15.stdout parsl.dataflow.dflow INFO: Standard error for task 805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/a38b8586-5a0d-454b-bf40-a1668e1869c0_isrForCrosstalkSources_996990_15.stderr parsl.dataflow.dflow DEBUG: Task 806 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 806 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 806 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 806 has memoization hash ea745a675a18f014157d1f292ea449b4 parsl.dataflow.memoization INFO: Task 806 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 806 try 0 launched on executor multi with executor id 807 parsl.dataflow.dflow INFO: Standard output for task 806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/29a80fad-a022-4229-8047-c000121b02f7_isrForCrosstalkSources_997025_57.stdout parsl.dataflow.dflow INFO: Standard error for task 806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/29a80fad-a022-4229-8047-c000121b02f7_isrForCrosstalkSources_997025_57.stderr parsl.dataflow.dflow DEBUG: Task 807 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 807 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 807 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 807 has memoization hash 06c0f3cfad7bf7936258ea812347595f parsl.dataflow.memoization INFO: Task 807 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 807 try 0 launched on executor multi with executor id 808 parsl.dataflow.dflow INFO: Standard output for task 807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1c4bd8cb-4563-4530-8cd3-0264d44aeac4_isrForCrosstalkSources_996968_8.stdout parsl.dataflow.dflow INFO: Standard error for task 807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1c4bd8cb-4563-4530-8cd3-0264d44aeac4_isrForCrosstalkSources_996968_8.stderr parsl.dataflow.dflow DEBUG: Task 808 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 808 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 808 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 808 has memoization hash 28e8b028efcc7bb6a0fdfae2892e335c parsl.dataflow.memoization INFO: Task 808 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 808 try 0 launched on executor multi with executor id 809 parsl.dataflow.dflow INFO: Standard output for task 808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/3ea77ef0-961d-471b-86a7-acd2d31e5b66_isrForCrosstalkSources_996983_5.stdout parsl.dataflow.dflow INFO: Standard error for task 808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/3ea77ef0-961d-471b-86a7-acd2d31e5b66_isrForCrosstalkSources_996983_5.stderr parsl.dataflow.dflow DEBUG: Task 809 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 809 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 809 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 809 has memoization hash 57974aedd4d91f86ba323c01d3d7890e parsl.dataflow.memoization INFO: Task 809 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808327a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 809 try 0 launched on executor multi with executor id 810 parsl.dataflow.dflow INFO: Standard output for task 809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/35c8333b-e080-4fc1-8a41-9b659503163e_isrForCrosstalkSources_996952_1.stdout parsl.dataflow.dflow INFO: Standard error for task 809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/35c8333b-e080-4fc1-8a41-9b659503163e_isrForCrosstalkSources_996952_1.stderr parsl.dataflow.dflow DEBUG: Task 810 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 810 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 810 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 810 has memoization hash e94e0b48e77e4dba39dddcd4d24e264a parsl.dataflow.memoization INFO: Task 810 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 810 try 0 launched on executor multi with executor id 811 parsl.dataflow.dflow INFO: Standard output for task 810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7b3501ae-d361-4032-a20f-355b031c92a4_isrForCrosstalkSources_996973_36.stdout parsl.dataflow.dflow INFO: Standard error for task 810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7b3501ae-d361-4032-a20f-355b031c92a4_isrForCrosstalkSources_996973_36.stderr parsl.dataflow.dflow DEBUG: Task 811 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 811 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 811 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 811 has memoization hash ed6f0c895b36d3773725dd7fd3a4e471 parsl.dataflow.memoization INFO: Task 811 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 811 try 0 launched on executor multi with executor id 812 parsl.dataflow.dflow INFO: Standard output for task 811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/efa72a08-f035-48b5-9263-a874ea514e72_isrForCrosstalkSources_997007_6.stdout parsl.dataflow.dflow INFO: Standard error for task 811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/efa72a08-f035-48b5-9263-a874ea514e72_isrForCrosstalkSources_997007_6.stderr parsl.dataflow.dflow DEBUG: Task 812 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 812 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 812 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 812 has memoization hash 6d4f5b30c304f5956784e81a10e369ab parsl.dataflow.memoization INFO: Task 812 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 812 try 0 launched on executor multi with executor id 813 parsl.dataflow.dflow INFO: Standard output for task 812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/818774ee-fa26-41d7-ac56-a5355bf18587_isrForCrosstalkSources_997002_41.stdout parsl.dataflow.dflow INFO: Standard error for task 812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/818774ee-fa26-41d7-ac56-a5355bf18587_isrForCrosstalkSources_997002_41.stderr parsl.dataflow.dflow DEBUG: Task 813 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 813 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 813 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 813 has memoization hash a8a555856c5c8527db9c2f8cbaa05627 parsl.dataflow.memoization INFO: Task 813 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 813 try 0 launched on executor multi with executor id 814 parsl.dataflow.dflow INFO: Standard output for task 813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/246e1e9b-918c-4077-a5b2-a62816bc909e_isrForCrosstalkSources_997024_23.stdout parsl.dataflow.dflow INFO: Standard error for task 813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/246e1e9b-918c-4077-a5b2-a62816bc909e_isrForCrosstalkSources_997024_23.stderr parsl.dataflow.dflow DEBUG: Task 814 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 814 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 814 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 814 has memoization hash 9aac46b8d656920c3375d3b4c7ac4d7f parsl.dataflow.memoization INFO: Task 814 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 814 try 0 launched on executor multi with executor id 815 parsl.dataflow.dflow INFO: Standard output for task 814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/48b63da0-497c-4602-ab9f-d80de55d6443_isrForCrosstalkSources_996966_28.stdout parsl.dataflow.dflow INFO: Standard error for task 814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/48b63da0-497c-4602-ab9f-d80de55d6443_isrForCrosstalkSources_996966_28.stderr parsl.dataflow.dflow DEBUG: Task 815 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 815 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 815 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 815 has memoization hash b545cbdb61142d89eedef653fe0b2691 parsl.dataflow.memoization INFO: Task 815 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 815 try 0 launched on executor multi with executor id 816 parsl.dataflow.dflow INFO: Standard output for task 815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/15d3b7ab-c217-4330-be82-ec92cce97aeb_isrForCrosstalkSources_997008_55.stdout parsl.dataflow.dflow INFO: Standard error for task 815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/15d3b7ab-c217-4330-be82-ec92cce97aeb_isrForCrosstalkSources_997008_55.stderr parsl.dataflow.dflow DEBUG: Task 816 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 816 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 816 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 816 has memoization hash a9faafaa2ae93ac9bacd172b49603c6a parsl.dataflow.memoization INFO: Task 816 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 816 try 0 launched on executor multi with executor id 817 parsl.dataflow.dflow INFO: Standard output for task 816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b7e635bb-0962-43f3-afde-f90d9fb9f4af_isrForCrosstalkSources_997003_46.stdout parsl.dataflow.dflow INFO: Standard error for task 816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b7e635bb-0962-43f3-afde-f90d9fb9f4af_isrForCrosstalkSources_997003_46.stderr parsl.dataflow.dflow DEBUG: Task 817 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 817 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 817 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 817 has memoization hash 7390a322ba03540a7e0a97cd94420339 parsl.dataflow.memoization INFO: Task 817 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 817 try 0 launched on executor multi with executor id 818 parsl.dataflow.dflow INFO: Standard output for task 817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/9be6e759-af82-4494-8656-bb07fe2defc2_isrForCrosstalkSources_996980_5.stdout parsl.dataflow.dflow INFO: Standard error for task 817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/9be6e759-af82-4494-8656-bb07fe2defc2_isrForCrosstalkSources_996980_5.stderr parsl.dataflow.dflow DEBUG: Task 818 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 818 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 818 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 818 has memoization hash d92bbc5a779b5e34d31d90866ef866c7 parsl.dataflow.memoization INFO: Task 818 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 818 try 0 launched on executor multi with executor id 819 parsl.dataflow.dflow INFO: Standard output for task 818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/afb6144d-047f-4e43-ab0f-826fae86a24e_isrForCrosstalkSources_996994_29.stdout parsl.dataflow.dflow INFO: Standard error for task 818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/afb6144d-047f-4e43-ab0f-826fae86a24e_isrForCrosstalkSources_996994_29.stderr parsl.dataflow.dflow DEBUG: Task 819 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 819 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 819 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 819 has memoization hash 62c21ec60dc4fb72298ab860af7c4834 parsl.dataflow.memoization INFO: Task 819 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 819 try 0 launched on executor multi with executor id 820 parsl.dataflow.dflow INFO: Standard output for task 819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/8f725df4-c9b0-4008-970c-2d4f2d609438_isrForCrosstalkSources_996990_50.stdout parsl.dataflow.dflow INFO: Standard error for task 819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/8f725df4-c9b0-4008-970c-2d4f2d609438_isrForCrosstalkSources_996990_50.stderr parsl.dataflow.dflow DEBUG: Task 820 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 820 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 820 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 820 has memoization hash 127cfb039c9f39e7b8c931c9164f72ed parsl.dataflow.memoization INFO: Task 820 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 820 try 0 launched on executor multi with executor id 821 parsl.dataflow.dflow INFO: Standard output for task 820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/34e4c126-96b9-43bb-8203-4d277e11f6bb_isrForCrosstalkSources_997007_19.stdout parsl.dataflow.dflow INFO: Standard error for task 820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/34e4c126-96b9-43bb-8203-4d277e11f6bb_isrForCrosstalkSources_997007_19.stderr parsl.dataflow.dflow DEBUG: Task 821 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 821 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 821 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 821 has memoization hash 66216a777dfb26b60f47509bcecbebd8 parsl.dataflow.memoization INFO: Task 821 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 821 try 0 launched on executor multi with executor id 822 parsl.dataflow.dflow INFO: Standard output for task 821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/97a69c8e-eb02-4e8b-a3cc-688f6bd7a023_isrForCrosstalkSources_996961_46.stdout parsl.dataflow.dflow INFO: Standard error for task 821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/97a69c8e-eb02-4e8b-a3cc-688f6bd7a023_isrForCrosstalkSources_996961_46.stderr parsl.dataflow.dflow DEBUG: Task 822 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 822 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 822 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 822 has memoization hash 083afae08706653b9de6adf13ca2d8ea parsl.dataflow.memoization INFO: Task 822 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 822 try 0 launched on executor multi with executor id 823 parsl.dataflow.dflow INFO: Standard output for task 822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/ebd52f2f-849b-4e7c-877f-a1c4cc38208e_isrForCrosstalkSources_997026_15.stdout parsl.dataflow.dflow INFO: Standard error for task 822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/ebd52f2f-849b-4e7c-877f-a1c4cc38208e_isrForCrosstalkSources_997026_15.stderr parsl.dataflow.dflow DEBUG: Task 823 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 823 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 823 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 823 has memoization hash cd9c7a396d361c44b6737b9468b971c4 parsl.dataflow.memoization INFO: Task 823 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 823 try 0 launched on executor multi with executor id 824 parsl.dataflow.dflow INFO: Standard output for task 823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c6e46aa4-8e53-4034-bc66-85a1a3c612dc_isrForCrosstalkSources_996994_43.stdout parsl.dataflow.dflow INFO: Standard error for task 823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c6e46aa4-8e53-4034-bc66-85a1a3c612dc_isrForCrosstalkSources_996994_43.stderr parsl.dataflow.dflow DEBUG: Task 824 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 824 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 824 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 824 has memoization hash d3187a681d635e87c0d035385903bbe2 parsl.dataflow.memoization INFO: Task 824 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 824 try 0 launched on executor multi with executor id 825 parsl.dataflow.dflow INFO: Standard output for task 824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d944a5e8-cb0c-4b53-8fc3-5af685ed39e4_isrForCrosstalkSources_996997_61.stdout parsl.dataflow.dflow INFO: Standard error for task 824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d944a5e8-cb0c-4b53-8fc3-5af685ed39e4_isrForCrosstalkSources_996997_61.stderr parsl.dataflow.dflow DEBUG: Task 825 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 825 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 825 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 825 has memoization hash 5c97f096253d598b98a5f79935bbdc40 parsl.dataflow.memoization INFO: Task 825 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 825 try 0 launched on executor multi with executor id 826 parsl.dataflow.dflow INFO: Standard output for task 825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ab44d992-939c-490d-b845-8fa7e925cc22_isrForCrosstalkSources_996998_57.stdout parsl.dataflow.dflow INFO: Standard error for task 825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ab44d992-939c-490d-b845-8fa7e925cc22_isrForCrosstalkSources_996998_57.stderr parsl.dataflow.dflow DEBUG: Task 826 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 826 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 826 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 826 has memoization hash a85efb35eba5608bd00f86afc2787021 parsl.dataflow.memoization INFO: Task 826 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 826 try 0 launched on executor multi with executor id 827 parsl.dataflow.dflow INFO: Standard output for task 826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/34eeafc9-861c-4550-9423-cbb389edea2d_isrForCrosstalkSources_996957_36.stdout parsl.dataflow.dflow INFO: Standard error for task 826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/34eeafc9-861c-4550-9423-cbb389edea2d_isrForCrosstalkSources_996957_36.stderr parsl.dataflow.dflow DEBUG: Task 827 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 827 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 827 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 827 has memoization hash 7d51822258f39e743c81378c028eb1db parsl.dataflow.memoization INFO: Task 827 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 827 try 0 launched on executor multi with executor id 828 parsl.dataflow.dflow INFO: Standard output for task 827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/04a5e965-7e86-4333-9950-38f947af7b29_isrForCrosstalkSources_997004_17.stdout parsl.dataflow.dflow INFO: Standard error for task 827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/04a5e965-7e86-4333-9950-38f947af7b29_isrForCrosstalkSources_997004_17.stderr parsl.dataflow.dflow DEBUG: Task 828 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 828 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 828 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 828 has memoization hash a5512db4d9395998ad90736ef97465a1 parsl.dataflow.memoization INFO: Task 828 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 828 try 0 launched on executor multi with executor id 829 parsl.dataflow.dflow INFO: Standard output for task 828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2aaa8ecd-1d9a-4c12-a67f-5d8d04bc3429_isrForCrosstalkSources_996962_11.stdout parsl.dataflow.dflow INFO: Standard error for task 828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2aaa8ecd-1d9a-4c12-a67f-5d8d04bc3429_isrForCrosstalkSources_996962_11.stderr parsl.dataflow.dflow DEBUG: Task 829 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 829 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 829 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 829 has memoization hash 30118957d46fb81a68e807260f2bcff5 parsl.dataflow.memoization INFO: Task 829 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 829 try 0 launched on executor multi with executor id 830 parsl.dataflow.dflow INFO: Standard output for task 829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/a22eaf35-29f7-4a92-ab0c-e7501fc5400e_isrForCrosstalkSources_996962_25.stdout parsl.dataflow.dflow INFO: Standard error for task 829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/a22eaf35-29f7-4a92-ab0c-e7501fc5400e_isrForCrosstalkSources_996962_25.stderr parsl.dataflow.dflow DEBUG: Task 830 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 830 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 830 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 830 has memoization hash 1c4e90831fb2a81b693812786fe7998e parsl.dataflow.memoization INFO: Task 830 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 830 try 0 launched on executor multi with executor id 831 parsl.dataflow.dflow INFO: Standard output for task 830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7fb78965-0fb9-442e-829e-a4b1d442f337_isrForCrosstalkSources_997028_22.stdout parsl.dataflow.dflow INFO: Standard error for task 830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7fb78965-0fb9-442e-829e-a4b1d442f337_isrForCrosstalkSources_997028_22.stderr parsl.dataflow.dflow DEBUG: Task 831 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 831 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 831 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 831 has memoization hash a08a9dfd46c4c802d7cde4cc1c54e974 parsl.dataflow.memoization INFO: Task 831 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 831 try 0 launched on executor multi with executor id 832 parsl.dataflow.dflow INFO: Standard output for task 831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e9a80a65-dfe8-4bc7-a1a9-f1201ed33657_isrForCrosstalkSources_996964_51.stdout parsl.dataflow.dflow INFO: Standard error for task 831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e9a80a65-dfe8-4bc7-a1a9-f1201ed33657_isrForCrosstalkSources_996964_51.stderr parsl.dataflow.dflow DEBUG: Task 832 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 832 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 832 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 832 has memoization hash aa734ab9b94b4a23c4feb2268c37ba88 parsl.dataflow.memoization INFO: Task 832 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 832 try 0 launched on executor multi with executor id 833 parsl.dataflow.dflow INFO: Standard output for task 832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9d5edf9a-f23b-4ab3-a4f0-955f9a8e3f0e_isrForCrosstalkSources_996986_40.stdout parsl.dataflow.dflow INFO: Standard error for task 832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9d5edf9a-f23b-4ab3-a4f0-955f9a8e3f0e_isrForCrosstalkSources_996986_40.stderr parsl.dataflow.dflow DEBUG: Task 833 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 833 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 833 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 833 has memoization hash f4dfc2b8d7abbf8e156f5a0154e0fa7c parsl.dataflow.memoization INFO: Task 833 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 833 try 0 launched on executor multi with executor id 834 parsl.dataflow.dflow INFO: Standard output for task 833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e228ea8a-7954-4b0b-ba98-88e02702bc48_isrForCrosstalkSources_996996_31.stdout parsl.dataflow.dflow INFO: Standard error for task 833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e228ea8a-7954-4b0b-ba98-88e02702bc48_isrForCrosstalkSources_996996_31.stderr parsl.dataflow.dflow DEBUG: Task 834 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 834 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 834 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 834 has memoization hash 6eb679667e6d666ef2eb61d9e0160825 parsl.dataflow.memoization INFO: Task 834 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 834 try 0 launched on executor multi with executor id 835 parsl.dataflow.dflow INFO: Standard output for task 834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/f6d2cea5-fc74-44da-984b-29c0176bedd2_isrForCrosstalkSources_996997_27.stdout parsl.dataflow.dflow INFO: Standard error for task 834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/f6d2cea5-fc74-44da-984b-29c0176bedd2_isrForCrosstalkSources_996997_27.stderr parsl.dataflow.dflow DEBUG: Task 835 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 835 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 835 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 835 has memoization hash c7cf6682dffb228d54f435d337c4699f parsl.dataflow.memoization INFO: Task 835 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 835 try 0 launched on executor multi with executor id 836 parsl.dataflow.dflow INFO: Standard output for task 835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/185e9fd6-e6a3-4784-a9bf-8449da8cd8ba_isrForCrosstalkSources_997014_40.stdout parsl.dataflow.dflow INFO: Standard error for task 835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/185e9fd6-e6a3-4784-a9bf-8449da8cd8ba_isrForCrosstalkSources_997014_40.stderr parsl.dataflow.dflow DEBUG: Task 836 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 836 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 836 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 836 has memoization hash 03beb55d75c02d131fc6bb23a5e63f51 parsl.dataflow.memoization INFO: Task 836 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 836 try 0 launched on executor multi with executor id 837 parsl.dataflow.dflow INFO: Standard output for task 836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/c13335b2-693e-4a28-9e8e-9f8728f6c372_isrForCrosstalkSources_997005_6.stdout parsl.dataflow.dflow INFO: Standard error for task 836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/c13335b2-693e-4a28-9e8e-9f8728f6c372_isrForCrosstalkSources_997005_6.stderr parsl.dataflow.dflow DEBUG: Task 837 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 837 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 837 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 837 has memoization hash e12968ac1f0bf5caaa658c5e76e53091 parsl.dataflow.memoization INFO: Task 837 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 837 try 0 launched on executor multi with executor id 838 parsl.dataflow.dflow INFO: Standard output for task 837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/2da20d0b-d507-42ae-937a-83373d5e7f6e_isrForCrosstalkSources_996967_16.stdout parsl.dataflow.dflow INFO: Standard error for task 837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/2da20d0b-d507-42ae-937a-83373d5e7f6e_isrForCrosstalkSources_996967_16.stderr parsl.dataflow.dflow DEBUG: Task 838 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 838 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 838 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 838 has memoization hash 983abc9ae1aa0d0ee3ccc9678d6cd34c parsl.dataflow.memoization INFO: Task 838 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 838 try 0 launched on executor multi with executor id 839 parsl.dataflow.dflow INFO: Standard output for task 838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2c2973c9-2af5-4305-b08d-f6a6b778fcb8_isrForCrosstalkSources_997006_28.stdout parsl.dataflow.dflow INFO: Standard error for task 838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2c2973c9-2af5-4305-b08d-f6a6b778fcb8_isrForCrosstalkSources_997006_28.stderr parsl.dataflow.dflow DEBUG: Task 839 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 839 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 839 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 839 has memoization hash 21cae9655f2acd3512a7fdda0318f9c8 parsl.dataflow.memoization INFO: Task 839 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 839 try 0 launched on executor multi with executor id 840 parsl.dataflow.dflow INFO: Standard output for task 839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4ed2f7ae-eae1-4ec7-9a29-4ed1bad01eaf_isrForCrosstalkSources_996957_39.stdout parsl.dataflow.dflow INFO: Standard error for task 839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4ed2f7ae-eae1-4ec7-9a29-4ed1bad01eaf_isrForCrosstalkSources_996957_39.stderr parsl.dataflow.dflow DEBUG: Task 840 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 840 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 840 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 840 has memoization hash 9fa04e4a2f8bed4d4e87fd50b3954599 parsl.dataflow.memoization INFO: Task 840 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 840 try 0 launched on executor multi with executor id 841 parsl.dataflow.dflow INFO: Standard output for task 840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/bf4daddf-5aad-46ca-8893-96027080de84_isrForCrosstalkSources_996980_1.stdout parsl.dataflow.dflow INFO: Standard error for task 840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/bf4daddf-5aad-46ca-8893-96027080de84_isrForCrosstalkSources_996980_1.stderr parsl.dataflow.dflow DEBUG: Task 841 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 841 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 841 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 841 has memoization hash 63740c5294d92374fefa9e5be90f8508 parsl.dataflow.memoization INFO: Task 841 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 841 try 0 launched on executor multi with executor id 842 parsl.dataflow.dflow INFO: Standard output for task 841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6658601f-552d-49ff-8931-1158da6844f3_isrForCrosstalkSources_996953_8.stdout parsl.dataflow.dflow INFO: Standard error for task 841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6658601f-552d-49ff-8931-1158da6844f3_isrForCrosstalkSources_996953_8.stderr parsl.dataflow.dflow DEBUG: Task 842 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 842 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 842 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 842 has memoization hash 3606504e08ba87c4547b0b8cd01359ff parsl.dataflow.memoization INFO: Task 842 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 842 try 0 launched on executor multi with executor id 843 parsl.dataflow.dflow INFO: Standard output for task 842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c9e6976a-92c0-4856-a2ed-21520273d226_isrForCrosstalkSources_997016_58.stdout parsl.dataflow.dflow INFO: Standard error for task 842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c9e6976a-92c0-4856-a2ed-21520273d226_isrForCrosstalkSources_997016_58.stderr parsl.dataflow.dflow DEBUG: Task 843 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 843 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 843 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 843 has memoization hash 24bd06dca7c9dd9657a89fc6b7b12fe5 parsl.dataflow.memoization INFO: Task 843 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 843 try 0 launched on executor multi with executor id 844 parsl.dataflow.dflow INFO: Standard output for task 843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/9d01e152-0937-479c-b08e-cd72618e5803_isrForCrosstalkSources_997007_18.stdout parsl.dataflow.dflow INFO: Standard error for task 843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/9d01e152-0937-479c-b08e-cd72618e5803_isrForCrosstalkSources_997007_18.stderr parsl.dataflow.dflow DEBUG: Task 844 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 844 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 844 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 844 has memoization hash 14a54057b41a2f95180e180c1c1e2eed parsl.dataflow.memoization INFO: Task 844 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 844 try 0 launched on executor multi with executor id 845 parsl.dataflow.dflow INFO: Standard output for task 844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6e3c1ab9-cf44-4e9d-8e03-125bdd2ea00c_isrForCrosstalkSources_997005_23.stdout parsl.dataflow.dflow INFO: Standard error for task 844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6e3c1ab9-cf44-4e9d-8e03-125bdd2ea00c_isrForCrosstalkSources_997005_23.stderr parsl.dataflow.dflow DEBUG: Task 845 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 845 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 845 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 845 has memoization hash 3502d2cfeea2e86f2b2612577da421f8 parsl.dataflow.memoization INFO: Task 845 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 845 try 0 launched on executor multi with executor id 846 parsl.dataflow.dflow INFO: Standard output for task 845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/9c55f14b-f320-4604-8769-c46ac3d164d4_isrForCrosstalkSources_996976_60.stdout parsl.dataflow.dflow INFO: Standard error for task 845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/9c55f14b-f320-4604-8769-c46ac3d164d4_isrForCrosstalkSources_996976_60.stderr parsl.dataflow.dflow DEBUG: Task 846 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 846 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 846 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 846 has memoization hash 0de30b96b7118663caaa98ecae48aeb4 parsl.dataflow.memoization INFO: Task 846 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 846 try 0 launched on executor multi with executor id 847 parsl.dataflow.dflow INFO: Standard output for task 846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d46630ab-2b66-48bb-a667-12dee0a04f11_isrForCrosstalkSources_997023_27.stdout parsl.dataflow.dflow INFO: Standard error for task 846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d46630ab-2b66-48bb-a667-12dee0a04f11_isrForCrosstalkSources_997023_27.stderr parsl.dataflow.dflow DEBUG: Task 847 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 847 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 847 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 847 has memoization hash 11e164e39c42f3857e5dd0fd16f5e207 parsl.dataflow.memoization INFO: Task 847 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 847 try 0 launched on executor multi with executor id 848 parsl.dataflow.dflow INFO: Standard output for task 847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b096b323-343f-4f50-aa25-7b1c92389a86_isrForCrosstalkSources_997025_36.stdout parsl.dataflow.dflow INFO: Standard error for task 847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b096b323-343f-4f50-aa25-7b1c92389a86_isrForCrosstalkSources_997025_36.stderr parsl.dataflow.dflow DEBUG: Task 848 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 848 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 848 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 848 has memoization hash 286cbb1460d79f9126ad3a3afebb00ae parsl.dataflow.memoization INFO: Task 848 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 848 try 0 launched on executor multi with executor id 849 parsl.dataflow.dflow INFO: Standard output for task 848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/0fcb4ad5-a808-4600-bd33-6bf8441f6544_isrForCrosstalkSources_997006_61.stdout parsl.dataflow.dflow INFO: Standard error for task 848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/0fcb4ad5-a808-4600-bd33-6bf8441f6544_isrForCrosstalkSources_997006_61.stderr parsl.dataflow.dflow DEBUG: Task 849 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 849 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 849 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 849 has memoization hash 943205567d13dbe12e95ca91bbca6a43 parsl.dataflow.memoization INFO: Task 849 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 849 try 0 launched on executor multi with executor id 850 parsl.dataflow.dflow INFO: Standard output for task 849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5f843ac6-a6f8-40ac-94ac-a36cea010fd2_isrForCrosstalkSources_997021_23.stdout parsl.dataflow.dflow INFO: Standard error for task 849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5f843ac6-a6f8-40ac-94ac-a36cea010fd2_isrForCrosstalkSources_997021_23.stderr parsl.dataflow.dflow DEBUG: Task 850 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 850 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 850 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 850 has memoization hash 517723fc197b5e3dbe3a16ace3723059 parsl.dataflow.memoization INFO: Task 850 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 850 try 0 launched on executor multi with executor id 851 parsl.dataflow.dflow INFO: Standard output for task 850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/1a2a22e5-6895-4062-91ba-34c3df47123c_isrForCrosstalkSources_996972_46.stdout parsl.dataflow.dflow INFO: Standard error for task 850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/1a2a22e5-6895-4062-91ba-34c3df47123c_isrForCrosstalkSources_996972_46.stderr parsl.dataflow.dflow DEBUG: Task 851 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 851 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 851 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 851 has memoization hash 13a97d8d239e36caddbc8a9a4fe88d00 parsl.dataflow.memoization INFO: Task 851 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 851 try 0 launched on executor multi with executor id 852 parsl.dataflow.dflow INFO: Standard output for task 851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d6e81ec2-6454-44bb-8b5d-eae93d50ef46_isrForCrosstalkSources_997024_30.stdout parsl.dataflow.dflow INFO: Standard error for task 851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d6e81ec2-6454-44bb-8b5d-eae93d50ef46_isrForCrosstalkSources_997024_30.stderr parsl.dataflow.dflow DEBUG: Task 852 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 852 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 852 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 852 has memoization hash 3bd8192bd4cc94b62cfd7c566b9838af parsl.dataflow.memoization INFO: Task 852 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 852 try 0 launched on executor multi with executor id 853 parsl.dataflow.dflow INFO: Standard output for task 852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/42a28c6a-255b-4c36-afc4-30528cf0b391_isrForCrosstalkSources_997001_20.stdout parsl.dataflow.dflow INFO: Standard error for task 852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/42a28c6a-255b-4c36-afc4-30528cf0b391_isrForCrosstalkSources_997001_20.stderr parsl.dataflow.dflow DEBUG: Task 853 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 853 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 853 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 853 has memoization hash 108d6bbd2d978d631576498d1965c79d parsl.dataflow.memoization INFO: Task 853 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 853 try 0 launched on executor multi with executor id 854 parsl.dataflow.dflow INFO: Standard output for task 853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e50c54d5-1d64-49b5-a7a4-0422c3c54774_isrForCrosstalkSources_996998_55.stdout parsl.dataflow.dflow INFO: Standard error for task 853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e50c54d5-1d64-49b5-a7a4-0422c3c54774_isrForCrosstalkSources_996998_55.stderr parsl.dataflow.dflow DEBUG: Task 854 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 854 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 854 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 854 has memoization hash 9b7e5fd1940b87b9f719896ace3c22fb parsl.dataflow.memoization INFO: Task 854 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 854 try 0 launched on executor multi with executor id 855 parsl.dataflow.dflow INFO: Standard output for task 854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/cd284f19-3ba6-4d71-a6eb-47d17fad29a0_isrForCrosstalkSources_997033_25.stdout parsl.dataflow.dflow INFO: Standard error for task 854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/cd284f19-3ba6-4d71-a6eb-47d17fad29a0_isrForCrosstalkSources_997033_25.stderr parsl.dataflow.dflow DEBUG: Task 855 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 855 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 855 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 855 has memoization hash 6997a60a76ff25a8dd03b53cce27cce1 parsl.dataflow.memoization INFO: Task 855 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 855 try 0 launched on executor multi with executor id 856 parsl.dataflow.dflow INFO: Standard output for task 855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/15761c2f-5f8b-4842-b424-d9dcd48f5355_isrForCrosstalkSources_996971_40.stdout parsl.dataflow.dflow INFO: Standard error for task 855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/15761c2f-5f8b-4842-b424-d9dcd48f5355_isrForCrosstalkSources_996971_40.stderr parsl.dataflow.dflow DEBUG: Task 856 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 856 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 856 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 856 has memoization hash 9db9b880dd992520737555988be6fa27 parsl.dataflow.memoization INFO: Task 856 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 856 try 0 launched on executor multi with executor id 857 parsl.dataflow.dflow INFO: Standard output for task 856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4bd46426-edf0-4290-b8ca-e4c850bc00d8_isrForCrosstalkSources_996980_19.stdout parsl.dataflow.dflow INFO: Standard error for task 856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4bd46426-edf0-4290-b8ca-e4c850bc00d8_isrForCrosstalkSources_996980_19.stderr parsl.dataflow.dflow DEBUG: Task 857 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 857 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 857 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 857 has memoization hash bf072a122c645a573e91abd591041c62 parsl.dataflow.memoization INFO: Task 857 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 857 try 0 launched on executor multi with executor id 858 parsl.dataflow.dflow INFO: Standard output for task 857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/50478ef0-320b-4825-9d41-caf945eed432_isrForCrosstalkSources_996998_36.stdout parsl.dataflow.dflow INFO: Standard error for task 857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/50478ef0-320b-4825-9d41-caf945eed432_isrForCrosstalkSources_996998_36.stderr parsl.dataflow.dflow DEBUG: Task 858 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 858 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 858 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 858 has memoization hash c53330bde1632434f304776caf45b595 parsl.dataflow.memoization INFO: Task 858 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 858 try 0 launched on executor multi with executor id 859 parsl.dataflow.dflow INFO: Standard output for task 858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/36416d78-1716-4301-812f-4838f5fafd5e_isrForCrosstalkSources_996985_51.stdout parsl.dataflow.dflow INFO: Standard error for task 858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/36416d78-1716-4301-812f-4838f5fafd5e_isrForCrosstalkSources_996985_51.stderr parsl.dataflow.dflow DEBUG: Task 859 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 859 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 859 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 859 has memoization hash 4a19ac2cd88fdfb294bf25da70ccc40a parsl.dataflow.memoization INFO: Task 859 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 859 try 0 launched on executor multi with executor id 860 parsl.dataflow.dflow INFO: Standard output for task 859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/23923201-b7b8-4937-993c-5d959842cbb9_isrForCrosstalkSources_996977_31.stdout parsl.dataflow.dflow INFO: Standard error for task 859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/23923201-b7b8-4937-993c-5d959842cbb9_isrForCrosstalkSources_996977_31.stderr parsl.dataflow.dflow DEBUG: Task 860 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 860 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 860 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 860 has memoization hash b198d6dd548fc07a9cd3d98cdd6a5b43 parsl.dataflow.memoization INFO: Task 860 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:32 proc_lsst.multi:170 [INFO] [multi] job_id 17074843 proc_lsst.multi INFO: [multi] job_id 17074843 2024-03-15 13:49:32 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 10 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 10 2024-03-15 13:49:32 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:32 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074843 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074843 parsl.executors.status_handling DEBUG: Launched block 10 on executor multi with job ID 17074843 parsl.executors.status_handling INFO: Allocated block ID 11 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-11 2024-03-15 13:49:32 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=11 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-11 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=11 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-11 2024-03-15 13:49:32 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=11 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=11 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:32 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=11 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=11 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 10 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 10 2024-03-15 13:49:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:32 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=11 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=11 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 860 try 0 launched on executor multi with executor id 861 parsl.dataflow.dflow INFO: Standard output for task 860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/33af21de-41ef-4aff-9c57-69232f5a3d80_isrForCrosstalkSources_997001_12.stdout parsl.dataflow.dflow INFO: Standard error for task 860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/33af21de-41ef-4aff-9c57-69232f5a3d80_isrForCrosstalkSources_997001_12.stderr parsl.dataflow.dflow DEBUG: Task 861 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 861 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 861 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 861 has memoization hash 528049d80139259db2ff20a99323ff89 parsl.dataflow.memoization INFO: Task 861 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 861 try 0 launched on executor multi with executor id 862 parsl.dataflow.dflow INFO: Standard output for task 861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/48fe4706-01ff-4eae-9b85-81da83342006_isrForCrosstalkSources_996998_42.stdout parsl.dataflow.dflow INFO: Standard error for task 861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/48fe4706-01ff-4eae-9b85-81da83342006_isrForCrosstalkSources_996998_42.stderr parsl.dataflow.dflow DEBUG: Task 862 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 862 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 862 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 862 has memoization hash 91c0f957d2bb4d67471348a5a42779ec parsl.dataflow.memoization INFO: Task 862 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808323e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 862 try 0 launched on executor multi with executor id 863 parsl.dataflow.dflow INFO: Standard output for task 862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/689785a9-9627-4338-982b-bfe7d9be7bb6_isrForCrosstalkSources_996981_23.stdout parsl.dataflow.dflow INFO: Standard error for task 862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/689785a9-9627-4338-982b-bfe7d9be7bb6_isrForCrosstalkSources_996981_23.stderr parsl.dataflow.dflow DEBUG: Task 863 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 863 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 863 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 863 has memoization hash 4b376361d0c9b261951e166ccd0023db parsl.dataflow.memoization INFO: Task 863 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806b8b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 863 try 0 launched on executor multi with executor id 864 parsl.dataflow.dflow INFO: Standard output for task 863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/5d41d401-6a3f-4ee1-b6c6-21388a081e72_isrForCrosstalkSources_997029_41.stdout parsl.dataflow.dflow INFO: Standard error for task 863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/5d41d401-6a3f-4ee1-b6c6-21388a081e72_isrForCrosstalkSources_997029_41.stderr parsl.dataflow.dflow DEBUG: Task 864 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 864 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 864 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 864 has memoization hash 896ca41621eaf5bd01bffd82c4e2f30a parsl.dataflow.memoization INFO: Task 864 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7806bae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 864 try 0 launched on executor multi with executor id 865 parsl.dataflow.dflow INFO: Standard output for task 864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6e2335ae-aa6e-4a63-842e-fc256d554263_isrForCrosstalkSources_997017_34.stdout parsl.dataflow.dflow INFO: Standard error for task 864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6e2335ae-aa6e-4a63-842e-fc256d554263_isrForCrosstalkSources_997017_34.stderr parsl.dataflow.dflow DEBUG: Task 865 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 865 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 865 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 865 has memoization hash d28e8cfc82ca2ca8d5043192902c33bf parsl.dataflow.memoization INFO: Task 865 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780832520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 865 try 0 launched on executor multi with executor id 866 parsl.dataflow.dflow INFO: Standard output for task 865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4952ac8e-eece-479e-9f84-32a5cf52589c_isrForCrosstalkSources_997023_44.stdout parsl.dataflow.dflow INFO: Standard error for task 865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4952ac8e-eece-479e-9f84-32a5cf52589c_isrForCrosstalkSources_997023_44.stderr parsl.dataflow.dflow DEBUG: Task 866 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 866 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 866 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 866 has memoization hash e9c9569119eaf919a143ec8a656989b2 parsl.dataflow.memoization INFO: Task 866 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 866 try 0 launched on executor multi with executor id 867 parsl.dataflow.dflow INFO: Standard output for task 866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2edc0ca3-5bba-4eb6-9d13-9c4a50929fe7_isrForCrosstalkSources_997015_41.stdout parsl.dataflow.dflow INFO: Standard error for task 866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2edc0ca3-5bba-4eb6-9d13-9c4a50929fe7_isrForCrosstalkSources_997015_41.stderr parsl.dataflow.dflow DEBUG: Task 867 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 867 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 867 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 867 has memoization hash 9d445df06c2a123b6728c1ad01c1f770 parsl.dataflow.memoization INFO: Task 867 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 867 try 0 launched on executor multi with executor id 868 parsl.dataflow.dflow INFO: Standard output for task 867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/e093d761-6ee3-4fe8-8053-12c6a8b01634_isrForCrosstalkSources_997014_61.stdout parsl.dataflow.dflow INFO: Standard error for task 867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/e093d761-6ee3-4fe8-8053-12c6a8b01634_isrForCrosstalkSources_997014_61.stderr parsl.dataflow.dflow DEBUG: Task 868 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 868 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 868 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 868 has memoization hash 7b10cf47d33b2a8e489b4dee39d255ff parsl.dataflow.memoization INFO: Task 868 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 868 try 0 launched on executor multi with executor id 869 parsl.dataflow.dflow INFO: Standard output for task 868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/73f958b4-a0fe-47c5-b3d9-f47cd9b05766_isrForCrosstalkSources_996984_56.stdout parsl.dataflow.dflow INFO: Standard error for task 868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/73f958b4-a0fe-47c5-b3d9-f47cd9b05766_isrForCrosstalkSources_996984_56.stderr parsl.dataflow.dflow DEBUG: Task 869 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 869 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 869 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 869 has memoization hash e487ef719eae24caca31ef3e86a57d88 parsl.dataflow.memoization INFO: Task 869 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 869 try 0 launched on executor multi with executor id 870 parsl.dataflow.dflow INFO: Standard output for task 869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e1c00d3e-fb58-42bd-98fa-d8f3bf11ff8f_isrForCrosstalkSources_996984_38.stdout parsl.dataflow.dflow INFO: Standard error for task 869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e1c00d3e-fb58-42bd-98fa-d8f3bf11ff8f_isrForCrosstalkSources_996984_38.stderr parsl.dataflow.dflow DEBUG: Task 870 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 870 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 870 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 870 has memoization hash cb54e92eb721e7840567890d898a5340 parsl.dataflow.memoization INFO: Task 870 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 870 try 0 launched on executor multi with executor id 871 parsl.dataflow.dflow INFO: Standard output for task 870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a37801f2-ba16-4ceb-8024-f10a781bae34_isrForCrosstalkSources_996952_6.stdout parsl.dataflow.dflow INFO: Standard error for task 870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a37801f2-ba16-4ceb-8024-f10a781bae34_isrForCrosstalkSources_996952_6.stderr parsl.dataflow.dflow DEBUG: Task 871 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 871 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 871 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 871 has memoization hash 048678a0f781e5c9d842328124125306 parsl.dataflow.memoization INFO: Task 871 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 871 try 0 launched on executor multi with executor id 872 parsl.dataflow.dflow INFO: Standard output for task 871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/11403d84-e6ae-4f71-b35a-9d068e881a29_isrForCrosstalkSources_996958_23.stdout parsl.dataflow.dflow INFO: Standard error for task 871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/11403d84-e6ae-4f71-b35a-9d068e881a29_isrForCrosstalkSources_996958_23.stderr parsl.dataflow.dflow DEBUG: Task 872 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 872 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 872 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 872 has memoization hash 8906d80646c22c66b4b148c8b96d8c16 parsl.dataflow.memoization INFO: Task 872 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 872 try 0 launched on executor multi with executor id 873 parsl.dataflow.dflow INFO: Standard output for task 872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f9e149c2-36c4-4ec8-950f-98df23e21f3a_isrForCrosstalkSources_996985_41.stdout parsl.dataflow.dflow INFO: Standard error for task 872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f9e149c2-36c4-4ec8-950f-98df23e21f3a_isrForCrosstalkSources_996985_41.stderr parsl.dataflow.dflow DEBUG: Task 873 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 873 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 873 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 873 has memoization hash 2e45b93f6ed6bd5e5959be1f66a03aac parsl.dataflow.memoization INFO: Task 873 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 873 try 0 launched on executor multi with executor id 874 parsl.dataflow.dflow INFO: Standard output for task 873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/67c58fca-9983-4ac5-8b81-c3e427823f83_isrForCrosstalkSources_997003_39.stdout parsl.dataflow.dflow INFO: Standard error for task 873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/67c58fca-9983-4ac5-8b81-c3e427823f83_isrForCrosstalkSources_997003_39.stderr parsl.dataflow.dflow DEBUG: Task 874 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 874 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 874 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 874 has memoization hash 8ea29cbfb5f923fad0520e284540028f parsl.dataflow.memoization INFO: Task 874 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 874 try 0 launched on executor multi with executor id 875 parsl.dataflow.dflow INFO: Standard output for task 874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/f2934c9f-84cc-446b-a378-698eb026f235_isrForCrosstalkSources_997011_38.stdout parsl.dataflow.dflow INFO: Standard error for task 874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/f2934c9f-84cc-446b-a378-698eb026f235_isrForCrosstalkSources_997011_38.stderr parsl.dataflow.dflow DEBUG: Task 875 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 875 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 875 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 875 has memoization hash bdfa010ebd8e7b1722636b7d371c4aa2 parsl.dataflow.memoization INFO: Task 875 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 875 try 0 launched on executor multi with executor id 876 parsl.dataflow.dflow INFO: Standard output for task 875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ec4a5c9f-aaa3-44ab-981d-e91b2c42f03e_isrForCrosstalkSources_997013_18.stdout parsl.dataflow.dflow INFO: Standard error for task 875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ec4a5c9f-aaa3-44ab-981d-e91b2c42f03e_isrForCrosstalkSources_997013_18.stderr parsl.dataflow.dflow DEBUG: Task 876 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 876 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 876 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 876 has memoization hash 7a0cd4e5f4052c26c892c643a78a5a3f parsl.dataflow.memoization INFO: Task 876 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 876 try 0 launched on executor multi with executor id 877 parsl.dataflow.dflow INFO: Standard output for task 876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/140fcced-5a0b-4934-ab88-eaa82c764c31_isrForCrosstalkSources_996962_35.stdout parsl.dataflow.dflow INFO: Standard error for task 876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/140fcced-5a0b-4934-ab88-eaa82c764c31_isrForCrosstalkSources_996962_35.stderr parsl.dataflow.dflow DEBUG: Task 877 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 877 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 877 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 877 has memoization hash ae3a98d47db5da13d6790f406915dd58 parsl.dataflow.memoization INFO: Task 877 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 877 try 0 launched on executor multi with executor id 878 parsl.dataflow.dflow INFO: Standard output for task 877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/64adc7cd-8992-415f-80fc-bbfd93498ac9_isrForCrosstalkSources_997024_53.stdout parsl.dataflow.dflow INFO: Standard error for task 877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/64adc7cd-8992-415f-80fc-bbfd93498ac9_isrForCrosstalkSources_997024_53.stderr parsl.dataflow.dflow DEBUG: Task 878 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 878 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 878 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 878 has memoization hash d7538fa45e1a983a15370074cb27cfe4 parsl.dataflow.memoization INFO: Task 878 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 878 try 0 launched on executor multi with executor id 879 parsl.dataflow.dflow INFO: Standard output for task 878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9badf252-476e-481c-b06e-2da33f4bd4dd_isrForCrosstalkSources_997000_11.stdout parsl.dataflow.dflow INFO: Standard error for task 878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9badf252-476e-481c-b06e-2da33f4bd4dd_isrForCrosstalkSources_997000_11.stderr parsl.dataflow.dflow DEBUG: Task 879 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 879 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 879 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 879 has memoization hash ca52247284882d7c29acd7b39efdb6cf parsl.dataflow.memoization INFO: Task 879 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 879 try 0 launched on executor multi with executor id 880 parsl.dataflow.dflow INFO: Standard output for task 879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/69ec2d5a-0744-4ff5-bb18-5c43f9ae9624_isrForCrosstalkSources_997022_44.stdout parsl.dataflow.dflow INFO: Standard error for task 879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/69ec2d5a-0744-4ff5-bb18-5c43f9ae9624_isrForCrosstalkSources_997022_44.stderr parsl.dataflow.dflow DEBUG: Task 880 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 880 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 880 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 880 has memoization hash 5fb4fc98c0f9568e373041933366c10e parsl.dataflow.memoization INFO: Task 880 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 880 try 0 launched on executor multi with executor id 881 parsl.dataflow.dflow INFO: Standard output for task 880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f0651d85-f576-4ba7-9643-a18a2a503875_isrForCrosstalkSources_996958_43.stdout parsl.dataflow.dflow INFO: Standard error for task 880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f0651d85-f576-4ba7-9643-a18a2a503875_isrForCrosstalkSources_996958_43.stderr parsl.dataflow.dflow DEBUG: Task 881 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 881 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 881 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 881 has memoization hash 5781bc3e80b769bde60b57fd8c143e3b parsl.dataflow.memoization INFO: Task 881 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 881 try 0 launched on executor multi with executor id 882 parsl.dataflow.dflow INFO: Standard output for task 881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/48b9f70b-51df-44cd-8f7d-b9a91415ef73_isrForCrosstalkSources_996974_29.stdout parsl.dataflow.dflow INFO: Standard error for task 881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/48b9f70b-51df-44cd-8f7d-b9a91415ef73_isrForCrosstalkSources_996974_29.stderr parsl.dataflow.dflow DEBUG: Task 882 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 882 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 882 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 882 has memoization hash 11d93210664b8cfc3e0afeeb71cfdca8 parsl.dataflow.memoization INFO: Task 882 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 882 try 0 launched on executor multi with executor id 883 parsl.dataflow.dflow INFO: Standard output for task 882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ca5003ea-f12f-43db-b9cf-9b09969b76e9_isrForCrosstalkSources_997001_57.stdout parsl.dataflow.dflow INFO: Standard error for task 882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ca5003ea-f12f-43db-b9cf-9b09969b76e9_isrForCrosstalkSources_997001_57.stderr parsl.dataflow.dflow DEBUG: Task 883 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 883 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 883 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 883 has memoization hash 56cb29e6dfe11d2ebdf9c01f5017a83e parsl.dataflow.memoization INFO: Task 883 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 883 try 0 launched on executor multi with executor id 884 parsl.dataflow.dflow INFO: Standard output for task 883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c4e8592d-cd9a-4848-b286-d692e6d05385_isrForCrosstalkSources_996961_26.stdout parsl.dataflow.dflow INFO: Standard error for task 883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c4e8592d-cd9a-4848-b286-d692e6d05385_isrForCrosstalkSources_996961_26.stderr parsl.dataflow.dflow DEBUG: Task 884 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 884 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 884 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 884 has memoization hash 1aecc53a54a648885b8d63ea3f16c30c parsl.dataflow.memoization INFO: Task 884 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 884 try 0 launched on executor multi with executor id 885 parsl.dataflow.dflow INFO: Standard output for task 884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/76b604b9-ce9d-49b0-8c3e-8222e0d94859_isrForCrosstalkSources_997015_56.stdout parsl.dataflow.dflow INFO: Standard error for task 884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/76b604b9-ce9d-49b0-8c3e-8222e0d94859_isrForCrosstalkSources_997015_56.stderr parsl.dataflow.dflow DEBUG: Task 885 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 885 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 885 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 885 has memoization hash c83e84a2ea4583102125bd7d112afae0 parsl.dataflow.memoization INFO: Task 885 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 885 try 0 launched on executor multi with executor id 886 parsl.dataflow.dflow INFO: Standard output for task 885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5f86934a-6c23-4622-b8c4-d083df4e64da_isrForCrosstalkSources_996981_10.stdout parsl.dataflow.dflow INFO: Standard error for task 885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5f86934a-6c23-4622-b8c4-d083df4e64da_isrForCrosstalkSources_996981_10.stderr parsl.dataflow.dflow DEBUG: Task 886 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 886 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 886 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 886 has memoization hash d82c2c17d2ca45d4bedd7a5dfd0708c8 parsl.dataflow.memoization INFO: Task 886 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 886 try 0 launched on executor multi with executor id 887 parsl.dataflow.dflow INFO: Standard output for task 886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/38a84872-3357-4a00-88ad-1da6b5d1c5f8_isrForCrosstalkSources_996980_12.stdout parsl.dataflow.dflow INFO: Standard error for task 886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/38a84872-3357-4a00-88ad-1da6b5d1c5f8_isrForCrosstalkSources_996980_12.stderr parsl.dataflow.dflow DEBUG: Task 887 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 887 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 887 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 887 has memoization hash 694f8ab1a0427b2e747ea4192cee735f parsl.dataflow.memoization INFO: Task 887 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 887 try 0 launched on executor multi with executor id 888 parsl.dataflow.dflow INFO: Standard output for task 887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cca988b9-6b14-4ca7-99cc-82e2e308309e_isrForCrosstalkSources_996957_48.stdout parsl.dataflow.dflow INFO: Standard error for task 887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cca988b9-6b14-4ca7-99cc-82e2e308309e_isrForCrosstalkSources_996957_48.stderr parsl.dataflow.dflow DEBUG: Task 888 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 888 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 888 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 888 has memoization hash 5780fd166239f198dbcbadf0bc88d73a parsl.dataflow.memoization INFO: Task 888 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 888 try 0 launched on executor multi with executor id 889 parsl.dataflow.dflow INFO: Standard output for task 888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/77334703-578e-4c8b-9a24-36fd39683198_isrForCrosstalkSources_996978_4.stdout parsl.dataflow.dflow INFO: Standard error for task 888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/77334703-578e-4c8b-9a24-36fd39683198_isrForCrosstalkSources_996978_4.stderr parsl.dataflow.dflow DEBUG: Task 889 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 889 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 889 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 889 has memoization hash 729af41cd3da3d252154e1117afae1fb parsl.dataflow.memoization INFO: Task 889 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 889 try 0 launched on executor multi with executor id 890 parsl.dataflow.dflow INFO: Standard output for task 889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d459c8fb-384e-40de-be28-77f93a7c8efc_isrForCrosstalkSources_997004_32.stdout parsl.dataflow.dflow INFO: Standard error for task 889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d459c8fb-384e-40de-be28-77f93a7c8efc_isrForCrosstalkSources_997004_32.stderr parsl.dataflow.dflow DEBUG: Task 890 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 890 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 890 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 890 has memoization hash 9e0894935fe867e46f70ed68b78142e4 parsl.dataflow.memoization INFO: Task 890 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 890 try 0 launched on executor multi with executor id 891 parsl.dataflow.dflow INFO: Standard output for task 890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/c23b3f79-7fcd-447f-8435-1f25f3137205_isrForCrosstalkSources_996971_2.stdout parsl.dataflow.dflow INFO: Standard error for task 890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/c23b3f79-7fcd-447f-8435-1f25f3137205_isrForCrosstalkSources_996971_2.stderr parsl.dataflow.dflow DEBUG: Task 891 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 891 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 891 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 891 has memoization hash 410883af4a2fb07a1ae03f7aff26d3e0 parsl.dataflow.memoization INFO: Task 891 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 891 try 0 launched on executor multi with executor id 892 parsl.dataflow.dflow INFO: Standard output for task 891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/078bb60e-9f36-45f0-bdd0-ab665851e6ab_isrForCrosstalkSources_996958_11.stdout parsl.dataflow.dflow INFO: Standard error for task 891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/078bb60e-9f36-45f0-bdd0-ab665851e6ab_isrForCrosstalkSources_996958_11.stderr parsl.dataflow.dflow DEBUG: Task 892 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 892 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 892 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 892 has memoization hash 34cd882dc65a5dc67c7a62b7875005f4 parsl.dataflow.memoization INFO: Task 892 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 892 try 0 launched on executor multi with executor id 893 parsl.dataflow.dflow INFO: Standard output for task 892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/af87b50b-6fab-4f2a-a259-203dd4af9cba_isrForCrosstalkSources_996996_2.stdout parsl.dataflow.dflow INFO: Standard error for task 892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/af87b50b-6fab-4f2a-a259-203dd4af9cba_isrForCrosstalkSources_996996_2.stderr parsl.dataflow.dflow DEBUG: Task 893 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 893 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 893 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 893 has memoization hash 39ec81c76a19bfe9a96dddb8f6947ebe parsl.dataflow.memoization INFO: Task 893 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 893 try 0 launched on executor multi with executor id 894 parsl.dataflow.dflow INFO: Standard output for task 893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4ce504f4-b2f5-48f0-9398-725b4f11cecb_isrForCrosstalkSources_996966_47.stdout parsl.dataflow.dflow INFO: Standard error for task 893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4ce504f4-b2f5-48f0-9398-725b4f11cecb_isrForCrosstalkSources_996966_47.stderr parsl.dataflow.dflow DEBUG: Task 894 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 894 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 894 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 894 has memoization hash 3778b3774d9d61af5f54ca6bb119607c parsl.dataflow.memoization INFO: Task 894 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 894 try 0 launched on executor multi with executor id 895 parsl.dataflow.dflow INFO: Standard output for task 894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8543486d-22f9-415b-96c3-51e5b7360b6c_isrForCrosstalkSources_996973_3.stdout parsl.dataflow.dflow INFO: Standard error for task 894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8543486d-22f9-415b-96c3-51e5b7360b6c_isrForCrosstalkSources_996973_3.stderr parsl.dataflow.dflow DEBUG: Task 895 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 895 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 895 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 895 has memoization hash c43a99b912a8d2c3d761a0f944550065 parsl.dataflow.memoization INFO: Task 895 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 895 try 0 launched on executor multi with executor id 896 parsl.dataflow.dflow INFO: Standard output for task 895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/db96f8d0-dafb-42a1-b45f-dcbc3798812e_isrForCrosstalkSources_997009_32.stdout parsl.dataflow.dflow INFO: Standard error for task 895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/db96f8d0-dafb-42a1-b45f-dcbc3798812e_isrForCrosstalkSources_997009_32.stderr parsl.dataflow.dflow DEBUG: Task 896 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 896 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 896 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 896 has memoization hash e24302ab3ed7dba1fd12aa6bb00792b6 parsl.dataflow.memoization INFO: Task 896 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 896 try 0 launched on executor multi with executor id 897 parsl.dataflow.dflow INFO: Standard output for task 896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/577b0b5b-f25a-4459-93bf-44ce45f67398_isrForCrosstalkSources_996980_11.stdout parsl.dataflow.dflow INFO: Standard error for task 896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/577b0b5b-f25a-4459-93bf-44ce45f67398_isrForCrosstalkSources_996980_11.stderr parsl.dataflow.dflow DEBUG: Task 897 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 897 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 897 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 897 has memoization hash 4d0278ea22ffe7a67f5432c7685c7f96 parsl.dataflow.memoization INFO: Task 897 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 897 try 0 launched on executor multi with executor id 898 parsl.dataflow.dflow INFO: Standard output for task 897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/51af2634-d344-403a-b1e5-2a55d5dd94a1_isrForCrosstalkSources_997025_49.stdout parsl.dataflow.dflow INFO: Standard error for task 897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/51af2634-d344-403a-b1e5-2a55d5dd94a1_isrForCrosstalkSources_997025_49.stderr parsl.dataflow.dflow DEBUG: Task 898 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 898 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 898 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 898 has memoization hash d77c3307d4c8e60a4723994ae126f0f7 parsl.dataflow.memoization INFO: Task 898 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 898 try 0 launched on executor multi with executor id 899 parsl.dataflow.dflow INFO: Standard output for task 898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e9f6317e-3d05-4641-8f85-b5833ff56456_isrForCrosstalkSources_997030_33.stdout parsl.dataflow.dflow INFO: Standard error for task 898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e9f6317e-3d05-4641-8f85-b5833ff56456_isrForCrosstalkSources_997030_33.stderr parsl.dataflow.dflow DEBUG: Task 899 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 899 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 899 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 899 has memoization hash dcfc383b90bc31544e46fb067eaf5b27 parsl.dataflow.memoization INFO: Task 899 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 899 try 0 launched on executor multi with executor id 900 parsl.dataflow.dflow INFO: Standard output for task 899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/69751147-9f54-490b-997e-a6ed8d44ce2d_isrForCrosstalkSources_996994_60.stdout parsl.dataflow.dflow INFO: Standard error for task 899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/69751147-9f54-490b-997e-a6ed8d44ce2d_isrForCrosstalkSources_996994_60.stderr parsl.dataflow.dflow DEBUG: Task 900 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 900 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 900 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 900 has memoization hash 9a18dd296a151d9331e17ae24c0eaded parsl.dataflow.memoization INFO: Task 900 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 900 try 0 launched on executor multi with executor id 901 parsl.dataflow.dflow INFO: Standard output for task 900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7eb842b6-5218-4889-9022-0090fa97523e_isrForCrosstalkSources_996988_20.stdout parsl.dataflow.dflow INFO: Standard error for task 900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7eb842b6-5218-4889-9022-0090fa97523e_isrForCrosstalkSources_996988_20.stderr parsl.dataflow.dflow DEBUG: Task 901 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 901 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 901 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 901 has memoization hash cf469f55167ad53062c75b6593d3e24c parsl.dataflow.memoization INFO: Task 901 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 901 try 0 launched on executor multi with executor id 902 parsl.dataflow.dflow INFO: Standard output for task 901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7e21db97-62ca-45c8-badc-a527bc7fe846_isrForCrosstalkSources_996963_43.stdout parsl.dataflow.dflow INFO: Standard error for task 901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7e21db97-62ca-45c8-badc-a527bc7fe846_isrForCrosstalkSources_996963_43.stderr parsl.dataflow.dflow DEBUG: Task 902 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 902 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 902 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 902 has memoization hash 23c66001caf9e0afa4dc16a1ba4504d1 parsl.dataflow.memoization INFO: Task 902 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 902 try 0 launched on executor multi with executor id 903 parsl.dataflow.dflow INFO: Standard output for task 902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5a22932e-b79d-4f9a-a9c7-b45b46f519c1_isrForCrosstalkSources_996964_49.stdout parsl.dataflow.dflow INFO: Standard error for task 902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5a22932e-b79d-4f9a-a9c7-b45b46f519c1_isrForCrosstalkSources_996964_49.stderr parsl.dataflow.dflow DEBUG: Task 903 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 903 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 903 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 903 has memoization hash 8af66b6097780988cdf937c69f56994f parsl.dataflow.memoization INFO: Task 903 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 903 try 0 launched on executor multi with executor id 904 parsl.dataflow.dflow INFO: Standard output for task 903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/3391b787-e7d6-413f-9360-b73f6a5a0ca6_isrForCrosstalkSources_997013_36.stdout parsl.dataflow.dflow INFO: Standard error for task 903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/3391b787-e7d6-413f-9360-b73f6a5a0ca6_isrForCrosstalkSources_997013_36.stderr parsl.dataflow.dflow DEBUG: Task 904 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 904 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 904 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 904 has memoization hash ce591dc577eaae47bbd13a462c78fff8 parsl.dataflow.memoization INFO: Task 904 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 904 try 0 launched on executor multi with executor id 905 parsl.dataflow.dflow INFO: Standard output for task 904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/b8a7c629-9f33-4cfa-be03-3995fc682ead_isrForCrosstalkSources_996978_36.stdout parsl.dataflow.dflow INFO: Standard error for task 904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/b8a7c629-9f33-4cfa-be03-3995fc682ead_isrForCrosstalkSources_996978_36.stderr parsl.dataflow.dflow DEBUG: Task 905 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 905 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 905 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 905 has memoization hash 9b25f278bff2577ea004f846a4d25003 parsl.dataflow.memoization INFO: Task 905 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 905 try 0 launched on executor multi with executor id 906 parsl.dataflow.dflow INFO: Standard output for task 905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7b234ece-b643-4e1a-a225-f281c59125c5_isrForCrosstalkSources_997013_48.stdout parsl.dataflow.dflow INFO: Standard error for task 905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7b234ece-b643-4e1a-a225-f281c59125c5_isrForCrosstalkSources_997013_48.stderr parsl.dataflow.dflow DEBUG: Task 906 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 906 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 906 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 906 has memoization hash 7a664e4326dc50c4d6012793209d25d9 parsl.dataflow.memoization INFO: Task 906 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 906 try 0 launched on executor multi with executor id 907 parsl.dataflow.dflow INFO: Standard output for task 906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/f995f856-fe08-4fe6-ad96-203b4b5c1746_isrForCrosstalkSources_997021_28.stdout parsl.dataflow.dflow INFO: Standard error for task 906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/f995f856-fe08-4fe6-ad96-203b4b5c1746_isrForCrosstalkSources_997021_28.stderr parsl.dataflow.dflow DEBUG: Task 907 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 907 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 907 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 907 has memoization hash d6d13c552f2b5fe83eb0fd74cc01b4e5 parsl.dataflow.memoization INFO: Task 907 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 907 try 0 launched on executor multi with executor id 908 parsl.dataflow.dflow INFO: Standard output for task 907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a8452c15-0488-4d97-a457-f46751b429e4_isrForCrosstalkSources_997015_60.stdout parsl.dataflow.dflow INFO: Standard error for task 907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a8452c15-0488-4d97-a457-f46751b429e4_isrForCrosstalkSources_997015_60.stderr parsl.dataflow.dflow DEBUG: Task 908 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 908 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 908 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 908 has memoization hash fca2e944a4925da6b996e045af6e8f9d parsl.dataflow.memoization INFO: Task 908 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 908 try 0 launched on executor multi with executor id 909 parsl.dataflow.dflow INFO: Standard output for task 908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/fde1c162-ee5f-467d-b6c0-ae62b9bb62a0_isrForCrosstalkSources_997020_7.stdout parsl.dataflow.dflow INFO: Standard error for task 908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/fde1c162-ee5f-467d-b6c0-ae62b9bb62a0_isrForCrosstalkSources_997020_7.stderr parsl.dataflow.dflow DEBUG: Task 909 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 909 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 909 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 909 has memoization hash 3a9d6323247980087320e36c901aafda parsl.dataflow.memoization INFO: Task 909 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 909 try 0 launched on executor multi with executor id 910 parsl.dataflow.dflow INFO: Standard output for task 909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fc58858c-81b5-4a78-b30d-f591547542d1_isrForCrosstalkSources_996961_53.stdout parsl.dataflow.dflow INFO: Standard error for task 909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fc58858c-81b5-4a78-b30d-f591547542d1_isrForCrosstalkSources_996961_53.stderr parsl.dataflow.dflow DEBUG: Task 910 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 910 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 910 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 910 has memoization hash 26b50c3732c8682eed2b3cd933d39cba parsl.dataflow.memoization INFO: Task 910 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a7808325c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 910 try 0 launched on executor multi with executor id 911 parsl.dataflow.dflow INFO: Standard output for task 910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e5fbbbf1-d1ba-495c-bc37-186355434bc6_isrForCrosstalkSources_996968_29.stdout parsl.dataflow.dflow INFO: Standard error for task 910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e5fbbbf1-d1ba-495c-bc37-186355434bc6_isrForCrosstalkSources_996968_29.stderr parsl.dataflow.dflow DEBUG: Task 911 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 911 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 911 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 911 has memoization hash af514919c93f6f68ff3405f31937332a parsl.dataflow.memoization INFO: Task 911 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e740e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 911 try 0 launched on executor multi with executor id 912 parsl.dataflow.dflow INFO: Standard output for task 911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/dbb5486f-ae04-4c75-b035-4bbb3e32b302_isrForCrosstalkSources_997027_46.stdout parsl.dataflow.dflow INFO: Standard error for task 911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/dbb5486f-ae04-4c75-b035-4bbb3e32b302_isrForCrosstalkSources_997027_46.stderr 2024-03-15 13:49:33 proc_lsst.multi:170 [INFO] [multi] job_id 17074850 proc_lsst.multi INFO: [multi] job_id 17074850 2024-03-15 13:49:33 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 11 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 11 2024-03-15 13:49:33 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:33 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074850 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074850 parsl.executors.status_handling DEBUG: Launched block 11 on executor multi with job ID 17074850 parsl.executors.status_handling INFO: Allocated block ID 12 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-12 2024-03-15 13:49:33 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=12 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-12 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=12 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-12 2024-03-15 13:49:33 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=12 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=12 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:33 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=12 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=12 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 11 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 11 2024-03-15 13:49:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:33 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=12 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=12 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 912 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 912 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 912 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 912 has memoization hash 339ce5edc321bcd5beab1644c5e500e9 parsl.dataflow.memoization INFO: Task 912 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 912 try 0 launched on executor multi with executor id 913 parsl.dataflow.dflow INFO: Standard output for task 912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e19a96f9-c988-4d6c-9e43-1bc79d3e2b48_isrForCrosstalkSources_997023_36.stdout parsl.dataflow.dflow INFO: Standard error for task 912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e19a96f9-c988-4d6c-9e43-1bc79d3e2b48_isrForCrosstalkSources_997023_36.stderr parsl.dataflow.dflow DEBUG: Task 913 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 913 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 913 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 913 has memoization hash 6b34bbd5ddcabe7ddda673ad7299d3e8 parsl.dataflow.memoization INFO: Task 913 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 913 try 0 launched on executor multi with executor id 914 parsl.dataflow.dflow INFO: Standard output for task 913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/997e7b38-4c40-42ef-b591-12d25ae07f44_isrForCrosstalkSources_997022_22.stdout parsl.dataflow.dflow INFO: Standard error for task 913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/997e7b38-4c40-42ef-b591-12d25ae07f44_isrForCrosstalkSources_997022_22.stderr parsl.dataflow.dflow DEBUG: Task 914 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 914 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 914 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 914 has memoization hash 282f91d765e9267938ae7cf76ff256b7 parsl.dataflow.memoization INFO: Task 914 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 914 try 0 launched on executor multi with executor id 915 parsl.dataflow.dflow INFO: Standard output for task 914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d93610fe-e27a-40f6-b543-1be9487f0f00_isrForCrosstalkSources_997030_49.stdout parsl.dataflow.dflow INFO: Standard error for task 914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d93610fe-e27a-40f6-b543-1be9487f0f00_isrForCrosstalkSources_997030_49.stderr parsl.dataflow.dflow DEBUG: Task 915 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 915 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 915 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 915 has memoization hash 52ea5a712ed06a63e18739021ad05a44 parsl.dataflow.memoization INFO: Task 915 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 915 try 0 launched on executor multi with executor id 916 parsl.dataflow.dflow INFO: Standard output for task 915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3b8aa232-48da-4b71-9bcd-4307a2e93b53_isrForCrosstalkSources_996955_14.stdout parsl.dataflow.dflow INFO: Standard error for task 915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3b8aa232-48da-4b71-9bcd-4307a2e93b53_isrForCrosstalkSources_996955_14.stderr parsl.dataflow.dflow DEBUG: Task 916 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 916 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 916 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 916 has memoization hash 3527e62dce6545272b4db0ac597fe59f parsl.dataflow.memoization INFO: Task 916 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 916 try 0 launched on executor multi with executor id 917 parsl.dataflow.dflow INFO: Standard output for task 916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d0360362-1b2d-40ae-9eb6-7b3308659148_isrForCrosstalkSources_996981_48.stdout parsl.dataflow.dflow INFO: Standard error for task 916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d0360362-1b2d-40ae-9eb6-7b3308659148_isrForCrosstalkSources_996981_48.stderr parsl.dataflow.dflow DEBUG: Task 917 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 917 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 917 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 917 has memoization hash 196be02bd45815fe7a6e3ba5d0bcf27d parsl.dataflow.memoization INFO: Task 917 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 917 try 0 launched on executor multi with executor id 918 parsl.dataflow.dflow INFO: Standard output for task 917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/1d1341b6-5b62-4ce2-8383-35c8348fafe3_isrForCrosstalkSources_997030_27.stdout parsl.dataflow.dflow INFO: Standard error for task 917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/1d1341b6-5b62-4ce2-8383-35c8348fafe3_isrForCrosstalkSources_997030_27.stderr parsl.dataflow.dflow DEBUG: Task 918 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 918 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 918 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 918 has memoization hash 5c05999fbfb9b5d0a1dea23a11c22dc7 parsl.dataflow.memoization INFO: Task 918 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 918 try 0 launched on executor multi with executor id 919 parsl.dataflow.dflow INFO: Standard output for task 918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c3133489-ce35-4e1b-9119-35853ac32603_isrForCrosstalkSources_997028_12.stdout parsl.dataflow.dflow INFO: Standard error for task 918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c3133489-ce35-4e1b-9119-35853ac32603_isrForCrosstalkSources_997028_12.stderr parsl.dataflow.dflow DEBUG: Task 919 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 919 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 919 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 919 has memoization hash 8cac75de1b3ccfc361e693d11f857e4e parsl.dataflow.memoization INFO: Task 919 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 919 try 0 launched on executor multi with executor id 920 parsl.dataflow.dflow INFO: Standard output for task 919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/dad2fd71-6043-457f-a8de-9d213df1eae4_isrForCrosstalkSources_996983_2.stdout parsl.dataflow.dflow INFO: Standard error for task 919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/dad2fd71-6043-457f-a8de-9d213df1eae4_isrForCrosstalkSources_996983_2.stderr parsl.dataflow.dflow DEBUG: Task 920 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 920 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 920 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 920 has memoization hash b81083c116ccff4a6926f7903a992934 parsl.dataflow.memoization INFO: Task 920 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 920 try 0 launched on executor multi with executor id 921 parsl.dataflow.dflow INFO: Standard output for task 920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/18dd25a3-5c0e-4a14-8a8a-ce0559cb4679_isrForCrosstalkSources_997014_59.stdout parsl.dataflow.dflow INFO: Standard error for task 920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/18dd25a3-5c0e-4a14-8a8a-ce0559cb4679_isrForCrosstalkSources_997014_59.stderr parsl.dataflow.dflow DEBUG: Task 921 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 921 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 921 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 921 has memoization hash 95009d483da348ad0a246e03abf661e3 parsl.dataflow.memoization INFO: Task 921 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 921 try 0 launched on executor multi with executor id 922 parsl.dataflow.dflow INFO: Standard output for task 921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c71431db-4def-49ca-9305-e39d306248a0_isrForCrosstalkSources_996965_57.stdout parsl.dataflow.dflow INFO: Standard error for task 921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c71431db-4def-49ca-9305-e39d306248a0_isrForCrosstalkSources_996965_57.stderr parsl.dataflow.dflow DEBUG: Task 922 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 922 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 922 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 922 has memoization hash 112a95e25a1a22a8d1c41b96392d31ab parsl.dataflow.memoization INFO: Task 922 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e777e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 922 try 0 launched on executor multi with executor id 923 parsl.dataflow.dflow INFO: Standard output for task 922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2fde8ca9-48a9-4773-ac52-2bc96d935b7f_isrForCrosstalkSources_996969_60.stdout parsl.dataflow.dflow INFO: Standard error for task 922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2fde8ca9-48a9-4773-ac52-2bc96d935b7f_isrForCrosstalkSources_996969_60.stderr parsl.dataflow.dflow DEBUG: Task 923 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 923 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 923 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 923 has memoization hash a1622607b3fa7c95b1b575b795f85627 parsl.dataflow.memoization INFO: Task 923 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 923 try 0 launched on executor multi with executor id 924 parsl.dataflow.dflow INFO: Standard output for task 923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/e385377b-55b4-4048-9da6-9277e9af94ad_isrForCrosstalkSources_996977_8.stdout parsl.dataflow.dflow INFO: Standard error for task 923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/e385377b-55b4-4048-9da6-9277e9af94ad_isrForCrosstalkSources_996977_8.stderr parsl.dataflow.dflow DEBUG: Task 924 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 924 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 924 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 924 has memoization hash 98edde3ce5384d4581f4af1b240d0ead parsl.dataflow.memoization INFO: Task 924 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 924 try 0 launched on executor multi with executor id 925 parsl.dataflow.dflow INFO: Standard output for task 924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/5085bafe-fe2f-4389-94d5-8fb1ca3b7acb_isrForCrosstalkSources_997029_3.stdout parsl.dataflow.dflow INFO: Standard error for task 924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/5085bafe-fe2f-4389-94d5-8fb1ca3b7acb_isrForCrosstalkSources_997029_3.stderr parsl.dataflow.dflow DEBUG: Task 925 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 925 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 925 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 925 has memoization hash 991f503a489c835bf546ff3287054671 parsl.dataflow.memoization INFO: Task 925 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 925 try 0 launched on executor multi with executor id 926 parsl.dataflow.dflow INFO: Standard output for task 925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c3ce31bc-355d-4d69-8b7f-98b70e992bae_isrForCrosstalkSources_996991_4.stdout parsl.dataflow.dflow INFO: Standard error for task 925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c3ce31bc-355d-4d69-8b7f-98b70e992bae_isrForCrosstalkSources_996991_4.stderr parsl.dataflow.dflow DEBUG: Task 926 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 926 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 926 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 926 has memoization hash be4dd31a6c80491de491edc54f8a8c90 parsl.dataflow.memoization INFO: Task 926 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 926 try 0 launched on executor multi with executor id 927 parsl.dataflow.dflow INFO: Standard output for task 926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7c70072d-e1b6-4599-945d-e8f1b2086a42_isrForCrosstalkSources_996991_25.stdout parsl.dataflow.dflow INFO: Standard error for task 926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7c70072d-e1b6-4599-945d-e8f1b2086a42_isrForCrosstalkSources_996991_25.stderr parsl.dataflow.dflow DEBUG: Task 927 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 927 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 927 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 927 has memoization hash 59e3f8e92218a472cca5f63d0c1c847a parsl.dataflow.memoization INFO: Task 927 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 927 try 0 launched on executor multi with executor id 928 parsl.dataflow.dflow INFO: Standard output for task 927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ef95b0da-aa84-4db1-9648-c6b5d84ba963_isrForCrosstalkSources_997029_12.stdout parsl.dataflow.dflow INFO: Standard error for task 927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ef95b0da-aa84-4db1-9648-c6b5d84ba963_isrForCrosstalkSources_997029_12.stderr parsl.dataflow.dflow DEBUG: Task 928 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 928 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 928 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 928 has memoization hash 5dbda31aaa162755b3a6a13ba496569b parsl.dataflow.memoization INFO: Task 928 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e749a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 928 try 0 launched on executor multi with executor id 929 parsl.dataflow.dflow INFO: Standard output for task 928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/9001ec41-bc01-48ca-8bfb-12f590ae50b7_isrForCrosstalkSources_997016_10.stdout parsl.dataflow.dflow INFO: Standard error for task 928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/9001ec41-bc01-48ca-8bfb-12f590ae50b7_isrForCrosstalkSources_997016_10.stderr parsl.dataflow.dflow DEBUG: Task 929 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 929 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 929 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 929 has memoization hash dd28dd8f12a0d0cd7cc7041edc40a37d parsl.dataflow.memoization INFO: Task 929 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 929 try 0 launched on executor multi with executor id 930 parsl.dataflow.dflow INFO: Standard output for task 929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/8a133999-f74f-4c7b-a11b-40897282332c_isrForCrosstalkSources_997033_26.stdout parsl.dataflow.dflow INFO: Standard error for task 929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/8a133999-f74f-4c7b-a11b-40897282332c_isrForCrosstalkSources_997033_26.stderr parsl.dataflow.dflow DEBUG: Task 930 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 930 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 930 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 930 has memoization hash 697b1d92b2f7903f2fab05e575cf379b parsl.dataflow.memoization INFO: Task 930 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e747c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 930 try 0 launched on executor multi with executor id 931 parsl.dataflow.dflow INFO: Standard output for task 930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/025e9d6c-a480-4ad9-91df-2931701b4066_isrForCrosstalkSources_996985_50.stdout parsl.dataflow.dflow INFO: Standard error for task 930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/025e9d6c-a480-4ad9-91df-2931701b4066_isrForCrosstalkSources_996985_50.stderr parsl.dataflow.dflow DEBUG: Task 931 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 931 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 931 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 931 has memoization hash 3ad9737b2c4b6e44402a483b793199b4 parsl.dataflow.memoization INFO: Task 931 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 931 try 0 launched on executor multi with executor id 932 parsl.dataflow.dflow INFO: Standard output for task 931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/1962d897-9a31-4aa2-9be4-4f9e6068c218_isrForCrosstalkSources_996985_16.stdout parsl.dataflow.dflow INFO: Standard error for task 931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/1962d897-9a31-4aa2-9be4-4f9e6068c218_isrForCrosstalkSources_996985_16.stderr parsl.dataflow.dflow DEBUG: Task 932 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 932 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 932 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 932 has memoization hash 86f3b0f19e80d0f4d0a2f366463af216 parsl.dataflow.memoization INFO: Task 932 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 932 try 0 launched on executor multi with executor id 933 parsl.dataflow.dflow INFO: Standard output for task 932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d13dc73f-ceb9-428b-82c2-69eac9d602a7_isrForCrosstalkSources_996981_41.stdout parsl.dataflow.dflow INFO: Standard error for task 932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d13dc73f-ceb9-428b-82c2-69eac9d602a7_isrForCrosstalkSources_996981_41.stderr parsl.dataflow.dflow DEBUG: Task 933 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 933 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 933 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 933 has memoization hash 75e4b9f853baa4be88053979757b322f parsl.dataflow.memoization INFO: Task 933 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 933 try 0 launched on executor multi with executor id 934 parsl.dataflow.dflow INFO: Standard output for task 933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/8717e401-5bba-4bf0-8bd9-3618ba143791_isrForCrosstalkSources_996975_19.stdout parsl.dataflow.dflow INFO: Standard error for task 933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/8717e401-5bba-4bf0-8bd9-3618ba143791_isrForCrosstalkSources_996975_19.stderr parsl.dataflow.dflow DEBUG: Task 934 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 934 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 934 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 934 has memoization hash c1b61b80e09a95eb6c6f02d36531b506 parsl.dataflow.memoization INFO: Task 934 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 934 try 0 launched on executor multi with executor id 935 parsl.dataflow.dflow INFO: Standard output for task 934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1a7a1d78-d8d4-4f91-af49-49d2dbfe9aa7_isrForCrosstalkSources_997032_51.stdout parsl.dataflow.dflow INFO: Standard error for task 934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1a7a1d78-d8d4-4f91-af49-49d2dbfe9aa7_isrForCrosstalkSources_997032_51.stderr parsl.dataflow.dflow DEBUG: Task 935 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 935 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 935 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 935 has memoization hash 8a530a14ceea4e321be4a71b85c804bf parsl.dataflow.memoization INFO: Task 935 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 935 try 0 launched on executor multi with executor id 936 parsl.dataflow.dflow INFO: Standard output for task 935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/aaf2b3bf-7086-49f7-b85e-099dbb31c755_isrForCrosstalkSources_997009_46.stdout parsl.dataflow.dflow INFO: Standard error for task 935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/aaf2b3bf-7086-49f7-b85e-099dbb31c755_isrForCrosstalkSources_997009_46.stderr parsl.dataflow.dflow DEBUG: Task 936 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 936 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 936 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 936 has memoization hash 4004ec68ada679c8adae2fa1824988b0 parsl.dataflow.memoization INFO: Task 936 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 936 try 0 launched on executor multi with executor id 937 parsl.dataflow.dflow INFO: Standard output for task 936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/111f3b68-51ab-4e85-8095-794eb6d8a311_isrForCrosstalkSources_997021_8.stdout parsl.dataflow.dflow INFO: Standard error for task 936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/111f3b68-51ab-4e85-8095-794eb6d8a311_isrForCrosstalkSources_997021_8.stderr parsl.dataflow.dflow DEBUG: Task 937 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 937 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 937 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 937 has memoization hash 9e73616af61dc6e3122c60ab820dd6e3 parsl.dataflow.memoization INFO: Task 937 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 937 try 0 launched on executor multi with executor id 938 parsl.dataflow.dflow INFO: Standard output for task 937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/2a1fd48f-c9cd-48d4-a285-e84959d050a6_isrForCrosstalkSources_996992_24.stdout parsl.dataflow.dflow INFO: Standard error for task 937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/2a1fd48f-c9cd-48d4-a285-e84959d050a6_isrForCrosstalkSources_996992_24.stderr parsl.dataflow.dflow DEBUG: Task 938 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 938 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 938 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 938 has memoization hash 426bfe04e5176bb97a0df22f7fe7e907 parsl.dataflow.memoization INFO: Task 938 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 938 try 0 launched on executor multi with executor id 939 parsl.dataflow.dflow INFO: Standard output for task 938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/728fdc0c-67b6-4d91-9ee9-45a9eab62ac6_isrForCrosstalkSources_996993_43.stdout parsl.dataflow.dflow INFO: Standard error for task 938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/728fdc0c-67b6-4d91-9ee9-45a9eab62ac6_isrForCrosstalkSources_996993_43.stderr parsl.dataflow.dflow DEBUG: Task 939 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 939 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 939 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 939 has memoization hash 44b9c7524a65b091f98d7b3b571878d9 parsl.dataflow.memoization INFO: Task 939 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 939 try 0 launched on executor multi with executor id 940 parsl.dataflow.dflow INFO: Standard output for task 939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a446093a-fba3-43c4-9e09-9bb6ed38f921_isrForCrosstalkSources_996999_31.stdout parsl.dataflow.dflow INFO: Standard error for task 939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a446093a-fba3-43c4-9e09-9bb6ed38f921_isrForCrosstalkSources_996999_31.stderr parsl.dataflow.dflow DEBUG: Task 940 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 940 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 940 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 940 has memoization hash 9c7125a7fbefe0b126dad48f6730a262 parsl.dataflow.memoization INFO: Task 940 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 940 try 0 launched on executor multi with executor id 941 parsl.dataflow.dflow INFO: Standard output for task 940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fbeb28bb-2796-4912-8da8-c8eea1b5ce9f_isrForCrosstalkSources_996955_7.stdout parsl.dataflow.dflow INFO: Standard error for task 940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fbeb28bb-2796-4912-8da8-c8eea1b5ce9f_isrForCrosstalkSources_996955_7.stderr parsl.dataflow.dflow DEBUG: Task 941 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 941 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 941 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 941 has memoization hash 33e852df8ccea38610343c6cafc5f5b4 parsl.dataflow.memoization INFO: Task 941 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e751c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 941 try 0 launched on executor multi with executor id 942 parsl.dataflow.dflow INFO: Standard output for task 941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/32cf3899-2294-4ec6-9913-cdc29a062131_isrForCrosstalkSources_997014_39.stdout parsl.dataflow.dflow INFO: Standard error for task 941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/32cf3899-2294-4ec6-9913-cdc29a062131_isrForCrosstalkSources_997014_39.stderr parsl.dataflow.dflow DEBUG: Task 942 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 942 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 942 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 942 has memoization hash 81bfb60ad69c4d13b90b033da4a5a1b5 parsl.dataflow.memoization INFO: Task 942 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 942 try 0 launched on executor multi with executor id 943 parsl.dataflow.dflow INFO: Standard output for task 942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c0404c2d-d74a-4436-a668-3320043bf59f_isrForCrosstalkSources_996955_32.stdout parsl.dataflow.dflow INFO: Standard error for task 942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c0404c2d-d74a-4436-a668-3320043bf59f_isrForCrosstalkSources_996955_32.stderr parsl.dataflow.dflow DEBUG: Task 943 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 943 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 943 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 943 has memoization hash 5466914f9a830ea76da0688f4af11b2f parsl.dataflow.memoization INFO: Task 943 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 943 try 0 launched on executor multi with executor id 944 parsl.dataflow.dflow INFO: Standard output for task 943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/29ba8115-abf7-4e9c-8471-7e4ce157a2a8_isrForCrosstalkSources_997004_36.stdout parsl.dataflow.dflow INFO: Standard error for task 943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/29ba8115-abf7-4e9c-8471-7e4ce157a2a8_isrForCrosstalkSources_997004_36.stderr parsl.dataflow.dflow DEBUG: Task 944 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 944 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 944 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 944 has memoization hash ceff5de246c62815b14b7e346ac76f2d parsl.dataflow.memoization INFO: Task 944 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 944 try 0 launched on executor multi with executor id 945 parsl.dataflow.dflow INFO: Standard output for task 944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/785d3d84-4a30-410b-8fbb-7b42c7ed8f24_isrForCrosstalkSources_996965_1.stdout parsl.dataflow.dflow INFO: Standard error for task 944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/785d3d84-4a30-410b-8fbb-7b42c7ed8f24_isrForCrosstalkSources_996965_1.stderr parsl.dataflow.dflow DEBUG: Task 945 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 945 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 945 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 945 has memoization hash 2b122dc71b12d421082fc17cc1fe87fb parsl.dataflow.memoization INFO: Task 945 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e753a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 945 try 0 launched on executor multi with executor id 946 parsl.dataflow.dflow INFO: Standard output for task 945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9453da9d-f41a-4dd9-90a4-fea80089a8b5_isrForCrosstalkSources_996986_35.stdout parsl.dataflow.dflow INFO: Standard error for task 945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9453da9d-f41a-4dd9-90a4-fea80089a8b5_isrForCrosstalkSources_996986_35.stderr parsl.dataflow.dflow DEBUG: Task 946 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 946 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 946 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 946 has memoization hash 03da7ef3449ac6db1adf5c86736ccccd parsl.dataflow.memoization INFO: Task 946 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e754e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 946 try 0 launched on executor multi with executor id 947 parsl.dataflow.dflow INFO: Standard output for task 946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/4424fceb-97a4-4029-aad6-7a770827164a_isrForCrosstalkSources_996953_23.stdout parsl.dataflow.dflow INFO: Standard error for task 946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/4424fceb-97a4-4029-aad6-7a770827164a_isrForCrosstalkSources_996953_23.stderr parsl.dataflow.dflow DEBUG: Task 947 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 947 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 947 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 947 has memoization hash 981900bd5ab851d7b44fd9bf4879bae7 parsl.dataflow.memoization INFO: Task 947 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 947 try 0 launched on executor multi with executor id 948 parsl.dataflow.dflow INFO: Standard output for task 947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/25584aba-5fed-4a74-ab62-e9e70a4843ff_isrForCrosstalkSources_996996_27.stdout parsl.dataflow.dflow INFO: Standard error for task 947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/25584aba-5fed-4a74-ab62-e9e70a4843ff_isrForCrosstalkSources_996996_27.stderr parsl.dataflow.dflow DEBUG: Task 948 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 948 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 948 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 948 has memoization hash 204b7173cd885069b3d43891208d0764 parsl.dataflow.memoization INFO: Task 948 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 948 try 0 launched on executor multi with executor id 949 parsl.dataflow.dflow INFO: Standard output for task 948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/0bda1630-816b-4915-a9b9-d4fabb3a4efd_isrForCrosstalkSources_996955_1.stdout parsl.dataflow.dflow INFO: Standard error for task 948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/0bda1630-816b-4915-a9b9-d4fabb3a4efd_isrForCrosstalkSources_996955_1.stderr parsl.dataflow.dflow DEBUG: Task 949 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 949 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 949 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 949 has memoization hash 2f2be0d0f77498dc04d41bb92a358d55 parsl.dataflow.memoization INFO: Task 949 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e756c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 949 try 0 launched on executor multi with executor id 950 parsl.dataflow.dflow INFO: Standard output for task 949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f479662c-592c-4942-91f9-928e35774d61_isrForCrosstalkSources_996995_49.stdout parsl.dataflow.dflow INFO: Standard error for task 949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f479662c-592c-4942-91f9-928e35774d61_isrForCrosstalkSources_996995_49.stderr parsl.dataflow.dflow DEBUG: Task 950 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 950 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 950 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 950 has memoization hash fc524459d75f4fc851f9f067cdf6c1e3 parsl.dataflow.memoization INFO: Task 950 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 950 try 0 launched on executor multi with executor id 951 parsl.dataflow.dflow INFO: Standard output for task 950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/19b8f33f-7551-406c-a2eb-cf15e1ac1448_isrForCrosstalkSources_997031_44.stdout parsl.dataflow.dflow INFO: Standard error for task 950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/19b8f33f-7551-406c-a2eb-cf15e1ac1448_isrForCrosstalkSources_997031_44.stderr parsl.dataflow.dflow DEBUG: Task 951 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 951 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 951 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 951 has memoization hash ef75ca4546eda875893cc623c8ac441d parsl.dataflow.memoization INFO: Task 951 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 951 try 0 launched on executor multi with executor id 952 parsl.dataflow.dflow INFO: Standard output for task 951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/00e97e3c-a66c-47df-b116-9588638758c2_isrForCrosstalkSources_997021_12.stdout parsl.dataflow.dflow INFO: Standard error for task 951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/00e97e3c-a66c-47df-b116-9588638758c2_isrForCrosstalkSources_997021_12.stderr parsl.dataflow.dflow DEBUG: Task 952 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 952 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 952 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 952 has memoization hash d461e79f2fca97707de0d88e9afd5ada parsl.dataflow.memoization INFO: Task 952 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 952 try 0 launched on executor multi with executor id 953 parsl.dataflow.dflow INFO: Standard output for task 952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7124b386-05b0-40dd-b684-10c86bb690bc_isrForCrosstalkSources_996993_6.stdout parsl.dataflow.dflow INFO: Standard error for task 952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7124b386-05b0-40dd-b684-10c86bb690bc_isrForCrosstalkSources_996993_6.stderr parsl.dataflow.dflow DEBUG: Task 953 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 953 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 953 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 953 has memoization hash f38f7008a940f5ed44dce9e6c7416e20 parsl.dataflow.memoization INFO: Task 953 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 953 try 0 launched on executor multi with executor id 954 parsl.dataflow.dflow INFO: Standard output for task 953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/db6ef0cf-6d43-415a-9adc-ef9828138cbc_isrForCrosstalkSources_996961_57.stdout parsl.dataflow.dflow INFO: Standard error for task 953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/db6ef0cf-6d43-415a-9adc-ef9828138cbc_isrForCrosstalkSources_996961_57.stderr parsl.dataflow.dflow DEBUG: Task 954 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 954 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 954 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 954 has memoization hash 41beb22dba8367d953106f6e1642bfea parsl.dataflow.memoization INFO: Task 954 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e759e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 954 try 0 launched on executor multi with executor id 955 parsl.dataflow.dflow INFO: Standard output for task 954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/61f6b43e-b955-4f96-a1e3-68adc9ec9b02_isrForCrosstalkSources_997004_14.stdout parsl.dataflow.dflow INFO: Standard error for task 954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/61f6b43e-b955-4f96-a1e3-68adc9ec9b02_isrForCrosstalkSources_997004_14.stderr parsl.dataflow.dflow DEBUG: Task 955 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 955 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 955 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 955 has memoization hash 54faf8c9d9e17260ce9a3bb035cf7e6a parsl.dataflow.memoization INFO: Task 955 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e758a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 955 try 0 launched on executor multi with executor id 956 parsl.dataflow.dflow INFO: Standard output for task 955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/1d6f811d-28b6-4dc9-bcbf-79076b3b2982_isrForCrosstalkSources_997023_58.stdout parsl.dataflow.dflow INFO: Standard error for task 955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/1d6f811d-28b6-4dc9-bcbf-79076b3b2982_isrForCrosstalkSources_997023_58.stderr parsl.dataflow.dflow DEBUG: Task 956 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 956 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 956 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 956 has memoization hash a2df30a26ea0c5ed45fc540ecb4d79ae parsl.dataflow.memoization INFO: Task 956 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 956 try 0 launched on executor multi with executor id 957 parsl.dataflow.dflow INFO: Standard output for task 956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/aff0a313-e807-4580-a899-79bc95e86bb5_isrForCrosstalkSources_996986_13.stdout parsl.dataflow.dflow INFO: Standard error for task 956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/aff0a313-e807-4580-a899-79bc95e86bb5_isrForCrosstalkSources_996986_13.stderr parsl.dataflow.dflow DEBUG: Task 957 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 957 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 957 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 957 has memoization hash da29ee98e03cacdaa0cf78bcadbdc322 parsl.dataflow.memoization INFO: Task 957 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 957 try 0 launched on executor multi with executor id 958 parsl.dataflow.dflow INFO: Standard output for task 957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/0f5c2fba-7ea8-4fde-87da-ad7063e898ef_isrForCrosstalkSources_997011_33.stdout parsl.dataflow.dflow INFO: Standard error for task 957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/0f5c2fba-7ea8-4fde-87da-ad7063e898ef_isrForCrosstalkSources_997011_33.stderr parsl.dataflow.dflow DEBUG: Task 958 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 958 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 958 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 958 has memoization hash 7c9e498ae087aefbdb7ab4ac8795b4f5 parsl.dataflow.memoization INFO: Task 958 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 958 try 0 launched on executor multi with executor id 959 parsl.dataflow.dflow INFO: Standard output for task 958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/1f30fa2e-d1a0-48ef-8ae3-af0f0a90d06a_isrForCrosstalkSources_996976_53.stdout parsl.dataflow.dflow INFO: Standard error for task 958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/1f30fa2e-d1a0-48ef-8ae3-af0f0a90d06a_isrForCrosstalkSources_996976_53.stderr parsl.dataflow.dflow DEBUG: Task 959 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 959 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 959 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 959 has memoization hash f002b410ccebd385422b147d5271b175 parsl.dataflow.memoization INFO: Task 959 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 959 try 0 launched on executor multi with executor id 960 parsl.dataflow.dflow INFO: Standard output for task 959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3fcaab23-b11e-4367-9a7d-a38ba8adfca3_isrForCrosstalkSources_996959_43.stdout parsl.dataflow.dflow INFO: Standard error for task 959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3fcaab23-b11e-4367-9a7d-a38ba8adfca3_isrForCrosstalkSources_996959_43.stderr parsl.dataflow.dflow DEBUG: Task 960 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 960 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 960 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 960 has memoization hash 071cc1db5379cd51efea2e60dd6df0eb parsl.dataflow.memoization INFO: Task 960 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 960 try 0 launched on executor multi with executor id 961 parsl.dataflow.dflow INFO: Standard output for task 960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/f2a5a851-3b91-4ae5-a229-7719bc13c087_isrForCrosstalkSources_996994_57.stdout parsl.dataflow.dflow INFO: Standard error for task 960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/f2a5a851-3b91-4ae5-a229-7719bc13c087_isrForCrosstalkSources_996994_57.stderr parsl.dataflow.dflow DEBUG: Task 961 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 961 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 961 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 961 has memoization hash 8770315730ceaf41b910f2b1f3c3b6a6 parsl.dataflow.memoization INFO: Task 961 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 961 try 0 launched on executor multi with executor id 962 parsl.dataflow.dflow INFO: Standard output for task 961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/38181a03-a93a-4c44-a4c1-d6aab1bcb5cd_isrForCrosstalkSources_996953_37.stdout parsl.dataflow.dflow INFO: Standard error for task 961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/38181a03-a93a-4c44-a4c1-d6aab1bcb5cd_isrForCrosstalkSources_996953_37.stderr parsl.dataflow.dflow DEBUG: Task 962 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 962 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 962 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 962 has memoization hash 27c73a527f08759bd23eea611a0cc4a5 parsl.dataflow.memoization INFO: Task 962 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 962 try 0 launched on executor multi with executor id 963 parsl.dataflow.dflow INFO: Standard output for task 962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0c05e74c-f6ea-4c5b-b600-f994ab6eb528_isrForCrosstalkSources_997023_18.stdout parsl.dataflow.dflow INFO: Standard error for task 962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0c05e74c-f6ea-4c5b-b600-f994ab6eb528_isrForCrosstalkSources_997023_18.stderr parsl.dataflow.dflow DEBUG: Task 963 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 963 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 963 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 963 has memoization hash 4a203555b33e93f8faaf6a7756b903ab parsl.dataflow.memoization INFO: Task 963 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 963 try 0 launched on executor multi with executor id 964 parsl.dataflow.dflow INFO: Standard output for task 963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/c3516cc3-03a2-4e2d-9243-444ddb00266b_isrForCrosstalkSources_996953_45.stdout parsl.dataflow.dflow INFO: Standard error for task 963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/c3516cc3-03a2-4e2d-9243-444ddb00266b_isrForCrosstalkSources_996953_45.stderr parsl.dataflow.dflow DEBUG: Task 964 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 964 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 964 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 964 has memoization hash 0c77259d64e4b02c4f4b1ef8248eee65 parsl.dataflow.memoization INFO: Task 964 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e760c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 964 try 0 launched on executor multi with executor id 965 parsl.dataflow.dflow INFO: Standard output for task 964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b30d7062-9b36-4f3e-a053-075412539de9_isrForCrosstalkSources_996997_58.stdout parsl.dataflow.dflow INFO: Standard error for task 964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b30d7062-9b36-4f3e-a053-075412539de9_isrForCrosstalkSources_996997_58.stderr parsl.dataflow.dflow DEBUG: Task 965 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 965 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 965 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 965 has memoization hash 9cdeb76d793d116bad3554d68f5e0d31 parsl.dataflow.memoization INFO: Task 965 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 965 try 0 launched on executor multi with executor id 966 parsl.dataflow.dflow INFO: Standard output for task 965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/857f8d43-09d6-479c-a7ee-25463438db04_isrForCrosstalkSources_996959_53.stdout parsl.dataflow.dflow INFO: Standard error for task 965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/857f8d43-09d6-479c-a7ee-25463438db04_isrForCrosstalkSources_996959_53.stderr parsl.dataflow.dflow DEBUG: Task 966 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 966 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 966 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 966 has memoization hash 0345043f73492bf3129fb6c06c112c35 parsl.dataflow.memoization INFO: Task 966 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e75ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 966 try 0 launched on executor multi with executor id 967 parsl.dataflow.dflow INFO: Standard output for task 966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b8a52336-afaf-4efa-8c7a-338f61b3200d_isrForCrosstalkSources_996975_33.stdout parsl.dataflow.dflow INFO: Standard error for task 966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b8a52336-afaf-4efa-8c7a-338f61b3200d_isrForCrosstalkSources_996975_33.stderr parsl.dataflow.dflow DEBUG: Task 967 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 967 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 967 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 967 has memoization hash 01e62e60fef330dd0ea3a3241dff6ec3 parsl.dataflow.memoization INFO: Task 967 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 967 try 0 launched on executor multi with executor id 968 parsl.dataflow.dflow INFO: Standard output for task 967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/0a991ddc-2e7d-46f0-ac8c-92ac0c1de089_isrForCrosstalkSources_996996_5.stdout parsl.dataflow.dflow INFO: Standard error for task 967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/0a991ddc-2e7d-46f0-ac8c-92ac0c1de089_isrForCrosstalkSources_996996_5.stderr parsl.dataflow.dflow DEBUG: Task 968 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 968 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 968 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 968 has memoization hash 167dd9fc91c8d20d5d32c3a1f3f1bac6 parsl.dataflow.memoization INFO: Task 968 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e762a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 968 try 0 launched on executor multi with executor id 969 parsl.dataflow.dflow INFO: Standard output for task 968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a5acff24-9cd8-470e-8432-9a6d95da5f77_isrForCrosstalkSources_996965_52.stdout parsl.dataflow.dflow INFO: Standard error for task 968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a5acff24-9cd8-470e-8432-9a6d95da5f77_isrForCrosstalkSources_996965_52.stderr parsl.dataflow.dflow DEBUG: Task 969 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 969 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 969 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 969 has memoization hash 7ae9f139c462e00f34ced9acb8c8755b parsl.dataflow.memoization INFO: Task 969 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 969 try 0 launched on executor multi with executor id 970 parsl.dataflow.dflow INFO: Standard output for task 969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/90fb61e7-5746-42a0-bf4a-2499756302f6_isrForCrosstalkSources_997032_58.stdout parsl.dataflow.dflow INFO: Standard error for task 969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/90fb61e7-5746-42a0-bf4a-2499756302f6_isrForCrosstalkSources_997032_58.stderr parsl.dataflow.dflow DEBUG: Task 970 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 970 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 970 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 970 has memoization hash 5335296dc5a9287b29ccd49d7c0efcdf parsl.dataflow.memoization INFO: Task 970 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 970 try 0 launched on executor multi with executor id 971 parsl.dataflow.dflow INFO: Standard output for task 970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/5c54f6b0-03bd-431a-85c1-d6bcb1d1ff3b_isrForCrosstalkSources_996958_38.stdout parsl.dataflow.dflow INFO: Standard error for task 970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/5c54f6b0-03bd-431a-85c1-d6bcb1d1ff3b_isrForCrosstalkSources_996958_38.stderr parsl.dataflow.dflow DEBUG: Task 971 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 971 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 971 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 971 has memoization hash 5a5274dbb273aa8fdb8e95a6ca9cae2d parsl.dataflow.memoization INFO: Task 971 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e763e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 971 try 0 launched on executor multi with executor id 972 parsl.dataflow.dflow INFO: Standard output for task 971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d2598782-d453-4711-bb5a-e6206682df75_isrForCrosstalkSources_996979_23.stdout parsl.dataflow.dflow INFO: Standard error for task 971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d2598782-d453-4711-bb5a-e6206682df75_isrForCrosstalkSources_996979_23.stderr parsl.dataflow.dflow DEBUG: Task 972 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 972 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 972 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 972 has memoization hash 048b5d9fc3b2b0a753e7d7ab9b5cb80c parsl.dataflow.memoization INFO: Task 972 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 972 try 0 launched on executor multi with executor id 973 parsl.dataflow.dflow INFO: Standard output for task 972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/56d95c02-2ca7-43dd-a260-359961adf940_isrForCrosstalkSources_997005_2.stdout parsl.dataflow.dflow INFO: Standard error for task 972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/56d95c02-2ca7-43dd-a260-359961adf940_isrForCrosstalkSources_997005_2.stderr parsl.dataflow.dflow DEBUG: Task 973 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 973 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 973 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 973 has memoization hash a9b66b37cd87cdd536b922cd3f00e03a parsl.dataflow.memoization INFO: Task 973 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e765c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 973 try 0 launched on executor multi with executor id 974 parsl.dataflow.dflow INFO: Standard output for task 973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e34f723e-63c4-46e9-8bc5-58a7fd8e60bd_isrForCrosstalkSources_997032_45.stdout parsl.dataflow.dflow INFO: Standard error for task 973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e34f723e-63c4-46e9-8bc5-58a7fd8e60bd_isrForCrosstalkSources_997032_45.stderr parsl.dataflow.dflow DEBUG: Task 974 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 974 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 974 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 974 has memoization hash 2d2d46b0160b941e626aa902ee884143 parsl.dataflow.memoization INFO: Task 974 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 974 try 0 launched on executor multi with executor id 975 parsl.dataflow.dflow INFO: Standard output for task 974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/de958e11-141a-4c2b-ba32-24fc726d36b5_isrForCrosstalkSources_996982_3.stdout parsl.dataflow.dflow INFO: Standard error for task 974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/de958e11-141a-4c2b-ba32-24fc726d36b5_isrForCrosstalkSources_996982_3.stderr parsl.dataflow.dflow DEBUG: Task 975 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 975 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 975 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 975 has memoization hash 340f525da12e201db3b895478853dac4 parsl.dataflow.memoization INFO: Task 975 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e776a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 975 try 0 launched on executor multi with executor id 976 parsl.dataflow.dflow INFO: Standard output for task 975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/896b5f44-57f5-457a-a256-764a3de2ec54_isrForCrosstalkSources_996994_1.stdout parsl.dataflow.dflow INFO: Standard error for task 975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/896b5f44-57f5-457a-a256-764a3de2ec54_isrForCrosstalkSources_996994_1.stderr parsl.dataflow.dflow DEBUG: Task 976 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 976 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 976 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 976 has memoization hash 34eaf940942d182cdb6ec5098a9713eb parsl.dataflow.memoization INFO: Task 976 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 976 try 0 launched on executor multi with executor id 977 parsl.dataflow.dflow INFO: Standard output for task 976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/9262851d-aa2b-453c-aebc-6d2499f3ee44_isrForCrosstalkSources_996992_53.stdout parsl.dataflow.dflow INFO: Standard error for task 976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/9262851d-aa2b-453c-aebc-6d2499f3ee44_isrForCrosstalkSources_996992_53.stderr parsl.dataflow.dflow DEBUG: Task 977 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 977 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 977 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 977 has memoization hash 3bc5e5134ad80f510ab7df8b185b197c parsl.dataflow.memoization INFO: Task 977 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e774c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 977 try 0 launched on executor multi with executor id 978 parsl.dataflow.dflow INFO: Standard output for task 977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/3ee415a1-fc3b-410c-8224-1ced65a98fdd_isrForCrosstalkSources_997013_46.stdout parsl.dataflow.dflow INFO: Standard error for task 977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/3ee415a1-fc3b-410c-8224-1ced65a98fdd_isrForCrosstalkSources_997013_46.stderr parsl.dataflow.dflow DEBUG: Task 978 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 978 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 978 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 978 has memoization hash 9dbb096357ec7201a6e46e9ccc95ff99 parsl.dataflow.memoization INFO: Task 978 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 978 try 0 launched on executor multi with executor id 979 parsl.dataflow.dflow INFO: Standard output for task 978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/5b18c679-e874-4398-b98f-fb537dfafb7e_isrForCrosstalkSources_997032_42.stdout parsl.dataflow.dflow INFO: Standard error for task 978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/5b18c679-e874-4398-b98f-fb537dfafb7e_isrForCrosstalkSources_997032_42.stderr parsl.dataflow.dflow DEBUG: Task 979 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 979 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 979 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 979 has memoization hash 4938d14cc043fc9b44a8d6b75a0727eb parsl.dataflow.memoization INFO: Task 979 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e772e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 979 try 0 launched on executor multi with executor id 980 parsl.dataflow.dflow INFO: Standard output for task 979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/d734e273-f818-4767-bd07-21473beb5cec_isrForCrosstalkSources_997028_57.stdout parsl.dataflow.dflow INFO: Standard error for task 979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/d734e273-f818-4767-bd07-21473beb5cec_isrForCrosstalkSources_997028_57.stderr parsl.dataflow.dflow DEBUG: Task 980 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 980 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 980 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 980 has memoization hash 78d546967b2993e3715e99bc61129a68 parsl.dataflow.memoization INFO: Task 980 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 980 try 0 launched on executor multi with executor id 981 parsl.dataflow.dflow INFO: Standard output for task 980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/09ba95d0-37cd-4dc9-ae9b-90c2d96faa14_isrForCrosstalkSources_996965_4.stdout parsl.dataflow.dflow INFO: Standard error for task 980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/09ba95d0-37cd-4dc9-ae9b-90c2d96faa14_isrForCrosstalkSources_996965_4.stderr parsl.dataflow.dflow DEBUG: Task 981 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 981 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 981 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 981 has memoization hash 4232769ef59cce868c72581ac0dda9b8 parsl.dataflow.memoization INFO: Task 981 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 981 try 0 launched on executor multi with executor id 982 parsl.dataflow.dflow INFO: Standard output for task 981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/91536af8-bcea-4317-8a53-7b525e037f84_isrForCrosstalkSources_997017_4.stdout parsl.dataflow.dflow INFO: Standard error for task 981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/91536af8-bcea-4317-8a53-7b525e037f84_isrForCrosstalkSources_997017_4.stderr parsl.dataflow.dflow DEBUG: Task 982 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 982 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 982 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 982 has memoization hash 7d73c2930db83ab87089607aa6ef9c2c parsl.dataflow.memoization INFO: Task 982 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e771a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 982 try 0 launched on executor multi with executor id 983 parsl.dataflow.dflow INFO: Standard output for task 982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/d514d75b-dc56-46f8-94bd-085054f8e06c_isrForCrosstalkSources_997029_2.stdout parsl.dataflow.dflow INFO: Standard error for task 982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/d514d75b-dc56-46f8-94bd-085054f8e06c_isrForCrosstalkSources_997029_2.stderr parsl.dataflow.dflow DEBUG: Task 983 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 983 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 983 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 983 has memoization hash ab093286dc68716a84b45981671a7857 parsl.dataflow.memoization INFO: Task 983 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 983 try 0 launched on executor multi with executor id 984 parsl.dataflow.dflow INFO: Standard output for task 983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/19984382-a14a-4a2f-a9f0-4f20261346e1_isrForCrosstalkSources_997028_1.stdout parsl.dataflow.dflow INFO: Standard error for task 983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/19984382-a14a-4a2f-a9f0-4f20261346e1_isrForCrosstalkSources_997028_1.stderr parsl.dataflow.dflow DEBUG: Task 984 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 984 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 984 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 984 has memoization hash 8d0c1534b8a33cbe602cf4e9440e3e85 parsl.dataflow.memoization INFO: Task 984 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 984 try 0 launched on executor multi with executor id 985 parsl.dataflow.dflow INFO: Standard output for task 984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f4762ba3-ffad-4b61-84d8-bbce4d473d20_isrForCrosstalkSources_996966_9.stdout parsl.dataflow.dflow INFO: Standard error for task 984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f4762ba3-ffad-4b61-84d8-bbce4d473d20_isrForCrosstalkSources_996966_9.stderr parsl.dataflow.dflow DEBUG: Task 985 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 985 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 985 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 985 has memoization hash c6e817430ac188ae1ee06a9d15985aee parsl.dataflow.memoization INFO: Task 985 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 985 try 0 launched on executor multi with executor id 986 parsl.dataflow.dflow INFO: Standard output for task 985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/9a4f0394-75a2-4967-a976-c0c1008ecd10_isrForCrosstalkSources_997025_3.stdout parsl.dataflow.dflow INFO: Standard error for task 985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/9a4f0394-75a2-4967-a976-c0c1008ecd10_isrForCrosstalkSources_997025_3.stderr parsl.dataflow.dflow DEBUG: Task 986 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 986 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 986 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 986 has memoization hash c0b95f674e1e89457d05b87cf01c2647 parsl.dataflow.memoization INFO: Task 986 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 986 try 0 launched on executor multi with executor id 987 parsl.dataflow.dflow INFO: Standard output for task 986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b68369c8-c0f1-477a-adbb-2db7efa65c72_isrForCrosstalkSources_996997_43.stdout parsl.dataflow.dflow INFO: Standard error for task 986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b68369c8-c0f1-477a-adbb-2db7efa65c72_isrForCrosstalkSources_996997_43.stderr parsl.dataflow.dflow DEBUG: Task 987 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 987 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 987 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 987 has memoization hash 160ddecd94855bff5c5673e96e2346ae parsl.dataflow.memoization INFO: Task 987 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 987 try 0 launched on executor multi with executor id 988 parsl.dataflow.dflow INFO: Standard output for task 987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2acf96d9-5db0-4d6b-aad0-696a43a12d13_isrForCrosstalkSources_996974_57.stdout parsl.dataflow.dflow INFO: Standard error for task 987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2acf96d9-5db0-4d6b-aad0-696a43a12d13_isrForCrosstalkSources_996974_57.stderr parsl.dataflow.dflow DEBUG: Task 988 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 988 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 988 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 988 has memoization hash a76f6bd94638768342b98200667b1329 parsl.dataflow.memoization INFO: Task 988 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 988 try 0 launched on executor multi with executor id 989 parsl.dataflow.dflow INFO: Standard output for task 988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/60f36c43-cac2-4c61-8d4a-2455f24eb560_isrForCrosstalkSources_996973_18.stdout parsl.dataflow.dflow INFO: Standard error for task 988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/60f36c43-cac2-4c61-8d4a-2455f24eb560_isrForCrosstalkSources_996973_18.stderr parsl.dataflow.dflow DEBUG: Task 989 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 989 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 989 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 989 has memoization hash 5e51af4503ad4ca39d849cca173eae88 parsl.dataflow.memoization INFO: Task 989 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 989 try 0 launched on executor multi with executor id 990 parsl.dataflow.dflow INFO: Standard output for task 989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/10569c16-8a35-4310-ab81-123817ed8006_isrForCrosstalkSources_997011_28.stdout parsl.dataflow.dflow INFO: Standard error for task 989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/10569c16-8a35-4310-ab81-123817ed8006_isrForCrosstalkSources_997011_28.stderr parsl.dataflow.dflow DEBUG: Task 990 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 990 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 990 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 990 has memoization hash 4f09596d8cf3b21aa188122519e7dbee parsl.dataflow.memoization INFO: Task 990 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 990 try 0 launched on executor multi with executor id 991 parsl.dataflow.dflow INFO: Standard output for task 990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7cbe3ea6-f0d7-4e1b-a1ca-5311e26b2794_isrForCrosstalkSources_997029_28.stdout parsl.dataflow.dflow INFO: Standard error for task 990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7cbe3ea6-f0d7-4e1b-a1ca-5311e26b2794_isrForCrosstalkSources_997029_28.stderr parsl.dataflow.dflow DEBUG: Task 991 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 991 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 991 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 991 has memoization hash 408cde82e7c50b6252351c317a120278 parsl.dataflow.memoization INFO: Task 991 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 991 try 0 launched on executor multi with executor id 992 parsl.dataflow.dflow INFO: Standard output for task 991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2f225d23-beb4-4747-937e-f815b20bde4a_isrForCrosstalkSources_997002_42.stdout parsl.dataflow.dflow INFO: Standard error for task 991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2f225d23-beb4-4747-937e-f815b20bde4a_isrForCrosstalkSources_997002_42.stderr parsl.dataflow.dflow DEBUG: Task 992 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 992 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 992 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 992 has memoization hash ea06cb919fdf4dc1c2f5acfba5a19dde parsl.dataflow.memoization INFO: Task 992 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 992 try 0 launched on executor multi with executor id 993 parsl.dataflow.dflow INFO: Standard output for task 992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2f3b592f-e4c6-4201-9ace-f8a419f8887b_isrForCrosstalkSources_997002_24.stdout parsl.dataflow.dflow INFO: Standard error for task 992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2f3b592f-e4c6-4201-9ace-f8a419f8887b_isrForCrosstalkSources_997002_24.stderr parsl.dataflow.dflow DEBUG: Task 993 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 993 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 993 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 993 has memoization hash 9c16b55a6a6fc960aa2cb51a29f2f8bc parsl.dataflow.memoization INFO: Task 993 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 993 try 0 launched on executor multi with executor id 994 parsl.dataflow.dflow INFO: Standard output for task 993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/45328959-e1e2-4dcb-9f8a-bfe0cc80aa2b_isrForCrosstalkSources_996965_32.stdout parsl.dataflow.dflow INFO: Standard error for task 993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/45328959-e1e2-4dcb-9f8a-bfe0cc80aa2b_isrForCrosstalkSources_996965_32.stderr parsl.dataflow.dflow DEBUG: Task 994 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 994 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 994 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 994 has memoization hash d474cddd7034f857119d62e0607c2234 parsl.dataflow.memoization INFO: Task 994 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 994 try 0 launched on executor multi with executor id 995 parsl.dataflow.dflow INFO: Standard output for task 994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/baf870db-1a53-4c27-8217-bdcea29ab72f_isrForCrosstalkSources_997024_37.stdout parsl.dataflow.dflow INFO: Standard error for task 994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/baf870db-1a53-4c27-8217-bdcea29ab72f_isrForCrosstalkSources_997024_37.stderr parsl.dataflow.dflow DEBUG: Task 995 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 995 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 995 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 995 has memoization hash ba4875a73dcf88250ac77d6ede2181c7 parsl.dataflow.memoization INFO: Task 995 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 995 try 0 launched on executor multi with executor id 996 parsl.dataflow.dflow INFO: Standard output for task 995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/67cf5956-c253-4bc1-8551-2494e2b4ba73_isrForCrosstalkSources_996973_10.stdout parsl.dataflow.dflow INFO: Standard error for task 995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/67cf5956-c253-4bc1-8551-2494e2b4ba73_isrForCrosstalkSources_996973_10.stderr parsl.dataflow.dflow DEBUG: Task 996 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 996 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 996 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 996 has memoization hash 932f6f2bc02de2d9c7d23e059f2201b6 parsl.dataflow.memoization INFO: Task 996 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e768e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 996 try 0 launched on executor multi with executor id 997 parsl.dataflow.dflow INFO: Standard output for task 996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/5894c832-b65e-4854-8ba2-b85516b355f0_isrForCrosstalkSources_996984_40.stdout parsl.dataflow.dflow INFO: Standard error for task 996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/5894c832-b65e-4854-8ba2-b85516b355f0_isrForCrosstalkSources_996984_40.stderr parsl.dataflow.dflow DEBUG: Task 997 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 997 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 997 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 997 has memoization hash 78822c7ea41c84297afb80c73b5825c4 parsl.dataflow.memoization INFO: Task 997 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:33 proc_lsst.multi:170 [INFO] [multi] job_id 17074851 proc_lsst.multi INFO: [multi] job_id 17074851 2024-03-15 13:49:33 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 12 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 12 2024-03-15 13:49:33 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:33 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074851 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074851 parsl.executors.status_handling DEBUG: Launched block 12 on executor multi with job ID 17074851 parsl.executors.status_handling INFO: Allocated block ID 13 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-13 2024-03-15 13:49:33 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=13 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-13 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=13 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-13 2024-03-15 13:49:33 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=13 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=13 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:33 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=13 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=13 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 12 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 12 2024-03-15 13:49:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:33 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=13 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=13 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 997 try 0 launched on executor multi with executor id 998 parsl.dataflow.dflow INFO: Standard output for task 997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/0cae0463-6a35-443d-a154-7e1bf55ff6be_isrForCrosstalkSources_997027_50.stdout parsl.dataflow.dflow INFO: Standard error for task 997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/0cae0463-6a35-443d-a154-7e1bf55ff6be_isrForCrosstalkSources_997027_50.stderr parsl.dataflow.dflow DEBUG: Task 998 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 998 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 998 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 998 has memoization hash 03e31216959be796b5aa78a8672e9892 parsl.dataflow.memoization INFO: Task 998 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e767a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 998 try 0 launched on executor multi with executor id 999 parsl.dataflow.dflow INFO: Standard output for task 998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/64d0b8fd-aee3-413c-ba72-b9b8efe60eb2_isrForCrosstalkSources_996964_54.stdout parsl.dataflow.dflow INFO: Standard error for task 998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/64d0b8fd-aee3-413c-ba72-b9b8efe60eb2_isrForCrosstalkSources_996964_54.stderr parsl.dataflow.dflow DEBUG: Task 999 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 999 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 999 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 999 has memoization hash 96c137b9a5d35d2f7c341c0aefb9cb71 parsl.dataflow.memoization INFO: Task 999 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e745e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 999 try 0 launched on executor multi with executor id 1000 parsl.dataflow.dflow INFO: Standard output for task 999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cefa7ecc-e766-4409-81a7-2ff5146ef836_isrForCrosstalkSources_997024_58.stdout parsl.dataflow.dflow INFO: Standard error for task 999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cefa7ecc-e766-4409-81a7-2ff5146ef836_isrForCrosstalkSources_997024_58.stderr parsl.dataflow.dflow DEBUG: Task 1000 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1000 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1000 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1000 has memoization hash be78e7f0a228e3d3c2bbfdb315798dc1 parsl.dataflow.memoization INFO: Task 1000 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1000 try 0 launched on executor multi with executor id 1001 parsl.dataflow.dflow INFO: Standard output for task 1000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4a2bbdc3-6400-4c09-8a8c-68609e66b859_isrForCrosstalkSources_996972_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4a2bbdc3-6400-4c09-8a8c-68609e66b859_isrForCrosstalkSources_996972_45.stderr parsl.dataflow.dflow DEBUG: Task 1001 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1001 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1001 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1001 has memoization hash d41aa72dc9e99805510505a288b8a46c parsl.dataflow.memoization INFO: Task 1001 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e779c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1001 try 0 launched on executor multi with executor id 1002 parsl.dataflow.dflow INFO: Standard output for task 1001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f8b87dd6-81eb-45d1-a93a-116aa4a969f1_isrForCrosstalkSources_996958_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f8b87dd6-81eb-45d1-a93a-116aa4a969f1_isrForCrosstalkSources_996958_16.stderr parsl.dataflow.dflow DEBUG: Task 1002 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1002 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1002 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1002 has memoization hash 3c8354b73fb744645bcac61c094f7cde parsl.dataflow.memoization INFO: Task 1002 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1002 try 0 launched on executor multi with executor id 1003 parsl.dataflow.dflow INFO: Standard output for task 1002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a0aceac6-ce33-45d5-9d89-c35859ac69eb_isrForCrosstalkSources_996994_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a0aceac6-ce33-45d5-9d89-c35859ac69eb_isrForCrosstalkSources_996994_17.stderr parsl.dataflow.dflow DEBUG: Task 1003 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1003 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1003 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1003 has memoization hash ff5e63bd767ce839f709ea63974c21d2 parsl.dataflow.memoization INFO: Task 1003 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e76520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1003 try 0 launched on executor multi with executor id 1004 parsl.dataflow.dflow INFO: Standard output for task 1003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a20996f2-b061-4d02-8a84-43f8406d308a_isrForCrosstalkSources_996965_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a20996f2-b061-4d02-8a84-43f8406d308a_isrForCrosstalkSources_996965_49.stderr parsl.dataflow.dflow DEBUG: Task 1004 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1004 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1004 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1004 has memoization hash ba6f603a99634578e7e3f4122973e907 parsl.dataflow.memoization INFO: Task 1004 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e744a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1004 try 0 launched on executor multi with executor id 1005 parsl.dataflow.dflow INFO: Standard output for task 1004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/8cf58eac-e2c1-4078-9dfa-5fe597f014e7_isrForCrosstalkSources_997014_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/8cf58eac-e2c1-4078-9dfa-5fe597f014e7_isrForCrosstalkSources_997014_48.stderr parsl.dataflow.dflow DEBUG: Task 1005 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1005 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1005 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1005 has memoization hash 3fcfe59c45d2e59c7ec997c3096abc49 parsl.dataflow.memoization INFO: Task 1005 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780a7e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1005 try 0 launched on executor multi with executor id 1006 parsl.dataflow.dflow INFO: Standard output for task 1005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/64133487-7aae-4b3b-89af-a22b5bbf3ee7_isrForCrosstalkSources_996977_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/64133487-7aae-4b3b-89af-a22b5bbf3ee7_isrForCrosstalkSources_996977_6.stderr parsl.dataflow.dflow DEBUG: Task 1006 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1006 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1006 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1006 has memoization hash e464d55b9f367e5f07eb34ea6dad658e parsl.dataflow.memoization INFO: Task 1006 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1006 try 0 launched on executor multi with executor id 1007 parsl.dataflow.dflow INFO: Standard output for task 1006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/0ff3723d-e379-4e7e-9a12-56bc5412a4e9_isrForCrosstalkSources_996974_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/0ff3723d-e379-4e7e-9a12-56bc5412a4e9_isrForCrosstalkSources_996974_43.stderr parsl.dataflow.dflow DEBUG: Task 1007 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1007 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1007 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1007 has memoization hash b8cbb0fe8d3a6973f582f8a027e4230a parsl.dataflow.memoization INFO: Task 1007 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1007 try 0 launched on executor multi with executor id 1008 parsl.dataflow.dflow INFO: Standard output for task 1007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f27091cd-4b22-4ae4-b72f-4d932870c187_isrForCrosstalkSources_997005_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f27091cd-4b22-4ae4-b72f-4d932870c187_isrForCrosstalkSources_997005_51.stderr parsl.dataflow.dflow DEBUG: Task 1008 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1008 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1008 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1008 has memoization hash 58772f85d1263984dc8266d45e6de0df parsl.dataflow.memoization INFO: Task 1008 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1008 try 0 launched on executor multi with executor id 1009 parsl.dataflow.dflow INFO: Standard output for task 1008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/790381ee-f65f-4ec0-b1d1-1e3b0c57a2ca_isrForCrosstalkSources_996982_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/790381ee-f65f-4ec0-b1d1-1e3b0c57a2ca_isrForCrosstalkSources_996982_21.stderr parsl.dataflow.dflow DEBUG: Task 1009 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1009 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1009 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1009 has memoization hash 434ecc5ed78f265bcf9af96a42905478 parsl.dataflow.memoization INFO: Task 1009 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1009 try 0 launched on executor multi with executor id 1010 parsl.dataflow.dflow INFO: Standard output for task 1009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/40575bad-dd1c-4a56-ba7a-8b4bf0c7c62c_isrForCrosstalkSources_996954_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/40575bad-dd1c-4a56-ba7a-8b4bf0c7c62c_isrForCrosstalkSources_996954_32.stderr parsl.dataflow.dflow DEBUG: Task 1010 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1010 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1010 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1010 has memoization hash 787cb914c58fe99bc4266bc9978bf55a parsl.dataflow.memoization INFO: Task 1010 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1010 try 0 launched on executor multi with executor id 1011 parsl.dataflow.dflow INFO: Standard output for task 1010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/42189ab0-7b0d-4a49-9ea8-c360cea2c695_isrForCrosstalkSources_997011_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/42189ab0-7b0d-4a49-9ea8-c360cea2c695_isrForCrosstalkSources_997011_41.stderr parsl.dataflow.dflow DEBUG: Task 1011 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1011 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1011 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1011 has memoization hash 828c5743d0239fc537b4b6b1d44135bd parsl.dataflow.memoization INFO: Task 1011 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1011 try 0 launched on executor multi with executor id 1012 parsl.dataflow.dflow INFO: Standard output for task 1011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b67fc318-170b-4f53-9502-2faded0d9e27_isrForCrosstalkSources_996972_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b67fc318-170b-4f53-9502-2faded0d9e27_isrForCrosstalkSources_996972_9.stderr parsl.dataflow.dflow DEBUG: Task 1012 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1012 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1012 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1012 has memoization hash 892994c111ed05564811b8fe00f45eb4 parsl.dataflow.memoization INFO: Task 1012 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1012 try 0 launched on executor multi with executor id 1013 parsl.dataflow.dflow INFO: Standard output for task 1012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/63ca5720-1bb2-4dd1-ad21-c408785f2a55_isrForCrosstalkSources_996988_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/63ca5720-1bb2-4dd1-ad21-c408785f2a55_isrForCrosstalkSources_996988_30.stderr parsl.dataflow.dflow DEBUG: Task 1013 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1013 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1013 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1013 has memoization hash 3b3d5f097addf33a4f4a8e55a9d22e7e parsl.dataflow.memoization INFO: Task 1013 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f640e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1013 try 0 launched on executor multi with executor id 1014 parsl.dataflow.dflow INFO: Standard output for task 1013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b5174a84-2e66-4630-9127-6ce0460d6750_isrForCrosstalkSources_997033_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b5174a84-2e66-4630-9127-6ce0460d6750_isrForCrosstalkSources_997033_34.stderr parsl.dataflow.dflow DEBUG: Task 1014 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1014 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1014 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1014 has memoization hash f6096727f07cc0dbc6242a0183f285ba parsl.dataflow.memoization INFO: Task 1014 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1014 try 0 launched on executor multi with executor id 1015 parsl.dataflow.dflow INFO: Standard output for task 1014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/f410303f-ce0d-427f-a951-070cd02104d2_isrForCrosstalkSources_997000_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/f410303f-ce0d-427f-a951-070cd02104d2_isrForCrosstalkSources_997000_39.stderr parsl.dataflow.dflow DEBUG: Task 1015 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1015 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1015 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1015 has memoization hash 791f9d8880da23e60304db0089e8b8c2 parsl.dataflow.memoization INFO: Task 1015 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1015 try 0 launched on executor multi with executor id 1016 parsl.dataflow.dflow INFO: Standard output for task 1015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1420c740-b583-4719-8133-c8f812eee047_isrForCrosstalkSources_996961_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1420c740-b583-4719-8133-c8f812eee047_isrForCrosstalkSources_996961_24.stderr parsl.dataflow.dflow DEBUG: Task 1016 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1016 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1016 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1016 has memoization hash ea633a7aa7607555a5e3a6d17a59468f parsl.dataflow.memoization INFO: Task 1016 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f642c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1016 try 0 launched on executor multi with executor id 1017 parsl.dataflow.dflow INFO: Standard output for task 1016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d3d6f35f-352b-4904-b7d3-03519cded0c0_isrForCrosstalkSources_996991_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d3d6f35f-352b-4904-b7d3-03519cded0c0_isrForCrosstalkSources_996991_39.stderr parsl.dataflow.dflow DEBUG: Task 1017 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1017 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1017 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1017 has memoization hash 3d5170a7d167873189f299228cd6f9d5 parsl.dataflow.memoization INFO: Task 1017 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1017 try 0 launched on executor multi with executor id 1018 parsl.dataflow.dflow INFO: Standard output for task 1017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d078fc2d-b6af-4c89-9616-6606a629b4a4_isrForCrosstalkSources_996966_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d078fc2d-b6af-4c89-9616-6606a629b4a4_isrForCrosstalkSources_996966_44.stderr parsl.dataflow.dflow DEBUG: Task 1018 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1018 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1018 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1018 has memoization hash dd14daf8ae3560686010868d41308b77 parsl.dataflow.memoization INFO: Task 1018 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1018 try 0 launched on executor multi with executor id 1019 parsl.dataflow.dflow INFO: Standard output for task 1018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/ab43e665-6862-4ff0-96ab-9037d10a9975_isrForCrosstalkSources_997008_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/ab43e665-6862-4ff0-96ab-9037d10a9975_isrForCrosstalkSources_997008_35.stderr parsl.dataflow.dflow DEBUG: Task 1019 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1019 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1019 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1019 has memoization hash 368e51c228b79d09f284dba3c8d058b8 parsl.dataflow.memoization INFO: Task 1019 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1019 try 0 launched on executor multi with executor id 1020 parsl.dataflow.dflow INFO: Standard output for task 1019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/d7cd5b54-0403-4f9b-8f5d-c0893dde9c17_isrForCrosstalkSources_997013_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/d7cd5b54-0403-4f9b-8f5d-c0893dde9c17_isrForCrosstalkSources_997013_9.stderr parsl.dataflow.dflow DEBUG: Task 1020 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1020 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1020 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1020 has memoization hash aca989ca66353c0958cdfb9de06aed29 parsl.dataflow.memoization INFO: Task 1020 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f644a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1020 try 0 launched on executor multi with executor id 1021 parsl.dataflow.dflow INFO: Standard output for task 1020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7da26623-734d-4265-966b-446581b285e1_isrForCrosstalkSources_996968_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7da26623-734d-4265-966b-446581b285e1_isrForCrosstalkSources_996968_58.stderr parsl.dataflow.dflow DEBUG: Task 1021 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1021 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1021 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1021 has memoization hash e8cfde0a841964e9868afb127f681238 parsl.dataflow.memoization INFO: Task 1021 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f645e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1021 try 0 launched on executor multi with executor id 1022 parsl.dataflow.dflow INFO: Standard output for task 1021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a332ed7d-aec8-4953-a5e8-a025b3cbce6a_isrForCrosstalkSources_996978_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a332ed7d-aec8-4953-a5e8-a025b3cbce6a_isrForCrosstalkSources_996978_39.stderr parsl.dataflow.dflow DEBUG: Task 1022 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1022 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1022 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1022 has memoization hash 0712032f201e6749dc5fb8373999b59b parsl.dataflow.memoization INFO: Task 1022 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1022 try 0 launched on executor multi with executor id 1023 parsl.dataflow.dflow INFO: Standard output for task 1022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/43d87b41-531e-4119-b95f-99dc9b5d9053_isrForCrosstalkSources_996971_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/43d87b41-531e-4119-b95f-99dc9b5d9053_isrForCrosstalkSources_996971_34.stderr parsl.dataflow.dflow DEBUG: Task 1023 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1023 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1023 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1023 has memoization hash d188a74c607376bc4169d385328d9031 parsl.dataflow.memoization INFO: Task 1023 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1023 try 0 launched on executor multi with executor id 1024 parsl.dataflow.dflow INFO: Standard output for task 1023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/4e70c73f-e627-46b1-a80d-f1a86afe63a9_isrForCrosstalkSources_997004_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/4e70c73f-e627-46b1-a80d-f1a86afe63a9_isrForCrosstalkSources_997004_42.stderr parsl.dataflow.dflow DEBUG: Task 1024 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1024 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1024 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1024 has memoization hash 4b2b81ee2aa2f41e0f9cac70b2ae61c3 parsl.dataflow.memoization INFO: Task 1024 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1024 try 0 launched on executor multi with executor id 1025 parsl.dataflow.dflow INFO: Standard output for task 1024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d8218b56-96f6-40e4-a347-93de56259202_isrForCrosstalkSources_996987_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d8218b56-96f6-40e4-a347-93de56259202_isrForCrosstalkSources_996987_55.stderr parsl.dataflow.dflow DEBUG: Task 1025 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1025 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1025 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1025 has memoization hash dd8a5d6157df224c3fc1f744d0fae7ea parsl.dataflow.memoization INFO: Task 1025 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f647c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1025 try 0 launched on executor multi with executor id 1026 parsl.dataflow.dflow INFO: Standard output for task 1025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/984f7ebf-9eba-4eac-a804-dd8b240023de_isrForCrosstalkSources_996982_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/984f7ebf-9eba-4eac-a804-dd8b240023de_isrForCrosstalkSources_996982_25.stderr parsl.dataflow.dflow DEBUG: Task 1026 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1026 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1026 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1026 has memoization hash 5b62975285c537a7b8493f48dfe59e01 parsl.dataflow.memoization INFO: Task 1026 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1026 try 0 launched on executor multi with executor id 1027 parsl.dataflow.dflow INFO: Standard output for task 1026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/11f627bd-f78a-46d7-bb5a-1c10bf42e2a7_isrForCrosstalkSources_996975_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/11f627bd-f78a-46d7-bb5a-1c10bf42e2a7_isrForCrosstalkSources_996975_26.stderr parsl.dataflow.dflow DEBUG: Task 1027 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1027 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1027 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1027 has memoization hash 38a4864723a1cb681f7165a53bcdd623 parsl.dataflow.memoization INFO: Task 1027 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f649a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1027 try 0 launched on executor multi with executor id 1028 parsl.dataflow.dflow INFO: Standard output for task 1027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bf70c3b8-187d-4003-873f-cd169b799936_isrForCrosstalkSources_996959_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bf70c3b8-187d-4003-873f-cd169b799936_isrForCrosstalkSources_996959_13.stderr parsl.dataflow.dflow DEBUG: Task 1028 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1028 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1028 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1028 has memoization hash 97e23bb656ad250e0832fabd58e4d993 parsl.dataflow.memoization INFO: Task 1028 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1028 try 0 launched on executor multi with executor id 1029 parsl.dataflow.dflow INFO: Standard output for task 1028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cea42da8-b66c-43e2-a890-a22c29d17a3b_isrForCrosstalkSources_996983_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cea42da8-b66c-43e2-a890-a22c29d17a3b_isrForCrosstalkSources_996983_57.stderr parsl.dataflow.dflow DEBUG: Task 1029 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1029 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1029 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1029 has memoization hash d10e6402de9d797af41901fa97586d39 parsl.dataflow.memoization INFO: Task 1029 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1029 try 0 launched on executor multi with executor id 1030 parsl.dataflow.dflow INFO: Standard output for task 1029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ac2bbdfe-1776-43f8-81e6-cfb29dd6f9a7_isrForCrosstalkSources_996981_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ac2bbdfe-1776-43f8-81e6-cfb29dd6f9a7_isrForCrosstalkSources_996981_51.stderr parsl.dataflow.dflow DEBUG: Task 1030 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1030 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1030 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1030 has memoization hash 18cebfca1aa9a6b4f6637b1b819644b1 parsl.dataflow.memoization INFO: Task 1030 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1030 try 0 launched on executor multi with executor id 1031 parsl.dataflow.dflow INFO: Standard output for task 1030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/6cd0833c-bb02-449d-937b-36bfb541f4a2_isrForCrosstalkSources_997014_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/6cd0833c-bb02-449d-937b-36bfb541f4a2_isrForCrosstalkSources_997014_30.stderr parsl.dataflow.dflow DEBUG: Task 1031 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1031 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1031 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1031 has memoization hash 74fcb469bc0059a2fff2a5803d7edb4a parsl.dataflow.memoization INFO: Task 1031 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1031 try 0 launched on executor multi with executor id 1032 parsl.dataflow.dflow INFO: Standard output for task 1031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/314a81ea-0985-4b99-8f71-bc48df093cf0_isrForCrosstalkSources_997031_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/314a81ea-0985-4b99-8f71-bc48df093cf0_isrForCrosstalkSources_997031_49.stderr parsl.dataflow.dflow DEBUG: Task 1032 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1032 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1032 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1032 has memoization hash a5cff2846a399b20edc195b5e219a738 parsl.dataflow.memoization INFO: Task 1032 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1032 try 0 launched on executor multi with executor id 1033 parsl.dataflow.dflow INFO: Standard output for task 1032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/cfd0c747-71f3-40b6-a6cc-d35103ea13e5_isrForCrosstalkSources_997025_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/cfd0c747-71f3-40b6-a6cc-d35103ea13e5_isrForCrosstalkSources_997025_11.stderr parsl.dataflow.dflow DEBUG: Task 1033 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1033 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1033 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1033 has memoization hash de23b1e2fe88828edca083beaabfb890 parsl.dataflow.memoization INFO: Task 1033 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1033 try 0 launched on executor multi with executor id 1034 parsl.dataflow.dflow INFO: Standard output for task 1033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/245475c8-48dc-4a31-88c4-c497de45483a_isrForCrosstalkSources_997008_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/245475c8-48dc-4a31-88c4-c497de45483a_isrForCrosstalkSources_997008_2.stderr parsl.dataflow.dflow DEBUG: Task 1034 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1034 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1034 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1034 has memoization hash ee29b6071ee6d7c0666e3fc5b3487937 parsl.dataflow.memoization INFO: Task 1034 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1034 try 0 launched on executor multi with executor id 1035 parsl.dataflow.dflow INFO: Standard output for task 1034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/bb88595a-5800-4383-98b4-83b513157d8d_isrForCrosstalkSources_996972_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/bb88595a-5800-4383-98b4-83b513157d8d_isrForCrosstalkSources_996972_31.stderr parsl.dataflow.dflow DEBUG: Task 1035 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1035 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1035 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1035 has memoization hash f876ce87487cacbc31b5ebb2a746fa1f parsl.dataflow.memoization INFO: Task 1035 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1035 try 0 launched on executor multi with executor id 1036 parsl.dataflow.dflow INFO: Standard output for task 1035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/75fa1e19-71fd-4d95-9e2b-04a4a7ca2d14_isrForCrosstalkSources_996985_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/75fa1e19-71fd-4d95-9e2b-04a4a7ca2d14_isrForCrosstalkSources_996985_8.stderr parsl.dataflow.dflow DEBUG: Task 1036 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1036 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1036 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1036 has memoization hash 209afe62c75fa33f9217b4f99965726a parsl.dataflow.memoization INFO: Task 1036 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1036 try 0 launched on executor multi with executor id 1037 parsl.dataflow.dflow INFO: Standard output for task 1036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/22559425-67fa-4727-942a-0576f6a9609a_isrForCrosstalkSources_997030_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/22559425-67fa-4727-942a-0576f6a9609a_isrForCrosstalkSources_997030_43.stderr parsl.dataflow.dflow DEBUG: Task 1037 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1037 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1037 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1037 has memoization hash f6684a0d881ed8334eb5e0b2820cc1db parsl.dataflow.memoization INFO: Task 1037 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1037 try 0 launched on executor multi with executor id 1038 parsl.dataflow.dflow INFO: Standard output for task 1037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/ff99ca74-62c3-4bbc-8c00-2a739d41bbbe_isrForCrosstalkSources_997012_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/ff99ca74-62c3-4bbc-8c00-2a739d41bbbe_isrForCrosstalkSources_997012_50.stderr parsl.dataflow.dflow DEBUG: Task 1038 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1038 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1038 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1038 has memoization hash a906711ab11fac132f5bf4ab65fc9f48 parsl.dataflow.memoization INFO: Task 1038 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1038 try 0 launched on executor multi with executor id 1039 parsl.dataflow.dflow INFO: Standard output for task 1038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/1f8aed8d-c2d6-42ac-9e9c-30d8fdf58479_isrForCrosstalkSources_997026_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/1f8aed8d-c2d6-42ac-9e9c-30d8fdf58479_isrForCrosstalkSources_997026_51.stderr parsl.dataflow.dflow DEBUG: Task 1039 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1039 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1039 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1039 has memoization hash 5ea5669006e0a732472c255b59263899 parsl.dataflow.memoization INFO: Task 1039 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f651c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1039 try 0 launched on executor multi with executor id 1040 parsl.dataflow.dflow INFO: Standard output for task 1039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e0dbcf7b-8a38-4609-a427-1f42c3746965_isrForCrosstalkSources_996961_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e0dbcf7b-8a38-4609-a427-1f42c3746965_isrForCrosstalkSources_996961_59.stderr parsl.dataflow.dflow DEBUG: Task 1040 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1040 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1040 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1040 has memoization hash 65e09efeac3fe64f171f5e1780979386 parsl.dataflow.memoization INFO: Task 1040 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1040 try 0 launched on executor multi with executor id 1041 parsl.dataflow.dflow INFO: Standard output for task 1040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/cb0cc08b-6ec5-4e8d-b56c-986d8eec255e_isrForCrosstalkSources_996981_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/cb0cc08b-6ec5-4e8d-b56c-986d8eec255e_isrForCrosstalkSources_996981_32.stderr parsl.dataflow.dflow DEBUG: Task 1041 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1041 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1041 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1041 has memoization hash 249c4d6430c6b5436b3e3b63d9908309 parsl.dataflow.memoization INFO: Task 1041 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1041 try 0 launched on executor multi with executor id 1042 parsl.dataflow.dflow INFO: Standard output for task 1041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/59bfe8bd-3760-43c1-bbfe-806ccd0c7bdc_isrForCrosstalkSources_997027_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/59bfe8bd-3760-43c1-bbfe-806ccd0c7bdc_isrForCrosstalkSources_997027_28.stderr parsl.dataflow.dflow DEBUG: Task 1042 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1042 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1042 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1042 has memoization hash 99ad1b7a5e1ef2029cdbd22df20b6388 parsl.dataflow.memoization INFO: Task 1042 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1042 try 0 launched on executor multi with executor id 1043 parsl.dataflow.dflow INFO: Standard output for task 1042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d419ddf5-fe04-4bd4-90a1-6d7a84b5bb29_isrForCrosstalkSources_996981_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d419ddf5-fe04-4bd4-90a1-6d7a84b5bb29_isrForCrosstalkSources_996981_7.stderr parsl.dataflow.dflow DEBUG: Task 1043 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1043 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1043 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1043 has memoization hash 58b21958eae28dcda2e0f117681ef7da parsl.dataflow.memoization INFO: Task 1043 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f653a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1043 try 0 launched on executor multi with executor id 1044 parsl.dataflow.dflow INFO: Standard output for task 1043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e4b3fbbd-75ca-4359-87d5-c355efee7f05_isrForCrosstalkSources_996954_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e4b3fbbd-75ca-4359-87d5-c355efee7f05_isrForCrosstalkSources_996954_9.stderr parsl.dataflow.dflow DEBUG: Task 1044 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1044 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1044 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1044 has memoization hash 518458aa6b25229761314b525c775736 parsl.dataflow.memoization INFO: Task 1044 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f654e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1044 try 0 launched on executor multi with executor id 1045 parsl.dataflow.dflow INFO: Standard output for task 1044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/28018e0a-fb60-43cd-ab9f-51925779fb95_isrForCrosstalkSources_996996_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/28018e0a-fb60-43cd-ab9f-51925779fb95_isrForCrosstalkSources_996996_39.stderr parsl.dataflow.dflow DEBUG: Task 1045 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1045 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1045 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1045 has memoization hash cacc47fa145ed6f0798ba1a455a10c49 parsl.dataflow.memoization INFO: Task 1045 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1045 try 0 launched on executor multi with executor id 1046 parsl.dataflow.dflow INFO: Standard output for task 1045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/8163d928-6f89-4f52-8d01-a7d2f4f09f5f_isrForCrosstalkSources_996978_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/8163d928-6f89-4f52-8d01-a7d2f4f09f5f_isrForCrosstalkSources_996978_58.stderr parsl.dataflow.dflow DEBUG: Task 1046 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1046 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1046 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1046 has memoization hash 174fc165cbdeb1a5c296be70b286325b parsl.dataflow.memoization INFO: Task 1046 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1046 try 0 launched on executor multi with executor id 1047 parsl.dataflow.dflow INFO: Standard output for task 1046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f84438b6-6749-4118-befa-6fa3b3481111_isrForCrosstalkSources_996971_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f84438b6-6749-4118-befa-6fa3b3481111_isrForCrosstalkSources_996971_62.stderr parsl.dataflow.dflow DEBUG: Task 1047 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1047 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1047 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1047 has memoization hash b9cf49c5bf02d59d178482c0f24d4f0b parsl.dataflow.memoization INFO: Task 1047 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1047 try 0 launched on executor multi with executor id 1048 parsl.dataflow.dflow INFO: Standard output for task 1047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/48429f34-1150-4ce4-a5aa-91cadea7b2e3_isrForCrosstalkSources_996965_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/48429f34-1150-4ce4-a5aa-91cadea7b2e3_isrForCrosstalkSources_996965_38.stderr parsl.dataflow.dflow DEBUG: Task 1048 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1048 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1048 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1048 has memoization hash 0151cfd8b017936c3474cb4c39b9dd63 parsl.dataflow.memoization INFO: Task 1048 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f656c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1048 try 0 launched on executor multi with executor id 1049 parsl.dataflow.dflow INFO: Standard output for task 1048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/707049ab-7b2e-4945-bb92-91652d324fd9_isrForCrosstalkSources_997007_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/707049ab-7b2e-4945-bb92-91652d324fd9_isrForCrosstalkSources_997007_14.stderr parsl.dataflow.dflow DEBUG: Task 1049 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1049 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1049 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1049 has memoization hash 037eb83588913ecd53a3eebecb0aef10 parsl.dataflow.memoization INFO: Task 1049 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1049 try 0 launched on executor multi with executor id 1050 parsl.dataflow.dflow INFO: Standard output for task 1049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6f3c43ae-20c6-4e36-abcd-503e9e802852_isrForCrosstalkSources_996983_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6f3c43ae-20c6-4e36-abcd-503e9e802852_isrForCrosstalkSources_996983_34.stderr parsl.dataflow.dflow DEBUG: Task 1050 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1050 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1050 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1050 has memoization hash 8019d3dba20a264a4a75899862bc7aa7 parsl.dataflow.memoization INFO: Task 1050 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1050 try 0 launched on executor multi with executor id 1051 parsl.dataflow.dflow INFO: Standard output for task 1050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/9502f89d-f46c-4b94-ad22-536b67e03d61_isrForCrosstalkSources_997013_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/9502f89d-f46c-4b94-ad22-536b67e03d61_isrForCrosstalkSources_997013_45.stderr parsl.dataflow.dflow DEBUG: Task 1051 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1051 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1051 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1051 has memoization hash 47cef778df8972038c5b91ff578d63f4 parsl.dataflow.memoization INFO: Task 1051 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f658a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1051 try 0 launched on executor multi with executor id 1052 parsl.dataflow.dflow INFO: Standard output for task 1051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b3842d3f-203c-4221-978f-c5b75be588f5_isrForCrosstalkSources_996980_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b3842d3f-203c-4221-978f-c5b75be588f5_isrForCrosstalkSources_996980_22.stderr parsl.dataflow.dflow DEBUG: Task 1052 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1052 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1052 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1052 has memoization hash 25eb84e46bd0de29c2727584e2819f4d parsl.dataflow.memoization INFO: Task 1052 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1052 try 0 launched on executor multi with executor id 1053 parsl.dataflow.dflow INFO: Standard output for task 1052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/213fc11d-81fa-44a2-9cf8-3ff61cf27512_isrForCrosstalkSources_997025_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/213fc11d-81fa-44a2-9cf8-3ff61cf27512_isrForCrosstalkSources_997025_56.stderr parsl.dataflow.dflow DEBUG: Task 1053 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1053 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1053 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1053 has memoization hash e45c0346b5a48edacf1d38bbf1f60251 parsl.dataflow.memoization INFO: Task 1053 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f659e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1053 try 0 launched on executor multi with executor id 1054 parsl.dataflow.dflow INFO: Standard output for task 1053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/26700e75-540a-4599-98d0-21bdeedded5c_isrForCrosstalkSources_996987_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/26700e75-540a-4599-98d0-21bdeedded5c_isrForCrosstalkSources_996987_40.stderr parsl.dataflow.dflow DEBUG: Task 1054 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1054 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1054 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1054 has memoization hash bddf30f64eb816e26321127b7c229293 parsl.dataflow.memoization INFO: Task 1054 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1054 try 0 launched on executor multi with executor id 1055 parsl.dataflow.dflow INFO: Standard output for task 1054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c1169a95-95bf-4361-b017-a2639bf9fef7_isrForCrosstalkSources_996995_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c1169a95-95bf-4361-b017-a2639bf9fef7_isrForCrosstalkSources_996995_16.stderr parsl.dataflow.dflow DEBUG: Task 1055 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1055 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1055 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1055 has memoization hash 0b5dae1426f082eed6190241a1dcc859 parsl.dataflow.memoization INFO: Task 1055 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1055 try 0 launched on executor multi with executor id 1056 parsl.dataflow.dflow INFO: Standard output for task 1055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d405770a-d8cd-4db9-af73-c18104d92c32_isrForCrosstalkSources_997026_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d405770a-d8cd-4db9-af73-c18104d92c32_isrForCrosstalkSources_997026_44.stderr parsl.dataflow.dflow DEBUG: Task 1056 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1056 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1056 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1056 has memoization hash 0ad76d4dac1d4a9d6d3cc7f4e5af0bc7 parsl.dataflow.memoization INFO: Task 1056 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1056 try 0 launched on executor multi with executor id 1057 parsl.dataflow.dflow INFO: Standard output for task 1056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/d849dd15-cb64-4647-952f-a2457ed167c4_isrForCrosstalkSources_996961_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/d849dd15-cb64-4647-952f-a2457ed167c4_isrForCrosstalkSources_996961_61.stderr parsl.dataflow.dflow DEBUG: Task 1057 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1057 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1057 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1057 has memoization hash 4e7b003fab546611bfacaeb85ab4c16b parsl.dataflow.memoization INFO: Task 1057 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1057 try 0 launched on executor multi with executor id 1058 parsl.dataflow.dflow INFO: Standard output for task 1057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f6fbbdbc-f9c9-4d13-b914-ea9c5d033162_isrForCrosstalkSources_996973_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f6fbbdbc-f9c9-4d13-b914-ea9c5d033162_isrForCrosstalkSources_996973_60.stderr parsl.dataflow.dflow DEBUG: Task 1058 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1058 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1058 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1058 has memoization hash fd9f7817d58ae29d3d801306a5eaf604 parsl.dataflow.memoization INFO: Task 1058 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1058 try 0 launched on executor multi with executor id 1059 parsl.dataflow.dflow INFO: Standard output for task 1058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d7f2a9e0-8e8a-433c-a901-1e721bb88022_isrForCrosstalkSources_996981_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d7f2a9e0-8e8a-433c-a901-1e721bb88022_isrForCrosstalkSources_996981_27.stderr parsl.dataflow.dflow DEBUG: Task 1059 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1059 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1059 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1059 has memoization hash dd85e4889554438dbb9d65e22ece57be parsl.dataflow.memoization INFO: Task 1059 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1059 try 0 launched on executor multi with executor id 1060 parsl.dataflow.dflow INFO: Standard output for task 1059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3d456af9-b89b-417d-9404-265a8e320e46_isrForCrosstalkSources_996988_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3d456af9-b89b-417d-9404-265a8e320e46_isrForCrosstalkSources_996988_32.stderr parsl.dataflow.dflow DEBUG: Task 1060 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1060 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1060 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1060 has memoization hash eb27831e09bd473d3464645a43130203 parsl.dataflow.memoization INFO: Task 1060 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1060 try 0 launched on executor multi with executor id 1061 parsl.dataflow.dflow INFO: Standard output for task 1060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ea2b423e-bf43-47ab-b386-b39905bcdcab_isrForCrosstalkSources_996992_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ea2b423e-bf43-47ab-b386-b39905bcdcab_isrForCrosstalkSources_996992_16.stderr parsl.dataflow.dflow DEBUG: Task 1061 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1061 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1061 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1061 has memoization hash 19902662f8ef35afdd13c08ff2a1f3d3 parsl.dataflow.memoization INFO: Task 1061 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1061 try 0 launched on executor multi with executor id 1062 parsl.dataflow.dflow INFO: Standard output for task 1061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c55301a5-84ef-43cb-9131-651fbddd7bbe_isrForCrosstalkSources_997000_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c55301a5-84ef-43cb-9131-651fbddd7bbe_isrForCrosstalkSources_997000_9.stderr parsl.dataflow.dflow DEBUG: Task 1062 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1062 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1062 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1062 has memoization hash 76ff027d2d2c42d8643e2b08e12318d2 parsl.dataflow.memoization INFO: Task 1062 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f65f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1062 try 0 launched on executor multi with executor id 1063 parsl.dataflow.dflow INFO: Standard output for task 1062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/c919d350-9c16-4228-a5a5-576a766808d5_isrForCrosstalkSources_997022_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/c919d350-9c16-4228-a5a5-576a766808d5_isrForCrosstalkSources_997022_16.stderr parsl.dataflow.dflow DEBUG: Task 1063 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1063 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1063 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1063 has memoization hash 90862a848ba6410c373946cfe1f960fd parsl.dataflow.memoization INFO: Task 1063 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1063 try 0 launched on executor multi with executor id 1064 parsl.dataflow.dflow INFO: Standard output for task 1063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f4681f4-5833-4ac3-b7ff-d48b1b16c415_isrForCrosstalkSources_996994_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f4681f4-5833-4ac3-b7ff-d48b1b16c415_isrForCrosstalkSources_996994_25.stderr parsl.dataflow.dflow DEBUG: Task 1064 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1064 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1064 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1064 has memoization hash f2ba32a6a1bd73ab2a90d66fcbb9e2a7 parsl.dataflow.memoization INFO: Task 1064 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1064 try 0 launched on executor multi with executor id 1065 parsl.dataflow.dflow INFO: Standard output for task 1064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d27d987a-ee4d-4bb7-88b3-4996962dbc65_isrForCrosstalkSources_996985_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d27d987a-ee4d-4bb7-88b3-4996962dbc65_isrForCrosstalkSources_996985_27.stderr parsl.dataflow.dflow DEBUG: Task 1065 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1065 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1065 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1065 has memoization hash 93dcde49c2b2611b6bbec6b736a3b20c parsl.dataflow.memoization INFO: Task 1065 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f660c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1065 try 0 launched on executor multi with executor id 1066 parsl.dataflow.dflow INFO: Standard output for task 1065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/41773c26-932f-4c59-a10a-7628dbde377a_isrForCrosstalkSources_996990_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/41773c26-932f-4c59-a10a-7628dbde377a_isrForCrosstalkSources_996990_39.stderr parsl.dataflow.dflow DEBUG: Task 1066 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1066 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1066 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1066 has memoization hash 18b252e4cf20a375d7a7edbfbc8dc7f2 parsl.dataflow.memoization INFO: Task 1066 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1066 try 0 launched on executor multi with executor id 1067 parsl.dataflow.dflow INFO: Standard output for task 1066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b8c7af25-ab61-4c7e-87e7-c704ee26363e_isrForCrosstalkSources_996953_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b8c7af25-ab61-4c7e-87e7-c704ee26363e_isrForCrosstalkSources_996953_43.stderr parsl.dataflow.dflow DEBUG: Task 1067 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1067 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1067 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1067 has memoization hash 5fd5178e044e465432c4bc76335ebc5a parsl.dataflow.memoization INFO: Task 1067 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f662a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1067 try 0 launched on executor multi with executor id 1068 parsl.dataflow.dflow INFO: Standard output for task 1067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d29dc867-1e90-4e68-bf73-1a0e6af726fa_isrForCrosstalkSources_996971_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d29dc867-1e90-4e68-bf73-1a0e6af726fa_isrForCrosstalkSources_996971_7.stderr parsl.dataflow.dflow DEBUG: Task 1068 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1068 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1068 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1068 has memoization hash 6d6acc7c55907196f635e2e2b74c0c95 parsl.dataflow.memoization INFO: Task 1068 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1068 try 0 launched on executor multi with executor id 1069 parsl.dataflow.dflow INFO: Standard output for task 1068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/805f8b5a-937d-42f1-a0d5-fe4757e002c8_isrForCrosstalkSources_996965_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/805f8b5a-937d-42f1-a0d5-fe4757e002c8_isrForCrosstalkSources_996965_21.stderr parsl.dataflow.dflow DEBUG: Task 1069 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1069 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1069 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1069 has memoization hash a299593769788dbae782b1d5c203f33f parsl.dataflow.memoization INFO: Task 1069 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1069 try 0 launched on executor multi with executor id 1070 parsl.dataflow.dflow INFO: Standard output for task 1069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0cd7a8fd-01a1-4da2-b085-aedcc3dd2d5e_isrForCrosstalkSources_997012_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0cd7a8fd-01a1-4da2-b085-aedcc3dd2d5e_isrForCrosstalkSources_997012_1.stderr parsl.dataflow.dflow DEBUG: Task 1070 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1070 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1070 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1070 has memoization hash ad38e7a40c75c575a5699d7f15df223b parsl.dataflow.memoization INFO: Task 1070 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f663e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1070 try 0 launched on executor multi with executor id 1071 parsl.dataflow.dflow INFO: Standard output for task 1070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/a93283c9-82c2-4b3f-aaf2-d36601834058_isrForCrosstalkSources_997004_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/a93283c9-82c2-4b3f-aaf2-d36601834058_isrForCrosstalkSources_997004_26.stderr parsl.dataflow.dflow DEBUG: Task 1071 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1071 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1071 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1071 has memoization hash e42975613d0d7db46850b4c51712bc6d parsl.dataflow.memoization INFO: Task 1071 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1071 try 0 launched on executor multi with executor id 1072 parsl.dataflow.dflow INFO: Standard output for task 1071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/de56bb61-8e41-4044-9098-5ffd5634982e_isrForCrosstalkSources_996996_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/de56bb61-8e41-4044-9098-5ffd5634982e_isrForCrosstalkSources_996996_34.stderr parsl.dataflow.dflow DEBUG: Task 1072 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1072 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1072 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1072 has memoization hash 6c99af1620e0142e9c62b0c9deb414b3 parsl.dataflow.memoization INFO: Task 1072 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f665c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1072 try 0 launched on executor multi with executor id 1073 parsl.dataflow.dflow INFO: Standard output for task 1072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/0f0e289a-12f7-4c54-a6c4-6233c62cb7ea_isrForCrosstalkSources_997033_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/0f0e289a-12f7-4c54-a6c4-6233c62cb7ea_isrForCrosstalkSources_997033_47.stderr parsl.dataflow.dflow DEBUG: Task 1073 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1073 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1073 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1073 has memoization hash 1ff97752d9a542b5f1b4cb1969460768 parsl.dataflow.memoization INFO: Task 1073 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1073 try 0 launched on executor multi with executor id 1074 parsl.dataflow.dflow INFO: Standard output for task 1073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d1ae974e-38fe-42c8-8f13-18b49c72841a_isrForCrosstalkSources_997017_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d1ae974e-38fe-42c8-8f13-18b49c72841a_isrForCrosstalkSources_997017_33.stderr parsl.dataflow.dflow DEBUG: Task 1074 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1074 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1074 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1074 has memoization hash 26613f7620d15b4ff6ea1d4c3552da77 parsl.dataflow.memoization INFO: Task 1074 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f667a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1074 try 0 launched on executor multi with executor id 1075 parsl.dataflow.dflow INFO: Standard output for task 1074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/6e7ade40-ec51-428f-814a-311b36300613_isrForCrosstalkSources_997000_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/6e7ade40-ec51-428f-814a-311b36300613_isrForCrosstalkSources_997000_45.stderr parsl.dataflow.dflow DEBUG: Task 1075 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1075 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1075 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1075 has memoization hash 01cd70e4b69cd0efdfdae7dda25eaab7 parsl.dataflow.memoization INFO: Task 1075 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1075 try 0 launched on executor multi with executor id 1076 parsl.dataflow.dflow INFO: Standard output for task 1075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9eaedfff-ff44-4955-b80f-a2614e93cfdf_isrForCrosstalkSources_996995_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9eaedfff-ff44-4955-b80f-a2614e93cfdf_isrForCrosstalkSources_996995_53.stderr parsl.dataflow.dflow DEBUG: Task 1076 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1076 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1076 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1076 has memoization hash 386d9682889cbb194c85260eaf26fe90 parsl.dataflow.memoization INFO: Task 1076 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1076 try 0 launched on executor multi with executor id 1077 parsl.dataflow.dflow INFO: Standard output for task 1076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3b4c81f7-cb3d-43c0-ae7a-7bb97a3bfca7_isrForCrosstalkSources_996959_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3b4c81f7-cb3d-43c0-ae7a-7bb97a3bfca7_isrForCrosstalkSources_996959_32.stderr parsl.dataflow.dflow DEBUG: Task 1077 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1077 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1077 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1077 has memoization hash 75ec9a7757ae60f592020f261ec888d2 parsl.dataflow.memoization INFO: Task 1077 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f668e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1077 try 0 launched on executor multi with executor id 1078 parsl.dataflow.dflow INFO: Standard output for task 1077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/17e28cfa-e183-4420-bf75-5e68da6bf9bf_isrForCrosstalkSources_996982_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/17e28cfa-e183-4420-bf75-5e68da6bf9bf_isrForCrosstalkSources_996982_39.stderr parsl.dataflow.dflow DEBUG: Task 1078 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1078 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1078 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1078 has memoization hash 393dd813baab0bd01b62b8bdef91d79e parsl.dataflow.memoization INFO: Task 1078 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1078 try 0 launched on executor multi with executor id 1079 parsl.dataflow.dflow INFO: Standard output for task 1078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/47c09d99-21b7-4c6a-9692-a2008d0d3765_isrForCrosstalkSources_996966_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/47c09d99-21b7-4c6a-9692-a2008d0d3765_isrForCrosstalkSources_996966_29.stderr parsl.dataflow.dflow DEBUG: Task 1079 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1079 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1079 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1079 has memoization hash 7e81d7dd0879c98da52f934e903ad034 parsl.dataflow.memoization INFO: Task 1079 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1079 try 0 launched on executor multi with executor id 1080 parsl.dataflow.dflow INFO: Standard output for task 1079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4d3dada1-ce8c-4401-af86-57d70f127aac_isrForCrosstalkSources_997018_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4d3dada1-ce8c-4401-af86-57d70f127aac_isrForCrosstalkSources_997018_37.stderr parsl.dataflow.dflow DEBUG: Task 1080 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1080 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1080 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1080 has memoization hash 2833162be16b0cfb42c39f8a4aaf88c1 parsl.dataflow.memoization INFO: Task 1080 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:34 proc_lsst.multi:170 [INFO] [multi] job_id 17074852 proc_lsst.multi INFO: [multi] job_id 17074852 2024-03-15 13:49:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 13 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 13 2024-03-15 13:49:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:34 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074852 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074852 parsl.executors.status_handling DEBUG: Launched block 13 on executor multi with job ID 17074852 parsl.executors.status_handling INFO: Allocated block ID 14 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-14 2024-03-15 13:49:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=14 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-14 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=14 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-14 2024-03-15 13:49:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=14 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=14 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=14 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=14 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 13 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 13 2024-03-15 13:49:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=14 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=14 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1080 try 0 launched on executor multi with executor id 1081 parsl.dataflow.dflow INFO: Standard output for task 1080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/bc893f69-0e9c-4267-8ac3-8e3b9c4b0765_isrForCrosstalkSources_996982_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/bc893f69-0e9c-4267-8ac3-8e3b9c4b0765_isrForCrosstalkSources_996982_45.stderr parsl.dataflow.dflow DEBUG: Task 1081 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1081 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1081 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1081 has memoization hash d545d4141ce96b796a85a355a4c2d6a6 parsl.dataflow.memoization INFO: Task 1081 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1081 try 0 launched on executor multi with executor id 1082 parsl.dataflow.dflow INFO: Standard output for task 1081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/08d7c441-5637-4f9e-b855-61f0651b9dd9_isrForCrosstalkSources_997024_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/08d7c441-5637-4f9e-b855-61f0651b9dd9_isrForCrosstalkSources_997024_7.stderr parsl.dataflow.dflow DEBUG: Task 1082 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1082 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1082 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1082 has memoization hash 6418cc50550a4eb311fc681ca2c0951a parsl.dataflow.memoization INFO: Task 1082 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1082 try 0 launched on executor multi with executor id 1083 parsl.dataflow.dflow INFO: Standard output for task 1082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/7b74aa89-d895-4bb1-8b3e-46b4e03b3d2a_isrForCrosstalkSources_997032_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/7b74aa89-d895-4bb1-8b3e-46b4e03b3d2a_isrForCrosstalkSources_997032_34.stderr parsl.dataflow.dflow DEBUG: Task 1083 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1083 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1083 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1083 has memoization hash 5b5d7910b219262c463f6bfdef82d859 parsl.dataflow.memoization INFO: Task 1083 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1083 try 0 launched on executor multi with executor id 1084 parsl.dataflow.dflow INFO: Standard output for task 1083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/ae99867e-84f5-4de2-898b-6b29453dab97_isrForCrosstalkSources_997025_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/ae99867e-84f5-4de2-898b-6b29453dab97_isrForCrosstalkSources_997025_62.stderr parsl.dataflow.dflow DEBUG: Task 1084 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1084 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1084 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1084 has memoization hash 78800121da82ac7da8c6aeade1da7d15 parsl.dataflow.memoization INFO: Task 1084 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f64040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1084 try 0 launched on executor multi with executor id 1085 parsl.dataflow.dflow INFO: Standard output for task 1084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/dbaf200c-a880-4485-962a-d5f337d5f674_isrForCrosstalkSources_996981_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/dbaf200c-a880-4485-962a-d5f337d5f674_isrForCrosstalkSources_996981_9.stderr parsl.dataflow.dflow DEBUG: Task 1085 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1085 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1085 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1085 has memoization hash a4bd205300ac0fd82704019d33ec9e1f parsl.dataflow.memoization INFO: Task 1085 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1085 try 0 launched on executor multi with executor id 1086 parsl.dataflow.dflow INFO: Standard output for task 1085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5168432e-c6d1-438e-9a56-763bdf74fe4f_isrForCrosstalkSources_996987_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5168432e-c6d1-438e-9a56-763bdf74fe4f_isrForCrosstalkSources_996987_6.stderr parsl.dataflow.dflow DEBUG: Task 1086 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1086 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1086 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1086 has memoization hash bc96c19c6e72492780a3495a16fa1bf9 parsl.dataflow.memoization INFO: Task 1086 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1086 try 0 launched on executor multi with executor id 1087 parsl.dataflow.dflow INFO: Standard output for task 1086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/98cef2ec-2bd5-443f-8c74-a11a09f3f8c7_isrForCrosstalkSources_996967_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/98cef2ec-2bd5-443f-8c74-a11a09f3f8c7_isrForCrosstalkSources_996967_58.stderr parsl.dataflow.dflow DEBUG: Task 1087 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1087 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1087 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1087 has memoization hash 73226312a59f1daafd014fc04446a216 parsl.dataflow.memoization INFO: Task 1087 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1087 try 0 launched on executor multi with executor id 1088 parsl.dataflow.dflow INFO: Standard output for task 1087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/732c7440-1e97-43c2-9383-48565eeffcba_isrForCrosstalkSources_997016_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/732c7440-1e97-43c2-9383-48565eeffcba_isrForCrosstalkSources_997016_40.stderr parsl.dataflow.dflow DEBUG: Task 1088 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1088 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1088 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1088 has memoization hash f290a2ada76f136400a680e10e8f36e7 parsl.dataflow.memoization INFO: Task 1088 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f671a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1088 try 0 launched on executor multi with executor id 1089 parsl.dataflow.dflow INFO: Standard output for task 1088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/bf975c85-7b94-40ef-9516-2ee260959340_isrForCrosstalkSources_996990_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/bf975c85-7b94-40ef-9516-2ee260959340_isrForCrosstalkSources_996990_37.stderr parsl.dataflow.dflow DEBUG: Task 1089 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1089 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1089 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1089 has memoization hash f1b9b349d58a46072af6a1524a82ebe2 parsl.dataflow.memoization INFO: Task 1089 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1089 try 0 launched on executor multi with executor id 1090 parsl.dataflow.dflow INFO: Standard output for task 1089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c3918c1d-3c72-496e-8d68-13b17fd0cf57_isrForCrosstalkSources_996977_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c3918c1d-3c72-496e-8d68-13b17fd0cf57_isrForCrosstalkSources_996977_16.stderr parsl.dataflow.dflow DEBUG: Task 1090 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1090 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1090 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1090 has memoization hash 95731eb4364c2678801e1d240e988dc9 parsl.dataflow.memoization INFO: Task 1090 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1090 try 0 launched on executor multi with executor id 1091 parsl.dataflow.dflow INFO: Standard output for task 1090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/1815ab39-727e-4c85-94c7-180f146b36ca_isrForCrosstalkSources_996954_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/1815ab39-727e-4c85-94c7-180f146b36ca_isrForCrosstalkSources_996954_11.stderr parsl.dataflow.dflow DEBUG: Task 1091 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1091 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1091 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1091 has memoization hash 9fca19318a864578867ad4640d80f2af parsl.dataflow.memoization INFO: Task 1091 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1091 try 0 launched on executor multi with executor id 1092 parsl.dataflow.dflow INFO: Standard output for task 1091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/543c9b31-9a37-457b-8ab4-c29f31626f14_isrForCrosstalkSources_996994_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/543c9b31-9a37-457b-8ab4-c29f31626f14_isrForCrosstalkSources_996994_49.stderr parsl.dataflow.dflow DEBUG: Task 1092 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1092 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1092 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1092 has memoization hash 8ab72f45a0a24f6888b788559e9f71cc parsl.dataflow.memoization INFO: Task 1092 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1092 try 0 launched on executor multi with executor id 1093 parsl.dataflow.dflow INFO: Standard output for task 1092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/89e82789-e4c6-4953-b531-bba6ecf3d058_isrForCrosstalkSources_997020_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/89e82789-e4c6-4953-b531-bba6ecf3d058_isrForCrosstalkSources_997020_34.stderr parsl.dataflow.dflow DEBUG: Task 1093 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1093 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1093 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1093 has memoization hash babc269d09fcaa6f86f239134dd6cab5 parsl.dataflow.memoization INFO: Task 1093 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f672e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1093 try 0 launched on executor multi with executor id 1094 parsl.dataflow.dflow INFO: Standard output for task 1093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/0de5cd82-cfb5-4b94-b28c-761cc7193e77_isrForCrosstalkSources_996969_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/0de5cd82-cfb5-4b94-b28c-761cc7193e77_isrForCrosstalkSources_996969_59.stderr parsl.dataflow.dflow DEBUG: Task 1094 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1094 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1094 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1094 has memoization hash 15a68c04c7d432c975f6b89cf1c1788b parsl.dataflow.memoization INFO: Task 1094 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f674c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1094 try 0 launched on executor multi with executor id 1095 parsl.dataflow.dflow INFO: Standard output for task 1094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ffbbf283-35c3-474e-8169-da64432f608d_isrForCrosstalkSources_996986_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ffbbf283-35c3-474e-8169-da64432f608d_isrForCrosstalkSources_996986_41.stderr parsl.dataflow.dflow DEBUG: Task 1095 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1095 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1095 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1095 has memoization hash 62ac55e4558dc70a358a899d18727c7c parsl.dataflow.memoization INFO: Task 1095 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1095 try 0 launched on executor multi with executor id 1096 parsl.dataflow.dflow INFO: Standard output for task 1095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/b9b8c5cc-6252-4141-867e-bd0424ae9109_isrForCrosstalkSources_997018_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/b9b8c5cc-6252-4141-867e-bd0424ae9109_isrForCrosstalkSources_997018_4.stderr parsl.dataflow.dflow DEBUG: Task 1096 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1096 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1096 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1096 has memoization hash 50f092b4438426e64e5319970f82f412 parsl.dataflow.memoization INFO: Task 1096 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1096 try 0 launched on executor multi with executor id 1097 parsl.dataflow.dflow INFO: Standard output for task 1096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/58c5c2dd-a3ad-4d87-89f4-c3e973d52ef8_isrForCrosstalkSources_996977_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/58c5c2dd-a3ad-4d87-89f4-c3e973d52ef8_isrForCrosstalkSources_996977_48.stderr parsl.dataflow.dflow DEBUG: Task 1097 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1097 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1097 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1097 has memoization hash 13ff227fd0573203ec5f53a610cf6df1 parsl.dataflow.memoization INFO: Task 1097 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1097 try 0 launched on executor multi with executor id 1098 parsl.dataflow.dflow INFO: Standard output for task 1097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/26d0c6b3-cacb-4c17-83af-41005d8dba8a_isrForCrosstalkSources_996983_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/26d0c6b3-cacb-4c17-83af-41005d8dba8a_isrForCrosstalkSources_996983_6.stderr parsl.dataflow.dflow DEBUG: Task 1098 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1098 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1098 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1098 has memoization hash e2e78fa7f28842b0375b9babe6040e41 parsl.dataflow.memoization INFO: Task 1098 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f677e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1098 try 0 launched on executor multi with executor id 1099 parsl.dataflow.dflow INFO: Standard output for task 1098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d4a70fd8-325e-494b-b693-58dd87f950bf_isrForCrosstalkSources_996982_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d4a70fd8-325e-494b-b693-58dd87f950bf_isrForCrosstalkSources_996982_12.stderr parsl.dataflow.dflow DEBUG: Task 1099 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1099 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1099 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1099 has memoization hash 617b487a05ef4bfc7b1326c32225aea2 parsl.dataflow.memoization INFO: Task 1099 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1099 try 0 launched on executor multi with executor id 1100 parsl.dataflow.dflow INFO: Standard output for task 1099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/97b6692c-ad34-48c1-ac2d-ed373151552e_isrForCrosstalkSources_996974_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/97b6692c-ad34-48c1-ac2d-ed373151552e_isrForCrosstalkSources_996974_8.stderr parsl.dataflow.dflow DEBUG: Task 1100 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1100 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1100 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1100 has memoization hash 18f6560d317f1a3d92e6983ba3619208 parsl.dataflow.memoization INFO: Task 1100 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1100 try 0 launched on executor multi with executor id 1101 parsl.dataflow.dflow INFO: Standard output for task 1100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0ab9e3cc-448b-48a7-9d9f-5bba644ed622_isrForCrosstalkSources_997010_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0ab9e3cc-448b-48a7-9d9f-5bba644ed622_isrForCrosstalkSources_997010_25.stderr parsl.dataflow.dflow DEBUG: Task 1101 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1101 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1101 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1101 has memoization hash 1fdeaa90cc7610e17812510a37cfe075 parsl.dataflow.memoization INFO: Task 1101 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f676a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1101 try 0 launched on executor multi with executor id 1102 parsl.dataflow.dflow INFO: Standard output for task 1101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b49430e5-607f-411f-8100-6a46baf9123a_isrForCrosstalkSources_996964_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b49430e5-607f-411f-8100-6a46baf9123a_isrForCrosstalkSources_996964_29.stderr parsl.dataflow.dflow DEBUG: Task 1102 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1102 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1102 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1102 has memoization hash 2ab03a9958263030d5e5fc6d65833145 parsl.dataflow.memoization INFO: Task 1102 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1102 try 0 launched on executor multi with executor id 1103 parsl.dataflow.dflow INFO: Standard output for task 1102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/47764378-8ef0-4390-a62e-81b0d3317ecd_isrForCrosstalkSources_997017_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/47764378-8ef0-4390-a62e-81b0d3317ecd_isrForCrosstalkSources_997017_18.stderr parsl.dataflow.dflow DEBUG: Task 1103 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1103 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1103 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1103 has memoization hash 5c0adf99d92505a33c748b23f1c59eae parsl.dataflow.memoization INFO: Task 1103 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1103 try 0 launched on executor multi with executor id 1104 parsl.dataflow.dflow INFO: Standard output for task 1103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/819ebf35-579a-4098-9a6c-e13c54999685_isrForCrosstalkSources_997025_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/819ebf35-579a-4098-9a6c-e13c54999685_isrForCrosstalkSources_997025_42.stderr parsl.dataflow.dflow DEBUG: Task 1104 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1104 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1104 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1104 has memoization hash 13dab32d19f808c55ef210f1025857f9 parsl.dataflow.memoization INFO: Task 1104 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1104 try 0 launched on executor multi with executor id 1105 parsl.dataflow.dflow INFO: Standard output for task 1104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/d08e9b66-c334-457b-984f-a496427d6f5d_isrForCrosstalkSources_996955_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/d08e9b66-c334-457b-984f-a496427d6f5d_isrForCrosstalkSources_996955_42.stderr parsl.dataflow.dflow DEBUG: Task 1105 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1105 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1105 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1105 has memoization hash d445cfb2290127acca34327afd003823 parsl.dataflow.memoization INFO: Task 1105 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1105 try 0 launched on executor multi with executor id 1106 parsl.dataflow.dflow INFO: Standard output for task 1105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/1c368f6c-f1c7-4e6b-a687-5c75b31f2812_isrForCrosstalkSources_996976_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/1c368f6c-f1c7-4e6b-a687-5c75b31f2812_isrForCrosstalkSources_996976_46.stderr parsl.dataflow.dflow DEBUG: Task 1106 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1106 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1106 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1106 has memoization hash 8622d997382721d96fca59aedcb35b9f parsl.dataflow.memoization INFO: Task 1106 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f679c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1106 try 0 launched on executor multi with executor id 1107 parsl.dataflow.dflow INFO: Standard output for task 1106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/af76bdac-b9c4-4a5a-a6df-bcbf9e42487a_isrForCrosstalkSources_997011_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/af76bdac-b9c4-4a5a-a6df-bcbf9e42487a_isrForCrosstalkSources_997011_17.stderr parsl.dataflow.dflow DEBUG: Task 1107 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1107 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1107 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1107 has memoization hash 2731f8cf9b45fbc3a8ed0e253baebe23 parsl.dataflow.memoization INFO: Task 1107 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1107 try 0 launched on executor multi with executor id 1108 parsl.dataflow.dflow INFO: Standard output for task 1107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/dc7844b2-f034-4302-8c20-86fa14ea0331_isrForCrosstalkSources_997010_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/dc7844b2-f034-4302-8c20-86fa14ea0331_isrForCrosstalkSources_997010_33.stderr parsl.dataflow.dflow DEBUG: Task 1108 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1108 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1108 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1108 has memoization hash b6bfa488de71cd32a607864ba85a6a67 parsl.dataflow.memoization INFO: Task 1108 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1108 try 0 launched on executor multi with executor id 1109 parsl.dataflow.dflow INFO: Standard output for task 1108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/078cbd60-f777-4a7a-99bb-dc7a8052c7a7_isrForCrosstalkSources_996972_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/078cbd60-f777-4a7a-99bb-dc7a8052c7a7_isrForCrosstalkSources_996972_4.stderr parsl.dataflow.dflow DEBUG: Task 1109 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1109 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1109 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1109 has memoization hash e8ddec831732ea334a628f1aa32be2c3 parsl.dataflow.memoization INFO: Task 1109 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1109 try 0 launched on executor multi with executor id 1110 parsl.dataflow.dflow INFO: Standard output for task 1109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f4d14f27-3471-4c0b-ac04-7dbb7f74bae9_isrForCrosstalkSources_996977_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f4d14f27-3471-4c0b-ac04-7dbb7f74bae9_isrForCrosstalkSources_996977_43.stderr parsl.dataflow.dflow DEBUG: Task 1110 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1110 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1110 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1110 has memoization hash 1421fe6b6cc6497a4bdcb5131195fd8c parsl.dataflow.memoization INFO: Task 1110 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1110 try 0 launched on executor multi with executor id 1111 parsl.dataflow.dflow INFO: Standard output for task 1110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a55b7bbb-b782-4482-9ece-e9aa31c0f74b_isrForCrosstalkSources_997000_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a55b7bbb-b782-4482-9ece-e9aa31c0f74b_isrForCrosstalkSources_997000_48.stderr parsl.dataflow.dflow DEBUG: Task 1111 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1111 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1111 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1111 has memoization hash 51b966f34f315b3cace0c797da997203 parsl.dataflow.memoization INFO: Task 1111 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1111 try 0 launched on executor multi with executor id 1112 parsl.dataflow.dflow INFO: Standard output for task 1111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/66d0fb60-6bc9-4692-be4f-809e1c0c66db_isrForCrosstalkSources_996997_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/66d0fb60-6bc9-4692-be4f-809e1c0c66db_isrForCrosstalkSources_996997_4.stderr parsl.dataflow.dflow DEBUG: Task 1112 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1112 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1112 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1112 has memoization hash d345a7c4486675c71097bc19334301a0 parsl.dataflow.memoization INFO: Task 1112 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1112 try 0 launched on executor multi with executor id 1113 parsl.dataflow.dflow INFO: Standard output for task 1112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/7d1daedb-bc0c-4ef0-a801-d0bf7eb823e5_isrForCrosstalkSources_996992_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/7d1daedb-bc0c-4ef0-a801-d0bf7eb823e5_isrForCrosstalkSources_996992_34.stderr parsl.dataflow.dflow DEBUG: Task 1113 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1113 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1113 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1113 has memoization hash 08adae0ff424f155274ca0199569cea1 parsl.dataflow.memoization INFO: Task 1113 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e742c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1113 try 0 launched on executor multi with executor id 1114 parsl.dataflow.dflow INFO: Standard output for task 1113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/d094433f-cb56-40db-ad16-02e40e67947b_isrForCrosstalkSources_996963_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/d094433f-cb56-40db-ad16-02e40e67947b_isrForCrosstalkSources_996963_53.stderr parsl.dataflow.dflow DEBUG: Task 1114 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1114 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1114 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1114 has memoization hash 37d2a27ace51100dc00824afbe4c9885 parsl.dataflow.memoization INFO: Task 1114 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1114 try 0 launched on executor multi with executor id 1115 parsl.dataflow.dflow INFO: Standard output for task 1114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4ec175dd-a44b-4800-86ec-4feab3945b88_isrForCrosstalkSources_996995_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4ec175dd-a44b-4800-86ec-4feab3945b88_isrForCrosstalkSources_996995_37.stderr parsl.dataflow.dflow DEBUG: Task 1115 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1115 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1115 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1115 has memoization hash e2f257b5be253faf287ba73f337eca3c parsl.dataflow.memoization INFO: Task 1115 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e77b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1115 try 0 launched on executor multi with executor id 1116 parsl.dataflow.dflow INFO: Standard output for task 1115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/cc342ad8-55ee-4d3e-a078-b7c6d33c319f_isrForCrosstalkSources_996980_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/cc342ad8-55ee-4d3e-a078-b7c6d33c319f_isrForCrosstalkSources_996980_58.stderr parsl.dataflow.dflow DEBUG: Task 1116 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1116 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1116 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1116 has memoization hash ffe6633bfe59741a6a29b34e7c13c169 parsl.dataflow.memoization INFO: Task 1116 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1116 try 0 launched on executor multi with executor id 1117 parsl.dataflow.dflow INFO: Standard output for task 1116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8ab1b09c-d3b9-4138-a0d4-4b9b8ba811e1_isrForCrosstalkSources_996977_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8ab1b09c-d3b9-4138-a0d4-4b9b8ba811e1_isrForCrosstalkSources_996977_56.stderr parsl.dataflow.dflow DEBUG: Task 1117 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1117 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1117 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1117 has memoization hash dfe17e4fdc6af4cd064bf4ce4ad0f356 parsl.dataflow.memoization INFO: Task 1117 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1117 try 0 launched on executor multi with executor id 1118 parsl.dataflow.dflow INFO: Standard output for task 1117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/9caa13e0-2187-4ecf-9869-43ab061fa53b_isrForCrosstalkSources_996975_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/9caa13e0-2187-4ecf-9869-43ab061fa53b_isrForCrosstalkSources_996975_39.stderr parsl.dataflow.dflow DEBUG: Task 1118 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1118 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1118 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1118 has memoization hash 90755bfc3759cfb8ec89cbc2d0e87c2c parsl.dataflow.memoization INFO: Task 1118 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1118 try 0 launched on executor multi with executor id 1119 parsl.dataflow.dflow INFO: Standard output for task 1118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5b77f48f-5214-465e-8b88-e87aea8be8d5_isrForCrosstalkSources_997017_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5b77f48f-5214-465e-8b88-e87aea8be8d5_isrForCrosstalkSources_997017_45.stderr parsl.dataflow.dflow DEBUG: Task 1119 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1119 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1119 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1119 has memoization hash 33c7a8f237a95d6ee2a7a4dd2e793d49 parsl.dataflow.memoization INFO: Task 1119 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1119 try 0 launched on executor multi with executor id 1120 parsl.dataflow.dflow INFO: Standard output for task 1119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d13afab9-1758-4e01-af80-18ca3bff2ba0_isrForCrosstalkSources_997003_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d13afab9-1758-4e01-af80-18ca3bff2ba0_isrForCrosstalkSources_997003_57.stderr parsl.dataflow.dflow DEBUG: Task 1120 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1120 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1120 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1120 has memoization hash d598ca8065d5277061f1ce91bcff47e7 parsl.dataflow.memoization INFO: Task 1120 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1120 try 0 launched on executor multi with executor id 1121 parsl.dataflow.dflow INFO: Standard output for task 1120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/d8905dc6-fbe6-4137-a9a8-79a0894311ee_isrForCrosstalkSources_996956_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/d8905dc6-fbe6-4137-a9a8-79a0894311ee_isrForCrosstalkSources_996956_34.stderr parsl.dataflow.dflow DEBUG: Task 1121 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1121 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1121 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1121 has memoization hash 6f5391f66c976f5a59bdb108333d677b parsl.dataflow.memoization INFO: Task 1121 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1121 try 0 launched on executor multi with executor id 1122 parsl.dataflow.dflow INFO: Standard output for task 1121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/5b979aae-f932-405b-a9aa-24c1c60198fd_isrForCrosstalkSources_996986_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/5b979aae-f932-405b-a9aa-24c1c60198fd_isrForCrosstalkSources_996986_5.stderr parsl.dataflow.dflow DEBUG: Task 1122 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1122 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1122 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1122 has memoization hash 4db81a5ceaa69fe09c3173caa200c4c6 parsl.dataflow.memoization INFO: Task 1122 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1122 try 0 launched on executor multi with executor id 1123 parsl.dataflow.dflow INFO: Standard output for task 1122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c3bae945-6dec-4259-8a6b-107282cb88f9_isrForCrosstalkSources_997020_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c3bae945-6dec-4259-8a6b-107282cb88f9_isrForCrosstalkSources_997020_49.stderr parsl.dataflow.dflow DEBUG: Task 1123 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1123 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1123 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1123 has memoization hash 0b87c867498e7bf60718db03175f8239 parsl.dataflow.memoization INFO: Task 1123 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1123 try 0 launched on executor multi with executor id 1124 parsl.dataflow.dflow INFO: Standard output for task 1123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/ccbe72e2-fe76-4ec7-8c19-1e623d632590_isrForCrosstalkSources_997032_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/ccbe72e2-fe76-4ec7-8c19-1e623d632590_isrForCrosstalkSources_997032_24.stderr parsl.dataflow.dflow DEBUG: Task 1124 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1124 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1124 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1124 has memoization hash a61ff1144798c6d7a3b09265eef8721c parsl.dataflow.memoization INFO: Task 1124 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1124 try 0 launched on executor multi with executor id 1125 parsl.dataflow.dflow INFO: Standard output for task 1124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/ab58644f-4b15-485f-ae74-d58fcfe458dc_isrForCrosstalkSources_996985_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/ab58644f-4b15-485f-ae74-d58fcfe458dc_isrForCrosstalkSources_996985_34.stderr parsl.dataflow.dflow DEBUG: Task 1125 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1125 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1125 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1125 has memoization hash 2eda869d6e624d88e3a53a07f8376021 parsl.dataflow.memoization INFO: Task 1125 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1125 try 0 launched on executor multi with executor id 1126 parsl.dataflow.dflow INFO: Standard output for task 1125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d38c02b6-5d32-48d4-9eef-89e854b55534_isrForCrosstalkSources_996997_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d38c02b6-5d32-48d4-9eef-89e854b55534_isrForCrosstalkSources_996997_24.stderr parsl.dataflow.dflow DEBUG: Task 1126 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1126 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1126 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1126 has memoization hash 417d35ebc5d361fce61e7123a66bd806 parsl.dataflow.memoization INFO: Task 1126 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1126 try 0 launched on executor multi with executor id 1127 parsl.dataflow.dflow INFO: Standard output for task 1126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e3e36a87-ab49-404b-8419-33998435197a_isrForCrosstalkSources_997005_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e3e36a87-ab49-404b-8419-33998435197a_isrForCrosstalkSources_997005_37.stderr parsl.dataflow.dflow DEBUG: Task 1127 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1127 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1127 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1127 has memoization hash 7a2e3612d914ef9b184ba0a7089a3e25 parsl.dataflow.memoization INFO: Task 1127 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1127 try 0 launched on executor multi with executor id 1128 parsl.dataflow.dflow INFO: Standard output for task 1127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/a0c198e7-9ad0-4f1b-9266-11548a3a230d_isrForCrosstalkSources_996986_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/a0c198e7-9ad0-4f1b-9266-11548a3a230d_isrForCrosstalkSources_996986_43.stderr parsl.dataflow.dflow DEBUG: Task 1128 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1128 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1128 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1128 has memoization hash 335fe23d638b1cb21d39fecc3570e0ef parsl.dataflow.memoization INFO: Task 1128 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1128 try 0 launched on executor multi with executor id 1129 parsl.dataflow.dflow INFO: Standard output for task 1128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c6f0f579-e782-480d-84ca-a1d3d6d15b87_isrForCrosstalkSources_997020_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c6f0f579-e782-480d-84ca-a1d3d6d15b87_isrForCrosstalkSources_997020_45.stderr parsl.dataflow.dflow DEBUG: Task 1129 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1129 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1129 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1129 has memoization hash 5c1d8d5e53ca4dd8e8c14c20f478bf86 parsl.dataflow.memoization INFO: Task 1129 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1129 try 0 launched on executor multi with executor id 1130 parsl.dataflow.dflow INFO: Standard output for task 1129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/31a1548a-42de-404b-8383-604e5b8519bf_isrForCrosstalkSources_997001_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/31a1548a-42de-404b-8383-604e5b8519bf_isrForCrosstalkSources_997001_46.stderr parsl.dataflow.dflow DEBUG: Task 1130 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1130 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1130 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1130 has memoization hash c9d9267446fea07abdbb5a458fcedf67 parsl.dataflow.memoization INFO: Task 1130 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1130 try 0 launched on executor multi with executor id 1131 parsl.dataflow.dflow INFO: Standard output for task 1130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a942a5b2-95e7-4f86-b5ca-c7402e3f1fcb_isrForCrosstalkSources_997011_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a942a5b2-95e7-4f86-b5ca-c7402e3f1fcb_isrForCrosstalkSources_997011_3.stderr parsl.dataflow.dflow DEBUG: Task 1131 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1131 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1131 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1131 has memoization hash 5b2c7067ff1630e378d83fcfae7f5ec3 parsl.dataflow.memoization INFO: Task 1131 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1131 try 0 launched on executor multi with executor id 1132 parsl.dataflow.dflow INFO: Standard output for task 1131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c0c9a97f-2b12-4c32-b053-fe4e0a8f3a68_isrForCrosstalkSources_997004_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c0c9a97f-2b12-4c32-b053-fe4e0a8f3a68_isrForCrosstalkSources_997004_44.stderr parsl.dataflow.dflow DEBUG: Task 1132 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1132 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1132 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1132 has memoization hash 47ef86121f3869907ba189eeecc124ec parsl.dataflow.memoization INFO: Task 1132 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1132 try 0 launched on executor multi with executor id 1133 parsl.dataflow.dflow INFO: Standard output for task 1132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/abc005b3-085c-4e1b-9ec8-b7798158ab26_isrForCrosstalkSources_996995_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/abc005b3-085c-4e1b-9ec8-b7798158ab26_isrForCrosstalkSources_996995_50.stderr parsl.dataflow.dflow DEBUG: Task 1133 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1133 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1133 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1133 has memoization hash 591d48b5270fb855b6ec71717ef4fad7 parsl.dataflow.memoization INFO: Task 1133 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1133 try 0 launched on executor multi with executor id 1134 parsl.dataflow.dflow INFO: Standard output for task 1133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a56d2b43-ab2a-4695-96cd-51627cdd6761_isrForCrosstalkSources_996970_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a56d2b43-ab2a-4695-96cd-51627cdd6761_isrForCrosstalkSources_996970_51.stderr parsl.dataflow.dflow DEBUG: Task 1134 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1134 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1134 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1134 has memoization hash d83a3a75971287efe2341499ae4a1342 parsl.dataflow.memoization INFO: Task 1134 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1134 try 0 launched on executor multi with executor id 1135 parsl.dataflow.dflow INFO: Standard output for task 1134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/805db196-a535-4b99-8887-2267bd0f3f55_isrForCrosstalkSources_996991_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/805db196-a535-4b99-8887-2267bd0f3f55_isrForCrosstalkSources_996991_62.stderr parsl.dataflow.dflow DEBUG: Task 1135 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1135 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1135 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1135 has memoization hash 629ae47a2cd0ebe8d2ca3f0115b6ffa6 parsl.dataflow.memoization INFO: Task 1135 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1135 try 0 launched on executor multi with executor id 1136 parsl.dataflow.dflow INFO: Standard output for task 1135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4f85b213-46a5-47c9-8856-ca1c4f505de2_isrForCrosstalkSources_996976_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4f85b213-46a5-47c9-8856-ca1c4f505de2_isrForCrosstalkSources_996976_9.stderr parsl.dataflow.dflow DEBUG: Task 1136 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1136 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1136 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1136 has memoization hash fb087f41af8774723121368863bfd756 parsl.dataflow.memoization INFO: Task 1136 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1136 try 0 launched on executor multi with executor id 1137 parsl.dataflow.dflow INFO: Standard output for task 1136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b7bb596b-38fd-48da-a8fb-a45a4f31d541_isrForCrosstalkSources_997003_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b7bb596b-38fd-48da-a8fb-a45a4f31d541_isrForCrosstalkSources_997003_7.stderr parsl.dataflow.dflow DEBUG: Task 1137 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1137 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1137 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1137 has memoization hash 7c99767be51ebaa1dbdb6c1926116a0c parsl.dataflow.memoization INFO: Task 1137 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1137 try 0 launched on executor multi with executor id 1138 parsl.dataflow.dflow INFO: Standard output for task 1137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1b4dd511-db2e-44ab-ac56-ebadd32576e2_isrForCrosstalkSources_997005_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1b4dd511-db2e-44ab-ac56-ebadd32576e2_isrForCrosstalkSources_997005_22.stderr parsl.dataflow.dflow DEBUG: Task 1138 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1138 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1138 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1138 has memoization hash d8579b86e9e83e8498c66d60e4570f0e parsl.dataflow.memoization INFO: Task 1138 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1138 try 0 launched on executor multi with executor id 1139 parsl.dataflow.dflow INFO: Standard output for task 1138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/64c89996-0dc1-4f0b-914c-cf9dbf9aa3d5_isrForCrosstalkSources_996954_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/64c89996-0dc1-4f0b-914c-cf9dbf9aa3d5_isrForCrosstalkSources_996954_58.stderr parsl.dataflow.dflow DEBUG: Task 1139 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1139 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1139 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1139 has memoization hash bd7a63d130d120ab778e8e6afd3594c3 parsl.dataflow.memoization INFO: Task 1139 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1139 try 0 launched on executor multi with executor id 1140 parsl.dataflow.dflow INFO: Standard output for task 1139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/aa709e19-52fb-44e0-a9e5-a2388db2f529_isrForCrosstalkSources_996956_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/aa709e19-52fb-44e0-a9e5-a2388db2f529_isrForCrosstalkSources_996956_53.stderr parsl.dataflow.dflow DEBUG: Task 1140 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1140 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1140 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1140 has memoization hash 0cd7d8f9f057afa0c25b62adf70ee18a parsl.dataflow.memoization INFO: Task 1140 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1140 try 0 launched on executor multi with executor id 1141 parsl.dataflow.dflow INFO: Standard output for task 1140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/78108d1a-ad6a-47da-9141-12e17d6a5d69_isrForCrosstalkSources_996962_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/78108d1a-ad6a-47da-9141-12e17d6a5d69_isrForCrosstalkSources_996962_29.stderr parsl.dataflow.dflow DEBUG: Task 1141 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1141 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1141 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1141 has memoization hash 0fe1f1ff10850dee871793d587e497bf parsl.dataflow.memoization INFO: Task 1141 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1141 try 0 launched on executor multi with executor id 1142 parsl.dataflow.dflow INFO: Standard output for task 1141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e2ae19ce-e8a1-4f58-b24a-5479bbeb2160_isrForCrosstalkSources_997026_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e2ae19ce-e8a1-4f58-b24a-5479bbeb2160_isrForCrosstalkSources_997026_5.stderr parsl.dataflow.dflow DEBUG: Task 1142 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1142 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1142 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1142 has memoization hash c250704ec5dffdc590635065535e1391 parsl.dataflow.memoization INFO: Task 1142 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1142 try 0 launched on executor multi with executor id 1143 parsl.dataflow.dflow INFO: Standard output for task 1142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b2dccd33-a1e1-4399-8e19-13f7272fff75_isrForCrosstalkSources_996962_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b2dccd33-a1e1-4399-8e19-13f7272fff75_isrForCrosstalkSources_996962_10.stderr parsl.dataflow.dflow DEBUG: Task 1143 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1143 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1143 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1143 has memoization hash 08eb23218fbe3989e8d405f7a1865562 parsl.dataflow.memoization INFO: Task 1143 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1143 try 0 launched on executor multi with executor id 1144 parsl.dataflow.dflow INFO: Standard output for task 1143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/28b7b053-8b8e-4e93-b09c-86e0f8f2a7d0_isrForCrosstalkSources_996993_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/28b7b053-8b8e-4e93-b09c-86e0f8f2a7d0_isrForCrosstalkSources_996993_20.stderr parsl.dataflow.dflow DEBUG: Task 1144 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1144 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1144 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1144 has memoization hash 2e8c6fe5949d6ad4da28654d4b70f979 parsl.dataflow.memoization INFO: Task 1144 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1144 try 0 launched on executor multi with executor id 1145 parsl.dataflow.dflow INFO: Standard output for task 1144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/e70921fc-c229-4b8a-8c1c-d0a6c8dc3fee_isrForCrosstalkSources_997013_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/e70921fc-c229-4b8a-8c1c-d0a6c8dc3fee_isrForCrosstalkSources_997013_11.stderr parsl.dataflow.dflow DEBUG: Task 1145 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1145 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1145 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1145 has memoization hash 194ef014250c74b3d8d53fad882a6bc5 parsl.dataflow.memoization INFO: Task 1145 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1145 try 0 launched on executor multi with executor id 1146 parsl.dataflow.dflow INFO: Standard output for task 1145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/2e92b2bd-5c4b-43d1-8e17-c4521691fde8_isrForCrosstalkSources_996985_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/2e92b2bd-5c4b-43d1-8e17-c4521691fde8_isrForCrosstalkSources_996985_61.stderr parsl.dataflow.dflow DEBUG: Task 1146 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1146 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1146 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1146 has memoization hash cf6248c05126bd50c4f2f3809e9755cf parsl.dataflow.memoization INFO: Task 1146 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1146 try 0 launched on executor multi with executor id 1147 parsl.dataflow.dflow INFO: Standard output for task 1146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/7fbeeba6-5452-4e76-a9d4-06a311cea8b4_isrForCrosstalkSources_996959_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/7fbeeba6-5452-4e76-a9d4-06a311cea8b4_isrForCrosstalkSources_996959_59.stderr parsl.dataflow.dflow DEBUG: Task 1147 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1147 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1147 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1147 has memoization hash fc67e839094c5f61f5246f09430a399a parsl.dataflow.memoization INFO: Task 1147 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1147 try 0 launched on executor multi with executor id 1148 parsl.dataflow.dflow INFO: Standard output for task 1147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/974eddc5-4261-467b-8be3-3350eb01e88b_isrForCrosstalkSources_996972_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/974eddc5-4261-467b-8be3-3350eb01e88b_isrForCrosstalkSources_996972_19.stderr parsl.dataflow.dflow DEBUG: Task 1148 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1148 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1148 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1148 has memoization hash 4b1284ffc892843ec941b99ad24875f7 parsl.dataflow.memoization INFO: Task 1148 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1148 try 0 launched on executor multi with executor id 1149 parsl.dataflow.dflow INFO: Standard output for task 1148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/822ab246-c73c-4a1c-af65-13c03a284b89_isrForCrosstalkSources_997032_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/822ab246-c73c-4a1c-af65-13c03a284b89_isrForCrosstalkSources_997032_44.stderr parsl.dataflow.dflow DEBUG: Task 1149 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1149 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1149 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1149 has memoization hash d542932144fbedab94f8749f32ebfc05 parsl.dataflow.memoization INFO: Task 1149 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1149 try 0 launched on executor multi with executor id 1150 parsl.dataflow.dflow INFO: Standard output for task 1149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/594d6f13-16bb-434f-9d4d-a321158a0201_isrForCrosstalkSources_996955_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/594d6f13-16bb-434f-9d4d-a321158a0201_isrForCrosstalkSources_996955_27.stderr parsl.dataflow.dflow DEBUG: Task 1150 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1150 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1150 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1150 has memoization hash a7296708174289708739315fa0a91d76 parsl.dataflow.memoization INFO: Task 1150 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1150 try 0 launched on executor multi with executor id 1151 parsl.dataflow.dflow INFO: Standard output for task 1150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/51859399-caba-4961-ad68-9ab6453b5181_isrForCrosstalkSources_996953_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/51859399-caba-4961-ad68-9ab6453b5181_isrForCrosstalkSources_996953_49.stderr parsl.dataflow.dflow DEBUG: Task 1151 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1151 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1151 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1151 has memoization hash cbcf2866811cc8e59fa3180ae55a11ec parsl.dataflow.memoization INFO: Task 1151 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1151 try 0 launched on executor multi with executor id 1152 parsl.dataflow.dflow INFO: Standard output for task 1151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/a63efe0d-7b0d-4b80-899d-f4438838409d_isrForCrosstalkSources_997010_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/a63efe0d-7b0d-4b80-899d-f4438838409d_isrForCrosstalkSources_997010_5.stderr parsl.dataflow.dflow DEBUG: Task 1152 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1152 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1152 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1152 has memoization hash e7f8f073407c0a990b9247bea838c6f8 parsl.dataflow.memoization INFO: Task 1152 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1152 try 0 launched on executor multi with executor id 1153 parsl.dataflow.dflow INFO: Standard output for task 1152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f35908ed-3989-4654-b277-1f27876a69f0_isrForCrosstalkSources_997020_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f35908ed-3989-4654-b277-1f27876a69f0_isrForCrosstalkSources_997020_35.stderr parsl.dataflow.dflow DEBUG: Task 1153 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1153 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1153 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1153 has memoization hash a34a7570bc4fb063937927d8dd79c83d parsl.dataflow.memoization INFO: Task 1153 had no result in cache 2024-03-15 13:49:35 proc_lsst.multi:170 [INFO] [multi] job_id 17074853 parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) proc_lsst.multi INFO: [multi] job_id 17074853 2024-03-15 13:49:35 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 14 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 14 2024-03-15 13:49:35 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:35 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074853 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074853 parsl.executors.status_handling DEBUG: Launched block 14 on executor multi with job ID 17074853 parsl.executors.status_handling INFO: Allocated block ID 15 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-15 2024-03-15 13:49:35 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=15 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-15 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=15 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-15 2024-03-15 13:49:35 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=15 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=15 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:35 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=15 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=15 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 14 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 14 2024-03-15 13:49:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:35 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=15 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=15 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1153 try 0 launched on executor multi with executor id 1154 parsl.dataflow.dflow INFO: Standard output for task 1153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d51ef99a-7877-4d06-9a3c-ced625a0b46d_isrForCrosstalkSources_996954_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d51ef99a-7877-4d06-9a3c-ced625a0b46d_isrForCrosstalkSources_996954_42.stderr parsl.dataflow.dflow DEBUG: Task 1154 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1154 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1154 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1154 has memoization hash 9f145782a85fe4ed29fb31d29cc141bb parsl.dataflow.memoization INFO: Task 1154 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77fee4040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1154 try 0 launched on executor multi with executor id 1155 parsl.dataflow.dflow INFO: Standard output for task 1154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7b56c0a5-2df3-4de7-9bd7-104eff6a2d85_isrForCrosstalkSources_996993_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7b56c0a5-2df3-4de7-9bd7-104eff6a2d85_isrForCrosstalkSources_996993_2.stderr parsl.dataflow.dflow DEBUG: Task 1155 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1155 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1155 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1155 has memoization hash 4fe6f653cd37763a4bb3eae35b989a76 parsl.dataflow.memoization INFO: Task 1155 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1155 try 0 launched on executor multi with executor id 1156 parsl.dataflow.dflow INFO: Standard output for task 1155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/db0a931f-3cfa-4c79-8ff0-c1b32d7cb8e8_isrForCrosstalkSources_996955_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/db0a931f-3cfa-4c79-8ff0-c1b32d7cb8e8_isrForCrosstalkSources_996955_46.stderr parsl.dataflow.dflow DEBUG: Task 1156 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1156 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1156 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1156 has memoization hash 8d940cd76dcf54f0f43e5a4258be66fb parsl.dataflow.memoization INFO: Task 1156 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1156 try 0 launched on executor multi with executor id 1157 parsl.dataflow.dflow INFO: Standard output for task 1156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b19d1ef0-5441-41f3-b8fb-e04e361b8d2b_isrForCrosstalkSources_997016_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b19d1ef0-5441-41f3-b8fb-e04e361b8d2b_isrForCrosstalkSources_997016_41.stderr parsl.dataflow.dflow DEBUG: Task 1157 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1157 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1157 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1157 has memoization hash 128a29c7d0c4fb905f0b895c71e5bd9c parsl.dataflow.memoization INFO: Task 1157 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1157 try 0 launched on executor multi with executor id 1158 parsl.dataflow.dflow INFO: Standard output for task 1157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/17ee4e50-ec0c-46c6-909f-d8d0730766bd_isrForCrosstalkSources_997020_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/17ee4e50-ec0c-46c6-909f-d8d0730766bd_isrForCrosstalkSources_997020_18.stderr parsl.dataflow.dflow DEBUG: Task 1158 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1158 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1158 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1158 has memoization hash d958c4bea070e570fb24b6d5898f1075 parsl.dataflow.memoization INFO: Task 1158 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1158 try 0 launched on executor multi with executor id 1159 parsl.dataflow.dflow INFO: Standard output for task 1158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e938bb47-0523-4446-a7e7-af83238ffa89_isrForCrosstalkSources_997022_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e938bb47-0523-4446-a7e7-af83238ffa89_isrForCrosstalkSources_997022_18.stderr parsl.dataflow.dflow DEBUG: Task 1159 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1159 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1159 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1159 has memoization hash 16d6695a3879e4d89dc85f43ba47ed13 parsl.dataflow.memoization INFO: Task 1159 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1159 try 0 launched on executor multi with executor id 1160 parsl.dataflow.dflow INFO: Standard output for task 1159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6946f26f-1efc-4c16-afb6-fd8efc143c46_isrForCrosstalkSources_997030_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6946f26f-1efc-4c16-afb6-fd8efc143c46_isrForCrosstalkSources_997030_26.stderr parsl.dataflow.dflow DEBUG: Task 1160 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1160 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1160 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1160 has memoization hash b4c0fba2427cc3939c8fb8db39ea4631 parsl.dataflow.memoization INFO: Task 1160 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1160 try 0 launched on executor multi with executor id 1161 parsl.dataflow.dflow INFO: Standard output for task 1160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d810fba2-d99b-4e9e-9476-e500e4b4a7ee_isrForCrosstalkSources_997010_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d810fba2-d99b-4e9e-9476-e500e4b4a7ee_isrForCrosstalkSources_997010_57.stderr parsl.dataflow.dflow DEBUG: Task 1161 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1161 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1161 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1161 has memoization hash 02c8bec20ad2ebabd449e7eaba470628 parsl.dataflow.memoization INFO: Task 1161 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1161 try 0 launched on executor multi with executor id 1162 parsl.dataflow.dflow INFO: Standard output for task 1161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/24d583d4-a8af-48f4-983a-76e164c75572_isrForCrosstalkSources_997019_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/24d583d4-a8af-48f4-983a-76e164c75572_isrForCrosstalkSources_997019_23.stderr parsl.dataflow.dflow DEBUG: Task 1162 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1162 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1162 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1162 has memoization hash 71130af273e81f915f892295eda3e7c8 parsl.dataflow.memoization INFO: Task 1162 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1162 try 0 launched on executor multi with executor id 1163 parsl.dataflow.dflow INFO: Standard output for task 1162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/94bffaba-2460-4e36-a5a4-845fa7a42b46_isrForCrosstalkSources_996989_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/94bffaba-2460-4e36-a5a4-845fa7a42b46_isrForCrosstalkSources_996989_42.stderr parsl.dataflow.dflow DEBUG: Task 1163 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1163 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1163 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1163 has memoization hash 32fc4049bd9aa606c61574d0897425f6 parsl.dataflow.memoization INFO: Task 1163 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1163 try 0 launched on executor multi with executor id 1164 parsl.dataflow.dflow INFO: Standard output for task 1163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/11f30f70-f027-4ee8-bfc6-f6e695faa5d9_isrForCrosstalkSources_997011_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/11f30f70-f027-4ee8-bfc6-f6e695faa5d9_isrForCrosstalkSources_997011_53.stderr parsl.dataflow.dflow DEBUG: Task 1164 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1164 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1164 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1164 has memoization hash 308555dd8c61a0ccfb590070a09a72ea parsl.dataflow.memoization INFO: Task 1164 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1164 try 0 launched on executor multi with executor id 1165 parsl.dataflow.dflow INFO: Standard output for task 1164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d98fb72f-67db-47ff-9cc7-a47386895cf1_isrForCrosstalkSources_997001_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d98fb72f-67db-47ff-9cc7-a47386895cf1_isrForCrosstalkSources_997001_39.stderr parsl.dataflow.dflow DEBUG: Task 1165 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1165 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1165 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1165 has memoization hash 7db8f61cbe2a346d37c32942f7239143 parsl.dataflow.memoization INFO: Task 1165 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1165 try 0 launched on executor multi with executor id 1166 parsl.dataflow.dflow INFO: Standard output for task 1165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6439a416-fe1d-4e98-a644-fc141dc5a846_isrForCrosstalkSources_996952_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6439a416-fe1d-4e98-a644-fc141dc5a846_isrForCrosstalkSources_996952_31.stderr parsl.dataflow.dflow DEBUG: Task 1166 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1166 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1166 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1166 has memoization hash d7c40f793fa70927938ef5efde4f723e parsl.dataflow.memoization INFO: Task 1166 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1166 try 0 launched on executor multi with executor id 1167 parsl.dataflow.dflow INFO: Standard output for task 1166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/7ab97a65-ac21-4c0b-8269-892678ff3cb3_isrForCrosstalkSources_996977_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/7ab97a65-ac21-4c0b-8269-892678ff3cb3_isrForCrosstalkSources_996977_58.stderr parsl.dataflow.dflow DEBUG: Task 1167 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1167 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1167 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1167 has memoization hash 6412466317878c36cc2e15d969d3aac8 parsl.dataflow.memoization INFO: Task 1167 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1167 try 0 launched on executor multi with executor id 1168 parsl.dataflow.dflow INFO: Standard output for task 1167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b2111f0c-297d-4d02-97c4-1226b1cab4ed_isrForCrosstalkSources_997030_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b2111f0c-297d-4d02-97c4-1226b1cab4ed_isrForCrosstalkSources_997030_24.stderr parsl.dataflow.dflow DEBUG: Task 1168 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1168 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1168 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1168 has memoization hash 813ce8f2468437a5818f73ffcc03e752 parsl.dataflow.memoization INFO: Task 1168 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1168 try 0 launched on executor multi with executor id 1169 parsl.dataflow.dflow INFO: Standard output for task 1168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/aad7d0cc-2490-4b27-9d7e-55bc9fe143cb_isrForCrosstalkSources_996996_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/aad7d0cc-2490-4b27-9d7e-55bc9fe143cb_isrForCrosstalkSources_996996_14.stderr parsl.dataflow.dflow DEBUG: Task 1169 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1169 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1169 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1169 has memoization hash ef8c448a0c4a76e03dd68aacc1bd8e57 parsl.dataflow.memoization INFO: Task 1169 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1169 try 0 launched on executor multi with executor id 1170 parsl.dataflow.dflow INFO: Standard output for task 1169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/2cc2f660-29d4-4d63-a1f5-d0b59eea3216_isrForCrosstalkSources_996992_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/2cc2f660-29d4-4d63-a1f5-d0b59eea3216_isrForCrosstalkSources_996992_58.stderr parsl.dataflow.dflow DEBUG: Task 1170 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1170 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1170 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1170 has memoization hash c4eed2730c31c749688ad20dc189ecd8 parsl.dataflow.memoization INFO: Task 1170 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1170 try 0 launched on executor multi with executor id 1171 parsl.dataflow.dflow INFO: Standard output for task 1170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8fb406ca-193b-4fde-92fb-7d7faff51e28_isrForCrosstalkSources_996963_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8fb406ca-193b-4fde-92fb-7d7faff51e28_isrForCrosstalkSources_996963_19.stderr parsl.dataflow.dflow DEBUG: Task 1171 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1171 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1171 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1171 has memoization hash 6218abd8760f15e396ce77548f1d5568 parsl.dataflow.memoization INFO: Task 1171 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1171 try 0 launched on executor multi with executor id 1172 parsl.dataflow.dflow INFO: Standard output for task 1171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/bd4859e9-b8aa-43f2-8587-5ca72fc9fd7e_isrForCrosstalkSources_996989_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/bd4859e9-b8aa-43f2-8587-5ca72fc9fd7e_isrForCrosstalkSources_996989_55.stderr parsl.dataflow.dflow DEBUG: Task 1172 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1172 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1172 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1172 has memoization hash f6fe8917f12b800150e284790ba8cd21 parsl.dataflow.memoization INFO: Task 1172 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1172 try 0 launched on executor multi with executor id 1173 parsl.dataflow.dflow INFO: Standard output for task 1172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/823e0feb-8628-4bbd-9b09-08299f51bf41_isrForCrosstalkSources_996962_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/823e0feb-8628-4bbd-9b09-08299f51bf41_isrForCrosstalkSources_996962_45.stderr parsl.dataflow.dflow DEBUG: Task 1173 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1173 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1173 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1173 has memoization hash 5f5a027c59cb05252cdeb86c72feee17 parsl.dataflow.memoization INFO: Task 1173 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1173 try 0 launched on executor multi with executor id 1174 parsl.dataflow.dflow INFO: Standard output for task 1173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2fadd61b-19dc-4ada-95b7-89a90bb9c66e_isrForCrosstalkSources_996956_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2fadd61b-19dc-4ada-95b7-89a90bb9c66e_isrForCrosstalkSources_996956_13.stderr parsl.dataflow.dflow DEBUG: Task 1174 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1174 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1174 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1174 has memoization hash a9d72c517ffcab5ca3af86b1594768fd parsl.dataflow.memoization INFO: Task 1174 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1174 try 0 launched on executor multi with executor id 1175 parsl.dataflow.dflow INFO: Standard output for task 1174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/21fcb382-9b78-4d6f-b456-a0e17531b1cb_isrForCrosstalkSources_997028_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/21fcb382-9b78-4d6f-b456-a0e17531b1cb_isrForCrosstalkSources_997028_58.stderr parsl.dataflow.dflow DEBUG: Task 1175 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1175 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1175 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1175 has memoization hash 23507354b9292c6f3d3d92f5bc6e9df8 parsl.dataflow.memoization INFO: Task 1175 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1175 try 0 launched on executor multi with executor id 1176 parsl.dataflow.dflow INFO: Standard output for task 1175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/44b348a3-ab23-463c-967a-485e737a6b6b_isrForCrosstalkSources_996964_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/44b348a3-ab23-463c-967a-485e737a6b6b_isrForCrosstalkSources_996964_42.stderr parsl.dataflow.dflow DEBUG: Task 1176 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1176 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1176 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1176 has memoization hash 5e8dcb502b07c7f45f50fc9f3e85d1da parsl.dataflow.memoization INFO: Task 1176 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1176 try 0 launched on executor multi with executor id 1177 parsl.dataflow.dflow INFO: Standard output for task 1176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/501ea7ee-9171-44ed-9c96-10189d3a75e5_isrForCrosstalkSources_997002_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/501ea7ee-9171-44ed-9c96-10189d3a75e5_isrForCrosstalkSources_997002_18.stderr parsl.dataflow.dflow DEBUG: Task 1177 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1177 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1177 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1177 has memoization hash 8ebd36a244d7c9a656475383ce719932 parsl.dataflow.memoization INFO: Task 1177 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1177 try 0 launched on executor multi with executor id 1178 parsl.dataflow.dflow INFO: Standard output for task 1177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0d2aebba-fb20-492d-a8fd-531fc9af53fa_isrForCrosstalkSources_996970_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0d2aebba-fb20-492d-a8fd-531fc9af53fa_isrForCrosstalkSources_996970_60.stderr parsl.dataflow.dflow DEBUG: Task 1178 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1178 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1178 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1178 has memoization hash 8ea2af3298cda08be1a5e4ab3a653275 parsl.dataflow.memoization INFO: Task 1178 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1178 try 0 launched on executor multi with executor id 1179 parsl.dataflow.dflow INFO: Standard output for task 1178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/f828a135-221c-4c54-921e-9af628af3a8b_isrForCrosstalkSources_996954_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/f828a135-221c-4c54-921e-9af628af3a8b_isrForCrosstalkSources_996954_46.stderr parsl.dataflow.dflow DEBUG: Task 1179 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1179 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1179 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1179 has memoization hash 1301e4b6c7546284b5a30f8357e11885 parsl.dataflow.memoization INFO: Task 1179 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1179 try 0 launched on executor multi with executor id 1180 parsl.dataflow.dflow INFO: Standard output for task 1179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7b755f60-0448-4c26-aefe-0a8fa6883c1a_isrForCrosstalkSources_996974_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7b755f60-0448-4c26-aefe-0a8fa6883c1a_isrForCrosstalkSources_996974_56.stderr parsl.dataflow.dflow DEBUG: Task 1180 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1180 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1180 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1180 has memoization hash a7cf1aaebff50501a5533f2dc23c389d parsl.dataflow.memoization INFO: Task 1180 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1180 try 0 launched on executor multi with executor id 1181 parsl.dataflow.dflow INFO: Standard output for task 1180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/061d4a9c-fbe4-432d-a179-102381ef8405_isrForCrosstalkSources_996955_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/061d4a9c-fbe4-432d-a179-102381ef8405_isrForCrosstalkSources_996955_28.stderr parsl.dataflow.dflow DEBUG: Task 1181 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1181 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1181 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1181 has memoization hash 42451e0d3c40a30b21504706e9cc7867 parsl.dataflow.memoization INFO: Task 1181 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1181 try 0 launched on executor multi with executor id 1182 parsl.dataflow.dflow INFO: Standard output for task 1181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/dc6e75b6-c543-470e-87f8-70a9b37964c9_isrForCrosstalkSources_996963_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/dc6e75b6-c543-470e-87f8-70a9b37964c9_isrForCrosstalkSources_996963_62.stderr parsl.dataflow.dflow DEBUG: Task 1182 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1182 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1182 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1182 has memoization hash 59ff57f35e04900a76c56b198d104791 parsl.dataflow.memoization INFO: Task 1182 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1182 try 0 launched on executor multi with executor id 1183 parsl.dataflow.dflow INFO: Standard output for task 1182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/29093b24-aac8-4290-9850-15a290c888e7_isrForCrosstalkSources_996979_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/29093b24-aac8-4290-9850-15a290c888e7_isrForCrosstalkSources_996979_47.stderr parsl.dataflow.dflow DEBUG: Task 1183 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1183 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1183 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1183 has memoization hash 54d07f18c5a9d36884b867d3f3841c50 parsl.dataflow.memoization INFO: Task 1183 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1183 try 0 launched on executor multi with executor id 1184 parsl.dataflow.dflow INFO: Standard output for task 1183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/48db3dfa-8f63-475c-94ac-72777aeae281_isrForCrosstalkSources_996998_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/48db3dfa-8f63-475c-94ac-72777aeae281_isrForCrosstalkSources_996998_59.stderr parsl.dataflow.dflow DEBUG: Task 1184 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1184 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1184 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1184 has memoization hash 4e0262539b8fe2382a001a23c1d8b2af parsl.dataflow.memoization INFO: Task 1184 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1184 try 0 launched on executor multi with executor id 1185 parsl.dataflow.dflow INFO: Standard output for task 1184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f43a009c-2b64-4d06-960f-7fe396fab50d_isrForCrosstalkSources_996971_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f43a009c-2b64-4d06-960f-7fe396fab50d_isrForCrosstalkSources_996971_38.stderr parsl.dataflow.dflow DEBUG: Task 1185 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1185 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1185 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1185 has memoization hash 1c92bdfafaf907e25f610378919b9bed parsl.dataflow.memoization INFO: Task 1185 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1185 try 0 launched on executor multi with executor id 1186 parsl.dataflow.dflow INFO: Standard output for task 1185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/77a72adf-b9a7-44be-a2c5-ee12523ff770_isrForCrosstalkSources_996954_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/77a72adf-b9a7-44be-a2c5-ee12523ff770_isrForCrosstalkSources_996954_3.stderr parsl.dataflow.dflow DEBUG: Task 1186 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1186 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1186 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1186 has memoization hash c4bf766cc0bc348e72be368bcbaa9b45 parsl.dataflow.memoization INFO: Task 1186 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1186 try 0 launched on executor multi with executor id 1187 parsl.dataflow.dflow INFO: Standard output for task 1186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/42e6f484-c6da-4b24-9809-5ab452f2c455_isrForCrosstalkSources_997029_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/42e6f484-c6da-4b24-9809-5ab452f2c455_isrForCrosstalkSources_997029_33.stderr parsl.dataflow.dflow DEBUG: Task 1187 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1187 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1187 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1187 has memoization hash 4fd43f2f617b8b180e2f8bd1db561796 parsl.dataflow.memoization INFO: Task 1187 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1187 try 0 launched on executor multi with executor id 1188 parsl.dataflow.dflow INFO: Standard output for task 1187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6332bfa1-0f00-4f5e-8bed-ec6aacc4227f_isrForCrosstalkSources_996983_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6332bfa1-0f00-4f5e-8bed-ec6aacc4227f_isrForCrosstalkSources_996983_38.stderr parsl.dataflow.dflow DEBUG: Task 1188 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1188 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1188 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1188 has memoization hash f6456678f96933bf649089ef84303f44 parsl.dataflow.memoization INFO: Task 1188 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1188 try 0 launched on executor multi with executor id 1189 parsl.dataflow.dflow INFO: Standard output for task 1188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/065afaa0-15e3-4c2d-8f7a-9bee862ebc23_isrForCrosstalkSources_996967_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/065afaa0-15e3-4c2d-8f7a-9bee862ebc23_isrForCrosstalkSources_996967_1.stderr parsl.dataflow.dflow DEBUG: Task 1189 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1189 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1189 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1189 has memoization hash 9e42ca05a76ade63b31c3a2eea8fa90e parsl.dataflow.memoization INFO: Task 1189 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1189 try 0 launched on executor multi with executor id 1190 parsl.dataflow.dflow INFO: Standard output for task 1189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fafaf523-fc1b-466e-b836-f4a0d2a169e4_isrForCrosstalkSources_996955_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fafaf523-fc1b-466e-b836-f4a0d2a169e4_isrForCrosstalkSources_996955_21.stderr parsl.dataflow.dflow DEBUG: Task 1190 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1190 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1190 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1190 has memoization hash 91350a0b6ff653ea47fbe736b97bc243 parsl.dataflow.memoization INFO: Task 1190 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1190 try 0 launched on executor multi with executor id 1191 parsl.dataflow.dflow INFO: Standard output for task 1190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/887801d3-f83f-4caf-9b02-cf4b276a3807_isrForCrosstalkSources_997003_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/887801d3-f83f-4caf-9b02-cf4b276a3807_isrForCrosstalkSources_997003_13.stderr parsl.dataflow.dflow DEBUG: Task 1191 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1191 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1191 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1191 has memoization hash 3be1e36faf897267a8de437707e00eb4 parsl.dataflow.memoization INFO: Task 1191 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1191 try 0 launched on executor multi with executor id 1192 parsl.dataflow.dflow INFO: Standard output for task 1191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/97122055-3346-4709-95c4-0150403e6538_isrForCrosstalkSources_997010_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/97122055-3346-4709-95c4-0150403e6538_isrForCrosstalkSources_997010_11.stderr parsl.dataflow.dflow DEBUG: Task 1192 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1192 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1192 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1192 has memoization hash 9820298e7a5a858cdc038d95fae81940 parsl.dataflow.memoization INFO: Task 1192 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1192 try 0 launched on executor multi with executor id 1193 parsl.dataflow.dflow INFO: Standard output for task 1192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/11d83f7b-b722-4734-b4e4-a2712138e405_isrForCrosstalkSources_997013_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/11d83f7b-b722-4734-b4e4-a2712138e405_isrForCrosstalkSources_997013_53.stderr parsl.dataflow.dflow DEBUG: Task 1193 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1193 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1193 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1193 has memoization hash 4735563ea45b17957e63b225126aadf7 parsl.dataflow.memoization INFO: Task 1193 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1193 try 0 launched on executor multi with executor id 1194 parsl.dataflow.dflow INFO: Standard output for task 1193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/fd47f819-cf57-478b-a574-6835a5de6117_isrForCrosstalkSources_996971_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/fd47f819-cf57-478b-a574-6835a5de6117_isrForCrosstalkSources_996971_46.stderr parsl.dataflow.dflow DEBUG: Task 1194 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1194 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1194 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1194 has memoization hash 5804a610837afa3f96e4df2d1f7c9ad7 parsl.dataflow.memoization INFO: Task 1194 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1194 try 0 launched on executor multi with executor id 1195 parsl.dataflow.dflow INFO: Standard output for task 1194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ca0df4df-3c0b-4c5e-85e9-40164d27ba93_isrForCrosstalkSources_996982_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ca0df4df-3c0b-4c5e-85e9-40164d27ba93_isrForCrosstalkSources_996982_27.stderr parsl.dataflow.dflow DEBUG: Task 1195 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1195 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1195 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1195 has memoization hash ae33bda43c60ddda022ed4e812f0ea56 parsl.dataflow.memoization INFO: Task 1195 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1195 try 0 launched on executor multi with executor id 1196 parsl.dataflow.dflow INFO: Standard output for task 1195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/8e043630-0666-4bc6-89c3-1b9f92b1bf65_isrForCrosstalkSources_997004_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/8e043630-0666-4bc6-89c3-1b9f92b1bf65_isrForCrosstalkSources_997004_61.stderr parsl.dataflow.dflow DEBUG: Task 1196 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1196 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1196 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1196 has memoization hash ab889be10fab4fbe686ea7d2568c475e parsl.dataflow.memoization INFO: Task 1196 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1196 try 0 launched on executor multi with executor id 1197 parsl.dataflow.dflow INFO: Standard output for task 1196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/184027d2-008b-4f5b-a807-28cc8eaa22e3_isrForCrosstalkSources_996958_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/184027d2-008b-4f5b-a807-28cc8eaa22e3_isrForCrosstalkSources_996958_25.stderr parsl.dataflow.dflow DEBUG: Task 1197 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1197 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1197 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1197 has memoization hash aa3928a4192612002faab8dfdfca272d parsl.dataflow.memoization INFO: Task 1197 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1197 try 0 launched on executor multi with executor id 1198 parsl.dataflow.dflow INFO: Standard output for task 1197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/03bab785-8838-493b-bed4-b503221cd3ec_isrForCrosstalkSources_997011_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/03bab785-8838-493b-bed4-b503221cd3ec_isrForCrosstalkSources_997011_2.stderr parsl.dataflow.dflow DEBUG: Task 1198 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1198 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1198 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1198 has memoization hash 347a91337ee09da38ef0c643f5dc3689 parsl.dataflow.memoization INFO: Task 1198 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1198 try 0 launched on executor multi with executor id 1199 parsl.dataflow.dflow INFO: Standard output for task 1198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/30431b6a-f8fc-444f-bab9-e41c5df87b42_isrForCrosstalkSources_997033_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/30431b6a-f8fc-444f-bab9-e41c5df87b42_isrForCrosstalkSources_997033_62.stderr parsl.dataflow.dflow DEBUG: Task 1199 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1199 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1199 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1199 has memoization hash 00f55431d55822dea215d8c731eedaf3 parsl.dataflow.memoization INFO: Task 1199 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1199 try 0 launched on executor multi with executor id 1200 parsl.dataflow.dflow INFO: Standard output for task 1199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/bcb74919-14b8-4d8f-9f15-f3ed29823127_isrForCrosstalkSources_997020_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/bcb74919-14b8-4d8f-9f15-f3ed29823127_isrForCrosstalkSources_997020_51.stderr parsl.dataflow.dflow DEBUG: Task 1200 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1200 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1200 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1200 has memoization hash 805e484fe4f8596ccefa31ac95e904b8 parsl.dataflow.memoization INFO: Task 1200 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1200 try 0 launched on executor multi with executor id 1201 parsl.dataflow.dflow INFO: Standard output for task 1200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7314f359-5bd9-4d62-9cb3-97cebcefb2ab_isrForCrosstalkSources_997025_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7314f359-5bd9-4d62-9cb3-97cebcefb2ab_isrForCrosstalkSources_997025_16.stderr parsl.dataflow.dflow DEBUG: Task 1201 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1201 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1201 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1201 has memoization hash cfde1d9f70fbca26f6353fda72452cf2 parsl.dataflow.memoization INFO: Task 1201 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1201 try 0 launched on executor multi with executor id 1202 parsl.dataflow.dflow INFO: Standard output for task 1201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/23d25687-64f6-4101-a9e0-6b8238c715cd_isrForCrosstalkSources_996983_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/23d25687-64f6-4101-a9e0-6b8238c715cd_isrForCrosstalkSources_996983_45.stderr parsl.dataflow.dflow DEBUG: Task 1202 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1202 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1202 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1202 has memoization hash d14f7756d148539a04dfb8f993a20db2 parsl.dataflow.memoization INFO: Task 1202 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1202 try 0 launched on executor multi with executor id 1203 parsl.dataflow.dflow INFO: Standard output for task 1202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/0eb02d11-02ff-40ef-8f49-3f17d9f7966a_isrForCrosstalkSources_996967_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/0eb02d11-02ff-40ef-8f49-3f17d9f7966a_isrForCrosstalkSources_996967_33.stderr parsl.dataflow.dflow DEBUG: Task 1203 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1203 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1203 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1203 has memoization hash a658f6bd77b52e788b0dd775978b5151 parsl.dataflow.memoization INFO: Task 1203 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1203 try 0 launched on executor multi with executor id 1204 parsl.dataflow.dflow INFO: Standard output for task 1203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c0ada6bf-91f5-4071-afdb-52f307ca9a2e_isrForCrosstalkSources_996973_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c0ada6bf-91f5-4071-afdb-52f307ca9a2e_isrForCrosstalkSources_996973_47.stderr parsl.dataflow.dflow DEBUG: Task 1204 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1204 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1204 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1204 has memoization hash 5563f1a68fe2ba71b49a5135837a3f7d parsl.dataflow.memoization INFO: Task 1204 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1204 try 0 launched on executor multi with executor id 1205 parsl.dataflow.dflow INFO: Standard output for task 1204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/34636636-b1e3-4de7-b813-ef2550382ea3_isrForCrosstalkSources_997003_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/34636636-b1e3-4de7-b813-ef2550382ea3_isrForCrosstalkSources_997003_1.stderr parsl.dataflow.dflow DEBUG: Task 1205 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1205 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1205 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1205 has memoization hash 820cb2ecb5208636577a882ee546bbd2 parsl.dataflow.memoization INFO: Task 1205 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1205 try 0 launched on executor multi with executor id 1206 parsl.dataflow.dflow INFO: Standard output for task 1205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/0eb4ea75-99ce-433d-ab38-879a4c7b83fd_isrForCrosstalkSources_997031_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/0eb4ea75-99ce-433d-ab38-879a4c7b83fd_isrForCrosstalkSources_997031_56.stderr parsl.dataflow.dflow DEBUG: Task 1206 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1206 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1206 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1206 has memoization hash ac478fd24779e551e09f0cc6690a182e parsl.dataflow.memoization INFO: Task 1206 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1206 try 0 launched on executor multi with executor id 1207 parsl.dataflow.dflow INFO: Standard output for task 1206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a3ca6510-a34a-4a02-8f39-ec82d6654db5_isrForCrosstalkSources_996961_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a3ca6510-a34a-4a02-8f39-ec82d6654db5_isrForCrosstalkSources_996961_17.stderr parsl.dataflow.dflow DEBUG: Task 1207 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1207 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1207 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1207 has memoization hash 686968aa0f6742d93eeb66402af6e222 parsl.dataflow.memoization INFO: Task 1207 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1207 try 0 launched on executor multi with executor id 1208 parsl.dataflow.dflow INFO: Standard output for task 1207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/526e8cc8-50ed-48e9-bfb9-e467d7cfd690_isrForCrosstalkSources_997033_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/526e8cc8-50ed-48e9-bfb9-e467d7cfd690_isrForCrosstalkSources_997033_57.stderr parsl.dataflow.dflow DEBUG: Task 1208 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1208 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1208 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1208 has memoization hash 802451328e01a3d65ca1d1a27a858882 parsl.dataflow.memoization INFO: Task 1208 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1208 try 0 launched on executor multi with executor id 1209 parsl.dataflow.dflow INFO: Standard output for task 1208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/ead171b2-cef5-4568-9f6e-3a84fc52f274_isrForCrosstalkSources_997018_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/ead171b2-cef5-4568-9f6e-3a84fc52f274_isrForCrosstalkSources_997018_34.stderr parsl.dataflow.dflow DEBUG: Task 1209 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1209 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1209 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1209 has memoization hash e9ccc716059e7f1558a2f94cdc3a9167 parsl.dataflow.memoization INFO: Task 1209 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1209 try 0 launched on executor multi with executor id 1210 parsl.dataflow.dflow INFO: Standard output for task 1209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3a45069f-0b9e-4a60-9414-87d2ee54caf1_isrForCrosstalkSources_996999_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3a45069f-0b9e-4a60-9414-87d2ee54caf1_isrForCrosstalkSources_996999_55.stderr parsl.dataflow.dflow DEBUG: Task 1210 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1210 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1210 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1210 has memoization hash 578ed6751e07724172b067cbc1b47996 parsl.dataflow.memoization INFO: Task 1210 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1210 try 0 launched on executor multi with executor id 1211 parsl.dataflow.dflow INFO: Standard output for task 1210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d8e71428-b76d-4604-863a-5e96068eedb5_isrForCrosstalkSources_996995_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d8e71428-b76d-4604-863a-5e96068eedb5_isrForCrosstalkSources_996995_1.stderr parsl.dataflow.dflow DEBUG: Task 1211 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1211 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1211 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1211 has memoization hash 348b76d4717039d910006c578c02a911 parsl.dataflow.memoization INFO: Task 1211 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1211 try 0 launched on executor multi with executor id 1212 parsl.dataflow.dflow INFO: Standard output for task 1211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/aca5642c-6c2a-4791-8dd4-0f737f8c6162_isrForCrosstalkSources_997000_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/aca5642c-6c2a-4791-8dd4-0f737f8c6162_isrForCrosstalkSources_997000_16.stderr parsl.dataflow.dflow DEBUG: Task 1212 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1212 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1212 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1212 has memoization hash ac87167752830cffc03db1bd8e35a93d parsl.dataflow.memoization INFO: Task 1212 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1212 try 0 launched on executor multi with executor id 1213 parsl.dataflow.dflow INFO: Standard output for task 1212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/39894ad4-2e88-4e09-a808-6f0eba5f2dc2_isrForCrosstalkSources_996966_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/39894ad4-2e88-4e09-a808-6f0eba5f2dc2_isrForCrosstalkSources_996966_21.stderr parsl.dataflow.dflow DEBUG: Task 1213 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1213 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1213 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1213 has memoization hash f67dcf0f6d805576e16bf78ca0c23d29 parsl.dataflow.memoization INFO: Task 1213 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1213 try 0 launched on executor multi with executor id 1214 parsl.dataflow.dflow INFO: Standard output for task 1213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60f7d0e9-7c12-4892-8db6-e840ace9d97b_isrForCrosstalkSources_996968_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60f7d0e9-7c12-4892-8db6-e840ace9d97b_isrForCrosstalkSources_996968_42.stderr parsl.dataflow.dflow DEBUG: Task 1214 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1214 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1214 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1214 has memoization hash 738b6df3fd1238e0f0c597119e386589 parsl.dataflow.memoization INFO: Task 1214 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f67e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1214 try 0 launched on executor multi with executor id 1215 parsl.dataflow.dflow INFO: Standard output for task 1214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c1733764-93ee-4793-899d-f1bd0b68e160_isrForCrosstalkSources_997020_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c1733764-93ee-4793-899d-f1bd0b68e160_isrForCrosstalkSources_997020_62.stderr parsl.dataflow.dflow DEBUG: Task 1215 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1215 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1215 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1215 has memoization hash 5f46da0aad1059b87e0526330666954e parsl.dataflow.memoization INFO: Task 1215 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1215 try 0 launched on executor multi with executor id 1216 parsl.dataflow.dflow INFO: Standard output for task 1215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fe080c47-3a4d-469b-b9b8-12c6f5d76912_isrForCrosstalkSources_996961_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fe080c47-3a4d-469b-b9b8-12c6f5d76912_isrForCrosstalkSources_996961_48.stderr parsl.dataflow.dflow DEBUG: Task 1216 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1216 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1216 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1216 has memoization hash cb288db349c3ebe8ba7008cac42853e3 parsl.dataflow.memoization INFO: Task 1216 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1216 try 0 launched on executor multi with executor id 1217 parsl.dataflow.dflow INFO: Standard output for task 1216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/d99f32b3-4edf-4558-948b-a1517925f342_isrForCrosstalkSources_996953_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/d99f32b3-4edf-4558-948b-a1517925f342_isrForCrosstalkSources_996953_10.stderr parsl.dataflow.dflow DEBUG: Task 1217 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1217 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1217 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1217 has memoization hash b5d37516740c4f621ccfad93358832d8 parsl.dataflow.memoization INFO: Task 1217 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1217 try 0 launched on executor multi with executor id 1218 parsl.dataflow.dflow INFO: Standard output for task 1217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/e6ea7906-0e50-44e9-b4d4-17a60bfb76cf_isrForCrosstalkSources_997009_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/e6ea7906-0e50-44e9-b4d4-17a60bfb76cf_isrForCrosstalkSources_997009_20.stderr parsl.dataflow.dflow DEBUG: Task 1218 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1218 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1218 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1218 has memoization hash 220cdc5834e335c0c8a8ef7a9eb85895 parsl.dataflow.memoization INFO: Task 1218 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1218 try 0 launched on executor multi with executor id 1219 parsl.dataflow.dflow INFO: Standard output for task 1218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d31ccfcc-731d-47d2-8838-d9f24c695f8d_isrForCrosstalkSources_996991_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d31ccfcc-731d-47d2-8838-d9f24c695f8d_isrForCrosstalkSources_996991_40.stderr parsl.dataflow.dflow DEBUG: Task 1219 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1219 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1219 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1219 has memoization hash 981d3563ae56585f9cb5b02b0be613bd parsl.dataflow.memoization INFO: Task 1219 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1219 try 0 launched on executor multi with executor id 1220 parsl.dataflow.dflow INFO: Standard output for task 1219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/cda0079f-9047-4a2b-bdd2-a9aa621d4cc6_isrForCrosstalkSources_997007_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/cda0079f-9047-4a2b-bdd2-a9aa621d4cc6_isrForCrosstalkSources_997007_12.stderr parsl.dataflow.dflow DEBUG: Task 1220 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1220 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1220 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1220 has memoization hash 39ac8370274e50e5a57973b8859f1978 parsl.dataflow.memoization INFO: Task 1220 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1220 try 0 launched on executor multi with executor id 1221 parsl.dataflow.dflow INFO: Standard output for task 1220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/2a995f59-aecb-43e8-a77e-e18f4935ff0e_isrForCrosstalkSources_996985_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/2a995f59-aecb-43e8-a77e-e18f4935ff0e_isrForCrosstalkSources_996985_58.stderr parsl.dataflow.dflow DEBUG: Task 1221 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1221 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1221 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1221 has memoization hash 8ae0bc4655941ffd6200c026976b4aa6 parsl.dataflow.memoization INFO: Task 1221 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1221 try 0 launched on executor multi with executor id 1222 parsl.dataflow.dflow INFO: Standard output for task 1221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/421f31cb-9c21-4531-8873-d4692508f4c6_isrForCrosstalkSources_996961_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/421f31cb-9c21-4531-8873-d4692508f4c6_isrForCrosstalkSources_996961_13.stderr parsl.dataflow.dflow DEBUG: Task 1222 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1222 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1222 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1222 has memoization hash 21e30f5661fa628dc023b1cec68eac29 parsl.dataflow.memoization INFO: Task 1222 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1222 try 0 launched on executor multi with executor id 1223 parsl.dataflow.dflow INFO: Standard output for task 1222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/d15d1658-95e7-4b35-ae58-3aca3d3dc485_isrForCrosstalkSources_996957_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/d15d1658-95e7-4b35-ae58-3aca3d3dc485_isrForCrosstalkSources_996957_14.stderr parsl.dataflow.dflow DEBUG: Task 1223 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1223 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1223 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1223 has memoization hash 2575469d36389b520d57a5fec4a83c1e parsl.dataflow.memoization INFO: Task 1223 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1223 try 0 launched on executor multi with executor id 1224 parsl.dataflow.dflow INFO: Standard output for task 1223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6595d0fb-e243-4208-84d1-6b2958fb0e1b_isrForCrosstalkSources_996982_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6595d0fb-e243-4208-84d1-6b2958fb0e1b_isrForCrosstalkSources_996982_51.stderr parsl.dataflow.dflow DEBUG: Task 1224 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1224 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1224 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1224 has memoization hash a45f199ae56e7a30122faa1afba898eb parsl.dataflow.memoization INFO: Task 1224 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1224 try 0 launched on executor multi with executor id 1225 parsl.dataflow.dflow INFO: Standard output for task 1224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/73fd87e0-0866-489e-9f0c-f41444ecd967_isrForCrosstalkSources_997009_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/73fd87e0-0866-489e-9f0c-f41444ecd967_isrForCrosstalkSources_997009_41.stderr parsl.dataflow.dflow DEBUG: Task 1225 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1225 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1225 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1225 has memoization hash 0e481c44d9778b2b79314e5196f44177 parsl.dataflow.memoization INFO: Task 1225 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1225 try 0 launched on executor multi with executor id 1226 parsl.dataflow.dflow INFO: Standard output for task 1225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/9d317245-990f-4eab-bda7-75a8bb0828f9_isrForCrosstalkSources_996996_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/9d317245-990f-4eab-bda7-75a8bb0828f9_isrForCrosstalkSources_996996_36.stderr parsl.dataflow.dflow DEBUG: Task 1226 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1226 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1226 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1226 has memoization hash 513b6682d86cc6e86fa80a6dad3b7d7c parsl.dataflow.memoization INFO: Task 1226 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1226 try 0 launched on executor multi with executor id 1227 parsl.dataflow.dflow INFO: Standard output for task 1226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/d7ee40c8-5419-4a15-80ab-0f866e8e4da5_isrForCrosstalkSources_997019_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/d7ee40c8-5419-4a15-80ab-0f866e8e4da5_isrForCrosstalkSources_997019_30.stderr parsl.dataflow.dflow DEBUG: Task 1227 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1227 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1227 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1227 has memoization hash d304641628670634683b847a4d3dbb67 parsl.dataflow.memoization INFO: Task 1227 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1227 try 0 launched on executor multi with executor id 1228 parsl.dataflow.dflow INFO: Standard output for task 1227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9056e112-aee5-451d-8342-90fdf5fcf584_isrForCrosstalkSources_996997_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9056e112-aee5-451d-8342-90fdf5fcf584_isrForCrosstalkSources_996997_17.stderr parsl.dataflow.dflow DEBUG: Task 1228 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1228 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1228 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1228 has memoization hash db1dfb8831c8199c782e971da3431e42 parsl.dataflow.memoization INFO: Task 1228 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1228 try 0 launched on executor multi with executor id 1229 parsl.dataflow.dflow INFO: Standard output for task 1228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/9c2b0afc-03f9-43b7-bfee-a8d98e3c80b1_isrForCrosstalkSources_996994_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/9c2b0afc-03f9-43b7-bfee-a8d98e3c80b1_isrForCrosstalkSources_996994_4.stderr parsl.dataflow.dflow DEBUG: Task 1229 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1229 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1229 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1229 has memoization hash 0f8f534f961bc04a78cb4daedb45a2ee parsl.dataflow.memoization INFO: Task 1229 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1229 try 0 launched on executor multi with executor id 1230 parsl.dataflow.dflow INFO: Standard output for task 1229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f6342740-da7d-42f9-8dd5-dd0436aa8692_isrForCrosstalkSources_996953_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f6342740-da7d-42f9-8dd5-dd0436aa8692_isrForCrosstalkSources_996953_13.stderr parsl.dataflow.dflow DEBUG: Task 1230 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1230 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1230 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1230 has memoization hash 7096ce7c3f10a969bc7e7ba73e3eb69c parsl.dataflow.memoization INFO: Task 1230 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1230 try 0 launched on executor multi with executor id 1231 parsl.dataflow.dflow INFO: Standard output for task 1230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/c90fd4c8-50b0-44ab-9ed0-0e7ef528f916_isrForCrosstalkSources_996963_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/c90fd4c8-50b0-44ab-9ed0-0e7ef528f916_isrForCrosstalkSources_996963_11.stderr parsl.dataflow.dflow DEBUG: Task 1231 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1231 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1231 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1231 has memoization hash a1dca868625bc8a490086149556b974f parsl.dataflow.memoization INFO: Task 1231 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1231 try 0 launched on executor multi with executor id 1232 parsl.dataflow.dflow INFO: Standard output for task 1231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/6a03c5af-d51e-448d-bfc2-a21331087a1f_isrForCrosstalkSources_996967_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/6a03c5af-d51e-448d-bfc2-a21331087a1f_isrForCrosstalkSources_996967_43.stderr parsl.dataflow.dflow DEBUG: Task 1232 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1232 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1232 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1232 has memoization hash e238b7d729220a128dd1813ebfd7f9a7 parsl.dataflow.memoization INFO: Task 1232 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1232 try 0 launched on executor multi with executor id 1233 parsl.dataflow.dflow INFO: Standard output for task 1232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/58527194-663b-4808-adef-d269525b5e53_isrForCrosstalkSources_996994_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/58527194-663b-4808-adef-d269525b5e53_isrForCrosstalkSources_996994_15.stderr parsl.dataflow.dflow DEBUG: Task 1233 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1233 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1233 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1233 has memoization hash 6fe6b05f48582155aa7e6ede5a35620a parsl.dataflow.memoization INFO: Task 1233 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:36 proc_lsst.multi:170 [INFO] [multi] job_id 17074854 proc_lsst.multi INFO: [multi] job_id 17074854 2024-03-15 13:49:36 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 15 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 15 2024-03-15 13:49:36 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:36 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074854 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074854 parsl.executors.status_handling DEBUG: Launched block 15 on executor multi with job ID 17074854 parsl.executors.status_handling INFO: Allocated block ID 16 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-16 2024-03-15 13:49:36 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=16 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-16 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=16 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-16 2024-03-15 13:49:36 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=16 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=16 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:36 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=16 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=16 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 15 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 15 2024-03-15 13:49:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:36 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=16 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=16 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 1233 try 0 launched on executor multi with executor id 1234 parsl.dataflow.dflow INFO: Standard output for task 1233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c98af136-f874-44e5-86b8-eb79913df754_isrForCrosstalkSources_996969_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c98af136-f874-44e5-86b8-eb79913df754_isrForCrosstalkSources_996969_41.stderr parsl.dataflow.dflow DEBUG: Task 1234 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1234 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1234 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1234 has memoization hash 071c76547716a3632010097b1243882a parsl.dataflow.memoization INFO: Task 1234 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1234 try 0 launched on executor multi with executor id 1235 parsl.dataflow.dflow INFO: Standard output for task 1234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b79768bf-1e7b-4f9c-b4be-a582692aa1a2_isrForCrosstalkSources_997007_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b79768bf-1e7b-4f9c-b4be-a582692aa1a2_isrForCrosstalkSources_997007_53.stderr parsl.dataflow.dflow DEBUG: Task 1235 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1235 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1235 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1235 has memoization hash b6a05a3c99ea7ef12caf19883d2bc452 parsl.dataflow.memoization INFO: Task 1235 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1235 try 0 launched on executor multi with executor id 1236 parsl.dataflow.dflow INFO: Standard output for task 1235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c9f24141-ed61-4dcb-b9c5-52b5f357b025_isrForCrosstalkSources_996985_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c9f24141-ed61-4dcb-b9c5-52b5f357b025_isrForCrosstalkSources_996985_55.stderr parsl.dataflow.dflow DEBUG: Task 1236 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1236 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1236 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1236 has memoization hash 28874969eaf0442206a847922a6eb091 parsl.dataflow.memoization INFO: Task 1236 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780321580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1236 try 0 launched on executor multi with executor id 1237 parsl.dataflow.dflow INFO: Standard output for task 1236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6ef02598-c8ba-46e5-88ad-93c51a5fe772_isrForCrosstalkSources_997017_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6ef02598-c8ba-46e5-88ad-93c51a5fe772_isrForCrosstalkSources_997017_57.stderr parsl.dataflow.dflow DEBUG: Task 1237 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1237 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1237 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1237 has memoization hash c4852461472deee8d7872cfe875feff4 parsl.dataflow.memoization INFO: Task 1237 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1237 try 0 launched on executor multi with executor id 1238 parsl.dataflow.dflow INFO: Standard output for task 1237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4714a952-867b-49ec-b322-924de3a3cf4f_isrForCrosstalkSources_996962_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4714a952-867b-49ec-b322-924de3a3cf4f_isrForCrosstalkSources_996962_31.stderr parsl.dataflow.dflow DEBUG: Task 1238 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1238 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1238 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1238 has memoization hash 980f821c69b9480600ad634ae2fdaf07 parsl.dataflow.memoization INFO: Task 1238 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1238 try 0 launched on executor multi with executor id 1239 parsl.dataflow.dflow INFO: Standard output for task 1238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9e4b4475-fdb1-4d9f-9993-30353bb7b1a9_isrForCrosstalkSources_996986_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9e4b4475-fdb1-4d9f-9993-30353bb7b1a9_isrForCrosstalkSources_996986_49.stderr parsl.dataflow.dflow DEBUG: Task 1239 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1239 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1239 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1239 has memoization hash a227c7a10afc534cf5d20be585a5445f parsl.dataflow.memoization INFO: Task 1239 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1239 try 0 launched on executor multi with executor id 1240 parsl.dataflow.dflow INFO: Standard output for task 1239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/515fd2b9-83cc-4fe0-a7b3-f03ba6b09fbb_isrForCrosstalkSources_997011_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/515fd2b9-83cc-4fe0-a7b3-f03ba6b09fbb_isrForCrosstalkSources_997011_37.stderr parsl.dataflow.dflow DEBUG: Task 1240 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1240 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1240 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1240 has memoization hash 7d20f200078e227fbd94adfd9aba67e4 parsl.dataflow.memoization INFO: Task 1240 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1240 try 0 launched on executor multi with executor id 1241 parsl.dataflow.dflow INFO: Standard output for task 1240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/3887aac0-b62f-400b-a0bf-0c25a2a81e78_isrForCrosstalkSources_997000_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/3887aac0-b62f-400b-a0bf-0c25a2a81e78_isrForCrosstalkSources_997000_19.stderr parsl.dataflow.dflow DEBUG: Task 1241 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1241 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1241 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1241 has memoization hash ceccb1974555595857e144b69962f2e8 parsl.dataflow.memoization INFO: Task 1241 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1241 try 0 launched on executor multi with executor id 1242 parsl.dataflow.dflow INFO: Standard output for task 1241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d502e38d-5503-45bd-a35a-0bb35d6af0cd_isrForCrosstalkSources_996995_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d502e38d-5503-45bd-a35a-0bb35d6af0cd_isrForCrosstalkSources_996995_29.stderr parsl.dataflow.dflow DEBUG: Task 1242 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1242 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1242 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1242 has memoization hash 85fb539be53940b15283474b5dd93e9c parsl.dataflow.memoization INFO: Task 1242 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1242 try 0 launched on executor multi with executor id 1243 parsl.dataflow.dflow INFO: Standard output for task 1242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/51384153-cd47-4990-802b-03982355d26d_isrForCrosstalkSources_996983_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/51384153-cd47-4990-802b-03982355d26d_isrForCrosstalkSources_996983_32.stderr parsl.dataflow.dflow DEBUG: Task 1243 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1243 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1243 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1243 has memoization hash b0fee61f5fcfce96841c6d2c3b0bdbf7 parsl.dataflow.memoization INFO: Task 1243 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1243 try 0 launched on executor multi with executor id 1244 parsl.dataflow.dflow INFO: Standard output for task 1243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/82c26b7c-be60-4471-b912-cd52c1917b5f_isrForCrosstalkSources_996979_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/82c26b7c-be60-4471-b912-cd52c1917b5f_isrForCrosstalkSources_996979_11.stderr parsl.dataflow.dflow DEBUG: Task 1244 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1244 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1244 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1244 has memoization hash 74e3a31a6484e5946c7c11e78554f4aa parsl.dataflow.memoization INFO: Task 1244 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1244 try 0 launched on executor multi with executor id 1245 parsl.dataflow.dflow INFO: Standard output for task 1244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1d294798-9131-4650-9cbb-d2eef4f16c83_isrForCrosstalkSources_996964_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1d294798-9131-4650-9cbb-d2eef4f16c83_isrForCrosstalkSources_996964_5.stderr parsl.dataflow.dflow DEBUG: Task 1245 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1245 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1245 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1245 has memoization hash f336b059ad1025a7d8770a2f5c01b415 parsl.dataflow.memoization INFO: Task 1245 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1245 try 0 launched on executor multi with executor id 1246 parsl.dataflow.dflow INFO: Standard output for task 1245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/575b4c42-a842-4953-b74d-213ba3155460_isrForCrosstalkSources_997026_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/575b4c42-a842-4953-b74d-213ba3155460_isrForCrosstalkSources_997026_24.stderr parsl.dataflow.dflow DEBUG: Task 1246 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1246 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1246 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1246 has memoization hash 037a20aba751f1ff7bebce0f7663ce38 parsl.dataflow.memoization INFO: Task 1246 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1246 try 0 launched on executor multi with executor id 1247 parsl.dataflow.dflow INFO: Standard output for task 1246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8574ae50-2678-41e9-b97e-9d90c0f28fac_isrForCrosstalkSources_996971_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8574ae50-2678-41e9-b97e-9d90c0f28fac_isrForCrosstalkSources_996971_28.stderr parsl.dataflow.dflow DEBUG: Task 1247 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1247 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1247 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1247 has memoization hash 516f5f35002cc583051ac26804880ab1 parsl.dataflow.memoization INFO: Task 1247 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1247 try 0 launched on executor multi with executor id 1248 parsl.dataflow.dflow INFO: Standard output for task 1247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/e87b722c-8a66-44ad-a5a1-29f20c9af976_isrForCrosstalkSources_996972_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/e87b722c-8a66-44ad-a5a1-29f20c9af976_isrForCrosstalkSources_996972_1.stderr parsl.dataflow.dflow DEBUG: Task 1248 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1248 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1248 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1248 has memoization hash ec01ede764e9f33ff1de6a8f8029bc2c parsl.dataflow.memoization INFO: Task 1248 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1248 try 0 launched on executor multi with executor id 1249 parsl.dataflow.dflow INFO: Standard output for task 1248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/98a8b7c6-9ded-4f84-bdba-4482988922d4_isrForCrosstalkSources_996952_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/98a8b7c6-9ded-4f84-bdba-4482988922d4_isrForCrosstalkSources_996952_49.stderr parsl.dataflow.dflow DEBUG: Task 1249 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1249 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1249 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1249 has memoization hash e7d6222174a7435a2515a8f1b73a2817 parsl.dataflow.memoization INFO: Task 1249 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1249 try 0 launched on executor multi with executor id 1250 parsl.dataflow.dflow INFO: Standard output for task 1249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/506f3c55-a429-42f8-bc81-3115e242075a_isrForCrosstalkSources_997017_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/506f3c55-a429-42f8-bc81-3115e242075a_isrForCrosstalkSources_997017_40.stderr parsl.dataflow.dflow DEBUG: Task 1250 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1250 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1250 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1250 has memoization hash e895ca3e28aef9ae312f9f465c0a4dcd parsl.dataflow.memoization INFO: Task 1250 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1250 try 0 launched on executor multi with executor id 1251 parsl.dataflow.dflow INFO: Standard output for task 1250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/85150cea-c614-4bcd-8b25-a3e7719340ab_isrForCrosstalkSources_997017_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/85150cea-c614-4bcd-8b25-a3e7719340ab_isrForCrosstalkSources_997017_14.stderr parsl.dataflow.dflow DEBUG: Task 1251 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1251 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1251 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1251 has memoization hash 922002c5aa0ec1357f31a068bf726ca9 parsl.dataflow.memoization INFO: Task 1251 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1251 try 0 launched on executor multi with executor id 1252 parsl.dataflow.dflow INFO: Standard output for task 1251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6185a494-953e-4992-9eb7-3488ebe0d28c_isrForCrosstalkSources_997010_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6185a494-953e-4992-9eb7-3488ebe0d28c_isrForCrosstalkSources_997010_51.stderr parsl.dataflow.dflow DEBUG: Task 1252 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1252 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1252 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1252 has memoization hash a6237539a0ce8b4de230d0106578409c parsl.dataflow.memoization INFO: Task 1252 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1252 try 0 launched on executor multi with executor id 1253 parsl.dataflow.dflow INFO: Standard output for task 1252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/df909d65-8ed8-431e-968b-e9c7fb520613_isrForCrosstalkSources_996962_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/df909d65-8ed8-431e-968b-e9c7fb520613_isrForCrosstalkSources_996962_12.stderr parsl.dataflow.dflow DEBUG: Task 1253 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1253 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1253 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1253 has memoization hash 62ebbbfd596d1751756baa02fa37f4e2 parsl.dataflow.memoization INFO: Task 1253 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1253 try 0 launched on executor multi with executor id 1254 parsl.dataflow.dflow INFO: Standard output for task 1253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/931f9440-cfdc-41d2-9ebb-188be10d9093_isrForCrosstalkSources_996965_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/931f9440-cfdc-41d2-9ebb-188be10d9093_isrForCrosstalkSources_996965_30.stderr parsl.dataflow.dflow DEBUG: Task 1254 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1254 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1254 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1254 has memoization hash 5cd12e8411f7586efb805aa055a0ca65 parsl.dataflow.memoization INFO: Task 1254 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1254 try 0 launched on executor multi with executor id 1255 parsl.dataflow.dflow INFO: Standard output for task 1254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/2d04903f-10eb-4163-93e4-238dc05c9428_isrForCrosstalkSources_996997_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/2d04903f-10eb-4163-93e4-238dc05c9428_isrForCrosstalkSources_996997_23.stderr parsl.dataflow.dflow DEBUG: Task 1255 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1255 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1255 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1255 has memoization hash 572fe4260e79bae26d202270003ff004 parsl.dataflow.memoization INFO: Task 1255 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1255 try 0 launched on executor multi with executor id 1256 parsl.dataflow.dflow INFO: Standard output for task 1255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e152647a-a845-4eda-9b09-164037dd2846_isrForCrosstalkSources_996985_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e152647a-a845-4eda-9b09-164037dd2846_isrForCrosstalkSources_996985_2.stderr parsl.dataflow.dflow DEBUG: Task 1256 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1256 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1256 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1256 has memoization hash 51e0d5c60a9677acb8e45b62f5f3b655 parsl.dataflow.memoization INFO: Task 1256 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1256 try 0 launched on executor multi with executor id 1257 parsl.dataflow.dflow INFO: Standard output for task 1256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/93508c79-65d6-41cb-b9d7-3711956710d1_isrForCrosstalkSources_996984_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/93508c79-65d6-41cb-b9d7-3711956710d1_isrForCrosstalkSources_996984_20.stderr parsl.dataflow.dflow DEBUG: Task 1257 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1257 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1257 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1257 has memoization hash 90a7f2277d3a849c8d043eb75dc4feec parsl.dataflow.memoization INFO: Task 1257 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1257 try 0 launched on executor multi with executor id 1258 parsl.dataflow.dflow INFO: Standard output for task 1257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/89917c3f-f4e7-41b7-976b-fd87fb551174_isrForCrosstalkSources_996996_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/89917c3f-f4e7-41b7-976b-fd87fb551174_isrForCrosstalkSources_996996_22.stderr parsl.dataflow.dflow DEBUG: Task 1258 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1258 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1258 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1258 has memoization hash 40b697ae8ebd4275de470bb595304aa8 parsl.dataflow.memoization INFO: Task 1258 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1258 try 0 launched on executor multi with executor id 1259 parsl.dataflow.dflow INFO: Standard output for task 1258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0bebb7c0-077b-49fa-b7be-40283d8b4f81_isrForCrosstalkSources_996991_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0bebb7c0-077b-49fa-b7be-40283d8b4f81_isrForCrosstalkSources_996991_16.stderr parsl.dataflow.dflow DEBUG: Task 1259 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1259 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1259 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1259 has memoization hash bb41c42283789126a88a5ffd6cf52197 parsl.dataflow.memoization INFO: Task 1259 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1259 try 0 launched on executor multi with executor id 1260 parsl.dataflow.dflow INFO: Standard output for task 1259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e7785ecd-3395-4285-877f-1070d9818af5_isrForCrosstalkSources_997002_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e7785ecd-3395-4285-877f-1070d9818af5_isrForCrosstalkSources_997002_17.stderr parsl.dataflow.dflow DEBUG: Task 1260 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1260 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1260 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1260 has memoization hash b8e7f34b82741e6f28b26a2719a1ff8b parsl.dataflow.memoization INFO: Task 1260 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1260 try 0 launched on executor multi with executor id 1261 parsl.dataflow.dflow INFO: Standard output for task 1260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c84f3837-7d7b-4f5b-8751-774a697b2454_isrForCrosstalkSources_996966_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c84f3837-7d7b-4f5b-8751-774a697b2454_isrForCrosstalkSources_996966_24.stderr parsl.dataflow.dflow DEBUG: Task 1261 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1261 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1261 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1261 has memoization hash 48eb2b629e1a157bdc5e3e00dfa56b17 parsl.dataflow.memoization INFO: Task 1261 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1261 try 0 launched on executor multi with executor id 1262 parsl.dataflow.dflow INFO: Standard output for task 1261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/bf4198ce-99fe-411f-bb20-26d848799635_isrForCrosstalkSources_996961_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/bf4198ce-99fe-411f-bb20-26d848799635_isrForCrosstalkSources_996961_16.stderr parsl.dataflow.dflow DEBUG: Task 1262 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1262 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1262 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1262 has memoization hash 5f91c1584c65778a2d1d6c61907e2865 parsl.dataflow.memoization INFO: Task 1262 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1262 try 0 launched on executor multi with executor id 1263 parsl.dataflow.dflow INFO: Standard output for task 1262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/22fa8b9f-a47c-47bb-aaf2-281abd50617e_isrForCrosstalkSources_996991_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/22fa8b9f-a47c-47bb-aaf2-281abd50617e_isrForCrosstalkSources_996991_3.stderr parsl.dataflow.dflow DEBUG: Task 1263 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1263 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1263 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1263 has memoization hash 10d5c075d7ef178dca61dc98c7eb5851 parsl.dataflow.memoization INFO: Task 1263 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1263 try 0 launched on executor multi with executor id 1264 parsl.dataflow.dflow INFO: Standard output for task 1263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/359315b4-cf2b-4bc5-926f-d3bde6c4413b_isrForCrosstalkSources_997010_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/359315b4-cf2b-4bc5-926f-d3bde6c4413b_isrForCrosstalkSources_997010_61.stderr parsl.dataflow.dflow DEBUG: Task 1264 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1264 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1264 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1264 has memoization hash 046bb81ae58217e602f6b1b2f62b871a parsl.dataflow.memoization INFO: Task 1264 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1264 try 0 launched on executor multi with executor id 1265 parsl.dataflow.dflow INFO: Standard output for task 1264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/217b0639-cadf-4d20-bed7-73adc057b671_isrForCrosstalkSources_996977_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/217b0639-cadf-4d20-bed7-73adc057b671_isrForCrosstalkSources_996977_4.stderr parsl.dataflow.dflow DEBUG: Task 1265 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1265 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1265 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1265 has memoization hash 109c49211d76cb970c9b807a18d33b18 parsl.dataflow.memoization INFO: Task 1265 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1265 try 0 launched on executor multi with executor id 1266 parsl.dataflow.dflow INFO: Standard output for task 1265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4fce2dce-b4ec-4d58-967a-11306098b4ba_isrForCrosstalkSources_997015_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4fce2dce-b4ec-4d58-967a-11306098b4ba_isrForCrosstalkSources_997015_1.stderr parsl.dataflow.dflow DEBUG: Task 1266 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1266 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1266 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1266 has memoization hash 1534eb0384ffd0e754a657760cbbf62c parsl.dataflow.memoization INFO: Task 1266 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1266 try 0 launched on executor multi with executor id 1267 parsl.dataflow.dflow INFO: Standard output for task 1266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/830dd92a-dedb-4fea-9d5c-7408583c201d_isrForCrosstalkSources_996965_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/830dd92a-dedb-4fea-9d5c-7408583c201d_isrForCrosstalkSources_996965_50.stderr parsl.dataflow.dflow DEBUG: Task 1267 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1267 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1267 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1267 has memoization hash 9ccbf8317418675fd66b12ddea512eaf parsl.dataflow.memoization INFO: Task 1267 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1267 try 0 launched on executor multi with executor id 1268 parsl.dataflow.dflow INFO: Standard output for task 1267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/59fa65e4-eb51-4ff8-b8b8-d741c6461fc1_isrForCrosstalkSources_996969_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/59fa65e4-eb51-4ff8-b8b8-d741c6461fc1_isrForCrosstalkSources_996969_42.stderr parsl.dataflow.dflow DEBUG: Task 1268 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1268 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1268 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1268 has memoization hash 0816196881daa4c77d9b284c57f77943 parsl.dataflow.memoization INFO: Task 1268 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1268 try 0 launched on executor multi with executor id 1269 parsl.dataflow.dflow INFO: Standard output for task 1268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/97930b67-466f-4029-8c4f-b74191b01e2d_isrForCrosstalkSources_996956_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/97930b67-466f-4029-8c4f-b74191b01e2d_isrForCrosstalkSources_996956_23.stderr parsl.dataflow.dflow DEBUG: Task 1269 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1269 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1269 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1269 has memoization hash d0c336531256d58c9f071ee0d7211bed parsl.dataflow.memoization INFO: Task 1269 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1269 try 0 launched on executor multi with executor id 1270 parsl.dataflow.dflow INFO: Standard output for task 1269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4a7b1532-05f8-42f4-a8b0-4805bd3407f1_isrForCrosstalkSources_996966_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4a7b1532-05f8-42f4-a8b0-4805bd3407f1_isrForCrosstalkSources_996966_31.stderr parsl.dataflow.dflow DEBUG: Task 1270 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1270 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1270 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1270 has memoization hash 82ddbdb2e84c57009d597f296667b22c parsl.dataflow.memoization INFO: Task 1270 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1270 try 0 launched on executor multi with executor id 1271 parsl.dataflow.dflow INFO: Standard output for task 1270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7eb32139-4a18-41d3-b141-3dad6ed510f7_isrForCrosstalkSources_996975_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7eb32139-4a18-41d3-b141-3dad6ed510f7_isrForCrosstalkSources_996975_13.stderr parsl.dataflow.dflow DEBUG: Task 1271 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1271 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1271 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1271 has memoization hash 81d72838d59e1ec23c17761900b44f51 parsl.dataflow.memoization INFO: Task 1271 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1271 try 0 launched on executor multi with executor id 1272 parsl.dataflow.dflow INFO: Standard output for task 1271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/344423a0-0b05-45be-8fb9-12c8daed71b5_isrForCrosstalkSources_997026_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/344423a0-0b05-45be-8fb9-12c8daed71b5_isrForCrosstalkSources_997026_41.stderr parsl.dataflow.dflow DEBUG: Task 1272 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1272 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1272 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1272 has memoization hash dc38f632caa8b32529551a887d0e17c2 parsl.dataflow.memoization INFO: Task 1272 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1272 try 0 launched on executor multi with executor id 1273 parsl.dataflow.dflow INFO: Standard output for task 1272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f1f4d51a-8d5b-425e-a9dd-83b3bee80c86_isrForCrosstalkSources_996961_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f1f4d51a-8d5b-425e-a9dd-83b3bee80c86_isrForCrosstalkSources_996961_35.stderr parsl.dataflow.dflow DEBUG: Task 1273 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1273 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1273 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1273 has memoization hash 17517d9c9bad4b1a06d6c72aeb2934ed parsl.dataflow.memoization INFO: Task 1273 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1273 try 0 launched on executor multi with executor id 1274 parsl.dataflow.dflow INFO: Standard output for task 1273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2392d6f-b6f4-4484-bc9a-b8f8c2ea7b69_isrForCrosstalkSources_996996_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2392d6f-b6f4-4484-bc9a-b8f8c2ea7b69_isrForCrosstalkSources_996996_37.stderr parsl.dataflow.dflow DEBUG: Task 1274 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1274 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1274 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1274 has memoization hash 02646a2f714ea7ca7b23dda4dc58a399 parsl.dataflow.memoization INFO: Task 1274 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1274 try 0 launched on executor multi with executor id 1275 parsl.dataflow.dflow INFO: Standard output for task 1274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/faec719c-cc80-4d22-99ac-e511c26b367a_isrForCrosstalkSources_997000_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/faec719c-cc80-4d22-99ac-e511c26b367a_isrForCrosstalkSources_997000_23.stderr parsl.dataflow.dflow DEBUG: Task 1275 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1275 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1275 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1275 has memoization hash 177b187c22023222a6b3a4096c71dca4 parsl.dataflow.memoization INFO: Task 1275 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1275 try 0 launched on executor multi with executor id 1276 parsl.dataflow.dflow INFO: Standard output for task 1275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/834d848f-cea3-496f-86e4-c4b80a22543b_isrForCrosstalkSources_996980_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/834d848f-cea3-496f-86e4-c4b80a22543b_isrForCrosstalkSources_996980_41.stderr parsl.dataflow.dflow DEBUG: Task 1276 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1276 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1276 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1276 has memoization hash c98d59ac788f858ad94537fd88a7a80f parsl.dataflow.memoization INFO: Task 1276 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1276 try 0 launched on executor multi with executor id 1277 parsl.dataflow.dflow INFO: Standard output for task 1276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0425416a-d00d-4d78-83f6-2fc4d1d8b413_isrForCrosstalkSources_997020_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0425416a-d00d-4d78-83f6-2fc4d1d8b413_isrForCrosstalkSources_997020_58.stderr parsl.dataflow.dflow DEBUG: Task 1277 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1277 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1277 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1277 has memoization hash 0e838239541bb50151f981c14e96e373 parsl.dataflow.memoization INFO: Task 1277 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1277 try 0 launched on executor multi with executor id 1278 parsl.dataflow.dflow INFO: Standard output for task 1277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/9be88a99-6663-4aff-a935-113b9503c7c1_isrForCrosstalkSources_997009_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/9be88a99-6663-4aff-a935-113b9503c7c1_isrForCrosstalkSources_997009_33.stderr parsl.dataflow.dflow DEBUG: Task 1278 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1278 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1278 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1278 has memoization hash b3738f6b20c9a05ca02a5028086dbeab parsl.dataflow.memoization INFO: Task 1278 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1278 try 0 launched on executor multi with executor id 1279 parsl.dataflow.dflow INFO: Standard output for task 1278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1d81105a-a7db-4993-97c2-ed281d48d342_isrForCrosstalkSources_997022_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1d81105a-a7db-4993-97c2-ed281d48d342_isrForCrosstalkSources_997022_48.stderr parsl.dataflow.dflow DEBUG: Task 1279 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1279 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1279 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1279 has memoization hash 499e342d97b5eacfd85083addafc6389 parsl.dataflow.memoization INFO: Task 1279 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1279 try 0 launched on executor multi with executor id 1280 parsl.dataflow.dflow INFO: Standard output for task 1279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b1ee44f1-4c6b-4394-86e4-dfeadba08318_isrForCrosstalkSources_996972_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b1ee44f1-4c6b-4394-86e4-dfeadba08318_isrForCrosstalkSources_996972_8.stderr parsl.dataflow.dflow DEBUG: Task 1280 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1280 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1280 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1280 has memoization hash 5854b1a41e1f99555caf1a42d2562491 parsl.dataflow.memoization INFO: Task 1280 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1280 try 0 launched on executor multi with executor id 1281 parsl.dataflow.dflow INFO: Standard output for task 1280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c273b913-5dfd-4ee5-b33a-023d0061288a_isrForCrosstalkSources_996992_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c273b913-5dfd-4ee5-b33a-023d0061288a_isrForCrosstalkSources_996992_7.stderr parsl.dataflow.dflow DEBUG: Task 1281 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1281 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1281 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1281 has memoization hash 4c6297f6f2090bee8c0a03e010cbe7be parsl.dataflow.memoization INFO: Task 1281 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1281 try 0 launched on executor multi with executor id 1282 parsl.dataflow.dflow INFO: Standard output for task 1281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/94029ffa-dcb1-4d57-a8bd-89112b22b50c_isrForCrosstalkSources_996989_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/94029ffa-dcb1-4d57-a8bd-89112b22b50c_isrForCrosstalkSources_996989_60.stderr parsl.dataflow.dflow DEBUG: Task 1282 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1282 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1282 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1282 has memoization hash e03874dba5e7a66d1ee500ec2d5a51c3 parsl.dataflow.memoization INFO: Task 1282 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1282 try 0 launched on executor multi with executor id 1283 parsl.dataflow.dflow INFO: Standard output for task 1282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/afffa461-e355-42d8-aa84-5e29e97b090a_isrForCrosstalkSources_997014_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/afffa461-e355-42d8-aa84-5e29e97b090a_isrForCrosstalkSources_997014_9.stderr parsl.dataflow.dflow DEBUG: Task 1283 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1283 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1283 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1283 has memoization hash 34996a8bb437af436e390519e609aa5c parsl.dataflow.memoization INFO: Task 1283 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1283 try 0 launched on executor multi with executor id 1284 parsl.dataflow.dflow INFO: Standard output for task 1283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c436a085-14d6-437e-be3f-57b61c4283da_isrForCrosstalkSources_996960_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c436a085-14d6-437e-be3f-57b61c4283da_isrForCrosstalkSources_996960_7.stderr parsl.dataflow.dflow DEBUG: Task 1284 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1284 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1284 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1284 has memoization hash b11c5e2de031bb08cc0f7781a9e82e58 parsl.dataflow.memoization INFO: Task 1284 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1284 try 0 launched on executor multi with executor id 1285 parsl.dataflow.dflow INFO: Standard output for task 1284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/687cea21-e716-4f86-b411-275f9391a3b9_isrForCrosstalkSources_997018_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/687cea21-e716-4f86-b411-275f9391a3b9_isrForCrosstalkSources_997018_28.stderr parsl.dataflow.dflow DEBUG: Task 1285 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1285 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1285 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1285 has memoization hash 40fcb8eb47158b61aae7a39c4b3248a7 parsl.dataflow.memoization INFO: Task 1285 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1285 try 0 launched on executor multi with executor id 1286 parsl.dataflow.dflow INFO: Standard output for task 1285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/137fcb53-4790-45b9-81d3-6554b3bd723f_isrForCrosstalkSources_996967_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/137fcb53-4790-45b9-81d3-6554b3bd723f_isrForCrosstalkSources_996967_34.stderr parsl.dataflow.dflow DEBUG: Task 1286 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1286 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1286 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1286 has memoization hash a384d299fd0e9537c4579f6969ba7fc6 parsl.dataflow.memoization INFO: Task 1286 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1286 try 0 launched on executor multi with executor id 1287 parsl.dataflow.dflow INFO: Standard output for task 1286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/79c87c47-242f-4cfc-b90c-5766dc31b80c_isrForCrosstalkSources_996997_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/79c87c47-242f-4cfc-b90c-5766dc31b80c_isrForCrosstalkSources_996997_45.stderr parsl.dataflow.dflow DEBUG: Task 1287 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1287 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1287 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1287 has memoization hash a496e3f857efb87b0066d23b6bcc901d parsl.dataflow.memoization INFO: Task 1287 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1287 try 0 launched on executor multi with executor id 1288 parsl.dataflow.dflow INFO: Standard output for task 1287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5e203a11-4052-4eb8-8962-132b07342a3e_isrForCrosstalkSources_997010_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5e203a11-4052-4eb8-8962-132b07342a3e_isrForCrosstalkSources_997010_21.stderr parsl.dataflow.dflow DEBUG: Task 1288 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1288 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1288 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1288 has memoization hash dd479a2d4865e77794541468da42c762 parsl.dataflow.memoization INFO: Task 1288 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1288 try 0 launched on executor multi with executor id 1289 parsl.dataflow.dflow INFO: Standard output for task 1288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/31dbc9b7-d3b7-46f4-b6f4-2a29de68e867_isrForCrosstalkSources_997017_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/31dbc9b7-d3b7-46f4-b6f4-2a29de68e867_isrForCrosstalkSources_997017_25.stderr parsl.dataflow.dflow DEBUG: Task 1289 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1289 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1289 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1289 has memoization hash 2e3a6cfaf6b9ad6ee6d5709fcc083258 parsl.dataflow.memoization INFO: Task 1289 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1289 try 0 launched on executor multi with executor id 1290 parsl.dataflow.dflow INFO: Standard output for task 1289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/de58d4eb-b320-4d0c-9990-d6c899c6377d_isrForCrosstalkSources_996969_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/de58d4eb-b320-4d0c-9990-d6c899c6377d_isrForCrosstalkSources_996969_37.stderr parsl.dataflow.dflow DEBUG: Task 1290 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1290 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1290 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1290 has memoization hash 751f065466220a75f3e5815e9b1c0f8c parsl.dataflow.memoization INFO: Task 1290 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1290 try 0 launched on executor multi with executor id 1291 parsl.dataflow.dflow INFO: Standard output for task 1290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/49742881-bcff-41e0-adb5-6c25dd439105_isrForCrosstalkSources_997015_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/49742881-bcff-41e0-adb5-6c25dd439105_isrForCrosstalkSources_997015_30.stderr parsl.dataflow.dflow DEBUG: Task 1291 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1291 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1291 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1291 has memoization hash 82a8ead1bf4f4a4236c4c9cd35f4e819 parsl.dataflow.memoization INFO: Task 1291 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:37 proc_lsst.multi:170 [INFO] [multi] job_id 17074855 proc_lsst.multi INFO: [multi] job_id 17074855 2024-03-15 13:49:37 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 16 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 16 2024-03-15 13:49:37 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:37 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074855 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074855 parsl.executors.status_handling DEBUG: Launched block 16 on executor multi with job ID 17074855 parsl.executors.status_handling INFO: Allocated block ID 17 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-17 2024-03-15 13:49:37 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=17 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-17 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=17 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-17 2024-03-15 13:49:37 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=17 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=17 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:37 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=17 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=17 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 16 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 16 2024-03-15 13:49:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:37 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=17 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=17 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 1291 try 0 launched on executor multi with executor id 1292 parsl.dataflow.dflow INFO: Standard output for task 1291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e41f1800-c981-4603-a5ab-000e68026fcb_isrForCrosstalkSources_997033_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e41f1800-c981-4603-a5ab-000e68026fcb_isrForCrosstalkSources_997033_31.stderr parsl.dataflow.dflow DEBUG: Task 1292 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1292 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1292 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1292 has memoization hash d9eccffda882360b9af0ea0217b1f3c0 parsl.dataflow.memoization INFO: Task 1292 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1292 try 0 launched on executor multi with executor id 1293 parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Standard output for task 1292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/ca852a3d-401f-4c61-a53b-f3364061bcf4_isrForCrosstalkSources_997018_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/ca852a3d-401f-4c61-a53b-f3364061bcf4_isrForCrosstalkSources_997018_29.stderr parsl.dataflow.dflow DEBUG: Task 1293 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1293 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1293 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1293 has memoization hash 8a0b5e2345aeb89fa9cceb0a7dcf12a7 parsl.dataflow.memoization INFO: Task 1293 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1293 try 0 launched on executor multi with executor id 1294 parsl.dataflow.dflow INFO: Standard output for task 1293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/24fd5a22-e006-4f4e-b4f8-8cfe3cd4a5c6_isrForCrosstalkSources_997026_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/24fd5a22-e006-4f4e-b4f8-8cfe3cd4a5c6_isrForCrosstalkSources_997026_29.stderr parsl.dataflow.dflow DEBUG: Task 1294 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1294 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1294 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1294 has memoization hash 8c9f9b4e0c2b8f84d3768e9a0496e8c1 parsl.dataflow.memoization INFO: Task 1294 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1294 try 0 launched on executor multi with executor id 1295 parsl.dataflow.dflow INFO: Standard output for task 1294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0c807557-ed83-4b82-a960-78278892c434_isrForCrosstalkSources_997007_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0c807557-ed83-4b82-a960-78278892c434_isrForCrosstalkSources_997007_40.stderr parsl.dataflow.dflow DEBUG: Task 1295 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1295 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1295 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1295 has memoization hash 1b6f33b508b4785cd80005c76ae321fb parsl.dataflow.memoization INFO: Task 1295 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1295 try 0 launched on executor multi with executor id 1296 parsl.dataflow.dflow INFO: Standard output for task 1295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/aa3d11b2-cd5c-4413-ad56-fde7bf53ff89_isrForCrosstalkSources_997024_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/aa3d11b2-cd5c-4413-ad56-fde7bf53ff89_isrForCrosstalkSources_997024_40.stderr parsl.dataflow.dflow DEBUG: Task 1296 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1296 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1296 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1296 has memoization hash b5bd5296f4aa2eff4dddde08d5198c13 parsl.dataflow.memoization INFO: Task 1296 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1296 try 0 launched on executor multi with executor id 1297 parsl.dataflow.dflow INFO: Standard output for task 1296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6d16bd49-5365-4534-aab7-2ee0f515f698_isrForCrosstalkSources_996953_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6d16bd49-5365-4534-aab7-2ee0f515f698_isrForCrosstalkSources_996953_18.stderr parsl.dataflow.dflow DEBUG: Task 1297 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1297 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1297 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1297 has memoization hash d2d521131c177697235096891a2a2365 parsl.dataflow.memoization INFO: Task 1297 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1297 try 0 launched on executor multi with executor id 1298 parsl.dataflow.dflow INFO: Standard output for task 1297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/b0f192b6-af69-45a3-b597-71185d20f310_isrForCrosstalkSources_996955_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/b0f192b6-af69-45a3-b597-71185d20f310_isrForCrosstalkSources_996955_19.stderr parsl.dataflow.dflow DEBUG: Task 1298 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1298 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1298 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1298 has memoization hash 3a11d22e480eb97dbdfd5e2dcf46ef7a parsl.dataflow.memoization INFO: Task 1298 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1298 try 0 launched on executor multi with executor id 1299 parsl.dataflow.dflow INFO: Standard output for task 1298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/226bdfac-d83e-4f11-aa1f-63196d9ea7c4_isrForCrosstalkSources_997014_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/226bdfac-d83e-4f11-aa1f-63196d9ea7c4_isrForCrosstalkSources_997014_16.stderr parsl.dataflow.dflow DEBUG: Task 1299 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1299 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1299 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1299 has memoization hash cace9d0912d354e3be04ecffdb77292b parsl.dataflow.memoization INFO: Task 1299 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1299 try 0 launched on executor multi with executor id 1300 parsl.dataflow.dflow INFO: Standard output for task 1299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/df6670c5-4cf0-4b4b-b988-708afcd945c9_isrForCrosstalkSources_996955_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/df6670c5-4cf0-4b4b-b988-708afcd945c9_isrForCrosstalkSources_996955_37.stderr parsl.dataflow.dflow DEBUG: Task 1300 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1300 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1300 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1300 has memoization hash 10d48b5f03092af257c58f2c2cf1eb7c parsl.dataflow.memoization INFO: Task 1300 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1300 try 0 launched on executor multi with executor id 1301 parsl.dataflow.dflow INFO: Standard output for task 1300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/74887274-c16a-40bc-88f2-c481e9671a5f_isrForCrosstalkSources_997033_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/74887274-c16a-40bc-88f2-c481e9671a5f_isrForCrosstalkSources_997033_39.stderr parsl.dataflow.dflow DEBUG: Task 1301 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1301 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1301 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1301 has memoization hash 48bd0802a891ec51c4818e8f04b96b74 parsl.dataflow.memoization INFO: Task 1301 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1301 try 0 launched on executor multi with executor id 1302 parsl.dataflow.dflow INFO: Standard output for task 1301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0dde1fdd-dfea-4d66-8c96-94b2d86c352f_isrForCrosstalkSources_996970_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0dde1fdd-dfea-4d66-8c96-94b2d86c352f_isrForCrosstalkSources_996970_6.stderr parsl.dataflow.dflow DEBUG: Task 1302 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1302 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1302 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1302 has memoization hash 2b457641dc1f9b2b85c6e8c866440c52 parsl.dataflow.memoization INFO: Task 1302 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1302 try 0 launched on executor multi with executor id 1303 parsl.dataflow.dflow INFO: Standard output for task 1302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/26cd148f-9f36-4c4b-8e22-162596923e08_isrForCrosstalkSources_997020_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/26cd148f-9f36-4c4b-8e22-162596923e08_isrForCrosstalkSources_997020_22.stderr parsl.dataflow.dflow DEBUG: Task 1303 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1303 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1303 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1303 has memoization hash 08615d3e08bd48bc006ae824a322a421 parsl.dataflow.memoization INFO: Task 1303 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1303 try 0 launched on executor multi with executor id 1304 parsl.dataflow.dflow INFO: Standard output for task 1303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/457fdc4b-2c2f-41d9-8f64-b18efc179194_isrForCrosstalkSources_996969_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/457fdc4b-2c2f-41d9-8f64-b18efc179194_isrForCrosstalkSources_996969_3.stderr parsl.dataflow.dflow DEBUG: Task 1304 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1304 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1304 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1304 has memoization hash 50c9f6572e2666edc9d442c0d987a7c9 parsl.dataflow.memoization INFO: Task 1304 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1304 try 0 launched on executor multi with executor id 1305 parsl.dataflow.dflow INFO: Standard output for task 1304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a3cecc5d-bdc0-4b9f-9847-d4a8251ec350_isrForCrosstalkSources_997033_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a3cecc5d-bdc0-4b9f-9847-d4a8251ec350_isrForCrosstalkSources_997033_42.stderr parsl.dataflow.dflow DEBUG: Task 1305 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1305 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1305 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1305 has memoization hash 9c7d686d6e37106e36b93b5d529182f9 parsl.dataflow.memoization INFO: Task 1305 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1305 try 0 launched on executor multi with executor id 1306 parsl.dataflow.dflow INFO: Standard output for task 1305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/487d083a-e295-45e3-b67f-3e493f2a7155_isrForCrosstalkSources_997028_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/487d083a-e295-45e3-b67f-3e493f2a7155_isrForCrosstalkSources_997028_39.stderr parsl.dataflow.dflow DEBUG: Task 1306 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1306 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1306 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1306 has memoization hash 8b6c6f82970b747e980336595f1a445c parsl.dataflow.memoization INFO: Task 1306 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1306 try 0 launched on executor multi with executor id 1307 parsl.dataflow.dflow INFO: Standard output for task 1306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/242fde74-344c-45e7-bfb9-3ea2a8c07141_isrForCrosstalkSources_996960_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/242fde74-344c-45e7-bfb9-3ea2a8c07141_isrForCrosstalkSources_996960_45.stderr parsl.dataflow.dflow DEBUG: Task 1307 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1307 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1307 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1307 has memoization hash 55b3d26dd5ff371320f635c8c8288c5e parsl.dataflow.memoization INFO: Task 1307 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1307 try 0 launched on executor multi with executor id 1308 parsl.dataflow.dflow INFO: Standard output for task 1307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/3af9d428-2174-485c-af41-a568d422dd9c_isrForCrosstalkSources_997002_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/3af9d428-2174-485c-af41-a568d422dd9c_isrForCrosstalkSources_997002_39.stderr parsl.dataflow.dflow DEBUG: Task 1308 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1308 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1308 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1308 has memoization hash b431235813293c94eed1bbb92f4a10f2 parsl.dataflow.memoization INFO: Task 1308 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1308 try 0 launched on executor multi with executor id 1309 parsl.dataflow.dflow INFO: Standard output for task 1308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/064ac10b-5c3e-49f4-802f-d914c06c50ef_isrForCrosstalkSources_996994_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/064ac10b-5c3e-49f4-802f-d914c06c50ef_isrForCrosstalkSources_996994_24.stderr parsl.dataflow.dflow DEBUG: Task 1309 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1309 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1309 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1309 has memoization hash d9b3d5f1b3ba738171a398b0acf8297b parsl.dataflow.memoization INFO: Task 1309 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1309 try 0 launched on executor multi with executor id 1310 parsl.dataflow.dflow INFO: Standard output for task 1309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/de3fd8a7-6533-4454-a5d4-78926fa77dc0_isrForCrosstalkSources_996990_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/de3fd8a7-6533-4454-a5d4-78926fa77dc0_isrForCrosstalkSources_996990_44.stderr parsl.dataflow.dflow DEBUG: Task 1310 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1310 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1310 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1310 has memoization hash a41b55e65f4c7bf3f7223ec8ad25a0b8 parsl.dataflow.memoization INFO: Task 1310 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1310 try 0 launched on executor multi with executor id 1311 parsl.dataflow.dflow INFO: Standard output for task 1310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/1e8e2d34-bfed-4bc2-a5d8-2d8a209c4281_isrForCrosstalkSources_996997_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/1e8e2d34-bfed-4bc2-a5d8-2d8a209c4281_isrForCrosstalkSources_996997_6.stderr parsl.dataflow.dflow DEBUG: Task 1311 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1311 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1311 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1311 has memoization hash d7bbf3854d3914720267ddb099209929 parsl.dataflow.memoization INFO: Task 1311 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1311 try 0 launched on executor multi with executor id 1312 parsl.dataflow.dflow INFO: Standard output for task 1311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/da14c176-7358-4465-bdef-b0d060dd7d3b_isrForCrosstalkSources_996978_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/da14c176-7358-4465-bdef-b0d060dd7d3b_isrForCrosstalkSources_996978_25.stderr parsl.dataflow.dflow DEBUG: Task 1312 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1312 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1312 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1312 has memoization hash bb0153a7d6dbc77cc0144227d9076ed7 parsl.dataflow.memoization INFO: Task 1312 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1312 try 0 launched on executor multi with executor id 1313 parsl.dataflow.dflow INFO: Standard output for task 1312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/59488eba-3101-4235-aaa3-92bc69fb0eea_isrForCrosstalkSources_997009_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/59488eba-3101-4235-aaa3-92bc69fb0eea_isrForCrosstalkSources_997009_40.stderr parsl.dataflow.dflow DEBUG: Task 1313 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1313 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1313 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1313 has memoization hash 3b123c46006d52ccd7b288850a7a8853 parsl.dataflow.memoization INFO: Task 1313 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1313 try 0 launched on executor multi with executor id 1314 parsl.dataflow.dflow INFO: Standard output for task 1313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/82be5258-d2f7-490b-9445-715cad7f5833_isrForCrosstalkSources_997017_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/82be5258-d2f7-490b-9445-715cad7f5833_isrForCrosstalkSources_997017_17.stderr parsl.dataflow.dflow DEBUG: Task 1314 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1314 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1314 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1314 has memoization hash 4f9fc271a547323a47ed351d2e334d15 parsl.dataflow.memoization INFO: Task 1314 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1314 try 0 launched on executor multi with executor id 1315 parsl.dataflow.dflow INFO: Standard output for task 1314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/5263f56e-2af3-4467-a1ad-95b1f5de7d60_isrForCrosstalkSources_997032_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/5263f56e-2af3-4467-a1ad-95b1f5de7d60_isrForCrosstalkSources_997032_31.stderr parsl.dataflow.dflow DEBUG: Task 1315 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1315 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1315 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1315 has memoization hash 3f91688c5ad6120087220b8ad2852fae parsl.dataflow.memoization INFO: Task 1315 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1315 try 0 launched on executor multi with executor id 1316 parsl.dataflow.dflow INFO: Standard output for task 1315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/de9dabb8-99b6-4574-999c-59c4b4cd74dd_isrForCrosstalkSources_996990_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/de9dabb8-99b6-4574-999c-59c4b4cd74dd_isrForCrosstalkSources_996990_36.stderr parsl.dataflow.dflow DEBUG: Task 1316 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1316 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1316 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1316 has memoization hash 32ef575d4f8c7e847b3b104aa552a554 parsl.dataflow.memoization INFO: Task 1316 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1316 try 0 launched on executor multi with executor id 1317 parsl.dataflow.dflow INFO: Standard output for task 1316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/7a537a40-2e7d-4c12-8a64-5d44d2524007_isrForCrosstalkSources_997020_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/7a537a40-2e7d-4c12-8a64-5d44d2524007_isrForCrosstalkSources_997020_38.stderr parsl.dataflow.dflow DEBUG: Task 1317 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1317 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1317 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1317 has memoization hash 1839bc930270c079bb4ec5e876f11c00 parsl.dataflow.memoization INFO: Task 1317 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1317 try 0 launched on executor multi with executor id 1318 parsl.dataflow.dflow INFO: Standard output for task 1317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/66ee52df-5a24-4781-8b7a-3ca87aa7e49b_isrForCrosstalkSources_996973_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/66ee52df-5a24-4781-8b7a-3ca87aa7e49b_isrForCrosstalkSources_996973_37.stderr parsl.dataflow.dflow DEBUG: Task 1318 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1318 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1318 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1318 has memoization hash 56006e8cd2a18f272707a92f6ac8748d parsl.dataflow.memoization INFO: Task 1318 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1318 try 0 launched on executor multi with executor id 1319 parsl.dataflow.dflow INFO: Standard output for task 1318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6dff968f-f8e1-45d2-b423-a31cb7cd7212_isrForCrosstalkSources_996970_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6dff968f-f8e1-45d2-b423-a31cb7cd7212_isrForCrosstalkSources_996970_57.stderr parsl.dataflow.dflow DEBUG: Task 1319 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1319 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1319 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1319 has memoization hash d1fcca5fa0f3abf2be7437b5ee358097 parsl.dataflow.memoization INFO: Task 1319 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1319 try 0 launched on executor multi with executor id 1320 parsl.dataflow.dflow INFO: Standard output for task 1319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f8fc7d37-0b42-4e56-b539-b0723368add4_isrForCrosstalkSources_997019_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f8fc7d37-0b42-4e56-b539-b0723368add4_isrForCrosstalkSources_997019_51.stderr parsl.dataflow.dflow DEBUG: Task 1320 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1320 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1320 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1320 has memoization hash e8f6e0446ca3d202b07369a419abe88a parsl.dataflow.memoization INFO: Task 1320 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1320 try 0 launched on executor multi with executor id 1321 parsl.dataflow.dflow INFO: Standard output for task 1320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/89227b92-74bc-42be-b188-544195de0ab9_isrForCrosstalkSources_997001_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/89227b92-74bc-42be-b188-544195de0ab9_isrForCrosstalkSources_997001_51.stderr parsl.dataflow.dflow DEBUG: Task 1321 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1321 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1321 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1321 has memoization hash df74be6534bac2da1daa1778aa209e13 parsl.dataflow.memoization INFO: Task 1321 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1321 try 0 launched on executor multi with executor id 1322 parsl.dataflow.dflow INFO: Standard output for task 1321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/5efb51f4-0f75-4bdb-a07e-9ac5afd841e1_isrForCrosstalkSources_996970_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/5efb51f4-0f75-4bdb-a07e-9ac5afd841e1_isrForCrosstalkSources_996970_2.stderr parsl.dataflow.dflow DEBUG: Task 1322 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1322 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1322 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1322 has memoization hash 123dfa8cd7db9cd556e5e3bc7c6f2742 parsl.dataflow.memoization INFO: Task 1322 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1322 try 0 launched on executor multi with executor id 1323 parsl.dataflow.dflow INFO: Standard output for task 1322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/1a1466af-dc60-4e78-a917-cb22f85dad0e_isrForCrosstalkSources_996982_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/1a1466af-dc60-4e78-a917-cb22f85dad0e_isrForCrosstalkSources_996982_26.stderr parsl.dataflow.dflow DEBUG: Task 1323 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1323 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1323 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1323 has memoization hash cbc723a87cc0baaf5b88896da84ba927 parsl.dataflow.memoization INFO: Task 1323 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1323 try 0 launched on executor multi with executor id 1324 parsl.dataflow.dflow INFO: Standard output for task 1323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/65923a1a-921f-4ad4-979e-6342213b4a20_isrForCrosstalkSources_996996_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/65923a1a-921f-4ad4-979e-6342213b4a20_isrForCrosstalkSources_996996_48.stderr parsl.dataflow.dflow DEBUG: Task 1324 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1324 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1324 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1324 has memoization hash a502b070bd0ec8dc55390369e89d7b84 parsl.dataflow.memoization INFO: Task 1324 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1324 try 0 launched on executor multi with executor id 1325 parsl.dataflow.dflow INFO: Standard output for task 1324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/32321300-18b0-4f2a-bde5-2187c9642783_isrForCrosstalkSources_996976_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/32321300-18b0-4f2a-bde5-2187c9642783_isrForCrosstalkSources_996976_32.stderr parsl.dataflow.dflow DEBUG: Task 1325 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1325 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1325 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1325 has memoization hash 9a53850b96bc77a6f42686c33c34cefe parsl.dataflow.memoization INFO: Task 1325 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1325 try 0 launched on executor multi with executor id 1326 parsl.dataflow.dflow INFO: Standard output for task 1325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3d4e266d-2fd4-4df3-8798-8d8aa46eb147_isrForCrosstalkSources_996969_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3d4e266d-2fd4-4df3-8798-8d8aa46eb147_isrForCrosstalkSources_996969_13.stderr parsl.dataflow.dflow DEBUG: Task 1326 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1326 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1326 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1326 has memoization hash 450d9d400bc14597c661e0404db34076 parsl.dataflow.memoization INFO: Task 1326 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9480e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1326 try 0 launched on executor multi with executor id 1327 parsl.dataflow.dflow INFO: Standard output for task 1326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ba15d08f-9d63-4b7b-846a-f5d7942ac4f1_isrForCrosstalkSources_996960_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ba15d08f-9d63-4b7b-846a-f5d7942ac4f1_isrForCrosstalkSources_996960_35.stderr parsl.dataflow.dflow DEBUG: Task 1327 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1327 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1327 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1327 has memoization hash 20268221b7f01d0943afa3c2501dc2e0 parsl.dataflow.memoization INFO: Task 1327 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1327 try 0 launched on executor multi with executor id 1328 parsl.dataflow.dflow INFO: Standard output for task 1327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/28c8cf10-919f-42f0-a6cb-ed6a39fe897d_isrForCrosstalkSources_997026_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/28c8cf10-919f-42f0-a6cb-ed6a39fe897d_isrForCrosstalkSources_997026_54.stderr parsl.dataflow.dflow DEBUG: Task 1328 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1328 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1328 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1328 has memoization hash a3403f84f56bb93aab7918aa7d9b6ee2 parsl.dataflow.memoization INFO: Task 1328 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1328 try 0 launched on executor multi with executor id 1329 parsl.dataflow.dflow INFO: Standard output for task 1328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/83fe8707-adf6-4c88-9c4b-7e698502bca8_isrForCrosstalkSources_997006_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/83fe8707-adf6-4c88-9c4b-7e698502bca8_isrForCrosstalkSources_997006_14.stderr parsl.dataflow.dflow DEBUG: Task 1329 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1329 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1329 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1329 has memoization hash c4607c7194d52b47910c42ea9c9dd80e parsl.dataflow.memoization INFO: Task 1329 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9484a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1329 try 0 launched on executor multi with executor id 1330 parsl.dataflow.dflow INFO: Standard output for task 1329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a5d4a18f-cb36-4cf0-8d56-b51aeffd397a_isrForCrosstalkSources_996984_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a5d4a18f-cb36-4cf0-8d56-b51aeffd397a_isrForCrosstalkSources_996984_3.stderr parsl.dataflow.dflow DEBUG: Task 1330 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1330 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1330 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1330 has memoization hash 6a14f1181f22da9f0b64e4087b8f1a85 parsl.dataflow.memoization INFO: Task 1330 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9482c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1330 try 0 launched on executor multi with executor id 1331 parsl.dataflow.dflow INFO: Standard output for task 1330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/416df719-3918-43c2-9187-6e5d7d31030b_isrForCrosstalkSources_996957_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/416df719-3918-43c2-9187-6e5d7d31030b_isrForCrosstalkSources_996957_33.stderr parsl.dataflow.dflow DEBUG: Task 1331 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1331 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1331 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1331 has memoization hash f695125201d8768c168fbb2e0071fd91 parsl.dataflow.memoization INFO: Task 1331 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1331 try 0 launched on executor multi with executor id 1332 parsl.dataflow.dflow INFO: Standard output for task 1331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5b127dcf-8c94-47a5-a746-4767b03e63ed_isrForCrosstalkSources_996987_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5b127dcf-8c94-47a5-a746-4767b03e63ed_isrForCrosstalkSources_996987_14.stderr parsl.dataflow.dflow DEBUG: Task 1332 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1332 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1332 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1332 has memoization hash 41f86c4ce569fa76c173cbbf9a7aee4e parsl.dataflow.memoization INFO: Task 1332 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9485e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1332 try 0 launched on executor multi with executor id 1333 parsl.dataflow.dflow INFO: Standard output for task 1332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a21bc2eb-3c8c-4d04-9a8c-8192aa1d38e9_isrForCrosstalkSources_997026_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a21bc2eb-3c8c-4d04-9a8c-8192aa1d38e9_isrForCrosstalkSources_997026_60.stderr parsl.dataflow.dflow DEBUG: Task 1333 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1333 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1333 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1333 has memoization hash fca170e3340f2cd7f5d82e04b9f0471e parsl.dataflow.memoization INFO: Task 1333 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1333 try 0 launched on executor multi with executor id 1334 parsl.dataflow.dflow INFO: Standard output for task 1333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0a52b2c0-9b70-4f4b-b9b1-f4d7611326dd_isrForCrosstalkSources_996953_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0a52b2c0-9b70-4f4b-b9b1-f4d7611326dd_isrForCrosstalkSources_996953_55.stderr parsl.dataflow.dflow DEBUG: Task 1334 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1334 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1334 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1334 has memoization hash 4f7bd6ec006cf981282988fb57bfcac1 parsl.dataflow.memoization INFO: Task 1334 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9487c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1334 try 0 launched on executor multi with executor id 1335 parsl.dataflow.dflow INFO: Standard output for task 1334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/240b246c-e75c-45ec-ada5-ebfa7cc7f1d1_isrForCrosstalkSources_997007_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/240b246c-e75c-45ec-ada5-ebfa7cc7f1d1_isrForCrosstalkSources_997007_49.stderr parsl.dataflow.dflow DEBUG: Task 1335 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1335 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1335 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1335 has memoization hash 0116391dea305913587f4eba65ee22aa parsl.dataflow.memoization INFO: Task 1335 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1335 try 0 launched on executor multi with executor id 1336 parsl.dataflow.dflow INFO: Standard output for task 1335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d796c913-4885-4562-9910-6643ed86f5f6_isrForCrosstalkSources_996973_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d796c913-4885-4562-9910-6643ed86f5f6_isrForCrosstalkSources_996973_32.stderr parsl.dataflow.dflow DEBUG: Task 1336 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1336 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1336 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1336 has memoization hash 420b5a9b6adf598c97ab1819a469f8b7 parsl.dataflow.memoization INFO: Task 1336 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1336 try 0 launched on executor multi with executor id 1337 parsl.dataflow.dflow INFO: Standard output for task 1336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/a21a7632-ca0d-4930-acbc-c61b0882d79a_isrForCrosstalkSources_996975_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/a21a7632-ca0d-4930-acbc-c61b0882d79a_isrForCrosstalkSources_996975_38.stderr parsl.dataflow.dflow DEBUG: Task 1337 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1337 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1337 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1337 has memoization hash b46486360d98f262d2a3bbd8ba926139 parsl.dataflow.memoization INFO: Task 1337 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1337 try 0 launched on executor multi with executor id 1338 parsl.dataflow.dflow INFO: Standard output for task 1337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f62efaf6-69c9-470c-9536-c0199ca768b1_isrForCrosstalkSources_997016_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f62efaf6-69c9-470c-9536-c0199ca768b1_isrForCrosstalkSources_997016_43.stderr parsl.dataflow.dflow DEBUG: Task 1338 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1338 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1338 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1338 has memoization hash 0aa75e9ee2c5065a06fb34d41ac609cc parsl.dataflow.memoization INFO: Task 1338 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1338 try 0 launched on executor multi with executor id 1339 parsl.dataflow.dflow INFO: Standard output for task 1338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/8e52bdbb-a210-40e4-aeb3-3a4a53871a1c_isrForCrosstalkSources_997022_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/8e52bdbb-a210-40e4-aeb3-3a4a53871a1c_isrForCrosstalkSources_997022_20.stderr parsl.dataflow.dflow DEBUG: Task 1339 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1339 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1339 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1339 has memoization hash d6bab3041861616a11c69d130eda30ed parsl.dataflow.memoization INFO: Task 1339 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1339 try 0 launched on executor multi with executor id 1340 parsl.dataflow.dflow INFO: Standard output for task 1339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/bab4ab96-dd47-4fe0-8c09-d7577102a235_isrForCrosstalkSources_996967_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/bab4ab96-dd47-4fe0-8c09-d7577102a235_isrForCrosstalkSources_996967_46.stderr parsl.dataflow.dflow DEBUG: Task 1340 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1340 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1340 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1340 has memoization hash c5d85a3203a8710d216885a568564ea5 parsl.dataflow.memoization INFO: Task 1340 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1340 try 0 launched on executor multi with executor id 1341 parsl.dataflow.dflow INFO: Standard output for task 1340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e551ecd5-d027-4155-8307-d3180ac31696_isrForCrosstalkSources_997020_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e551ecd5-d027-4155-8307-d3180ac31696_isrForCrosstalkSources_997020_6.stderr parsl.dataflow.dflow DEBUG: Task 1341 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1341 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1341 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1341 has memoization hash 1945fa391eb0ad5b82e5be5c41409060 parsl.dataflow.memoization INFO: Task 1341 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1341 try 0 launched on executor multi with executor id 1342 parsl.dataflow.dflow INFO: Standard output for task 1341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/32769c93-990a-48ac-81c8-52e1e5795c0c_isrForCrosstalkSources_996954_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/32769c93-990a-48ac-81c8-52e1e5795c0c_isrForCrosstalkSources_996954_57.stderr parsl.dataflow.dflow DEBUG: Task 1342 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1342 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1342 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1342 has memoization hash d5f376daff5d30d6662c2668b1df900c parsl.dataflow.memoization INFO: Task 1342 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9489a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1342 try 0 launched on executor multi with executor id 1343 parsl.dataflow.dflow INFO: Standard output for task 1342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/cabb6e59-41d8-4baa-a564-9b831117c418_isrForCrosstalkSources_997002_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/cabb6e59-41d8-4baa-a564-9b831117c418_isrForCrosstalkSources_997002_29.stderr parsl.dataflow.dflow DEBUG: Task 1343 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1343 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1343 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1343 has memoization hash 73f1ca1cc45d287762f0ee3009e74355 parsl.dataflow.memoization INFO: Task 1343 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1343 try 0 launched on executor multi with executor id 1344 parsl.dataflow.dflow INFO: Standard output for task 1343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/66560b0d-ae21-4c61-9c71-a2403f733e7c_isrForCrosstalkSources_996963_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/66560b0d-ae21-4c61-9c71-a2403f733e7c_isrForCrosstalkSources_996963_47.stderr parsl.dataflow.dflow DEBUG: Task 1344 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1344 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1344 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1344 has memoization hash 3829378929a8850305497eabc04c957e parsl.dataflow.memoization INFO: Task 1344 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1344 try 0 launched on executor multi with executor id 1345 parsl.dataflow.dflow INFO: Standard output for task 1344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/3384cda8-9b95-4fd7-9c10-d9419b3c22da_isrForCrosstalkSources_996989_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/3384cda8-9b95-4fd7-9c10-d9419b3c22da_isrForCrosstalkSources_996989_14.stderr parsl.dataflow.dflow DEBUG: Task 1345 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1345 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1345 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1345 has memoization hash be510ddc1ac34f485df7130bd00d2683 parsl.dataflow.memoization INFO: Task 1345 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1345 try 0 launched on executor multi with executor id 1346 parsl.dataflow.dflow INFO: Standard output for task 1345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/b814ea2a-52c4-44e4-b0b3-6d4125b87a54_isrForCrosstalkSources_996988_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/b814ea2a-52c4-44e4-b0b3-6d4125b87a54_isrForCrosstalkSources_996988_8.stderr parsl.dataflow.dflow DEBUG: Task 1346 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1346 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1346 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1346 has memoization hash ec53e2838f05ee31001a72cc3b38055e parsl.dataflow.memoization INFO: Task 1346 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1346 try 0 launched on executor multi with executor id 1347 parsl.dataflow.dflow INFO: Standard output for task 1346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/25293d16-91fd-4d4c-baf8-dbdb94feb743_isrForCrosstalkSources_996983_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/25293d16-91fd-4d4c-baf8-dbdb94feb743_isrForCrosstalkSources_996983_9.stderr parsl.dataflow.dflow DEBUG: Task 1347 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1347 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1347 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1347 has memoization hash 55fe375909be34a3e85fd20f498b1d8e parsl.dataflow.memoization INFO: Task 1347 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1347 try 0 launched on executor multi with executor id 1348 parsl.dataflow.dflow INFO: Standard output for task 1347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/cd8857a7-8aa7-41a1-b2d3-f764e8851472_isrForCrosstalkSources_996960_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/cd8857a7-8aa7-41a1-b2d3-f764e8851472_isrForCrosstalkSources_996960_52.stderr parsl.dataflow.dflow DEBUG: Task 1348 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1348 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1348 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1348 has memoization hash 237293d1f36a18f0f25ea1da8cee5cc5 parsl.dataflow.memoization INFO: Task 1348 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1348 try 0 launched on executor multi with executor id 1349 parsl.dataflow.dflow INFO: Standard output for task 1348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/89e4ecd4-9bd8-45a6-85cd-44233c76f46a_isrForCrosstalkSources_997029_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/89e4ecd4-9bd8-45a6-85cd-44233c76f46a_isrForCrosstalkSources_997029_1.stderr parsl.dataflow.dflow DEBUG: Task 1349 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1349 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1349 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1349 has memoization hash 043383cd307a4810ce23a2be9322d487 parsl.dataflow.memoization INFO: Task 1349 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1349 try 0 launched on executor multi with executor id 1350 parsl.dataflow.dflow INFO: Standard output for task 1349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/6a86c5e7-98d3-426b-bcdc-b5ed53688e4a_isrForCrosstalkSources_997026_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/6a86c5e7-98d3-426b-bcdc-b5ed53688e4a_isrForCrosstalkSources_997026_38.stderr parsl.dataflow.dflow DEBUG: Task 1350 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1350 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1350 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1350 has memoization hash 4c97915f2f45c024f4fac416b45c4534 parsl.dataflow.memoization INFO: Task 1350 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1350 try 0 launched on executor multi with executor id 1351 parsl.dataflow.dflow INFO: Standard output for task 1350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/256136dd-a48d-44a3-8da9-5b1909b97c9d_isrForCrosstalkSources_996967_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/256136dd-a48d-44a3-8da9-5b1909b97c9d_isrForCrosstalkSources_996967_26.stderr parsl.dataflow.dflow DEBUG: Task 1351 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1351 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1351 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1351 has memoization hash da553d95e50ff5bf96000d73924b9826 parsl.dataflow.memoization INFO: Task 1351 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1351 try 0 launched on executor multi with executor id 1352 parsl.dataflow.dflow INFO: Standard output for task 1351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4f1f41e1-f1e9-4c1e-bfe7-3011c987c0c8_isrForCrosstalkSources_996959_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4f1f41e1-f1e9-4c1e-bfe7-3011c987c0c8_isrForCrosstalkSources_996959_11.stderr parsl.dataflow.dflow DEBUG: Task 1352 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1352 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1352 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1352 has memoization hash 3396a26583fb5acee48583661489960e parsl.dataflow.memoization INFO: Task 1352 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1352 try 0 launched on executor multi with executor id 1353 parsl.dataflow.dflow INFO: Standard output for task 1352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/6443bc6e-8ec1-4dc2-be77-dc03fae7b3ad_isrForCrosstalkSources_997019_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/6443bc6e-8ec1-4dc2-be77-dc03fae7b3ad_isrForCrosstalkSources_997019_61.stderr parsl.dataflow.dflow DEBUG: Task 1353 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1353 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1353 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1353 has memoization hash 2ea9d9de9a8b85ad1f7251af80a60c01 parsl.dataflow.memoization INFO: Task 1353 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1353 try 0 launched on executor multi with executor id 1354 parsl.dataflow.dflow INFO: Standard output for task 1353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/82e6cc5e-8ddf-4a36-956f-6545f3d0d008_isrForCrosstalkSources_997032_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/82e6cc5e-8ddf-4a36-956f-6545f3d0d008_isrForCrosstalkSources_997032_43.stderr parsl.dataflow.dflow DEBUG: Task 1354 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1354 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1354 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1354 has memoization hash 519f3b1fdc9e20c25d8556c7cdeecdcb parsl.dataflow.memoization INFO: Task 1354 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9499e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1354 try 0 launched on executor multi with executor id 1355 parsl.dataflow.dflow INFO: Standard output for task 1354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/df396c51-8137-49c2-a500-3615095cd399_isrForCrosstalkSources_996967_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/df396c51-8137-49c2-a500-3615095cd399_isrForCrosstalkSources_996967_42.stderr parsl.dataflow.dflow DEBUG: Task 1355 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1355 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1355 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1355 has memoization hash 7ea0b89b162ed8140d71461d32bc80a0 parsl.dataflow.memoization INFO: Task 1355 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1355 try 0 launched on executor multi with executor id 1356 parsl.dataflow.dflow INFO: Standard output for task 1355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/5b123d46-8cee-47aa-b293-1d197ae2647a_isrForCrosstalkSources_997030_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/5b123d46-8cee-47aa-b293-1d197ae2647a_isrForCrosstalkSources_997030_6.stderr parsl.dataflow.dflow DEBUG: Task 1356 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1356 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1356 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1356 has memoization hash 5fb51675de9dbb6b8fa153ac813c4797 parsl.dataflow.memoization INFO: Task 1356 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1356 try 0 launched on executor multi with executor id 1357 parsl.dataflow.dflow INFO: Standard output for task 1356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/83c0172f-49d6-424a-b70f-a4594a080f74_isrForCrosstalkSources_996971_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/83c0172f-49d6-424a-b70f-a4594a080f74_isrForCrosstalkSources_996971_56.stderr parsl.dataflow.dflow DEBUG: Task 1357 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1357 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1357 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1357 has memoization hash 9eed0107cd127161be2c038666c6ac2e parsl.dataflow.memoization INFO: Task 1357 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9498a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1357 try 0 launched on executor multi with executor id 1358 parsl.dataflow.dflow INFO: Standard output for task 1357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c4df4a6b-3d01-4e1d-8b06-d8f9379630cf_isrForCrosstalkSources_996995_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c4df4a6b-3d01-4e1d-8b06-d8f9379630cf_isrForCrosstalkSources_996995_3.stderr parsl.dataflow.dflow DEBUG: Task 1358 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1358 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1358 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1358 has memoization hash 5958d7105d93f77fcd8c37722810bfa0 parsl.dataflow.memoization INFO: Task 1358 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:38 proc_lsst.multi:170 [INFO] [multi] job_id 17074856 proc_lsst.multi INFO: [multi] job_id 17074856 2024-03-15 13:49:38 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 17 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 17 2024-03-15 13:49:38 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:38 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074856 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074856 parsl.executors.status_handling DEBUG: Launched block 17 on executor multi with job ID 17074856 parsl.executors.status_handling INFO: Allocated block ID 18 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-18 2024-03-15 13:49:38 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=18 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-18 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=18 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-18 2024-03-15 13:49:38 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=18 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=18 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:38 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=18 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=18 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 17 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 17 2024-03-15 13:49:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:38 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=18 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=18 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1358 try 0 launched on executor multi with executor id 1359 parsl.dataflow.dflow INFO: Standard output for task 1358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/80716ad3-987e-4b76-9c34-441fb48fc620_isrForCrosstalkSources_997019_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/80716ad3-987e-4b76-9c34-441fb48fc620_isrForCrosstalkSources_997019_59.stderr parsl.dataflow.dflow DEBUG: Task 1359 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1359 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1359 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1359 has memoization hash 8325e92a93d7eca86c7daabcea59b402 parsl.dataflow.memoization INFO: Task 1359 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9496c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1359 try 0 launched on executor multi with executor id 1360 parsl.dataflow.dflow INFO: Standard output for task 1359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/63e17bf0-a30e-4b87-a51e-53902bff45a3_isrForCrosstalkSources_997023_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/63e17bf0-a30e-4b87-a51e-53902bff45a3_isrForCrosstalkSources_997023_43.stderr parsl.dataflow.dflow DEBUG: Task 1360 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1360 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1360 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1360 has memoization hash a92d79d560e33ba05a977390de2d07d3 parsl.dataflow.memoization INFO: Task 1360 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94be20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1360 try 0 launched on executor multi with executor id 1361 parsl.dataflow.dflow INFO: Standard output for task 1360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/36874233-be3b-45e9-a5bc-70ee8c2872a4_isrForCrosstalkSources_996954_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/36874233-be3b-45e9-a5bc-70ee8c2872a4_isrForCrosstalkSources_996954_34.stderr parsl.dataflow.dflow DEBUG: Task 1361 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1361 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1361 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1361 has memoization hash a1d08f3cba28a038b452f9b4998c74c1 parsl.dataflow.memoization INFO: Task 1361 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1361 try 0 launched on executor multi with executor id 1362 parsl.dataflow.dflow INFO: Standard output for task 1361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/84e05f91-ed6a-444e-b85b-a4b5d9aa7f6b_isrForCrosstalkSources_996999_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/84e05f91-ed6a-444e-b85b-a4b5d9aa7f6b_isrForCrosstalkSources_996999_21.stderr parsl.dataflow.dflow DEBUG: Task 1362 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1362 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1362 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1362 has memoization hash c75b2c47f72068798aed7359d589f9b4 parsl.dataflow.memoization INFO: Task 1362 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94bec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1362 try 0 launched on executor multi with executor id 1363 parsl.dataflow.dflow INFO: Standard output for task 1362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d0d40a2-cb5d-4423-8809-5d90e42880d5_isrForCrosstalkSources_996995_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d0d40a2-cb5d-4423-8809-5d90e42880d5_isrForCrosstalkSources_996995_22.stderr parsl.dataflow.dflow DEBUG: Task 1363 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1363 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1363 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1363 has memoization hash 6a26ccb198654ec7981c9f19d2353ff3 parsl.dataflow.memoization INFO: Task 1363 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94bd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1363 try 0 launched on executor multi with executor id 1364 parsl.dataflow.dflow INFO: Standard output for task 1363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/77d4c0e6-c86b-4112-b64c-26258e0a1952_isrForCrosstalkSources_996961_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/77d4c0e6-c86b-4112-b64c-26258e0a1952_isrForCrosstalkSources_996961_40.stderr parsl.dataflow.dflow DEBUG: Task 1364 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1364 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1364 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1364 has memoization hash 5d9e93c662e3c8374f591190480c4527 parsl.dataflow.memoization INFO: Task 1364 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9494e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1364 try 0 launched on executor multi with executor id 1365 parsl.dataflow.dflow INFO: Standard output for task 1364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3f3fd5fb-e653-4c86-9c11-7ba1236f3dc6_isrForCrosstalkSources_997033_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3f3fd5fb-e653-4c86-9c11-7ba1236f3dc6_isrForCrosstalkSources_997033_24.stderr parsl.dataflow.dflow DEBUG: Task 1365 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1365 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1365 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1365 has memoization hash 6ef1e1cd9a86ba180f40e6560e2bbc37 parsl.dataflow.memoization INFO: Task 1365 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94bce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1365 try 0 launched on executor multi with executor id 1366 parsl.dataflow.dflow INFO: Standard output for task 1365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/d185549d-7f51-4bbe-8a44-7f8028eac3e3_isrForCrosstalkSources_996952_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/d185549d-7f51-4bbe-8a44-7f8028eac3e3_isrForCrosstalkSources_996952_29.stderr parsl.dataflow.dflow DEBUG: Task 1366 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1366 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1366 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1366 has memoization hash 11d69a19704a79aaecc65f4dc84ead66 parsl.dataflow.memoization INFO: Task 1366 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1366 try 0 launched on executor multi with executor id 1367 parsl.dataflow.dflow INFO: Standard output for task 1366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/043cb00f-9507-4053-a34c-c2c8244c1f71_isrForCrosstalkSources_997008_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/043cb00f-9507-4053-a34c-c2c8244c1f71_isrForCrosstalkSources_997008_6.stderr parsl.dataflow.dflow DEBUG: Task 1367 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1367 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1367 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1367 has memoization hash 51c2b4e3c6cdb2c217426729cba7f093 parsl.dataflow.memoization INFO: Task 1367 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9493a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1367 try 0 launched on executor multi with executor id 1368 parsl.dataflow.dflow INFO: Standard output for task 1367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/32c90c0f-6145-460c-8e5c-b2178b84f1a7_isrForCrosstalkSources_996981_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/32c90c0f-6145-460c-8e5c-b2178b84f1a7_isrForCrosstalkSources_996981_14.stderr parsl.dataflow.dflow DEBUG: Task 1368 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1368 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1368 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1368 has memoization hash d13139be0681b09373fdd556107895f8 parsl.dataflow.memoization INFO: Task 1368 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1368 try 0 launched on executor multi with executor id 1369 parsl.dataflow.dflow INFO: Standard output for task 1368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/11631a4e-c027-4d36-91e5-dff9579cb863_isrForCrosstalkSources_997025_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/11631a4e-c027-4d36-91e5-dff9579cb863_isrForCrosstalkSources_997025_13.stderr parsl.dataflow.dflow DEBUG: Task 1369 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1369 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1369 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1369 has memoization hash 7ff1dc4abc946757d52ef7cdee7b5bb6 parsl.dataflow.memoization INFO: Task 1369 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1369 try 0 launched on executor multi with executor id 1370 parsl.dataflow.dflow INFO: Standard output for task 1369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/40c106e9-00db-4ba9-964e-ef716701c4c0_isrForCrosstalkSources_996956_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/40c106e9-00db-4ba9-964e-ef716701c4c0_isrForCrosstalkSources_996956_39.stderr parsl.dataflow.dflow DEBUG: Task 1370 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1370 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1370 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1370 has memoization hash f68f3b45c88026b6d635bb2516997de9 parsl.dataflow.memoization INFO: Task 1370 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1370 try 0 launched on executor multi with executor id 1371 parsl.dataflow.dflow INFO: Standard output for task 1370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/58c56853-4cac-492f-9716-18c61b1d5890_isrForCrosstalkSources_997025_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/58c56853-4cac-492f-9716-18c61b1d5890_isrForCrosstalkSources_997025_9.stderr parsl.dataflow.dflow DEBUG: Task 1371 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1371 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1371 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1371 has memoization hash 08cbd5e75c9fbdc004606b697852eeb4 parsl.dataflow.memoization INFO: Task 1371 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b9491c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1371 try 0 launched on executor multi with executor id 1372 parsl.dataflow.dflow INFO: Standard output for task 1371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/9264a22f-5fc3-41b7-9955-5b490d8f945b_isrForCrosstalkSources_996966_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/9264a22f-5fc3-41b7-9955-5b490d8f945b_isrForCrosstalkSources_996966_45.stderr parsl.dataflow.dflow DEBUG: Task 1372 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1372 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1372 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1372 has memoization hash 74f8079f09249597010249d33e1d569d parsl.dataflow.memoization INFO: Task 1372 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1372 try 0 launched on executor multi with executor id 1373 parsl.dataflow.dflow INFO: Standard output for task 1372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4bfab800-d624-4aef-9776-38f9e611cab3_isrForCrosstalkSources_997021_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4bfab800-d624-4aef-9776-38f9e611cab3_isrForCrosstalkSources_997021_58.stderr parsl.dataflow.dflow DEBUG: Task 1373 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1373 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1373 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1373 has memoization hash 4ae2561f177f5b68bb7ac8d7106286f4 parsl.dataflow.memoization INFO: Task 1373 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1373 try 0 launched on executor multi with executor id 1374 parsl.dataflow.dflow INFO: Standard output for task 1373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/18687f19-681a-4df7-91dc-6a9f1477b284_isrForCrosstalkSources_996992_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/18687f19-681a-4df7-91dc-6a9f1477b284_isrForCrosstalkSources_996992_11.stderr parsl.dataflow.dflow DEBUG: Task 1374 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1374 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1374 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1374 has memoization hash 094bcc2c667272b780cf11e37a68b823 parsl.dataflow.memoization INFO: Task 1374 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1374 try 0 launched on executor multi with executor id 1375 parsl.dataflow.dflow INFO: Standard output for task 1374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c52e06b8-1fab-46ad-aed7-8628d40aecb1_isrForCrosstalkSources_996985_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c52e06b8-1fab-46ad-aed7-8628d40aecb1_isrForCrosstalkSources_996985_19.stderr parsl.dataflow.dflow DEBUG: Task 1375 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1375 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1375 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1375 has memoization hash 91371c2a9279ddef657d39c8096ffd09 parsl.dataflow.memoization INFO: Task 1375 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1375 try 0 launched on executor multi with executor id 1376 parsl.dataflow.dflow INFO: Standard output for task 1375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4960308f-934f-4718-837b-90c76a24697d_isrForCrosstalkSources_996980_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4960308f-934f-4718-837b-90c76a24697d_isrForCrosstalkSources_996980_24.stderr parsl.dataflow.dflow DEBUG: Task 1376 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1376 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1376 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1376 has memoization hash 32992761400341cbdd4f061cd0b68c6c parsl.dataflow.memoization INFO: Task 1376 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1376 try 0 launched on executor multi with executor id 1377 parsl.dataflow.dflow INFO: Standard output for task 1376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f6ede271-3479-455f-95aa-7d4468b5771d_isrForCrosstalkSources_996955_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f6ede271-3479-455f-95aa-7d4468b5771d_isrForCrosstalkSources_996955_17.stderr parsl.dataflow.dflow DEBUG: Task 1377 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1377 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1377 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1377 has memoization hash 6d14810ea04fcc512e50cfb23e5d9612 parsl.dataflow.memoization INFO: Task 1377 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b948d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1377 try 0 launched on executor multi with executor id 1378 parsl.dataflow.dflow INFO: Standard output for task 1377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5cfdbcda-8edd-4a5d-bc2f-342849c14f56_isrForCrosstalkSources_996979_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5cfdbcda-8edd-4a5d-bc2f-342849c14f56_isrForCrosstalkSources_996979_43.stderr parsl.dataflow.dflow DEBUG: Task 1378 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1378 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1378 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1378 has memoization hash 7619ad3eca4e0aa7c5ac0898aca9faaa parsl.dataflow.memoization INFO: Task 1378 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1378 try 0 launched on executor multi with executor id 1379 parsl.dataflow.dflow INFO: Standard output for task 1378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/63ab8952-1687-426e-87f4-e478bdc90ece_isrForCrosstalkSources_997027_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/63ab8952-1687-426e-87f4-e478bdc90ece_isrForCrosstalkSources_997027_54.stderr parsl.dataflow.dflow DEBUG: Task 1379 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1379 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1379 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1379 has memoization hash e29b580380db42961e9408ff21da99e6 parsl.dataflow.memoization INFO: Task 1379 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1379 try 0 launched on executor multi with executor id 1380 parsl.dataflow.dflow INFO: Standard output for task 1379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0cb75091-38b3-40f5-89d6-d995dc091ab7_isrForCrosstalkSources_996973_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0cb75091-38b3-40f5-89d6-d995dc091ab7_isrForCrosstalkSources_996973_19.stderr parsl.dataflow.dflow DEBUG: Task 1380 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1380 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1380 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1380 has memoization hash 6d0327ddba5697bb129f41c85f3cdb9a parsl.dataflow.memoization INFO: Task 1380 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1380 try 0 launched on executor multi with executor id 1381 parsl.dataflow.dflow INFO: Standard output for task 1380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d72035be-9bc2-43c5-a687-253b271d840b_isrForCrosstalkSources_997027_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d72035be-9bc2-43c5-a687-253b271d840b_isrForCrosstalkSources_997027_32.stderr parsl.dataflow.dflow DEBUG: Task 1381 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1381 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1381 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1381 has memoization hash fed2c0d30bf1f9cf41e11c9d10343f9b parsl.dataflow.memoization INFO: Task 1381 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1381 try 0 launched on executor multi with executor id 1382 parsl.dataflow.dflow INFO: Standard output for task 1381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/0895b0ec-e4f9-49da-b64a-39e524c853a2_isrForCrosstalkSources_997000_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/0895b0ec-e4f9-49da-b64a-39e524c853a2_isrForCrosstalkSources_997000_30.stderr parsl.dataflow.dflow DEBUG: Task 1382 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1382 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1382 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1382 has memoization hash 05714e02edf8a5372294b08fcdbb428d parsl.dataflow.memoization INFO: Task 1382 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1382 try 0 launched on executor multi with executor id 1383 parsl.dataflow.dflow INFO: Standard output for task 1382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/a90c5157-90d6-42bc-9763-3bac58828c75_isrForCrosstalkSources_997031_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/a90c5157-90d6-42bc-9763-3bac58828c75_isrForCrosstalkSources_997031_4.stderr parsl.dataflow.dflow DEBUG: Task 1383 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1383 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1383 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1383 has memoization hash 80ae07328fc5c8b5a84df4cd8fc272b9 parsl.dataflow.memoization INFO: Task 1383 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1383 try 0 launched on executor multi with executor id 1384 parsl.dataflow.dflow INFO: Standard output for task 1383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/3f585083-a96d-4660-b8a2-6259452b6f77_isrForCrosstalkSources_997023_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/3f585083-a96d-4660-b8a2-6259452b6f77_isrForCrosstalkSources_997023_47.stderr parsl.dataflow.dflow DEBUG: Task 1384 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1384 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1384 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1384 has memoization hash ec062341699cd6566e7ca53024d67146 parsl.dataflow.memoization INFO: Task 1384 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1384 try 0 launched on executor multi with executor id 1385 parsl.dataflow.dflow INFO: Standard output for task 1384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f487a93c-c73e-473d-a3a2-ab6bf0286866_isrForCrosstalkSources_997017_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f487a93c-c73e-473d-a3a2-ab6bf0286866_isrForCrosstalkSources_997017_6.stderr parsl.dataflow.dflow DEBUG: Task 1385 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1385 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1385 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1385 has memoization hash eedc12e150076d72b79db2cf5735b9c0 parsl.dataflow.memoization INFO: Task 1385 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1385 try 0 launched on executor multi with executor id 1386 parsl.dataflow.dflow INFO: Standard output for task 1385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/2987827c-de3f-4d8e-8ddc-a928c9b03613_isrForCrosstalkSources_996986_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/2987827c-de3f-4d8e-8ddc-a928c9b03613_isrForCrosstalkSources_996986_21.stderr parsl.dataflow.dflow DEBUG: Task 1386 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1386 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1386 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1386 has memoization hash b133b696bbdcd7d306fc5a487d5bb946 parsl.dataflow.memoization INFO: Task 1386 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1386 try 0 launched on executor multi with executor id 1387 parsl.dataflow.dflow INFO: Standard output for task 1386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7a665c5b-174e-4acb-a179-3afa6aae4c1b_isrForCrosstalkSources_997029_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7a665c5b-174e-4acb-a179-3afa6aae4c1b_isrForCrosstalkSources_997029_22.stderr parsl.dataflow.dflow DEBUG: Task 1387 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1387 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1387 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1387 has memoization hash 4221e7f99ea754c01871e1cd590e8974 parsl.dataflow.memoization INFO: Task 1387 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1387 try 0 launched on executor multi with executor id 1388 parsl.dataflow.dflow INFO: Standard output for task 1387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/fb0797cb-7a01-4f99-8dd8-cad3e9afa593_isrForCrosstalkSources_996989_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/fb0797cb-7a01-4f99-8dd8-cad3e9afa593_isrForCrosstalkSources_996989_4.stderr parsl.dataflow.dflow DEBUG: Task 1388 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1388 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1388 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1388 has memoization hash c0ef4d3e6866adf9fb5d014583c8ea34 parsl.dataflow.memoization INFO: Task 1388 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1388 try 0 launched on executor multi with executor id 1389 parsl.dataflow.dflow INFO: Standard output for task 1388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b9a446b6-4f02-41cd-ae12-2d7873c01406_isrForCrosstalkSources_996957_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b9a446b6-4f02-41cd-ae12-2d7873c01406_isrForCrosstalkSources_996957_5.stderr parsl.dataflow.dflow DEBUG: Task 1389 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1389 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1389 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1389 has memoization hash bf721ce299aae462425042698261c545 parsl.dataflow.memoization INFO: Task 1389 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94a980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1389 try 0 launched on executor multi with executor id 1390 parsl.dataflow.dflow INFO: Standard output for task 1389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9316077c-55b7-4ef1-a82c-0ddba226aaf1_isrForCrosstalkSources_997004_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9316077c-55b7-4ef1-a82c-0ddba226aaf1_isrForCrosstalkSources_997004_30.stderr parsl.dataflow.dflow DEBUG: Task 1390 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1390 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1390 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1390 has memoization hash 5a687eb077449b49a69ef1723389c352 parsl.dataflow.memoization INFO: Task 1390 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94aa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1390 try 0 launched on executor multi with executor id 1391 parsl.dataflow.dflow INFO: Standard output for task 1390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/0a04c159-0612-4aef-a459-7339456bcb29_isrForCrosstalkSources_996994_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/0a04c159-0612-4aef-a459-7339456bcb29_isrForCrosstalkSources_996994_12.stderr parsl.dataflow.dflow DEBUG: Task 1391 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1391 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1391 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1391 has memoization hash bf47b680aa11f715f8f370b75ade4fb4 parsl.dataflow.memoization INFO: Task 1391 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94aac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1391 try 0 launched on executor multi with executor id 1392 parsl.dataflow.dflow INFO: Standard output for task 1391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/92384390-7534-4530-ba28-468aa034790f_isrForCrosstalkSources_997029_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/92384390-7534-4530-ba28-468aa034790f_isrForCrosstalkSources_997029_54.stderr parsl.dataflow.dflow DEBUG: Task 1392 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1392 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1392 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1392 has memoization hash 15d3e90e844c1f84eb55ee74df727cc2 parsl.dataflow.memoization INFO: Task 1392 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94ac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1392 try 0 launched on executor multi with executor id 1393 parsl.dataflow.dflow INFO: Standard output for task 1392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/8f56e55d-d91b-4dfc-a9fb-74f5dc64b2be_isrForCrosstalkSources_996986_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/8f56e55d-d91b-4dfc-a9fb-74f5dc64b2be_isrForCrosstalkSources_996986_9.stderr parsl.dataflow.dflow DEBUG: Task 1393 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1393 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1393 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1393 has memoization hash 12b7f6b0479ed7fda5cd23858723f535 parsl.dataflow.memoization INFO: Task 1393 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94ab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1393 try 0 launched on executor multi with executor id 1394 parsl.dataflow.dflow INFO: Standard output for task 1393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/f7bc368e-6763-427e-af8a-7974b6938e9c_isrForCrosstalkSources_996991_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/f7bc368e-6763-427e-af8a-7974b6938e9c_isrForCrosstalkSources_996991_32.stderr parsl.dataflow.dflow DEBUG: Task 1394 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1394 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1394 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1394 has memoization hash fb1c55b6e98df2c652929bf301005575 parsl.dataflow.memoization INFO: Task 1394 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94aca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1394 try 0 launched on executor multi with executor id 1395 parsl.dataflow.dflow INFO: Standard output for task 1394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/414194ed-bf69-4b74-8e9a-9c978e6ac727_isrForCrosstalkSources_997016_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/414194ed-bf69-4b74-8e9a-9c978e6ac727_isrForCrosstalkSources_997016_50.stderr parsl.dataflow.dflow DEBUG: Task 1395 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1395 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1395 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1395 has memoization hash 7399bbc347d91027cfb988d743aaaa4b parsl.dataflow.memoization INFO: Task 1395 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94ad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1395 try 0 launched on executor multi with executor id 1396 parsl.dataflow.dflow INFO: Standard output for task 1395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/16ec4af1-2f04-4137-a1cf-fc97ffc91923_isrForCrosstalkSources_996960_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/16ec4af1-2f04-4137-a1cf-fc97ffc91923_isrForCrosstalkSources_996960_14.stderr parsl.dataflow.dflow DEBUG: Task 1396 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1396 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1396 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1396 has memoization hash 4890712cdd87eec1965fe1e59aec1bbd parsl.dataflow.memoization INFO: Task 1396 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94ade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1396 try 0 launched on executor multi with executor id 1397 parsl.dataflow.dflow INFO: Standard output for task 1396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6a0d19b4-d024-44a1-acab-50d52ca77dbf_isrForCrosstalkSources_997015_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6a0d19b4-d024-44a1-acab-50d52ca77dbf_isrForCrosstalkSources_997015_25.stderr parsl.dataflow.dflow DEBUG: Task 1397 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1397 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1397 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1397 has memoization hash ba206af9550d5c1b6e6a5ee0a4740885 parsl.dataflow.memoization INFO: Task 1397 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94af20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1397 try 0 launched on executor multi with executor id 1398 parsl.dataflow.dflow INFO: Standard output for task 1397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/9698b4f4-5cf7-4f47-a439-f993aa9b2885_isrForCrosstalkSources_996999_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/9698b4f4-5cf7-4f47-a439-f993aa9b2885_isrForCrosstalkSources_996999_35.stderr parsl.dataflow.dflow DEBUG: Task 1398 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1398 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1398 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1398 has memoization hash 5095625fd8e4a54d07d2d99df62dcc30 parsl.dataflow.memoization INFO: Task 1398 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94ae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1398 try 0 launched on executor multi with executor id 1399 parsl.dataflow.dflow INFO: Standard output for task 1398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/c8159115-b233-40d9-a17d-cce4a6ca9bed_isrForCrosstalkSources_996953_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/c8159115-b233-40d9-a17d-cce4a6ca9bed_isrForCrosstalkSources_996953_61.stderr parsl.dataflow.dflow DEBUG: Task 1399 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1399 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1399 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1399 has memoization hash 62efe12f7f23f7ae81c8fd667a9d0b22 parsl.dataflow.memoization INFO: Task 1399 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94afc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1399 try 0 launched on executor multi with executor id 1400 parsl.dataflow.dflow INFO: Standard output for task 1399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/18c14827-03f2-436e-b7b8-0a7691298774_isrForCrosstalkSources_997020_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/18c14827-03f2-436e-b7b8-0a7691298774_isrForCrosstalkSources_997020_57.stderr parsl.dataflow.dflow DEBUG: Task 1400 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1400 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1400 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1400 has memoization hash 20913d264f337e2fd7a9b723c288a12b parsl.dataflow.memoization INFO: Task 1400 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1400 try 0 launched on executor multi with executor id 1401 parsl.dataflow.dflow INFO: Standard output for task 1400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/25688a2f-dc36-45c6-9aae-fd5be5e174b7_isrForCrosstalkSources_996955_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/25688a2f-dc36-45c6-9aae-fd5be5e174b7_isrForCrosstalkSources_996955_55.stderr parsl.dataflow.dflow DEBUG: Task 1401 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1401 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1401 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1401 has memoization hash c7d113270175c86603ce153e0a6bcb6b parsl.dataflow.memoization INFO: Task 1401 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1401 try 0 launched on executor multi with executor id 1402 parsl.dataflow.dflow INFO: Standard output for task 1401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5b85ef51-20f6-4dd0-b149-e1ba89f3345b_isrForCrosstalkSources_997018_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5b85ef51-20f6-4dd0-b149-e1ba89f3345b_isrForCrosstalkSources_997018_55.stderr parsl.dataflow.dflow DEBUG: Task 1402 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1402 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1402 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1402 has memoization hash d211e517c6b1e8208228299804103fd2 parsl.dataflow.memoization INFO: Task 1402 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1402 try 0 launched on executor multi with executor id 1403 parsl.dataflow.dflow INFO: Standard output for task 1402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9023dedb-9aae-49ad-8620-55c8757fd573_isrForCrosstalkSources_997004_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9023dedb-9aae-49ad-8620-55c8757fd573_isrForCrosstalkSources_997004_35.stderr parsl.dataflow.dflow DEBUG: Task 1403 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1403 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1403 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1403 has memoization hash 57005397032bc6b326cf6100da6e7196 parsl.dataflow.memoization INFO: Task 1403 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1403 try 0 launched on executor multi with executor id 1404 parsl.dataflow.dflow INFO: Standard output for task 1403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2801cc1a-59a4-4bea-a96b-fe7c3049c3a9_isrForCrosstalkSources_996963_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2801cc1a-59a4-4bea-a96b-fe7c3049c3a9_isrForCrosstalkSources_996963_16.stderr parsl.dataflow.dflow DEBUG: Task 1404 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1404 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1404 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1404 has memoization hash 7eecd3dfd62df61413f306b7aff9be53 parsl.dataflow.memoization INFO: Task 1404 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1404 try 0 launched on executor multi with executor id 1405 parsl.dataflow.dflow INFO: Standard output for task 1404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/eed05226-eeb0-4c76-985e-85e72a143f1a_isrForCrosstalkSources_996995_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/eed05226-eeb0-4c76-985e-85e72a143f1a_isrForCrosstalkSources_996995_20.stderr parsl.dataflow.dflow DEBUG: Task 1405 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1405 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1405 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1405 has memoization hash ff7cd3a91c7b44b6bb207de8e65e972f parsl.dataflow.memoization INFO: Task 1405 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1405 try 0 launched on executor multi with executor id 1406 parsl.dataflow.dflow INFO: Standard output for task 1405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/822a4c9f-d790-47e1-9a17-346730c563e2_isrForCrosstalkSources_996972_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/822a4c9f-d790-47e1-9a17-346730c563e2_isrForCrosstalkSources_996972_61.stderr parsl.dataflow.dflow DEBUG: Task 1406 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1406 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1406 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1406 has memoization hash e715d9f7b58bb2b5c22fbf46f6a0f299 parsl.dataflow.memoization INFO: Task 1406 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1406 try 0 launched on executor multi with executor id 1407 parsl.dataflow.dflow INFO: Standard output for task 1406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/47818732-8fe1-401b-b600-5cf5bb8fa224_isrForCrosstalkSources_997023_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/47818732-8fe1-401b-b600-5cf5bb8fa224_isrForCrosstalkSources_997023_53.stderr parsl.dataflow.dflow DEBUG: Task 1407 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1407 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1407 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1407 has memoization hash a8b9063be2f748564882b5f4edbc2048 parsl.dataflow.memoization INFO: Task 1407 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1407 try 0 launched on executor multi with executor id 1408 parsl.dataflow.dflow INFO: Standard output for task 1407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/19479e1f-749a-45b9-b97e-982eaea4f564_isrForCrosstalkSources_996974_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/19479e1f-749a-45b9-b97e-982eaea4f564_isrForCrosstalkSources_996974_13.stderr parsl.dataflow.dflow DEBUG: Task 1408 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1408 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1408 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1408 has memoization hash 67ca993e9f5be421661fbe536a1f0a92 parsl.dataflow.memoization INFO: Task 1408 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1408 try 0 launched on executor multi with executor id 1409 parsl.dataflow.dflow INFO: Standard output for task 1408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/33f5933e-4847-4e1a-916e-31293186b258_isrForCrosstalkSources_996960_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/33f5933e-4847-4e1a-916e-31293186b258_isrForCrosstalkSources_996960_48.stderr parsl.dataflow.dflow DEBUG: Task 1409 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1409 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1409 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1409 has memoization hash 98d29eb13687e55450de6ca164908dbe parsl.dataflow.memoization INFO: Task 1409 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1409 try 0 launched on executor multi with executor id 1410 parsl.dataflow.dflow INFO: Standard output for task 1409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2280f749-20ee-4d16-b7e2-e99784c2ba7e_isrForCrosstalkSources_996963_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2280f749-20ee-4d16-b7e2-e99784c2ba7e_isrForCrosstalkSources_996963_22.stderr parsl.dataflow.dflow DEBUG: Task 1410 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1410 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1410 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1410 has memoization hash ad28f06246dc0fb4905abfcea01bfd09 parsl.dataflow.memoization INFO: Task 1410 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1410 try 0 launched on executor multi with executor id 1411 parsl.dataflow.dflow INFO: Standard output for task 1410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e8f1bf39-b372-4ee9-b232-619e2b46abb8_isrForCrosstalkSources_996985_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e8f1bf39-b372-4ee9-b232-619e2b46abb8_isrForCrosstalkSources_996985_14.stderr parsl.dataflow.dflow DEBUG: Task 1411 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1411 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1411 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1411 has memoization hash f4a6bdc43f1cd6833a37f16c4129ed5f parsl.dataflow.memoization INFO: Task 1411 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1411 try 0 launched on executor multi with executor id 1412 parsl.dataflow.dflow INFO: Standard output for task 1411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/e2f4f58f-ddb3-4fd1-8f80-64282396a47d_isrForCrosstalkSources_997009_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/e2f4f58f-ddb3-4fd1-8f80-64282396a47d_isrForCrosstalkSources_997009_16.stderr parsl.dataflow.dflow DEBUG: Task 1412 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1412 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1412 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1412 has memoization hash 833452019e59752abd5a26971b331114 parsl.dataflow.memoization INFO: Task 1412 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1412 try 0 launched on executor multi with executor id 1413 parsl.dataflow.dflow INFO: Standard output for task 1412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/659bceb8-d19f-4820-9dde-ef368a426240_isrForCrosstalkSources_997033_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/659bceb8-d19f-4820-9dde-ef368a426240_isrForCrosstalkSources_997033_22.stderr parsl.dataflow.dflow DEBUG: Task 1413 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1413 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1413 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1413 has memoization hash ed7c587bbd45e48804c639b2233e57e2 parsl.dataflow.memoization INFO: Task 1413 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1413 try 0 launched on executor multi with executor id 1414 parsl.dataflow.dflow INFO: Standard output for task 1413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8f90ac0d-b200-4428-a52a-84f4f26401f6_isrForCrosstalkSources_997017_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8f90ac0d-b200-4428-a52a-84f4f26401f6_isrForCrosstalkSources_997017_16.stderr parsl.dataflow.dflow DEBUG: Task 1414 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1414 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1414 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1414 has memoization hash b069eb03f7cb92a70b70717cb7432ad5 parsl.dataflow.memoization INFO: Task 1414 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94bc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1414 try 0 launched on executor multi with executor id 1415 parsl.dataflow.dflow INFO: Standard output for task 1414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/93565768-9a9c-4bb5-99ca-7f2c1c5d459c_isrForCrosstalkSources_996978_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/93565768-9a9c-4bb5-99ca-7f2c1c5d459c_isrForCrosstalkSources_996978_3.stderr parsl.dataflow.dflow DEBUG: Task 1415 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1415 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1415 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1415 has memoization hash 450e4dcf7c12027164ffdebe479bf1b3 parsl.dataflow.memoization INFO: Task 1415 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1415 try 0 launched on executor multi with executor id 1416 parsl.dataflow.dflow INFO: Standard output for task 1415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/2de71e17-dc1c-4ee2-888e-34a601be2c4f_isrForCrosstalkSources_996978_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/2de71e17-dc1c-4ee2-888e-34a601be2c4f_isrForCrosstalkSources_996978_15.stderr parsl.dataflow.dflow DEBUG: Task 1416 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1416 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1416 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1416 has memoization hash 9d54c84c345104f90bdcbefaa3a74f7a parsl.dataflow.memoization INFO: Task 1416 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94bba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1416 try 0 launched on executor multi with executor id 1417 parsl.dataflow.dflow INFO: Standard output for task 1416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/5c0124ad-68d4-4a50-8c53-972c0cb5d0d4_isrForCrosstalkSources_997008_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/5c0124ad-68d4-4a50-8c53-972c0cb5d0d4_isrForCrosstalkSources_997008_62.stderr parsl.dataflow.dflow DEBUG: Task 1417 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1417 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1417 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1417 has memoization hash da9baeb60fdf6f71fb0c718b8a0afab3 parsl.dataflow.memoization INFO: Task 1417 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94bb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1417 try 0 launched on executor multi with executor id 1418 parsl.dataflow.dflow INFO: Standard output for task 1417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1b7a2568-6699-4e35-9de2-4449106e31c0_isrForCrosstalkSources_997022_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1b7a2568-6699-4e35-9de2-4449106e31c0_isrForCrosstalkSources_997022_36.stderr parsl.dataflow.dflow DEBUG: Task 1418 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1418 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1418 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1418 has memoization hash e5e66541d34a44b06411fc0d8ed06abd parsl.dataflow.memoization INFO: Task 1418 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94ba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:40 proc_lsst.multi:170 [INFO] [multi] job_id 17074857 proc_lsst.multi INFO: [multi] job_id 17074857 2024-03-15 13:49:40 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 18 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 18 2024-03-15 13:49:40 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:40 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074857 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074857 parsl.executors.status_handling DEBUG: Launched block 18 on executor multi with job ID 17074857 parsl.executors.status_handling INFO: Allocated block ID 19 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-19 2024-03-15 13:49:40 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=19 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-19 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=19 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-19 2024-03-15 13:49:40 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=19 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=19 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:40 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=19 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=19 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 18 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 18 2024-03-15 13:49:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:40 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=19 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=19 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1418 try 0 launched on executor multi with executor id 1419 parsl.dataflow.dflow INFO: Standard output for task 1418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7fce1d81-5833-42f4-9c1a-f7d5589c0bb0_isrForCrosstalkSources_997026_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7fce1d81-5833-42f4-9c1a-f7d5589c0bb0_isrForCrosstalkSources_997026_3.stderr parsl.dataflow.dflow DEBUG: Task 1419 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1419 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1419 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1419 has memoization hash 5921bd145e0d21c559a24d914fc5bc82 parsl.dataflow.memoization INFO: Task 1419 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e9c02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1419 try 0 launched on executor multi with executor id 1420 parsl.dataflow.dflow INFO: Standard output for task 1419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/db235a98-d7f9-4ba1-9828-aaed65b04635_isrForCrosstalkSources_997004_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/db235a98-d7f9-4ba1-9828-aaed65b04635_isrForCrosstalkSources_997004_20.stderr parsl.dataflow.dflow DEBUG: Task 1420 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1420 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1420 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1420 has memoization hash 3d9c4771e09c21ed5c586e4e5236b023 parsl.dataflow.memoization INFO: Task 1420 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a791d0b7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1420 try 0 launched on executor multi with executor id 1421 parsl.dataflow.dflow INFO: Standard output for task 1420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ebb7c666-a18b-4203-9f77-9da2e8301326_isrForCrosstalkSources_996986_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ebb7c666-a18b-4203-9f77-9da2e8301326_isrForCrosstalkSources_996986_52.stderr parsl.dataflow.dflow DEBUG: Task 1421 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1421 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1421 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1421 has memoization hash 6f139e8db0b7e944d66112f460528927 parsl.dataflow.memoization INFO: Task 1421 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1421 try 0 launched on executor multi with executor id 1422 parsl.dataflow.dflow INFO: Standard output for task 1421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/56ee6034-3834-408d-b5c1-b849fe471f02_isrForCrosstalkSources_997006_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/56ee6034-3834-408d-b5c1-b849fe471f02_isrForCrosstalkSources_997006_29.stderr parsl.dataflow.dflow DEBUG: Task 1422 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1422 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1422 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1422 has memoization hash 44594ebf3e84a9f54e8aee6b84f145fa parsl.dataflow.memoization INFO: Task 1422 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94b920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1422 try 0 launched on executor multi with executor id 1423 parsl.dataflow.dflow INFO: Standard output for task 1422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/35f8c91c-bbee-4882-8e87-4cab7ae55f82_isrForCrosstalkSources_996994_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/35f8c91c-bbee-4882-8e87-4cab7ae55f82_isrForCrosstalkSources_996994_52.stderr parsl.dataflow.dflow DEBUG: Task 1423 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1423 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1423 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1423 has memoization hash 73d6b589220411d1aa524db24bddf671 parsl.dataflow.memoization INFO: Task 1423 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b94bf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1423 try 0 launched on executor multi with executor id 1424 parsl.dataflow.dflow INFO: Standard output for task 1423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d5900cfa-5589-4615-bc59-2c4d93e172c4_isrForCrosstalkSources_996954_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d5900cfa-5589-4615-bc59-2c4d93e172c4_isrForCrosstalkSources_996954_23.stderr parsl.dataflow.dflow DEBUG: Task 1424 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1424 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1424 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1424 has memoization hash 427a81c43234651c4fd2add5a9fd5887 parsl.dataflow.memoization INFO: Task 1424 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2032e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1424 try 0 launched on executor multi with executor id 1425 parsl.dataflow.dflow INFO: Standard output for task 1424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/5c0be663-c3df-46da-85c1-0dd94d6decdd_isrForCrosstalkSources_996966_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/5c0be663-c3df-46da-85c1-0dd94d6decdd_isrForCrosstalkSources_996966_55.stderr parsl.dataflow.dflow DEBUG: Task 1425 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1425 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1425 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1425 has memoization hash 79c1486178072dc75487091d756bcefd parsl.dataflow.memoization INFO: Task 1425 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1425 try 0 launched on executor multi with executor id 1426 parsl.dataflow.dflow INFO: Standard output for task 1425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d8648b19-7986-4921-b48b-fbe28e395a88_isrForCrosstalkSources_996971_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d8648b19-7986-4921-b48b-fbe28e395a88_isrForCrosstalkSources_996971_14.stderr parsl.dataflow.dflow DEBUG: Task 1426 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1426 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1426 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1426 has memoization hash c835ee6a448f2704ad708594cefd1823 parsl.dataflow.memoization INFO: Task 1426 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1426 try 0 launched on executor multi with executor id 1427 parsl.dataflow.dflow INFO: Standard output for task 1426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/6ac4d563-708e-4b20-8146-81cceadccace_isrForCrosstalkSources_996981_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/6ac4d563-708e-4b20-8146-81cceadccace_isrForCrosstalkSources_996981_1.stderr parsl.dataflow.dflow DEBUG: Task 1427 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1427 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1427 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1427 has memoization hash 6b484f3655306f7d956439e1fd0736f3 parsl.dataflow.memoization INFO: Task 1427 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1427 try 0 launched on executor multi with executor id 1428 parsl.dataflow.dflow INFO: Standard output for task 1427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/0db53437-4385-4bdf-ae9c-8f339b70c4fb_isrForCrosstalkSources_996976_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/0db53437-4385-4bdf-ae9c-8f339b70c4fb_isrForCrosstalkSources_996976_27.stderr parsl.dataflow.dflow DEBUG: Task 1428 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1428 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1428 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1428 has memoization hash 6f542d40f51ff254b4815a66fa6d1a17 parsl.dataflow.memoization INFO: Task 1428 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2000e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1428 try 0 launched on executor multi with executor id 1429 parsl.dataflow.dflow INFO: Standard output for task 1428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0e6f4d16-643d-4a3e-9fa3-6b013649d20d_isrForCrosstalkSources_996973_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0e6f4d16-643d-4a3e-9fa3-6b013649d20d_isrForCrosstalkSources_996973_49.stderr parsl.dataflow.dflow DEBUG: Task 1429 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1429 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1429 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1429 has memoization hash a56643a067061c495a2e92de404993ad parsl.dataflow.memoization INFO: Task 1429 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1429 try 0 launched on executor multi with executor id 1430 parsl.dataflow.dflow INFO: Standard output for task 1429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e17bc60f-866c-4ba9-910b-c5b39d414685_isrForCrosstalkSources_996963_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e17bc60f-866c-4ba9-910b-c5b39d414685_isrForCrosstalkSources_996963_42.stderr parsl.dataflow.dflow DEBUG: Task 1430 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1430 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1430 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1430 has memoization hash 23bf5f8bbeb72596e100e36f29b10fb4 parsl.dataflow.memoization INFO: Task 1430 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2002c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1430 try 0 launched on executor multi with executor id 1431 parsl.dataflow.dflow INFO: Standard output for task 1430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/49579abe-5e87-42d9-a01f-e9edc10f8d2c_isrForCrosstalkSources_997017_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/49579abe-5e87-42d9-a01f-e9edc10f8d2c_isrForCrosstalkSources_997017_48.stderr parsl.dataflow.dflow DEBUG: Task 1431 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1431 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1431 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1431 has memoization hash d7bfdf04c2f6e3d79715411c424a25aa parsl.dataflow.memoization INFO: Task 1431 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1431 try 0 launched on executor multi with executor id 1432 parsl.dataflow.dflow INFO: Standard output for task 1431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/091f4dac-25c0-4485-8ff1-79d1cbc37fe9_isrForCrosstalkSources_997032_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/091f4dac-25c0-4485-8ff1-79d1cbc37fe9_isrForCrosstalkSources_997032_61.stderr parsl.dataflow.dflow DEBUG: Task 1432 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1432 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1432 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1432 has memoization hash dd050651c87c2dd48cb37bcde65323b5 parsl.dataflow.memoization INFO: Task 1432 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1432 try 0 launched on executor multi with executor id 1433 parsl.dataflow.dflow INFO: Standard output for task 1432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/cadc95bd-fab6-418c-b7a8-8599161e1785_isrForCrosstalkSources_996971_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/cadc95bd-fab6-418c-b7a8-8599161e1785_isrForCrosstalkSources_996971_26.stderr parsl.dataflow.dflow DEBUG: Task 1433 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1433 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1433 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1433 has memoization hash 06e76a83e6ba685a0f6c1a3515fe6e06 parsl.dataflow.memoization INFO: Task 1433 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2004a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1433 try 0 launched on executor multi with executor id 1434 parsl.dataflow.dflow INFO: Standard output for task 1433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c3e097b8-19df-4131-b342-e8aa1af89a6d_isrForCrosstalkSources_997014_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c3e097b8-19df-4131-b342-e8aa1af89a6d_isrForCrosstalkSources_997014_45.stderr parsl.dataflow.dflow DEBUG: Task 1434 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1434 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1434 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1434 has memoization hash 243886579a82c663a71dc25cd5e10a23 parsl.dataflow.memoization INFO: Task 1434 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2005e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1434 try 0 launched on executor multi with executor id 1435 parsl.dataflow.dflow INFO: Standard output for task 1434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/bdde0b14-ff4a-46af-8d7c-286ced3e6e81_isrForCrosstalkSources_996991_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/bdde0b14-ff4a-46af-8d7c-286ced3e6e81_isrForCrosstalkSources_996991_49.stderr parsl.dataflow.dflow DEBUG: Task 1435 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1435 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1435 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1435 has memoization hash 7e5a150098f4b823d2fa7c506315641b parsl.dataflow.memoization INFO: Task 1435 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1435 try 0 launched on executor multi with executor id 1436 parsl.dataflow.dflow INFO: Standard output for task 1435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/8a9296c4-43c5-487a-b6ca-a96a42646041_isrForCrosstalkSources_997022_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/8a9296c4-43c5-487a-b6ca-a96a42646041_isrForCrosstalkSources_997022_34.stderr parsl.dataflow.dflow DEBUG: Task 1436 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1436 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1436 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1436 has memoization hash c9798a5dc1316cf275b51cd1b89c390c parsl.dataflow.memoization INFO: Task 1436 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1436 try 0 launched on executor multi with executor id 1437 parsl.dataflow.dflow INFO: Standard output for task 1436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5263eab8-8ae6-4d7e-b268-cad356873830_isrForCrosstalkSources_996988_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5263eab8-8ae6-4d7e-b268-cad356873830_isrForCrosstalkSources_996988_6.stderr parsl.dataflow.dflow DEBUG: Task 1437 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1437 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1437 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1437 has memoization hash 9005d7ac52ac1628661863f900a7f766 parsl.dataflow.memoization INFO: Task 1437 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2007c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1437 try 0 launched on executor multi with executor id 1438 parsl.dataflow.dflow INFO: Standard output for task 1437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0bd0ed93-3097-4839-bbe7-f3506647e26e_isrForCrosstalkSources_997025_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0bd0ed93-3097-4839-bbe7-f3506647e26e_isrForCrosstalkSources_997025_4.stderr parsl.dataflow.dflow DEBUG: Task 1438 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1438 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1438 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1438 has memoization hash be027937c1a713ace8b5f166b37ec862 parsl.dataflow.memoization INFO: Task 1438 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1438 try 0 launched on executor multi with executor id 1439 parsl.dataflow.dflow INFO: Standard output for task 1438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/804181a3-e576-416a-be8c-2fd45bc198d5_isrForCrosstalkSources_997029_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/804181a3-e576-416a-be8c-2fd45bc198d5_isrForCrosstalkSources_997029_4.stderr parsl.dataflow.dflow DEBUG: Task 1439 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1439 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1439 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1439 has memoization hash 87a2119cfc3916a36c8cb88bfea1a5cd parsl.dataflow.memoization INFO: Task 1439 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1439 try 0 launched on executor multi with executor id 1440 parsl.dataflow.dflow INFO: Standard output for task 1439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a4087502-7e6b-4736-aee4-3c0f28dcea6f_isrForCrosstalkSources_997013_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a4087502-7e6b-4736-aee4-3c0f28dcea6f_isrForCrosstalkSources_997013_60.stderr parsl.dataflow.dflow DEBUG: Task 1440 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1440 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1440 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1440 has memoization hash 7f126f589e487844808c2f26d44f316c parsl.dataflow.memoization INFO: Task 1440 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1440 try 0 launched on executor multi with executor id 1441 parsl.dataflow.dflow INFO: Standard output for task 1440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5e2b28dd-62d1-403b-bf11-822eb635225c_isrForCrosstalkSources_997033_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5e2b28dd-62d1-403b-bf11-822eb635225c_isrForCrosstalkSources_997033_5.stderr parsl.dataflow.dflow DEBUG: Task 1441 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1441 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1441 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1441 has memoization hash b344e19ebac188831ad5b815121894d4 parsl.dataflow.memoization INFO: Task 1441 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2009a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1441 try 0 launched on executor multi with executor id 1442 parsl.dataflow.dflow INFO: Standard output for task 1441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/8ee93071-70bb-4821-8003-2144f25e632b_isrForCrosstalkSources_996975_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/8ee93071-70bb-4821-8003-2144f25e632b_isrForCrosstalkSources_996975_11.stderr parsl.dataflow.dflow DEBUG: Task 1442 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1442 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1442 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1442 has memoization hash cf2c934f9cad433a1dc92feda2eb3efe parsl.dataflow.memoization INFO: Task 1442 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1442 try 0 launched on executor multi with executor id 1443 parsl.dataflow.dflow INFO: Standard output for task 1442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/8e290043-a8ed-4c9c-b3b5-e568e2f3bf52_isrForCrosstalkSources_996983_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/8e290043-a8ed-4c9c-b3b5-e568e2f3bf52_isrForCrosstalkSources_996983_20.stderr parsl.dataflow.dflow DEBUG: Task 1443 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1443 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1443 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1443 has memoization hash a397295d804baf82f60a76cd7c3b042d parsl.dataflow.memoization INFO: Task 1443 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1443 try 0 launched on executor multi with executor id 1444 parsl.dataflow.dflow INFO: Standard output for task 1443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/5fe2c676-5a2c-4d9c-8c42-5a6b9e8bb768_isrForCrosstalkSources_996989_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/5fe2c676-5a2c-4d9c-8c42-5a6b9e8bb768_isrForCrosstalkSources_996989_61.stderr parsl.dataflow.dflow DEBUG: Task 1444 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1444 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1444 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1444 has memoization hash 6c108ddaa3b83768406f9102c21577ad parsl.dataflow.memoization INFO: Task 1444 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1444 try 0 launched on executor multi with executor id 1445 parsl.dataflow.dflow INFO: Standard output for task 1444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/46c978cf-bbd3-4948-8a7d-453ea6ee5a12_isrForCrosstalkSources_996955_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/46c978cf-bbd3-4948-8a7d-453ea6ee5a12_isrForCrosstalkSources_996955_61.stderr parsl.dataflow.dflow DEBUG: Task 1445 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1445 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1445 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1445 has memoization hash 8e2a340ab802d62285df871fb8922964 parsl.dataflow.memoization INFO: Task 1445 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1445 try 0 launched on executor multi with executor id 1446 parsl.dataflow.dflow INFO: Standard output for task 1445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/06ad926e-1cc3-4c30-b565-4b18252b42fb_isrForCrosstalkSources_997005_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/06ad926e-1cc3-4c30-b565-4b18252b42fb_isrForCrosstalkSources_997005_57.stderr parsl.dataflow.dflow DEBUG: Task 1446 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1446 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1446 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1446 has memoization hash 49c899e0a7c7b60f6829270dfbd9e210 parsl.dataflow.memoization INFO: Task 1446 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1446 try 0 launched on executor multi with executor id 1447 parsl.dataflow.dflow INFO: Standard output for task 1446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/877496fa-5f51-42d4-a303-0050b2e70dc8_isrForCrosstalkSources_997031_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/877496fa-5f51-42d4-a303-0050b2e70dc8_isrForCrosstalkSources_997031_59.stderr parsl.dataflow.dflow DEBUG: Task 1447 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1447 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1447 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1447 has memoization hash 5517f37334c2ff05784fc0c60334a8e3 parsl.dataflow.memoization INFO: Task 1447 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1447 try 0 launched on executor multi with executor id 1448 parsl.dataflow.dflow INFO: Standard output for task 1447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/82a2ff66-0393-4d04-be28-e30052f6bc4e_isrForCrosstalkSources_997006_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/82a2ff66-0393-4d04-be28-e30052f6bc4e_isrForCrosstalkSources_997006_3.stderr parsl.dataflow.dflow DEBUG: Task 1448 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1448 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1448 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1448 has memoization hash 878a40127a6cadbfb4b4bfc995d5ef86 parsl.dataflow.memoization INFO: Task 1448 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1448 try 0 launched on executor multi with executor id 1449 parsl.dataflow.dflow INFO: Standard output for task 1448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4b75651e-11a4-4610-bb1f-acb6d77af405_isrForCrosstalkSources_997020_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4b75651e-11a4-4610-bb1f-acb6d77af405_isrForCrosstalkSources_997020_33.stderr parsl.dataflow.dflow DEBUG: Task 1449 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1449 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1449 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1449 has memoization hash cf1d3faf47516e0e8de6a183341615e9 parsl.dataflow.memoization INFO: Task 1449 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1449 try 0 launched on executor multi with executor id 1450 parsl.dataflow.dflow INFO: Standard output for task 1449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/20b3ea58-0211-4f49-b973-2ac7f3e7aa4e_isrForCrosstalkSources_996954_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/20b3ea58-0211-4f49-b973-2ac7f3e7aa4e_isrForCrosstalkSources_996954_62.stderr parsl.dataflow.dflow DEBUG: Task 1450 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1450 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1450 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1450 has memoization hash b14f3708bd704a8193719db6268d4f23 parsl.dataflow.memoization INFO: Task 1450 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1450 try 0 launched on executor multi with executor id 1451 parsl.dataflow.dflow INFO: Standard output for task 1450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/575832a9-bb14-42b6-9079-b757a8050eda_isrForCrosstalkSources_997005_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/575832a9-bb14-42b6-9079-b757a8050eda_isrForCrosstalkSources_997005_26.stderr parsl.dataflow.dflow DEBUG: Task 1451 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1451 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1451 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1451 has memoization hash 65639748ee3712b4a899e89078e714b7 parsl.dataflow.memoization INFO: Task 1451 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1451 try 0 launched on executor multi with executor id 1452 parsl.dataflow.dflow INFO: Standard output for task 1451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9cacc2a1-3ff6-4767-b9bb-a9467da2e947_isrForCrosstalkSources_997008_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9cacc2a1-3ff6-4767-b9bb-a9467da2e947_isrForCrosstalkSources_997008_29.stderr parsl.dataflow.dflow DEBUG: Task 1452 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1452 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1452 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1452 has memoization hash 2d391cc12b4d8d548d6dd818e1941385 parsl.dataflow.memoization INFO: Task 1452 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1452 try 0 launched on executor multi with executor id 1453 parsl.dataflow.dflow INFO: Standard output for task 1452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/9f36cc29-7cd2-4d10-a648-dc4d2a8a3c00_isrForCrosstalkSources_996954_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/9f36cc29-7cd2-4d10-a648-dc4d2a8a3c00_isrForCrosstalkSources_996954_60.stderr parsl.dataflow.dflow DEBUG: Task 1453 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1453 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1453 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1453 has memoization hash fd52390ee9077bd463798d3c9102f3f2 parsl.dataflow.memoization INFO: Task 1453 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1453 try 0 launched on executor multi with executor id 1454 parsl.dataflow.dflow INFO: Standard output for task 1453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/82612bfa-46af-4e40-a5f6-cf89e396b28d_isrForCrosstalkSources_997019_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/82612bfa-46af-4e40-a5f6-cf89e396b28d_isrForCrosstalkSources_997019_47.stderr parsl.dataflow.dflow DEBUG: Task 1454 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1454 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1454 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1454 has memoization hash bfb454714d1c3c277c3c2568ad80c59f parsl.dataflow.memoization INFO: Task 1454 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2011c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1454 try 0 launched on executor multi with executor id 1455 parsl.dataflow.dflow INFO: Standard output for task 1454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2692bd89-11ba-48f8-9472-1cd715245cec_isrForCrosstalkSources_996995_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2692bd89-11ba-48f8-9472-1cd715245cec_isrForCrosstalkSources_996995_52.stderr parsl.dataflow.dflow DEBUG: Task 1455 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1455 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1455 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1455 has memoization hash 44f96d545d4b16eb276dcdb471cb6015 parsl.dataflow.memoization INFO: Task 1455 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1455 try 0 launched on executor multi with executor id 1456 parsl.dataflow.dflow INFO: Standard output for task 1455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/8b75221c-7769-481f-bce6-a1afd33c639f_isrForCrosstalkSources_996989_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/8b75221c-7769-481f-bce6-a1afd33c639f_isrForCrosstalkSources_996989_25.stderr parsl.dataflow.dflow DEBUG: Task 1456 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1456 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1456 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1456 has memoization hash 808ab62a3f501854713d1baecd85fdbb parsl.dataflow.memoization INFO: Task 1456 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2013a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1456 try 0 launched on executor multi with executor id 1457 parsl.dataflow.dflow INFO: Standard output for task 1456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c09a8288-8766-4664-b39d-e5819faf716f_isrForCrosstalkSources_996993_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c09a8288-8766-4664-b39d-e5819faf716f_isrForCrosstalkSources_996993_29.stderr parsl.dataflow.dflow DEBUG: Task 1457 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1457 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1457 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1457 has memoization hash b8265ff696fd469fafb401c668e60b19 parsl.dataflow.memoization INFO: Task 1457 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1457 try 0 launched on executor multi with executor id 1458 parsl.dataflow.dflow INFO: Standard output for task 1457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/238de23f-fab5-4f4d-bd17-a07859b0aa58_isrForCrosstalkSources_996963_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/238de23f-fab5-4f4d-bd17-a07859b0aa58_isrForCrosstalkSources_996963_10.stderr parsl.dataflow.dflow DEBUG: Task 1458 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1458 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1458 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1458 has memoization hash 7c2cc1030def1b7e37f26b719922148f parsl.dataflow.memoization INFO: Task 1458 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1458 try 0 launched on executor multi with executor id 1459 parsl.dataflow.dflow INFO: Standard output for task 1458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/68b04fa4-fa45-41d2-90c7-764fab9d962c_isrForCrosstalkSources_997033_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/68b04fa4-fa45-41d2-90c7-764fab9d962c_isrForCrosstalkSources_997033_51.stderr parsl.dataflow.dflow DEBUG: Task 1459 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1459 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1459 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1459 has memoization hash 340bfc370f10f2cc14d998c22e182b4b parsl.dataflow.memoization INFO: Task 1459 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2014e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1459 try 0 launched on executor multi with executor id 1460 parsl.dataflow.dflow INFO: Standard output for task 1459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/07b78570-8e68-483b-a9e8-0d81a6c662fe_isrForCrosstalkSources_996957_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/07b78570-8e68-483b-a9e8-0d81a6c662fe_isrForCrosstalkSources_996957_37.stderr parsl.dataflow.dflow DEBUG: Task 1460 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1460 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1460 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1460 has memoization hash 1cc1301c334c27dceb7cd56b75c472e1 parsl.dataflow.memoization INFO: Task 1460 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1460 try 0 launched on executor multi with executor id 1461 parsl.dataflow.dflow INFO: Standard output for task 1460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8eda186b-bccc-49f4-a8ea-ea27d9c35516_isrForCrosstalkSources_996960_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8eda186b-bccc-49f4-a8ea-ea27d9c35516_isrForCrosstalkSources_996960_55.stderr parsl.dataflow.dflow DEBUG: Task 1461 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1461 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1461 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1461 has memoization hash 125507546757ef2ffec96d26caa92fb8 parsl.dataflow.memoization INFO: Task 1461 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2016c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1461 try 0 launched on executor multi with executor id 1462 parsl.dataflow.dflow INFO: Standard output for task 1461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9e06c4bd-e478-4f6f-97e4-18168a27a93e_isrForCrosstalkSources_996986_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9e06c4bd-e478-4f6f-97e4-18168a27a93e_isrForCrosstalkSources_996986_53.stderr parsl.dataflow.dflow DEBUG: Task 1462 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1462 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1462 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1462 has memoization hash 4497b19203990a053cf2a76fa21444e8 parsl.dataflow.memoization INFO: Task 1462 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1462 try 0 launched on executor multi with executor id 1463 parsl.dataflow.dflow INFO: Standard output for task 1462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/b1ca0f5d-c56b-473b-a7af-77d0ce9aeade_isrForCrosstalkSources_996987_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/b1ca0f5d-c56b-473b-a7af-77d0ce9aeade_isrForCrosstalkSources_996987_39.stderr parsl.dataflow.dflow DEBUG: Task 1463 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1463 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1463 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1463 has memoization hash 13bc7bec38b4d858f1977e465f47febc parsl.dataflow.memoization INFO: Task 1463 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1463 try 0 launched on executor multi with executor id 1464 parsl.dataflow.dflow INFO: Standard output for task 1463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bc7bc3c0-f50d-476c-a7a2-c566c4e16966_isrForCrosstalkSources_997014_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bc7bc3c0-f50d-476c-a7a2-c566c4e16966_isrForCrosstalkSources_997014_62.stderr parsl.dataflow.dflow DEBUG: Task 1464 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1464 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1464 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1464 has memoization hash 5e859ef9aa5d28521d75dc44574fce99 parsl.dataflow.memoization INFO: Task 1464 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1464 try 0 launched on executor multi with executor id 1465 parsl.dataflow.dflow INFO: Standard output for task 1464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/d7df6361-4ed5-4a7e-aea0-e743b3a050e6_isrForCrosstalkSources_997012_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/d7df6361-4ed5-4a7e-aea0-e743b3a050e6_isrForCrosstalkSources_997012_41.stderr parsl.dataflow.dflow DEBUG: Task 1465 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1465 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1465 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1465 has memoization hash 4cf7670220d3412984a824c36a86afc2 parsl.dataflow.memoization INFO: Task 1465 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2018a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1465 try 0 launched on executor multi with executor id 1466 parsl.dataflow.dflow INFO: Standard output for task 1465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/60687ebc-5ce1-4411-b6ea-651b43960473_isrForCrosstalkSources_996976_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/60687ebc-5ce1-4411-b6ea-651b43960473_isrForCrosstalkSources_996976_28.stderr parsl.dataflow.dflow DEBUG: Task 1466 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1466 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1466 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1466 has memoization hash e3a6713910102f057b5d05b2b288a3bf parsl.dataflow.memoization INFO: Task 1466 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2019e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1466 try 0 launched on executor multi with executor id 1467 parsl.dataflow.dflow INFO: Standard output for task 1466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ce9c5f99-8541-4044-9e74-ea6c19afa2c1_isrForCrosstalkSources_996992_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ce9c5f99-8541-4044-9e74-ea6c19afa2c1_isrForCrosstalkSources_996992_3.stderr parsl.dataflow.dflow DEBUG: Task 1467 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1467 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1467 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1467 has memoization hash 749aed52bbd609196bf417f5c89dcff1 parsl.dataflow.memoization INFO: Task 1467 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1467 try 0 launched on executor multi with executor id 1468 parsl.dataflow.dflow INFO: Standard output for task 1467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ab822ff8-a111-46f8-9f18-5b5224cb6d32_isrForCrosstalkSources_997000_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ab822ff8-a111-46f8-9f18-5b5224cb6d32_isrForCrosstalkSources_997000_25.stderr parsl.dataflow.dflow DEBUG: Task 1468 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1468 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1468 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1468 has memoization hash 8f9bff68533bcb2ea9d35d8cc003ac50 parsl.dataflow.memoization INFO: Task 1468 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1468 try 0 launched on executor multi with executor id 1469 parsl.dataflow.dflow INFO: Standard output for task 1468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/294e7cc6-b5bf-4583-8371-91c5e8bbf90f_isrForCrosstalkSources_996999_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/294e7cc6-b5bf-4583-8371-91c5e8bbf90f_isrForCrosstalkSources_996999_53.stderr parsl.dataflow.dflow DEBUG: Task 1469 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1469 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1469 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1469 has memoization hash 8eda178aebc06f0d747517e9e7e7516e parsl.dataflow.memoization INFO: Task 1469 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1469 try 0 launched on executor multi with executor id 1470 parsl.dataflow.dflow INFO: Standard output for task 1469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/8adfd2bb-6df1-41ac-969e-e4456dfc17b8_isrForCrosstalkSources_996966_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/8adfd2bb-6df1-41ac-969e-e4456dfc17b8_isrForCrosstalkSources_996966_42.stderr parsl.dataflow.dflow DEBUG: Task 1470 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1470 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1470 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1470 has memoization hash 8732666ebc8ccd6143f3d9bc105fcfaa parsl.dataflow.memoization INFO: Task 1470 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1470 try 0 launched on executor multi with executor id 1471 parsl.dataflow.dflow INFO: Standard output for task 1470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/141df55d-41f3-474d-bc36-28ecbc6feacd_isrForCrosstalkSources_997018_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/141df55d-41f3-474d-bc36-28ecbc6feacd_isrForCrosstalkSources_997018_1.stderr parsl.dataflow.dflow DEBUG: Task 1471 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1471 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1471 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1471 has memoization hash e71bc14905838240bd57868e82916061 parsl.dataflow.memoization INFO: Task 1471 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1471 try 0 launched on executor multi with executor id 1472 parsl.dataflow.dflow INFO: Standard output for task 1471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/fa482bd4-364e-4e34-945e-15163e67b36b_isrForCrosstalkSources_997016_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/fa482bd4-364e-4e34-945e-15163e67b36b_isrForCrosstalkSources_997016_25.stderr parsl.dataflow.dflow DEBUG: Task 1472 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1472 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1472 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1472 has memoization hash e9882c6ebbd8c1fdb477ef48aa0c97d6 parsl.dataflow.memoization INFO: Task 1472 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1472 try 0 launched on executor multi with executor id 1473 parsl.dataflow.dflow INFO: Standard output for task 1472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/9502aa45-0bc2-4e84-a682-55f811923112_isrForCrosstalkSources_996990_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/9502aa45-0bc2-4e84-a682-55f811923112_isrForCrosstalkSources_996990_29.stderr parsl.dataflow.dflow DEBUG: Task 1473 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1473 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1473 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1473 has memoization hash 4f5f28c652a32d72f0642ecd8b616c8b parsl.dataflow.memoization INFO: Task 1473 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1473 try 0 launched on executor multi with executor id 1474 parsl.dataflow.dflow INFO: Standard output for task 1473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/bab4ce2f-9a0d-45c4-9c1b-b584de2f770f_isrForCrosstalkSources_996983_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/bab4ce2f-9a0d-45c4-9c1b-b584de2f770f_isrForCrosstalkSources_996983_13.stderr parsl.dataflow.dflow DEBUG: Task 1474 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1474 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1474 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1474 has memoization hash 934095b8a9f2f79992f36c27b48c2df7 parsl.dataflow.memoization INFO: Task 1474 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1474 try 0 launched on executor multi with executor id 1475 parsl.dataflow.dflow INFO: Standard output for task 1474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a53f53ef-c6c5-4bb7-a15d-e032a9f8478d_isrForCrosstalkSources_996952_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a53f53ef-c6c5-4bb7-a15d-e032a9f8478d_isrForCrosstalkSources_996952_62.stderr parsl.dataflow.dflow DEBUG: Task 1475 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1475 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1475 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1475 has memoization hash 8fd2cf7d8c96533f58083e45f0d2a744 parsl.dataflow.memoization INFO: Task 1475 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1475 try 0 launched on executor multi with executor id 1476 parsl.dataflow.dflow INFO: Standard output for task 1475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/859b595a-943e-4ce4-ac1c-095491edbe3a_isrForCrosstalkSources_996966_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/859b595a-943e-4ce4-ac1c-095491edbe3a_isrForCrosstalkSources_996966_18.stderr parsl.dataflow.dflow DEBUG: Task 1476 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1476 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1476 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1476 has memoization hash 1af31adb7ddea1b630e1aecef47d55a9 parsl.dataflow.memoization INFO: Task 1476 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1476 try 0 launched on executor multi with executor id 1477 parsl.dataflow.dflow INFO: Standard output for task 1476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/793a4c3b-3dff-4c9b-a436-d5dde2c6c08f_isrForCrosstalkSources_996989_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/793a4c3b-3dff-4c9b-a436-d5dde2c6c08f_isrForCrosstalkSources_996989_54.stderr parsl.dataflow.dflow DEBUG: Task 1477 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1477 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1477 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1477 has memoization hash 32767ce342b63d7d06270c27ed1829a9 parsl.dataflow.memoization INFO: Task 1477 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2031a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1477 try 0 launched on executor multi with executor id 1478 parsl.dataflow.dflow INFO: Standard output for task 1477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f640112d-8bd8-4b0e-b865-bef040b94cdb_isrForCrosstalkSources_996986_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f640112d-8bd8-4b0e-b865-bef040b94cdb_isrForCrosstalkSources_996986_26.stderr parsl.dataflow.dflow DEBUG: Task 1478 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1478 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1478 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1478 has memoization hash ceb6dcc52d1260299b60ef11b26eb5a9 parsl.dataflow.memoization INFO: Task 1478 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1478 try 0 launched on executor multi with executor id 1479 parsl.dataflow.dflow INFO: Standard output for task 1478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d8cebc9a-63ee-4871-bebd-001d96c2e1d0_isrForCrosstalkSources_997010_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d8cebc9a-63ee-4871-bebd-001d96c2e1d0_isrForCrosstalkSources_997010_54.stderr parsl.dataflow.dflow DEBUG: Task 1479 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1479 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1479 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1479 has memoization hash 19bbaf1e3be743f7a02b617ae6b3124a parsl.dataflow.memoization INFO: Task 1479 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1479 try 0 launched on executor multi with executor id 1480 parsl.dataflow.dflow INFO: Standard output for task 1479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4e6458bd-8b01-42da-b302-2dd80ab28eb2_isrForCrosstalkSources_997033_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4e6458bd-8b01-42da-b302-2dd80ab28eb2_isrForCrosstalkSources_997033_15.stderr parsl.dataflow.dflow DEBUG: Task 1480 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1480 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1480 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1480 has memoization hash cf9021cec93157d93e172cdec4b2d03f parsl.dataflow.memoization INFO: Task 1480 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1480 try 0 launched on executor multi with executor id 1481 parsl.dataflow.dflow INFO: Standard output for task 1480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a647783f-84a7-4d8a-b567-bc6aecf8b923_isrForCrosstalkSources_997013_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a647783f-84a7-4d8a-b567-bc6aecf8b923_isrForCrosstalkSources_997013_40.stderr parsl.dataflow.dflow DEBUG: Task 1481 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1481 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1481 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1481 has memoization hash 7d0942cce8670882e8b3a5edcdbeb85a parsl.dataflow.memoization INFO: Task 1481 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1481 try 0 launched on executor multi with executor id 1482 parsl.dataflow.dflow INFO: Standard output for task 1481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/c1e35aa5-9e54-4994-804d-e41e5c0258cf_isrForCrosstalkSources_996984_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/c1e35aa5-9e54-4994-804d-e41e5c0258cf_isrForCrosstalkSources_996984_43.stderr parsl.dataflow.dflow DEBUG: Task 1482 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1482 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1482 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1482 has memoization hash 4287ac668171ab145f786434e1e5a7d4 parsl.dataflow.memoization INFO: Task 1482 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1482 try 0 launched on executor multi with executor id 1483 parsl.dataflow.dflow INFO: Standard output for task 1482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/cffa0df5-8126-4621-9b61-f83919bbf476_isrForCrosstalkSources_996972_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/cffa0df5-8126-4621-9b61-f83919bbf476_isrForCrosstalkSources_996972_35.stderr parsl.dataflow.dflow DEBUG: Task 1483 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1483 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1483 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1483 has memoization hash 909eb0b7c39f886f85abbbe5eb13e238 parsl.dataflow.memoization INFO: Task 1483 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1483 try 0 launched on executor multi with executor id 1484 parsl.dataflow.dflow INFO: Standard output for task 1483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e4e4c221-6240-48a5-ab8d-541ecea86054_isrForCrosstalkSources_997019_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e4e4c221-6240-48a5-ab8d-541ecea86054_isrForCrosstalkSources_997019_20.stderr parsl.dataflow.dflow DEBUG: Task 1484 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1484 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1484 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1484 has memoization hash e66ae865b008c52f96a909aecdf8c9de parsl.dataflow.memoization INFO: Task 1484 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1484 try 0 launched on executor multi with executor id 1485 parsl.dataflow.dflow INFO: Standard output for task 1484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/679b074c-5daf-4ede-858e-0ba2a22a2eaf_isrForCrosstalkSources_997002_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/679b074c-5daf-4ede-858e-0ba2a22a2eaf_isrForCrosstalkSources_997002_11.stderr parsl.dataflow.dflow DEBUG: Task 1485 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1485 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1485 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1485 has memoization hash 337b894c9f6b8d71ab99aa205acf52ff parsl.dataflow.memoization INFO: Task 1485 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1485 try 0 launched on executor multi with executor id 1486 parsl.dataflow.dflow INFO: Standard output for task 1485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/085e3c5d-987f-4fec-b57e-fece8ac22659_isrForCrosstalkSources_997023_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/085e3c5d-987f-4fec-b57e-fece8ac22659_isrForCrosstalkSources_997023_33.stderr parsl.dataflow.dflow DEBUG: Task 1486 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1486 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1486 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1486 has memoization hash afb40e504b943ca5345f8930525b956b parsl.dataflow.memoization INFO: Task 1486 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1486 try 0 launched on executor multi with executor id 1487 parsl.dataflow.dflow INFO: Standard output for task 1486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/9abbef3f-917f-4d8d-902a-1c308264b93b_isrForCrosstalkSources_996962_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/9abbef3f-917f-4d8d-902a-1c308264b93b_isrForCrosstalkSources_996962_8.stderr parsl.dataflow.dflow DEBUG: Task 1487 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1487 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1487 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1487 has memoization hash 5aa7c960693c0ef04e0b3026374a2ced parsl.dataflow.memoization INFO: Task 1487 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1487 try 0 launched on executor multi with executor id 1488 parsl.dataflow.dflow INFO: Standard output for task 1487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/8c6aa64d-233c-4e65-b200-df3ffad5b27f_isrForCrosstalkSources_996986_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/8c6aa64d-233c-4e65-b200-df3ffad5b27f_isrForCrosstalkSources_996986_1.stderr parsl.dataflow.dflow DEBUG: Task 1488 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1488 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1488 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1488 has memoization hash 2120924a7db01a6810a1cdc8049a0b63 parsl.dataflow.memoization INFO: Task 1488 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1488 try 0 launched on executor multi with executor id 1489 parsl.dataflow.dflow INFO: Standard output for task 1488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9c070e48-49c2-4c60-8a54-86a5057b42ac_isrForCrosstalkSources_997008_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9c070e48-49c2-4c60-8a54-86a5057b42ac_isrForCrosstalkSources_997008_54.stderr parsl.dataflow.dflow DEBUG: Task 1489 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1489 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1489 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1489 has memoization hash 42dddcf69327b975664e8da93480a861 parsl.dataflow.memoization INFO: Task 1489 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1489 try 0 launched on executor multi with executor id 1490 parsl.dataflow.dflow INFO: Standard output for task 1489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/7a305445-3390-4e11-b11e-6237dec4b52c_isrForCrosstalkSources_997012_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/7a305445-3390-4e11-b11e-6237dec4b52c_isrForCrosstalkSources_997012_34.stderr parsl.dataflow.dflow DEBUG: Task 1490 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1490 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1490 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1490 has memoization hash 8b7bd26232f3f38b15920b34a314ca49 parsl.dataflow.memoization INFO: Task 1490 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1490 try 0 launched on executor multi with executor id 1491 parsl.dataflow.dflow INFO: Standard output for task 1490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/498826c6-ad2c-4dcc-9b29-b2301f0b376f_isrForCrosstalkSources_996983_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/498826c6-ad2c-4dcc-9b29-b2301f0b376f_isrForCrosstalkSources_996983_28.stderr parsl.dataflow.dflow DEBUG: Task 1491 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1491 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1491 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1491 has memoization hash 922e5cbd2e059086cac133e74b2d1096 parsl.dataflow.memoization INFO: Task 1491 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2028e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1491 try 0 launched on executor multi with executor id 1492 parsl.dataflow.dflow INFO: Standard output for task 1491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/32e5cf89-826a-40fd-984b-4fb023f09df1_isrForCrosstalkSources_997024_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/32e5cf89-826a-40fd-984b-4fb023f09df1_isrForCrosstalkSources_997024_59.stderr parsl.dataflow.dflow DEBUG: Task 1492 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1492 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1492 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1492 has memoization hash ca25ab544dcca7a23d7ca38a9f1fa4c6 parsl.dataflow.memoization INFO: Task 1492 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1492 try 0 launched on executor multi with executor id 1493 parsl.dataflow.dflow INFO: Standard output for task 1492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/dc252747-9ac5-4548-9124-537252dffb72_isrForCrosstalkSources_996970_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/dc252747-9ac5-4548-9124-537252dffb72_isrForCrosstalkSources_996970_25.stderr parsl.dataflow.dflow DEBUG: Task 1493 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1493 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1493 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1493 has memoization hash 686d2a7bee110a3bebe495945f222d06 parsl.dataflow.memoization INFO: Task 1493 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2027a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1493 try 0 launched on executor multi with executor id 1494 parsl.dataflow.dflow INFO: Standard output for task 1493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/65963caf-d448-41c9-a4b7-29672731979e_isrForCrosstalkSources_997006_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/65963caf-d448-41c9-a4b7-29672731979e_isrForCrosstalkSources_997006_7.stderr parsl.dataflow.dflow DEBUG: Task 1494 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1494 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1494 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1494 has memoization hash 220c4193615a10f1847c41e5428a0dba parsl.dataflow.memoization INFO: Task 1494 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1494 try 0 launched on executor multi with executor id 1495 parsl.dataflow.dflow INFO: Standard output for task 1494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/2fe348e2-4856-4a79-ba04-8651fa0e522d_isrForCrosstalkSources_997008_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/2fe348e2-4856-4a79-ba04-8651fa0e522d_isrForCrosstalkSources_997008_28.stderr parsl.dataflow.dflow DEBUG: Task 1495 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1495 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1495 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1495 has memoization hash 89273111cc62e1ab36880d03d633efce parsl.dataflow.memoization INFO: Task 1495 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1495 try 0 launched on executor multi with executor id 1496 parsl.dataflow.dflow INFO: Standard output for task 1495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/97f80376-cb70-42c7-bc52-4130d3278fa5_isrForCrosstalkSources_997007_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/97f80376-cb70-42c7-bc52-4130d3278fa5_isrForCrosstalkSources_997007_24.stderr parsl.dataflow.dflow DEBUG: Task 1496 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1496 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1496 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1496 has memoization hash 31b4d92dba3e8c3d6da3220090680de0 parsl.dataflow.memoization INFO: Task 1496 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2025c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1496 try 0 launched on executor multi with executor id 1497 parsl.dataflow.dflow INFO: Standard output for task 1496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/22dd9694-1925-4ea5-a14a-e07e27f4b0d4_isrForCrosstalkSources_997003_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/22dd9694-1925-4ea5-a14a-e07e27f4b0d4_isrForCrosstalkSources_997003_56.stderr parsl.dataflow.dflow DEBUG: Task 1497 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1497 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1497 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1497 has memoization hash 09322ca5330a5859cb5811b323bc5c62 parsl.dataflow.memoization INFO: Task 1497 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1497 try 0 launched on executor multi with executor id 1498 parsl.dataflow.dflow INFO: Standard output for task 1497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/d42dd075-c70e-4639-a89a-088c530b740f_isrForCrosstalkSources_997029_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/d42dd075-c70e-4639-a89a-088c530b740f_isrForCrosstalkSources_997029_25.stderr parsl.dataflow.dflow DEBUG: Task 1498 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1498 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1498 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1498 has memoization hash 0426d2cf195c11b68ed0b492d2270f98 parsl.dataflow.memoization INFO: Task 1498 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1498 try 0 launched on executor multi with executor id 1499 parsl.dataflow.dflow INFO: Standard output for task 1498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c9245c44-1c69-4896-a046-9917a5e0e15a_isrForCrosstalkSources_997033_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c9245c44-1c69-4896-a046-9917a5e0e15a_isrForCrosstalkSources_997033_60.stderr parsl.dataflow.dflow DEBUG: Task 1499 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1499 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1499 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1499 has memoization hash 21da6a01c2bd438064f2140253032549 parsl.dataflow.memoization INFO: Task 1499 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1499 try 0 launched on executor multi with executor id 1500 parsl.dataflow.dflow INFO: Standard output for task 1499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5c619962-58fd-4a36-ba6f-9d92a229547c_isrForCrosstalkSources_996991_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5c619962-58fd-4a36-ba6f-9d92a229547c_isrForCrosstalkSources_996991_43.stderr parsl.dataflow.dflow DEBUG: Task 1500 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1500 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1500 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1500 has memoization hash 27ca982883ade4fb5bee014681959470 parsl.dataflow.memoization INFO: Task 1500 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2023e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1500 try 0 launched on executor multi with executor id 1501 parsl.dataflow.dflow INFO: Standard output for task 1500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f42eb58c-d8b5-488e-bfb2-605b5153619e_isrForCrosstalkSources_996960_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f42eb58c-d8b5-488e-bfb2-605b5153619e_isrForCrosstalkSources_996960_46.stderr parsl.dataflow.dflow DEBUG: Task 1501 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1501 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1501 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1501 has memoization hash 2feaf32f5302929aa4cab029a3aed39b parsl.dataflow.memoization INFO: Task 1501 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2022a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1501 try 0 launched on executor multi with executor id 1502 parsl.dataflow.dflow INFO: Standard output for task 1501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/898a29d3-9b7b-406f-aa69-1c677bdca2b8_isrForCrosstalkSources_997018_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/898a29d3-9b7b-406f-aa69-1c677bdca2b8_isrForCrosstalkSources_997018_52.stderr parsl.dataflow.dflow DEBUG: Task 1502 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1502 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1502 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1502 has memoization hash b87f1736521f009daa0db331bac531f7 parsl.dataflow.memoization INFO: Task 1502 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1502 try 0 launched on executor multi with executor id 1503 parsl.dataflow.dflow INFO: Standard output for task 1502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/78de333d-ce91-4903-8431-b013b1ef5e29_isrForCrosstalkSources_996955_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/78de333d-ce91-4903-8431-b013b1ef5e29_isrForCrosstalkSources_996955_24.stderr parsl.dataflow.dflow DEBUG: Task 1503 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1503 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1503 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1503 has memoization hash 8fc59aa15861887142b7f905aec1e6c0 parsl.dataflow.memoization INFO: Task 1503 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e202160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1503 try 0 launched on executor multi with executor id 1504 parsl.dataflow.dflow INFO: Standard output for task 1503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/883fb164-105c-4169-993a-d3fc84d29af6_isrForCrosstalkSources_996991_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/883fb164-105c-4169-993a-d3fc84d29af6_isrForCrosstalkSources_996991_51.stderr parsl.dataflow.dflow DEBUG: Task 1504 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1504 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1504 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1504 has memoization hash 04e964ba407941235a8cf11cf988539b parsl.dataflow.memoization INFO: Task 1504 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e201f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1504 try 0 launched on executor multi with executor id 1505 parsl.dataflow.dflow INFO: Standard output for task 1504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dd8aff54-7c8c-45bb-a051-a01d73a37851_isrForCrosstalkSources_997001_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dd8aff54-7c8c-45bb-a051-a01d73a37851_isrForCrosstalkSources_997001_43.stderr parsl.dataflow.dflow DEBUG: Task 1505 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1505 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1505 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1505 has memoization hash 9c355d0df88d7db872634d81f6b1b501 parsl.dataflow.memoization INFO: Task 1505 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2020c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1505 try 0 launched on executor multi with executor id 1506 parsl.dataflow.dflow INFO: Standard output for task 1505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/4285129d-563b-436d-b7d5-807a746b8aea_isrForCrosstalkSources_996964_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/4285129d-563b-436d-b7d5-807a746b8aea_isrForCrosstalkSources_996964_57.stderr parsl.dataflow.dflow DEBUG: Task 1506 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1506 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1506 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1506 has memoization hash 1a74d75ab1229fd41201cfecc03f6305 parsl.dataflow.memoization INFO: Task 1506 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2034c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1506 try 0 launched on executor multi with executor id 1507 parsl.dataflow.dflow INFO: Standard output for task 1506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/65138cc8-3ee6-4c68-b338-c4b8e377c316_isrForCrosstalkSources_996990_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/65138cc8-3ee6-4c68-b338-c4b8e377c316_isrForCrosstalkSources_996990_54.stderr parsl.dataflow.dflow DEBUG: Task 1507 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1507 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1507 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1507 has memoization hash 946e1ab95fb5bab3db3595e0fc79e19f parsl.dataflow.memoization INFO: Task 1507 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1507 try 0 launched on executor multi with executor id 1508 parsl.dataflow.dflow INFO: Standard output for task 1507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/83f6a31f-5a73-4f85-8fca-04d1f5b8aa2d_isrForCrosstalkSources_996955_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/83f6a31f-5a73-4f85-8fca-04d1f5b8aa2d_isrForCrosstalkSources_996955_57.stderr parsl.dataflow.dflow DEBUG: Task 1508 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1508 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1508 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1508 has memoization hash 044a4dd6ae8bc49e57dc0331657eb41c parsl.dataflow.memoization INFO: Task 1508 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1508 try 0 launched on executor multi with executor id 1509 parsl.dataflow.dflow INFO: Standard output for task 1508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/612a3ca7-5e6b-4379-86dc-1626b2178a9c_isrForCrosstalkSources_996985_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/612a3ca7-5e6b-4379-86dc-1626b2178a9c_isrForCrosstalkSources_996985_7.stderr parsl.dataflow.dflow DEBUG: Task 1509 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1509 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1509 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1509 has memoization hash 724a40f85f293d693f15551d1674b59c parsl.dataflow.memoization INFO: Task 1509 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1509 try 0 launched on executor multi with executor id 1510 parsl.dataflow.dflow INFO: Standard output for task 1509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/7887f3d5-b59c-459a-923d-2a2889bff98e_isrForCrosstalkSources_996987_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/7887f3d5-b59c-459a-923d-2a2889bff98e_isrForCrosstalkSources_996987_53.stderr parsl.dataflow.dflow DEBUG: Task 1510 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1510 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1510 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1510 has memoization hash c1c096f4253d4092b754033ccfae57a7 parsl.dataflow.memoization INFO: Task 1510 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2036a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1510 try 0 launched on executor multi with executor id 1511 parsl.dataflow.dflow INFO: Standard output for task 1510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/220601d8-b795-46c1-a86a-7a43e5bff073_isrForCrosstalkSources_997002_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/220601d8-b795-46c1-a86a-7a43e5bff073_isrForCrosstalkSources_997002_21.stderr parsl.dataflow.dflow DEBUG: Task 1511 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1511 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1511 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1511 has memoization hash c4bd8b3ed97aeddbb11a9d07c40283a0 parsl.dataflow.memoization INFO: Task 1511 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2037e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1511 try 0 launched on executor multi with executor id 1512 parsl.dataflow.dflow INFO: Standard output for task 1511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9fe0f193-ce39-4f8e-9b8e-8e8808f529ee_isrForCrosstalkSources_996958_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9fe0f193-ce39-4f8e-9b8e-8e8808f529ee_isrForCrosstalkSources_996958_32.stderr parsl.dataflow.dflow DEBUG: Task 1512 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1512 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1512 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1512 has memoization hash 51069c415f667bc763f415f17b0ff6b4 parsl.dataflow.memoization INFO: Task 1512 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1512 try 0 launched on executor multi with executor id 1513 parsl.dataflow.dflow INFO: Standard output for task 1512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4b396fcf-c1dd-42ab-a5f2-ecadabe3111e_isrForCrosstalkSources_997033_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4b396fcf-c1dd-42ab-a5f2-ecadabe3111e_isrForCrosstalkSources_997033_18.stderr parsl.dataflow.dflow DEBUG: Task 1513 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1513 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1513 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1513 has memoization hash 5e2a5fd765808667a4271c3bda3269bd parsl.dataflow.memoization INFO: Task 1513 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1513 try 0 launched on executor multi with executor id 1514 parsl.dataflow.dflow INFO: Standard output for task 1513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/baff17c7-ee9a-4916-a827-ea1c16de5daf_isrForCrosstalkSources_996953_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/baff17c7-ee9a-4916-a827-ea1c16de5daf_isrForCrosstalkSources_996953_50.stderr parsl.dataflow.dflow DEBUG: Task 1514 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1514 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1514 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1514 has memoization hash 891789bac009b061be58f99e73764bb9 parsl.dataflow.memoization INFO: Task 1514 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1514 try 0 launched on executor multi with executor id 1515 parsl.dataflow.dflow INFO: Standard output for task 1514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/95626e66-9fd5-4650-ab98-288c2cc673ee_isrForCrosstalkSources_996977_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/95626e66-9fd5-4650-ab98-288c2cc673ee_isrForCrosstalkSources_996977_57.stderr parsl.dataflow.dflow DEBUG: Task 1515 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1515 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1515 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1515 has memoization hash b00c67363f3c777f6e065e2abb4a6366 parsl.dataflow.memoization INFO: Task 1515 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2039c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1515 try 0 launched on executor multi with executor id 1516 parsl.dataflow.dflow INFO: Standard output for task 1515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c1d454a6-b8c0-46a3-bd33-9f503cd07154_isrForCrosstalkSources_997020_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c1d454a6-b8c0-46a3-bd33-9f503cd07154_isrForCrosstalkSources_997020_41.stderr parsl.dataflow.dflow DEBUG: Task 1516 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1516 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1516 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1516 has memoization hash 8ba3f5ef1bc48daf230782e79ba68dd7 parsl.dataflow.memoization INFO: Task 1516 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1516 try 0 launched on executor multi with executor id 1517 parsl.dataflow.dflow INFO: Standard output for task 1516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e7f1f3fc-f164-4c6e-a69c-8158ab6642f7_isrForCrosstalkSources_997028_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e7f1f3fc-f164-4c6e-a69c-8158ab6642f7_isrForCrosstalkSources_997028_11.stderr parsl.dataflow.dflow DEBUG: Task 1517 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1517 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1517 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1517 has memoization hash d74386a05f0bcccd92275ee02e0e7f84 parsl.dataflow.memoization INFO: Task 1517 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1517 try 0 launched on executor multi with executor id 1518 parsl.dataflow.dflow INFO: Standard output for task 1517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b71d70af-a38f-4ae2-96a4-7965cf354f6f_isrForCrosstalkSources_996997_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b71d70af-a38f-4ae2-96a4-7965cf354f6f_isrForCrosstalkSources_996997_11.stderr parsl.dataflow.dflow DEBUG: Task 1518 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1518 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1518 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1518 has memoization hash efd0d6bb7ef314abdb0181144302827a parsl.dataflow.memoization INFO: Task 1518 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1518 try 0 launched on executor multi with executor id 1519 parsl.dataflow.dflow INFO: Standard output for task 1518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0d7da39f-ccbe-47f3-9e43-2d18da246717_isrForCrosstalkSources_996983_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0d7da39f-ccbe-47f3-9e43-2d18da246717_isrForCrosstalkSources_996983_62.stderr parsl.dataflow.dflow DEBUG: Task 1519 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1519 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1519 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1519 has memoization hash bc45015dd62a56ecc8056f9d54ebc87a parsl.dataflow.memoization INFO: Task 1519 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1519 try 0 launched on executor multi with executor id 1520 parsl.dataflow.dflow INFO: Standard output for task 1519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4ec7c032-2423-4f8c-86b0-e66cdbd1aa42_isrForCrosstalkSources_996987_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4ec7c032-2423-4f8c-86b0-e66cdbd1aa42_isrForCrosstalkSources_996987_59.stderr parsl.dataflow.dflow DEBUG: Task 1520 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1520 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1520 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1520 has memoization hash 879edcef9df0b28781bceb340d0e555c parsl.dataflow.memoization INFO: Task 1520 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:41 proc_lsst.multi:170 [INFO] [multi] job_id 17074859 proc_lsst.multi INFO: [multi] job_id 17074859 2024-03-15 13:49:41 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 19 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 19 2024-03-15 13:49:41 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:41 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074859 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074859 parsl.executors.status_handling DEBUG: Launched block 19 on executor multi with job ID 17074859 parsl.executors.status_handling INFO: Allocated block ID 20 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-20 2024-03-15 13:49:41 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=20 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-20 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=20 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-20 2024-03-15 13:49:41 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=20 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=20 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:41 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=20 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=20 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 19 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 19 2024-03-15 13:49:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:41 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=20 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=20 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 1520 try 0 launched on executor multi with executor id 1521 parsl.dataflow.dflow INFO: Standard output for task 1520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/238e5c4f-7234-451a-81e8-57c5762f7de2_isrForCrosstalkSources_997029_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/238e5c4f-7234-451a-81e8-57c5762f7de2_isrForCrosstalkSources_997029_52.stderr parsl.dataflow.dflow DEBUG: Task 1521 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1521 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1521 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1521 has memoization hash d487ba2d495dced271dadee9dcad5949 parsl.dataflow.memoization INFO: Task 1521 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1521 try 0 launched on executor multi with executor id 1522 parsl.dataflow.dflow INFO: Standard output for task 1521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9ecc5a08-a3bc-419e-8923-20c920fd99ac_isrForCrosstalkSources_996971_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9ecc5a08-a3bc-419e-8923-20c920fd99ac_isrForCrosstalkSources_996971_30.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Task 1522 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1522 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1522 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1522 has memoization hash 31cab6b218c23d97141c89f0bc3d49ae parsl.dataflow.memoization INFO: Task 1522 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b99e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1522 try 0 launched on executor multi with executor id 1523 parsl.dataflow.dflow INFO: Standard output for task 1522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5657cb6f-173a-4bc3-849c-c07816cc48ae_isrForCrosstalkSources_996981_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5657cb6f-173a-4bc3-849c-c07816cc48ae_isrForCrosstalkSources_996981_30.stderr parsl.dataflow.dflow DEBUG: Task 1523 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1523 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1523 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1523 has memoization hash 53889aec963af2e6441843f567a85e2a parsl.dataflow.memoization INFO: Task 1523 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78b949760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1523 try 0 launched on executor multi with executor id 1524 parsl.dataflow.dflow INFO: Standard output for task 1523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/6c68607b-2fc0-4ba1-8869-3c2ee49a49ba_isrForCrosstalkSources_996978_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/6c68607b-2fc0-4ba1-8869-3c2ee49a49ba_isrForCrosstalkSources_996978_44.stderr parsl.dataflow.dflow DEBUG: Task 1524 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1524 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1524 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1524 has memoization hash 10ac69014bf1520beb16458ffdd2e570 parsl.dataflow.memoization INFO: Task 1524 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1524 try 0 launched on executor multi with executor id 1525 parsl.dataflow.dflow INFO: Standard output for task 1524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c496dd36-8dcf-40c2-bc41-298231aca09b_isrForCrosstalkSources_997016_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c496dd36-8dcf-40c2-bc41-298231aca09b_isrForCrosstalkSources_997016_38.stderr parsl.dataflow.dflow DEBUG: Task 1525 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1525 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1525 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1525 has memoization hash 01bd16447baa958b97146764639c7e3a parsl.dataflow.memoization INFO: Task 1525 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1525 try 0 launched on executor multi with executor id 1526 parsl.dataflow.dflow INFO: Standard output for task 1525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/12dde2f6-5aa8-4f02-ad5e-617c6ab3aed1_isrForCrosstalkSources_996983_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/12dde2f6-5aa8-4f02-ad5e-617c6ab3aed1_isrForCrosstalkSources_996983_60.stderr parsl.dataflow.dflow DEBUG: Task 1526 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1526 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1526 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1526 has memoization hash 9c5e3486c353ef4857c1f2730cb7b24d parsl.dataflow.memoization INFO: Task 1526 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1526 try 0 launched on executor multi with executor id 1527 parsl.dataflow.dflow INFO: Standard output for task 1526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/b242fe10-067a-435f-891a-2790a3f90bc4_isrForCrosstalkSources_997010_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/b242fe10-067a-435f-891a-2790a3f90bc4_isrForCrosstalkSources_997010_41.stderr parsl.dataflow.dflow DEBUG: Task 1527 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1527 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1527 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1527 has memoization hash 4dfa10f372cb74f401091ade9b367528 parsl.dataflow.memoization INFO: Task 1527 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1527 try 0 launched on executor multi with executor id 1528 parsl.dataflow.dflow INFO: Standard output for task 1527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b3adb660-ffed-4c68-8d3d-ec5a3b554389_isrForCrosstalkSources_997004_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b3adb660-ffed-4c68-8d3d-ec5a3b554389_isrForCrosstalkSources_997004_4.stderr parsl.dataflow.dflow DEBUG: Task 1528 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1528 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1528 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1528 has memoization hash 9a1ddd1969ae20081b411ec82e78ae8c parsl.dataflow.memoization INFO: Task 1528 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1528 try 0 launched on executor multi with executor id 1529 parsl.dataflow.dflow INFO: Standard output for task 1528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/cfee817f-d45a-462a-9078-c13d4163f574_isrForCrosstalkSources_996981_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/cfee817f-d45a-462a-9078-c13d4163f574_isrForCrosstalkSources_996981_35.stderr parsl.dataflow.dflow DEBUG: Task 1529 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1529 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1529 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1529 has memoization hash 33b84512abadd16542df0432c6b7727b parsl.dataflow.memoization INFO: Task 1529 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1529 try 0 launched on executor multi with executor id 1530 parsl.dataflow.dflow INFO: Standard output for task 1529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/bacff6c0-fcb1-48c4-b6b1-3e4bfb77e0a9_isrForCrosstalkSources_997023_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/bacff6c0-fcb1-48c4-b6b1-3e4bfb77e0a9_isrForCrosstalkSources_997023_30.stderr parsl.dataflow.dflow DEBUG: Task 1530 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1530 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1530 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1530 has memoization hash dc5063d92c51737a4c7f8665d9e491b5 parsl.dataflow.memoization INFO: Task 1530 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1530 try 0 launched on executor multi with executor id 1531 parsl.dataflow.dflow INFO: Standard output for task 1530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/5c488b99-3be3-434d-a158-825c71e60c80_isrForCrosstalkSources_996982_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/5c488b99-3be3-434d-a158-825c71e60c80_isrForCrosstalkSources_996982_55.stderr parsl.dataflow.dflow DEBUG: Task 1531 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1531 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1531 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1531 has memoization hash 14c7a52c01d3fdb26cdefdfae99d9875 parsl.dataflow.memoization INFO: Task 1531 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1531 try 0 launched on executor multi with executor id 1532 parsl.dataflow.dflow INFO: Standard output for task 1531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/271b2f55-4281-46c6-9060-dec1c8cbf0cd_isrForCrosstalkSources_996972_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/271b2f55-4281-46c6-9060-dec1c8cbf0cd_isrForCrosstalkSources_996972_52.stderr parsl.dataflow.dflow DEBUG: Task 1532 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1532 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1532 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1532 has memoization hash ef42ed858502f986e9c8c963ddc85a85 parsl.dataflow.memoization INFO: Task 1532 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1532 try 0 launched on executor multi with executor id 1533 parsl.dataflow.dflow INFO: Standard output for task 1532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/84ad607d-58dc-40a6-a7a7-61f07279a1b5_isrForCrosstalkSources_996954_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/84ad607d-58dc-40a6-a7a7-61f07279a1b5_isrForCrosstalkSources_996954_35.stderr parsl.dataflow.dflow DEBUG: Task 1533 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1533 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1533 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1533 has memoization hash 35c5e302b542e7a37104240765a24979 parsl.dataflow.memoization INFO: Task 1533 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1533 try 0 launched on executor multi with executor id 1534 parsl.dataflow.dflow INFO: Standard output for task 1533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/cfc544f9-1a9e-47b2-980b-eeba3de2e94a_isrForCrosstalkSources_996966_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/cfc544f9-1a9e-47b2-980b-eeba3de2e94a_isrForCrosstalkSources_996966_57.stderr parsl.dataflow.dflow DEBUG: Task 1534 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1534 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1534 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1534 has memoization hash 840f74ad5c4584299b2b01971ea82ee8 parsl.dataflow.memoization INFO: Task 1534 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1534 try 0 launched on executor multi with executor id 1535 parsl.dataflow.dflow INFO: Standard output for task 1534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6bb6fe36-71a2-48d7-9888-85aa96d9603c_isrForCrosstalkSources_997005_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6bb6fe36-71a2-48d7-9888-85aa96d9603c_isrForCrosstalkSources_997005_62.stderr parsl.dataflow.dflow DEBUG: Task 1535 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1535 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1535 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1535 has memoization hash c302e82781b9b63d545c1a4ba773db81 parsl.dataflow.memoization INFO: Task 1535 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1535 try 0 launched on executor multi with executor id 1536 parsl.dataflow.dflow INFO: Standard output for task 1535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/1a949cdb-7765-45b9-b1b8-7a47246c7942_isrForCrosstalkSources_996955_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/1a949cdb-7765-45b9-b1b8-7a47246c7942_isrForCrosstalkSources_996955_51.stderr parsl.dataflow.dflow DEBUG: Task 1536 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1536 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1536 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1536 has memoization hash 0770464da6894da510b1f213050fd32f parsl.dataflow.memoization INFO: Task 1536 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1536 try 0 launched on executor multi with executor id 1537 parsl.dataflow.dflow INFO: Standard output for task 1536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/321be0e8-acab-4d5c-8fb7-6f6b44023c2a_isrForCrosstalkSources_996999_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/321be0e8-acab-4d5c-8fb7-6f6b44023c2a_isrForCrosstalkSources_996999_52.stderr parsl.dataflow.dflow DEBUG: Task 1537 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1537 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1537 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1537 has memoization hash f5c92b327554143137d359e50fe750a8 parsl.dataflow.memoization INFO: Task 1537 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1537 try 0 launched on executor multi with executor id 1538 parsl.dataflow.dflow INFO: Standard output for task 1537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/97238b31-7495-4f7d-872a-809417f42b1d_isrForCrosstalkSources_997028_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/97238b31-7495-4f7d-872a-809417f42b1d_isrForCrosstalkSources_997028_21.stderr parsl.dataflow.dflow DEBUG: Task 1538 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1538 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1538 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1538 has memoization hash 45e05fec8789406860d2e43ed0cb8993 parsl.dataflow.memoization INFO: Task 1538 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1538 try 0 launched on executor multi with executor id 1539 parsl.dataflow.dflow INFO: Standard output for task 1538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ccc7758c-7f1c-4216-a99c-d3179c44964d_isrForCrosstalkSources_996992_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ccc7758c-7f1c-4216-a99c-d3179c44964d_isrForCrosstalkSources_996992_25.stderr parsl.dataflow.dflow DEBUG: Task 1539 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1539 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1539 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1539 has memoization hash a6a586d1c0e20ae2e239c61f4ec83d0c parsl.dataflow.memoization INFO: Task 1539 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 1 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2285f91fdc50e60cb79b857fd7b01b32 with result from task 1 parsl.dataflow.dflow INFO: Parsl task 1539 try 0 launched on executor multi with executor id 1540 parsl.dataflow.dflow INFO: Standard output for task 1539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/e09f547f-bda9-4480-b965-d518fd6102fd_isrForCrosstalkSources_996982_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/e09f547f-bda9-4480-b965-d518fd6102fd_isrForCrosstalkSources_996982_54.stderr parsl.dataflow.dflow DEBUG: Task 1540 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1540 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1540 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1540 has memoization hash 5abe094ff11c74045bc4a82dd2055e68 parsl.dataflow.memoization INFO: Task 1540 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1540 try 0 launched on executor multi with executor id 1541 parsl.dataflow.dflow INFO: Standard output for task 1540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d19de0a3-65cc-46c6-a29d-e4c88f8cf0d3_isrForCrosstalkSources_996981_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d19de0a3-65cc-46c6-a29d-e4c88f8cf0d3_isrForCrosstalkSources_996981_6.stderr parsl.dataflow.dflow DEBUG: Task 1541 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1541 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1541 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1541 has memoization hash 327b9fe2df0e279ae77194d7556f5d9a parsl.dataflow.memoization INFO: Task 1541 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 1 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/f9c828f8-1533-49bf-bbf4-31f6b2e3b1cd_isrForCrosstalkSources_997033_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/f9c828f8-1533-49bf-bbf4-31f6b2e3b1cd_isrForCrosstalkSources_997033_14.stderr parsl.dataflow.dflow INFO: Parsl task 1541 try 0 launched on executor multi with executor id 1542 parsl.dataflow.dflow INFO: Standard output for task 1541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/e27b060b-d7ca-413b-9e6a-5f38d5661d27_isrForCrosstalkSources_996988_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/e27b060b-d7ca-413b-9e6a-5f38d5661d27_isrForCrosstalkSources_996988_36.stderr parsl.dataflow.dflow DEBUG: Task 1542 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1542 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1542 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1542 has memoization hash a3e513e2c42db53b0747ddf2f336eb86 parsl.dataflow.memoization INFO: Task 1542 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1542 try 0 launched on executor multi with executor id 1543 parsl.dataflow.dflow INFO: Standard output for task 1542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d156b8b6-22a9-42d0-9900-460364d29096_isrForCrosstalkSources_996990_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d156b8b6-22a9-42d0-9900-460364d29096_isrForCrosstalkSources_996990_40.stderr parsl.dataflow.dflow DEBUG: Task 1543 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1543 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1543 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1543 has memoization hash 69e95688da02372fd14e0a3cff36011f parsl.dataflow.memoization INFO: Task 1543 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1543 try 0 launched on executor multi with executor id 1544 parsl.dataflow.dflow INFO: Standard output for task 1543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/17eb8961-712f-4370-b3a8-6885751a0bc7_isrForCrosstalkSources_997010_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/17eb8961-712f-4370-b3a8-6885751a0bc7_isrForCrosstalkSources_997010_9.stderr parsl.dataflow.dflow DEBUG: Task 1544 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1544 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1544 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1544 has memoization hash ac60a1800630d5d4a9ad2d071dbc5e40 parsl.dataflow.memoization INFO: Task 1544 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1544 try 0 launched on executor multi with executor id 1545 parsl.dataflow.dflow INFO: Standard output for task 1544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/3811c446-afa0-43ae-ba11-88e4be8a89d6_isrForCrosstalkSources_997008_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/3811c446-afa0-43ae-ba11-88e4be8a89d6_isrForCrosstalkSources_997008_18.stderr parsl.dataflow.dflow DEBUG: Task 1545 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1545 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1545 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1545 has memoization hash 1591e9ab9f01eaf81805b99cb2333b10 parsl.dataflow.memoization INFO: Task 1545 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1545 try 0 launched on executor multi with executor id 1546 parsl.dataflow.dflow INFO: Standard output for task 1545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/93a47611-db30-47c8-9ce2-d4fdb6bb68f2_isrForCrosstalkSources_996972_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/93a47611-db30-47c8-9ce2-d4fdb6bb68f2_isrForCrosstalkSources_996972_16.stderr parsl.dataflow.dflow DEBUG: Task 1546 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1546 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1546 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1546 has memoization hash 19ba207b9afe8e417b53fb7395dc3198 parsl.dataflow.memoization INFO: Task 1546 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1546 try 0 launched on executor multi with executor id 1547 parsl.dataflow.dflow INFO: Standard output for task 1546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/8151e5cf-f633-42a2-a29f-2a1ed7381226_isrForCrosstalkSources_996965_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/8151e5cf-f633-42a2-a29f-2a1ed7381226_isrForCrosstalkSources_996965_53.stderr parsl.dataflow.dflow DEBUG: Task 1547 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1547 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1547 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1547 has memoization hash 1a3aacb8b94076bb5280790a681a3732 parsl.dataflow.memoization INFO: Task 1547 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1547 try 0 launched on executor multi with executor id 1548 parsl.dataflow.dflow INFO: Standard output for task 1547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4dbf54c2-079c-46b0-aa78-828a5b34f1ca_isrForCrosstalkSources_996980_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4dbf54c2-079c-46b0-aa78-828a5b34f1ca_isrForCrosstalkSources_996980_17.stderr parsl.dataflow.dflow DEBUG: Task 1548 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1548 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1548 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1548 has memoization hash e902971f6fe01106089fdbd117f1bc40 parsl.dataflow.memoization INFO: Task 1548 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1548 try 0 launched on executor multi with executor id 1549 parsl.dataflow.dflow INFO: Standard output for task 1548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/aba9cee1-3124-495a-8bc7-c85e95ebdc66_isrForCrosstalkSources_996971_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/aba9cee1-3124-495a-8bc7-c85e95ebdc66_isrForCrosstalkSources_996971_10.stderr parsl.dataflow.dflow DEBUG: Task 1549 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1549 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1549 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1549 has memoization hash 193f96c7f492ae44e11ea03e62bb55ca parsl.dataflow.memoization INFO: Task 1549 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1549 try 0 launched on executor multi with executor id 1550 parsl.dataflow.dflow INFO: Standard output for task 1549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/b08586b8-d960-4e08-a6e1-05a6179eb6f9_isrForCrosstalkSources_997019_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/b08586b8-d960-4e08-a6e1-05a6179eb6f9_isrForCrosstalkSources_997019_57.stderr parsl.dataflow.dflow DEBUG: Task 1550 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1550 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1550 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1550 has memoization hash 05b9d1004f74e8999571dc6a624d40d7 parsl.dataflow.memoization INFO: Task 1550 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1550 try 0 launched on executor multi with executor id 1551 parsl.dataflow.dflow INFO: Standard output for task 1550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/60c30f39-44c7-48c6-9fc1-f0f0b4e5e121_isrForCrosstalkSources_996996_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/60c30f39-44c7-48c6-9fc1-f0f0b4e5e121_isrForCrosstalkSources_996996_8.stderr parsl.dataflow.dflow DEBUG: Task 1551 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1551 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1551 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1551 has memoization hash 897c8e9105061f05dee99b7521511505 parsl.dataflow.memoization INFO: Task 1551 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1551 try 0 launched on executor multi with executor id 1552 parsl.dataflow.dflow INFO: Standard output for task 1551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8e083bb0-9571-442a-a1ab-f8f5a65885fc_isrForCrosstalkSources_997017_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8e083bb0-9571-442a-a1ab-f8f5a65885fc_isrForCrosstalkSources_997017_37.stderr parsl.dataflow.dflow DEBUG: Task 1552 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1552 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1552 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1552 has memoization hash 127fbe4a180b11fac81eacba8306c977 parsl.dataflow.memoization INFO: Task 1552 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1552 try 0 launched on executor multi with executor id 1553 parsl.dataflow.dflow INFO: Standard output for task 1552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9af38b87-ef5a-41ba-99bc-8e06967d997e_isrForCrosstalkSources_996964_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9af38b87-ef5a-41ba-99bc-8e06967d997e_isrForCrosstalkSources_996964_19.stderr parsl.dataflow.dflow DEBUG: Task 1553 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1553 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1553 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1553 has memoization hash eb08dd00f849dcf1bd3ab6ec6f3da751 parsl.dataflow.memoization INFO: Task 1553 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1553 try 0 launched on executor multi with executor id 1554 parsl.dataflow.dflow INFO: Standard output for task 1553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7296a330-1653-4095-b881-402fed49a2b3_isrForCrosstalkSources_997027_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7296a330-1653-4095-b881-402fed49a2b3_isrForCrosstalkSources_997027_22.stderr parsl.dataflow.dflow DEBUG: Task 1554 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1554 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1554 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1554 has memoization hash 75f46894bda3569358559ec2df282be4 parsl.dataflow.memoization INFO: Task 1554 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1554 try 0 launched on executor multi with executor id 1555 parsl.dataflow.dflow INFO: Standard output for task 1554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/3d4546bb-a1c9-45ab-bc38-5e926f677a1d_isrForCrosstalkSources_996962_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/3d4546bb-a1c9-45ab-bc38-5e926f677a1d_isrForCrosstalkSources_996962_43.stderr parsl.dataflow.dflow DEBUG: Task 1555 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1555 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1555 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1555 has memoization hash f72ee2868443c5685a8c89469ceaa715 parsl.dataflow.memoization INFO: Task 1555 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1555 try 0 launched on executor multi with executor id 1556 parsl.dataflow.dflow INFO: Standard output for task 1555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/53993f1f-7aa3-44a2-b430-7ca849750263_isrForCrosstalkSources_996967_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/53993f1f-7aa3-44a2-b430-7ca849750263_isrForCrosstalkSources_996967_8.stderr parsl.dataflow.dflow DEBUG: Task 1556 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1556 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1556 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1556 has memoization hash e2b3218a771cc47b283521cc896a599d parsl.dataflow.memoization INFO: Task 1556 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1556 try 0 launched on executor multi with executor id 1557 parsl.dataflow.dflow INFO: Standard output for task 1556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/3c12f4c2-e42e-4519-8d25-1ea9ca44feb2_isrForCrosstalkSources_996960_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/3c12f4c2-e42e-4519-8d25-1ea9ca44feb2_isrForCrosstalkSources_996960_3.stderr parsl.dataflow.dflow DEBUG: Task 1557 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1557 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1557 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1557 has memoization hash c8159c862ab8c9acaa104faec2c3d294 parsl.dataflow.memoization INFO: Task 1557 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1557 try 0 launched on executor multi with executor id 1558 parsl.dataflow.dflow INFO: Standard output for task 1557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7dc5849a-fe38-4565-b946-e400c1417c43_isrForCrosstalkSources_996984_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7dc5849a-fe38-4565-b946-e400c1417c43_isrForCrosstalkSources_996984_6.stderr parsl.dataflow.dflow DEBUG: Task 1558 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1558 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1558 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1558 has memoization hash 66aa61bb657e778aee07e2decb6e756a parsl.dataflow.memoization INFO: Task 1558 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1558 try 0 launched on executor multi with executor id 1559 parsl.dataflow.dflow INFO: Standard output for task 1558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/48fe998a-f70e-4afd-80c8-5947c5e84f98_isrForCrosstalkSources_997011_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/48fe998a-f70e-4afd-80c8-5947c5e84f98_isrForCrosstalkSources_997011_40.stderr parsl.dataflow.dflow DEBUG: Task 1559 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1559 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1559 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1559 has memoization hash 8d3c5218a4bed04a1e8de158d239c3bc parsl.dataflow.memoization INFO: Task 1559 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1559 try 0 launched on executor multi with executor id 1560 parsl.dataflow.dflow INFO: Standard output for task 1559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/3900dbd7-e0d0-4703-844f-33bd350b6f6e_isrForCrosstalkSources_997016_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/3900dbd7-e0d0-4703-844f-33bd350b6f6e_isrForCrosstalkSources_997016_35.stderr parsl.dataflow.dflow DEBUG: Task 1560 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1560 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1560 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1560 has memoization hash 6b3dc5c5f7a7f540592dfd6b56eb8d93 parsl.dataflow.memoization INFO: Task 1560 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1560 try 0 launched on executor multi with executor id 1561 parsl.dataflow.dflow INFO: Standard output for task 1560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e4e13630-f58c-4566-bec1-f9f9cdbd7c39_isrForCrosstalkSources_996956_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e4e13630-f58c-4566-bec1-f9f9cdbd7c39_isrForCrosstalkSources_996956_60.stderr parsl.dataflow.dflow DEBUG: Task 1561 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1561 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1561 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1561 has memoization hash 3bfa7f73ee3355f814cb7c8f689f61f7 parsl.dataflow.memoization INFO: Task 1561 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1561 try 0 launched on executor multi with executor id 1562 parsl.dataflow.dflow INFO: Standard output for task 1561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/e435dd66-2a59-4a24-93d2-b978de87564c_isrForCrosstalkSources_997004_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/e435dd66-2a59-4a24-93d2-b978de87564c_isrForCrosstalkSources_997004_41.stderr parsl.dataflow.dflow DEBUG: Task 1562 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1562 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1562 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1562 has memoization hash bc77f8bfc0ec4d1ee2af5279153ab281 parsl.dataflow.memoization INFO: Task 1562 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1562 try 0 launched on executor multi with executor id 1563 parsl.dataflow.dflow INFO: Standard output for task 1562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/e2adafdd-ddba-4052-a601-b5bbbfeaa4db_isrForCrosstalkSources_997027_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/e2adafdd-ddba-4052-a601-b5bbbfeaa4db_isrForCrosstalkSources_997027_49.stderr parsl.dataflow.dflow DEBUG: Task 1563 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1563 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1563 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1563 has memoization hash de09325355183b7c070c257eec9e562d parsl.dataflow.memoization INFO: Task 1563 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1563 try 0 launched on executor multi with executor id 1564 parsl.dataflow.dflow INFO: Standard output for task 1563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/29122da0-7dc6-4e24-9d26-76d4794b3684_isrForCrosstalkSources_996971_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/29122da0-7dc6-4e24-9d26-76d4794b3684_isrForCrosstalkSources_996971_44.stderr parsl.dataflow.dflow DEBUG: Task 1564 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1564 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1564 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1564 has memoization hash ab6cc377acaabf1cba376e83aa2a2212 parsl.dataflow.memoization INFO: Task 1564 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1564 try 0 launched on executor multi with executor id 1565 parsl.dataflow.dflow INFO: Standard output for task 1564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0079eb0a-d9ce-49c4-aa21-e89a0cecd990_isrForCrosstalkSources_997010_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0079eb0a-d9ce-49c4-aa21-e89a0cecd990_isrForCrosstalkSources_997010_29.stderr parsl.dataflow.dflow DEBUG: Task 1565 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1565 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1565 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1565 has memoization hash 5a8e541b2fd46f35fe078b0f08b1907a parsl.dataflow.memoization INFO: Task 1565 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1565 try 0 launched on executor multi with executor id 1566 parsl.dataflow.dflow INFO: Standard output for task 1565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ef82f429-f0c1-400e-a850-740f7a502707_isrForCrosstalkSources_997021_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ef82f429-f0c1-400e-a850-740f7a502707_isrForCrosstalkSources_997021_21.stderr parsl.dataflow.dflow DEBUG: Task 1566 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1566 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1566 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1566 has memoization hash 9cb31e73013f0051ef8528efa11fbf63 parsl.dataflow.memoization INFO: Task 1566 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1566 try 0 launched on executor multi with executor id 1567 parsl.dataflow.dflow INFO: Standard output for task 1566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/9f96640a-7bf3-43ba-939d-e5a8344789cf_isrForCrosstalkSources_996981_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/9f96640a-7bf3-43ba-939d-e5a8344789cf_isrForCrosstalkSources_996981_62.stderr parsl.dataflow.dflow DEBUG: Task 1567 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1567 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1567 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1567 has memoization hash b58d6268309cdc351254b348089f296e parsl.dataflow.memoization INFO: Task 1567 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1567 try 0 launched on executor multi with executor id 1568 parsl.dataflow.dflow INFO: Standard output for task 1567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/468790a7-792d-47df-aae0-e431983b68c6_isrForCrosstalkSources_997008_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/468790a7-792d-47df-aae0-e431983b68c6_isrForCrosstalkSources_997008_60.stderr parsl.dataflow.dflow DEBUG: Task 1568 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1568 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1568 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1568 has memoization hash 410ccd8a9cd860ceab59805d3756887f parsl.dataflow.memoization INFO: Task 1568 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1568 try 0 launched on executor multi with executor id 1569 parsl.dataflow.dflow INFO: Standard output for task 1568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cc5783b1-6607-4bee-b7dc-fafa8d564835_isrForCrosstalkSources_997015_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cc5783b1-6607-4bee-b7dc-fafa8d564835_isrForCrosstalkSources_997015_22.stderr parsl.dataflow.dflow DEBUG: Task 1569 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1569 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1569 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1569 has memoization hash aa8d5c253c1e3a4fed30f167f783007d parsl.dataflow.memoization INFO: Task 1569 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1569 try 0 launched on executor multi with executor id 1570 parsl.dataflow.dflow INFO: Standard output for task 1569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/67ae528d-8c27-4193-9a91-2bc9ef3b1b28_isrForCrosstalkSources_996982_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/67ae528d-8c27-4193-9a91-2bc9ef3b1b28_isrForCrosstalkSources_996982_11.stderr parsl.dataflow.dflow DEBUG: Task 1570 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1570 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1570 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1570 has memoization hash f833e47b3c01935f3de6fb2f51d9315f parsl.dataflow.memoization INFO: Task 1570 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1570 try 0 launched on executor multi with executor id 1571 parsl.dataflow.dflow INFO: Standard output for task 1570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/460cab89-dcd3-4a4c-b3f4-56c3debed576_isrForCrosstalkSources_996959_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/460cab89-dcd3-4a4c-b3f4-56c3debed576_isrForCrosstalkSources_996959_60.stderr parsl.dataflow.dflow DEBUG: Task 1571 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1571 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1571 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1571 has memoization hash f34469379aeeec6b4a40ddb22ac917a4 parsl.dataflow.memoization INFO: Task 1571 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1571 try 0 launched on executor multi with executor id 1572 parsl.dataflow.dflow INFO: Standard output for task 1571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ae23a244-7d79-4082-8bdb-1c78bb3d23c4_isrForCrosstalkSources_997030_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ae23a244-7d79-4082-8bdb-1c78bb3d23c4_isrForCrosstalkSources_997030_60.stderr parsl.dataflow.dflow DEBUG: Task 1572 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1572 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1572 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1572 has memoization hash 8c9d33efb2c2a3757a547915aa96f73b parsl.dataflow.memoization INFO: Task 1572 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1572 try 0 launched on executor multi with executor id 1573 parsl.dataflow.dflow INFO: Standard output for task 1572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/10a653c3-7cb9-4a2a-aa2b-88876f74d6bc_isrForCrosstalkSources_997011_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/10a653c3-7cb9-4a2a-aa2b-88876f74d6bc_isrForCrosstalkSources_997011_35.stderr parsl.dataflow.dflow DEBUG: Task 1573 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1573 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1573 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1573 has memoization hash 0284d1037441312f822a3f04775ceb6c parsl.dataflow.memoization INFO: Task 1573 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 0 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f870af3d45c9759f4928706318ab496 with result from task 0 parsl.dataflow.dflow INFO: Parsl task 1573 try 0 launched on executor multi with executor id 1574 parsl.dataflow.dflow INFO: Standard output for task 1573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4b8b4ee5-ee39-43ce-b546-c1231481098e_isrForCrosstalkSources_996998_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4b8b4ee5-ee39-43ce-b546-c1231481098e_isrForCrosstalkSources_996998_61.stderr parsl.dataflow.dflow DEBUG: Task 1574 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1574 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1574 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1574 has memoization hash a7f8cd333e7d691bd57050987d9cd116 parsl.dataflow.memoization INFO: Task 1574 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1574 try 0 launched on executor multi with executor id 1575 parsl.dataflow.dflow INFO: Standard output for task 1574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/9d8077e9-786d-4a65-bb57-9f149422d4c8_isrForCrosstalkSources_997012_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/9d8077e9-786d-4a65-bb57-9f149422d4c8_isrForCrosstalkSources_997012_31.stderr parsl.dataflow.dflow DEBUG: Task 1575 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1575 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1575 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1575 has memoization hash 7b99c8ee4dd8c96c62acdc7bde6abd81 parsl.dataflow.memoization INFO: Task 1575 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 0 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 0 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8faee67f-132b-42f2-b22f-6f323b61684f_isrForCrosstalkSources_997027_36.stdout parsl.dataflow.dflow INFO: Standard error for task 0 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8faee67f-132b-42f2-b22f-6f323b61684f_isrForCrosstalkSources_997027_36.stderr parsl.dataflow.dflow INFO: Parsl task 1575 try 0 launched on executor multi with executor id 1576 parsl.dataflow.dflow INFO: Standard output for task 1575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/34c22ff2-4c41-40f9-8190-3ac4730ca050_isrForCrosstalkSources_997026_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/34c22ff2-4c41-40f9-8190-3ac4730ca050_isrForCrosstalkSources_997026_21.stderr parsl.dataflow.dflow DEBUG: Task 1576 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1576 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1576 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1576 has memoization hash 7fe5246450f34bc7b30cba0da7783604 parsl.dataflow.memoization INFO: Task 1576 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1576 try 0 launched on executor multi with executor id 1577 parsl.dataflow.dflow INFO: Standard output for task 1576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/bcd792d9-a55b-4247-8bde-90b789b415bf_isrForCrosstalkSources_997001_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/bcd792d9-a55b-4247-8bde-90b789b415bf_isrForCrosstalkSources_997001_30.stderr parsl.dataflow.dflow DEBUG: Task 1577 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1577 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1577 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1577 has memoization hash 8fac901427a692b62bda40de65c3e463 parsl.dataflow.memoization INFO: Task 1577 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1577 try 0 launched on executor multi with executor id 1578 parsl.dataflow.dflow INFO: Standard output for task 1577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a9459040-d0d5-41b4-b099-4fab03b41a5c_isrForCrosstalkSources_996996_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a9459040-d0d5-41b4-b099-4fab03b41a5c_isrForCrosstalkSources_996996_49.stderr parsl.dataflow.dflow DEBUG: Task 1578 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1578 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1578 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1578 has memoization hash 9f4b9c03b0e4b150d66b393a400f11ed parsl.dataflow.memoization INFO: Task 1578 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1578 try 0 launched on executor multi with executor id 1579 parsl.dataflow.dflow INFO: Standard output for task 1578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/f6b9e0e9-c1e4-42ef-9283-3e16d0fd1072_isrForCrosstalkSources_996976_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/f6b9e0e9-c1e4-42ef-9283-3e16d0fd1072_isrForCrosstalkSources_996976_20.stderr parsl.dataflow.dflow DEBUG: Task 1579 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1579 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1579 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1579 has memoization hash 8482fd2a76b058114f7d4626096cb4b6 parsl.dataflow.memoization INFO: Task 1579 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1579 try 0 launched on executor multi with executor id 1580 parsl.dataflow.dflow INFO: Standard output for task 1579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1186c2d8-8727-48fa-829f-16fc3bbbe05a_isrForCrosstalkSources_997000_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1186c2d8-8727-48fa-829f-16fc3bbbe05a_isrForCrosstalkSources_997000_35.stderr parsl.dataflow.dflow DEBUG: Task 1580 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1580 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1580 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1580 has memoization hash 0185c843a42f5970189fc63d8463c48f parsl.dataflow.memoization INFO: Task 1580 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1580 try 0 launched on executor multi with executor id 1581 parsl.dataflow.dflow INFO: Standard output for task 1580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/dba5bb04-70e3-4b26-ad72-463bf5974ec6_isrForCrosstalkSources_997008_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/dba5bb04-70e3-4b26-ad72-463bf5974ec6_isrForCrosstalkSources_997008_1.stderr parsl.dataflow.dflow DEBUG: Task 1581 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1581 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1581 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1581 has memoization hash 9bfc549fd8a44dd192f8a9185f1d1f10 parsl.dataflow.memoization INFO: Task 1581 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1581 try 0 launched on executor multi with executor id 1582 parsl.dataflow.dflow INFO: Standard output for task 1581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7285f41b-f367-4648-a71c-e6eb354175df_isrForCrosstalkSources_997015_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7285f41b-f367-4648-a71c-e6eb354175df_isrForCrosstalkSources_997015_5.stderr parsl.dataflow.dflow DEBUG: Task 1582 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1582 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1582 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1582 has memoization hash e982bfef72b6fbd0f84042caab089164 parsl.dataflow.memoization INFO: Task 1582 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1582 try 0 launched on executor multi with executor id 1583 parsl.dataflow.dflow INFO: Standard output for task 1582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/3224f670-f29e-44b7-85c7-484e108e827c_isrForCrosstalkSources_997009_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/3224f670-f29e-44b7-85c7-484e108e827c_isrForCrosstalkSources_997009_13.stderr parsl.dataflow.dflow DEBUG: Task 1583 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1583 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1583 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1583 has memoization hash 27e548bb79eaf103a1ebf46d9b4c74e9 parsl.dataflow.memoization INFO: Task 1583 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1583 try 0 launched on executor multi with executor id 1584 parsl.dataflow.dflow INFO: Standard output for task 1583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/5793d7dd-81a7-43f2-945a-78f3cb17889d_isrForCrosstalkSources_996975_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/5793d7dd-81a7-43f2-945a-78f3cb17889d_isrForCrosstalkSources_996975_31.stderr parsl.dataflow.dflow DEBUG: Task 1584 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1584 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1584 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1584 has memoization hash eae92035dab637759d6f82f20fdca0bb parsl.dataflow.memoization INFO: Task 1584 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1584 try 0 launched on executor multi with executor id 1585 parsl.dataflow.dflow INFO: Standard output for task 1584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/50a8c756-3764-4376-a642-43932e938c82_isrForCrosstalkSources_996955_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/50a8c756-3764-4376-a642-43932e938c82_isrForCrosstalkSources_996955_33.stderr parsl.dataflow.dflow DEBUG: Task 1585 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1585 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1585 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1585 has memoization hash e647b666619e0fb99395233894bfba69 parsl.dataflow.memoization INFO: Task 1585 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1585 try 0 launched on executor multi with executor id 1586 parsl.dataflow.dflow INFO: Standard output for task 1585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/4b937ebf-3c37-4360-8b25-62ef78ac8d35_isrForCrosstalkSources_996960_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/4b937ebf-3c37-4360-8b25-62ef78ac8d35_isrForCrosstalkSources_996960_29.stderr parsl.dataflow.dflow DEBUG: Task 1586 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1586 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1586 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1586 has memoization hash bef6f82c7a3a560ed27f3bfac9979f9e parsl.dataflow.memoization INFO: Task 1586 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1586 try 0 launched on executor multi with executor id 1587 parsl.dataflow.dflow INFO: Standard output for task 1586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/16e1f6a5-993f-443a-93fb-a0f04af78673_isrForCrosstalkSources_997023_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/16e1f6a5-993f-443a-93fb-a0f04af78673_isrForCrosstalkSources_997023_23.stderr parsl.dataflow.dflow DEBUG: Task 1587 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1587 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1587 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1587 has memoization hash 9aecd14404d43ba1830d6362eabe2ba0 parsl.dataflow.memoization INFO: Task 1587 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1587 try 0 launched on executor multi with executor id 1588 parsl.dataflow.dflow INFO: Standard output for task 1587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/8aaadfff-c125-4f05-bedc-7019926c2fff_isrForCrosstalkSources_996968_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/8aaadfff-c125-4f05-bedc-7019926c2fff_isrForCrosstalkSources_996968_22.stderr parsl.dataflow.dflow DEBUG: Task 1588 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1588 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1588 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1588 has memoization hash 654c112a5d8142e342b54bbd3c5c8387 parsl.dataflow.memoization INFO: Task 1588 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1588 try 0 launched on executor multi with executor id 1589 parsl.dataflow.dflow INFO: Standard output for task 1588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/f1abf74e-8431-49d6-928d-90e5555b595f_isrForCrosstalkSources_997013_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/f1abf74e-8431-49d6-928d-90e5555b595f_isrForCrosstalkSources_997013_14.stderr parsl.dataflow.dflow DEBUG: Task 1589 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1589 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1589 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1589 has memoization hash a3501ca32f038a6b7a287507a5f3d96e parsl.dataflow.memoization INFO: Task 1589 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1589 try 0 launched on executor multi with executor id 1590 parsl.dataflow.dflow INFO: Standard output for task 1589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/03b43be3-3892-4c93-a14d-1cbb878dc3be_isrForCrosstalkSources_996958_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/03b43be3-3892-4c93-a14d-1cbb878dc3be_isrForCrosstalkSources_996958_27.stderr parsl.dataflow.dflow DEBUG: Task 1590 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1590 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1590 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1590 has memoization hash 9eefb937cfab32255c8a4002caef428f parsl.dataflow.memoization INFO: Task 1590 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1590 try 0 launched on executor multi with executor id 1591 parsl.dataflow.dflow INFO: Standard output for task 1590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/babdcd8a-c4c3-4ac6-af1c-e488732051f8_isrForCrosstalkSources_997005_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/babdcd8a-c4c3-4ac6-af1c-e488732051f8_isrForCrosstalkSources_997005_8.stderr parsl.dataflow.dflow DEBUG: Task 1591 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1591 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1591 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1591 has memoization hash 265dd09546851ae0fe44fcff00bd164a parsl.dataflow.memoization INFO: Task 1591 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1591 try 0 launched on executor multi with executor id 1592 parsl.dataflow.dflow INFO: Standard output for task 1591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c9e40796-4ea2-41c6-be5d-6b8b955f99f8_isrForCrosstalkSources_996955_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c9e40796-4ea2-41c6-be5d-6b8b955f99f8_isrForCrosstalkSources_996955_13.stderr parsl.dataflow.dflow DEBUG: Task 1592 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1592 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1592 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1592 has memoization hash 0a8c777204cbff8a551b839f06c13c69 parsl.dataflow.memoization INFO: Task 1592 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1592 try 0 launched on executor multi with executor id 1593 parsl.dataflow.dflow INFO: Standard output for task 1592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/92bbfd61-7ec0-49fa-9652-e12675d9586d_isrForCrosstalkSources_996990_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/92bbfd61-7ec0-49fa-9652-e12675d9586d_isrForCrosstalkSources_996990_22.stderr parsl.dataflow.dflow DEBUG: Task 1593 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1593 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1593 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1593 has memoization hash 77da7b0d9ea386f7f2e521b1b9407a78 parsl.dataflow.memoization INFO: Task 1593 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1593 try 0 launched on executor multi with executor id 1594 parsl.dataflow.dflow INFO: Standard output for task 1593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e94627ee-9450-4b02-956f-c64c4d0e5019_isrForCrosstalkSources_996985_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e94627ee-9450-4b02-956f-c64c4d0e5019_isrForCrosstalkSources_996985_54.stderr parsl.dataflow.dflow DEBUG: Task 1594 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1594 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1594 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1594 has memoization hash 0a8f8855a72fe9fde7b38b3993784def parsl.dataflow.memoization INFO: Task 1594 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:42 proc_lsst.multi:170 [INFO] [multi] job_id 17074861 proc_lsst.multi INFO: [multi] job_id 17074861 2024-03-15 13:49:42 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:49:42 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:42 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074861 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074861 parsl.executors.status_handling DEBUG: Launched block 20 on executor multi with job ID 17074861 parsl.executors.status_handling INFO: Allocated block ID 21 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-21 2024-03-15 13:49:42 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=21 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-21 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=21 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-21 2024-03-15 13:49:42 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=21 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=21 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:42 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=21 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=21 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:49:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:42 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=21 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=21 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1594 try 0 launched on executor multi with executor id 1595 parsl.dataflow.dflow INFO: Standard output for task 1594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0b014c23-ef61-4170-a2e8-912968de9ce1_isrForCrosstalkSources_996991_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0b014c23-ef61-4170-a2e8-912968de9ce1_isrForCrosstalkSources_996991_38.stderr parsl.dataflow.dflow DEBUG: Task 1595 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1595 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1595 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1595 has memoization hash bc6a9594ae1de64f44cde1494c09e43f parsl.dataflow.memoization INFO: Task 1595 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1595 try 0 launched on executor multi with executor id 1596 parsl.dataflow.dflow INFO: Standard output for task 1595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a7ca9ef0-4dee-422e-95e4-953acf7196cb_isrForCrosstalkSources_997033_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a7ca9ef0-4dee-422e-95e4-953acf7196cb_isrForCrosstalkSources_997033_19.stderr parsl.dataflow.dflow DEBUG: Task 1596 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1596 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1596 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1596 has memoization hash 948708923ae2b1a81f09f41a66550772 parsl.dataflow.memoization INFO: Task 1596 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1596 try 0 launched on executor multi with executor id 1597 parsl.dataflow.dflow INFO: Standard output for task 1596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7c30c948-99ab-4287-b74c-5307b159940d_isrForCrosstalkSources_996971_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7c30c948-99ab-4287-b74c-5307b159940d_isrForCrosstalkSources_996971_55.stderr parsl.dataflow.dflow DEBUG: Task 1597 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1597 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1597 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1597 has memoization hash 63d93f58f69596fce709a97229d00186 parsl.dataflow.memoization INFO: Task 1597 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1597 try 0 launched on executor multi with executor id 1598 parsl.dataflow.dflow INFO: Standard output for task 1597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/e9811915-206a-42bf-911e-3764d7b21245_isrForCrosstalkSources_997027_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/e9811915-206a-42bf-911e-3764d7b21245_isrForCrosstalkSources_997027_61.stderr parsl.dataflow.dflow DEBUG: Task 1598 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1598 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1598 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1598 has memoization hash 26933d1ba5e6bdae3197ceb0b427a0c3 parsl.dataflow.memoization INFO: Task 1598 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1598 try 0 launched on executor multi with executor id 1599 parsl.dataflow.dflow INFO: Standard output for task 1598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/dd6dcfdc-cf4e-4b4c-8634-1fc390b94de2_isrForCrosstalkSources_997002_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/dd6dcfdc-cf4e-4b4c-8634-1fc390b94de2_isrForCrosstalkSources_997002_55.stderr parsl.dataflow.dflow DEBUG: Task 1599 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1599 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1599 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1599 has memoization hash 1e4106f200a0dc5ee8442a6a09c3eb47 parsl.dataflow.memoization INFO: Task 1599 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1599 try 0 launched on executor multi with executor id 1600 parsl.dataflow.dflow INFO: Standard output for task 1599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/bf663ffa-b048-4f73-b990-3e49dd35acb6_isrForCrosstalkSources_996987_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/bf663ffa-b048-4f73-b990-3e49dd35acb6_isrForCrosstalkSources_996987_42.stderr parsl.dataflow.dflow DEBUG: Task 1600 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1600 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1600 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1600 has memoization hash 23e0dbcdb9a6c2d4af87aab15babda9d parsl.dataflow.memoization INFO: Task 1600 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1600 try 0 launched on executor multi with executor id 1601 parsl.dataflow.dflow INFO: Standard output for task 1600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b729d635-f3a7-4e89-8ec1-ecfb3711b40d_isrForCrosstalkSources_997014_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b729d635-f3a7-4e89-8ec1-ecfb3711b40d_isrForCrosstalkSources_997014_17.stderr parsl.dataflow.dflow DEBUG: Task 1601 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1601 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1601 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1601 has memoization hash c2de7566001eb115a765c89f95b78ca2 parsl.dataflow.memoization INFO: Task 1601 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1601 try 0 launched on executor multi with executor id 1602 parsl.dataflow.dflow INFO: Standard output for task 1601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/085a5886-7843-4a66-bead-b9769171d559_isrForCrosstalkSources_996977_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/085a5886-7843-4a66-bead-b9769171d559_isrForCrosstalkSources_996977_47.stderr parsl.dataflow.dflow DEBUG: Task 1602 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1602 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1602 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1602 has memoization hash 70a4059dbba552ec380796f9e1caa7ff parsl.dataflow.memoization INFO: Task 1602 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1602 try 0 launched on executor multi with executor id 1603 parsl.dataflow.dflow INFO: Standard output for task 1602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/41775b54-5980-4dcc-80b0-8b1ffddf3a25_isrForCrosstalkSources_997013_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/41775b54-5980-4dcc-80b0-8b1ffddf3a25_isrForCrosstalkSources_997013_59.stderr parsl.dataflow.dflow DEBUG: Task 1603 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1603 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1603 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1603 has memoization hash e9600bde5d9786c3fd78248358953418 parsl.dataflow.memoization INFO: Task 1603 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1603 try 0 launched on executor multi with executor id 1604 parsl.dataflow.dflow INFO: Standard output for task 1603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/01bd91ef-6c22-4f30-9ecd-a37d3b7daf18_isrForCrosstalkSources_997012_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/01bd91ef-6c22-4f30-9ecd-a37d3b7daf18_isrForCrosstalkSources_997012_56.stderr parsl.dataflow.dflow DEBUG: Task 1604 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1604 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1604 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1604 has memoization hash e45b8d3b7dd25dce7519237b8df9d06f parsl.dataflow.memoization INFO: Task 1604 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1604 try 0 launched on executor multi with executor id 1605 parsl.dataflow.dflow INFO: Standard output for task 1604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/5ed1126f-07d6-47c7-9b4f-424458ccea50_isrForCrosstalkSources_997020_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/5ed1126f-07d6-47c7-9b4f-424458ccea50_isrForCrosstalkSources_997020_15.stderr parsl.dataflow.dflow DEBUG: Task 1605 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1605 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1605 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1605 has memoization hash c611ca677d2da60d8a9ffafdedd055c7 parsl.dataflow.memoization INFO: Task 1605 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1605 try 0 launched on executor multi with executor id 1606 parsl.dataflow.dflow INFO: Standard output for task 1605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/74a90412-7cb6-47d3-a69c-7847c05669f4_isrForCrosstalkSources_997016_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/74a90412-7cb6-47d3-a69c-7847c05669f4_isrForCrosstalkSources_997016_36.stderr parsl.dataflow.dflow DEBUG: Task 1606 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1606 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1606 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1606 has memoization hash da81b1e6e05fb94a318c550a40a7de96 parsl.dataflow.memoization INFO: Task 1606 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1606 try 0 launched on executor multi with executor id 1607 parsl.dataflow.dflow INFO: Standard output for task 1606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/95b45f0d-38d3-43ff-93d2-ff3e8c61088f_isrForCrosstalkSources_997025_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/95b45f0d-38d3-43ff-93d2-ff3e8c61088f_isrForCrosstalkSources_997025_12.stderr parsl.dataflow.dflow DEBUG: Task 1607 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1607 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1607 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1607 has memoization hash feff27cd1a36a3b5b8140c218663621d parsl.dataflow.memoization INFO: Task 1607 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1607 try 0 launched on executor multi with executor id 1608 parsl.dataflow.dflow INFO: Standard output for task 1607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/c56ba9d5-f516-480c-bd25-47c0c325682d_isrForCrosstalkSources_997031_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/c56ba9d5-f516-480c-bd25-47c0c325682d_isrForCrosstalkSources_997031_19.stderr parsl.dataflow.dflow DEBUG: Task 1608 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1608 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1608 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1608 has memoization hash dbb1a06cd1cec34404322dc0366deebd parsl.dataflow.memoization INFO: Task 1608 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1608 try 0 launched on executor multi with executor id 1609 parsl.dataflow.dflow INFO: Standard output for task 1608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e875d7e8-898a-430d-b621-e1ec7398102b_isrForCrosstalkSources_997026_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e875d7e8-898a-430d-b621-e1ec7398102b_isrForCrosstalkSources_997026_25.stderr parsl.dataflow.dflow DEBUG: Task 1609 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1609 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1609 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1609 has memoization hash 94ce2abb51194a1fb72addc5af1bcc4d parsl.dataflow.memoization INFO: Task 1609 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1609 try 0 launched on executor multi with executor id 1610 parsl.dataflow.dflow INFO: Standard output for task 1609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/dd25be28-b1ba-4477-b85b-47e03b5f04ea_isrForCrosstalkSources_996973_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/dd25be28-b1ba-4477-b85b-47e03b5f04ea_isrForCrosstalkSources_996973_21.stderr parsl.dataflow.dflow DEBUG: Task 1610 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1610 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1610 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1610 has memoization hash cdba0a9893bdcfe8900cd13bd1fe22cc parsl.dataflow.memoization INFO: Task 1610 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1610 try 0 launched on executor multi with executor id 1611 parsl.dataflow.dflow INFO: Standard output for task 1610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/f0c714ae-5b4a-4462-866e-a6de713c0590_isrForCrosstalkSources_997022_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/f0c714ae-5b4a-4462-866e-a6de713c0590_isrForCrosstalkSources_997022_12.stderr parsl.dataflow.dflow DEBUG: Task 1611 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1611 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1611 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1611 has memoization hash bef5fdfa41323bfc3523b93f5907854c parsl.dataflow.memoization INFO: Task 1611 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1611 try 0 launched on executor multi with executor id 1612 parsl.dataflow.dflow INFO: Standard output for task 1611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a48b467b-3e7d-41e1-949e-da0d520f19ab_isrForCrosstalkSources_997011_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a48b467b-3e7d-41e1-949e-da0d520f19ab_isrForCrosstalkSources_997011_4.stderr parsl.dataflow.dflow DEBUG: Task 1612 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1612 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1612 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1612 has memoization hash 1295295b9bcbbbea637d71d0f7183261 parsl.dataflow.memoization INFO: Task 1612 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1612 try 0 launched on executor multi with executor id 1613 parsl.dataflow.dflow INFO: Standard output for task 1612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/57add925-48ba-4c39-92b3-7a8eadc94700_isrForCrosstalkSources_996987_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/57add925-48ba-4c39-92b3-7a8eadc94700_isrForCrosstalkSources_996987_62.stderr parsl.dataflow.dflow DEBUG: Task 1613 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1613 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1613 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1613 has memoization hash bf7bb336d43e91aadc8d23f315cc989f parsl.dataflow.memoization INFO: Task 1613 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1613 try 0 launched on executor multi with executor id 1614 parsl.dataflow.dflow INFO: Standard output for task 1613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/af5ebe60-181e-4c74-b60b-5242a29dc514_isrForCrosstalkSources_996971_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/af5ebe60-181e-4c74-b60b-5242a29dc514_isrForCrosstalkSources_996971_33.stderr parsl.dataflow.dflow DEBUG: Task 1614 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1614 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1614 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1614 has memoization hash 94a8c386493608c679f97f98d5426d34 parsl.dataflow.memoization INFO: Task 1614 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1614 try 0 launched on executor multi with executor id 1615 parsl.dataflow.dflow INFO: Standard output for task 1614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/a4bf2e22-cd8d-4cd3-87a8-f2be728c423a_isrForCrosstalkSources_997003_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/a4bf2e22-cd8d-4cd3-87a8-f2be728c423a_isrForCrosstalkSources_997003_45.stderr parsl.dataflow.dflow DEBUG: Task 1615 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1615 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1615 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1615 has memoization hash db37331e03295cc14b26a12100673f69 parsl.dataflow.memoization INFO: Task 1615 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1615 try 0 launched on executor multi with executor id 1616 parsl.dataflow.dflow INFO: Standard output for task 1615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/8c0f21f1-1e61-4d6e-871a-e73fe78a3d78_isrForCrosstalkSources_997030_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/8c0f21f1-1e61-4d6e-871a-e73fe78a3d78_isrForCrosstalkSources_997030_3.stderr parsl.dataflow.dflow DEBUG: Task 1616 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1616 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1616 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1616 has memoization hash 8eac9856d1a14f9c120631f34d062639 parsl.dataflow.memoization INFO: Task 1616 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1616 try 0 launched on executor multi with executor id 1617 parsl.dataflow.dflow INFO: Standard output for task 1616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/3816b224-72b8-4009-8f82-3275ffeea8b6_isrForCrosstalkSources_996965_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/3816b224-72b8-4009-8f82-3275ffeea8b6_isrForCrosstalkSources_996965_24.stderr parsl.dataflow.dflow DEBUG: Task 1617 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1617 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1617 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1617 has memoization hash 71ddc945807d3ac77f21e08e75c5b676 parsl.dataflow.memoization INFO: Task 1617 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1617 try 0 launched on executor multi with executor id 1618 parsl.dataflow.dflow INFO: Standard output for task 1617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/8e01f1aa-c888-4bbf-8eff-3ab77ae6caf1_isrForCrosstalkSources_996969_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/8e01f1aa-c888-4bbf-8eff-3ab77ae6caf1_isrForCrosstalkSources_996969_25.stderr parsl.dataflow.dflow DEBUG: Task 1618 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1618 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1618 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1618 has memoization hash 934d5d6526b3411ea016dadae2a8f610 parsl.dataflow.memoization INFO: Task 1618 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1618 try 0 launched on executor multi with executor id 1619 parsl.dataflow.dflow INFO: Standard output for task 1618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5ee7713a-d4ea-4bfd-a47a-964f31d04913_isrForCrosstalkSources_996954_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5ee7713a-d4ea-4bfd-a47a-964f31d04913_isrForCrosstalkSources_996954_10.stderr parsl.dataflow.dflow DEBUG: Task 1619 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1619 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1619 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1619 has memoization hash 1da0f188a376a071e0aa6f163870ecef parsl.dataflow.memoization INFO: Task 1619 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1619 try 0 launched on executor multi with executor id 1620 parsl.dataflow.dflow INFO: Standard output for task 1619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e143eece-9394-4641-9818-cc73cd9576d8_isrForCrosstalkSources_996984_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e143eece-9394-4641-9818-cc73cd9576d8_isrForCrosstalkSources_996984_7.stderr parsl.dataflow.dflow DEBUG: Task 1620 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1620 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1620 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1620 has memoization hash b400e5a343442417652bf401de3f1eb3 parsl.dataflow.memoization INFO: Task 1620 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1620 try 0 launched on executor multi with executor id 1621 parsl.dataflow.dflow INFO: Standard output for task 1620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/de6f77b6-b585-4a5f-84d7-d6cd3140d5ec_isrForCrosstalkSources_996966_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/de6f77b6-b585-4a5f-84d7-d6cd3140d5ec_isrForCrosstalkSources_996966_14.stderr parsl.dataflow.dflow DEBUG: Task 1621 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1621 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1621 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1621 has memoization hash 00de6fbb3ad888c1dd32735181858abd parsl.dataflow.memoization INFO: Task 1621 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1621 try 0 launched on executor multi with executor id 1622 parsl.dataflow.dflow INFO: Standard output for task 1621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c24d7cd0-20c9-4ce3-9f7f-fa31826541d4_isrForCrosstalkSources_996985_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c24d7cd0-20c9-4ce3-9f7f-fa31826541d4_isrForCrosstalkSources_996985_62.stderr parsl.dataflow.dflow DEBUG: Task 1622 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1622 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1622 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1622 has memoization hash 952281a98f0527848293d4ecc9ba6c23 parsl.dataflow.memoization INFO: Task 1622 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1622 try 0 launched on executor multi with executor id 1623 parsl.dataflow.dflow INFO: Standard output for task 1622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/315e9df4-9e9b-41cf-966b-9da910761a9c_isrForCrosstalkSources_997013_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/315e9df4-9e9b-41cf-966b-9da910761a9c_isrForCrosstalkSources_997013_30.stderr parsl.dataflow.dflow DEBUG: Task 1623 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1623 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1623 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1623 has memoization hash 93f54e062526e11cbf8157d279ec51fc parsl.dataflow.memoization INFO: Task 1623 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1623 try 0 launched on executor multi with executor id 1624 parsl.dataflow.dflow INFO: Standard output for task 1623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/25357b8a-d47d-4070-ade7-5ea411c6a599_isrForCrosstalkSources_996965_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/25357b8a-d47d-4070-ade7-5ea411c6a599_isrForCrosstalkSources_996965_43.stderr parsl.dataflow.dflow DEBUG: Task 1624 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1624 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1624 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1624 has memoization hash f856dd9d2469c8a5339a80140a4b3b07 parsl.dataflow.memoization INFO: Task 1624 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1624 try 0 launched on executor multi with executor id 1625 parsl.dataflow.dflow INFO: Standard output for task 1624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6148973a-0263-4ecb-b13f-bd4c014232d3_isrForCrosstalkSources_997017_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6148973a-0263-4ecb-b13f-bd4c014232d3_isrForCrosstalkSources_997017_31.stderr parsl.dataflow.dflow DEBUG: Task 1625 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1625 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1625 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1625 has memoization hash 85bc9fc59be21b5cb22fb09783660dc5 parsl.dataflow.memoization INFO: Task 1625 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1625 try 0 launched on executor multi with executor id 1626 parsl.dataflow.dflow INFO: Standard output for task 1625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/f3b27d9d-fc5b-4d50-9f6b-2c7586f0be90_isrForCrosstalkSources_996983_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/f3b27d9d-fc5b-4d50-9f6b-2c7586f0be90_isrForCrosstalkSources_996983_42.stderr parsl.dataflow.dflow DEBUG: Task 1626 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1626 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1626 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1626 has memoization hash 2f35f83094f38b54bfa21ea6737aef41 parsl.dataflow.memoization INFO: Task 1626 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b80e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1626 try 0 launched on executor multi with executor id 1627 parsl.dataflow.dflow INFO: Standard output for task 1626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ca164872-4693-4c12-a0fe-d3791246c892_isrForCrosstalkSources_996998_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ca164872-4693-4c12-a0fe-d3791246c892_isrForCrosstalkSources_996998_3.stderr parsl.dataflow.dflow DEBUG: Task 1627 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1627 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1627 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1627 has memoization hash fa9328cc48cd63e470513ac66ff70b98 parsl.dataflow.memoization INFO: Task 1627 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1627 try 0 launched on executor multi with executor id 1628 parsl.dataflow.dflow INFO: Standard output for task 1627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b11a649c-0a17-4544-9600-1fe04d7d7f13_isrForCrosstalkSources_996979_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b11a649c-0a17-4544-9600-1fe04d7d7f13_isrForCrosstalkSources_996979_10.stderr parsl.dataflow.dflow DEBUG: Task 1628 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1628 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1628 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1628 has memoization hash b697a203244cc5fabdca8dadb04c6a74 parsl.dataflow.memoization INFO: Task 1628 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1628 try 0 launched on executor multi with executor id 1629 parsl.dataflow.dflow INFO: Standard output for task 1628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/4bc7eae2-467b-4cf0-a4a0-a4621c4ec1cc_isrForCrosstalkSources_996984_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/4bc7eae2-467b-4cf0-a4a0-a4621c4ec1cc_isrForCrosstalkSources_996984_47.stderr parsl.dataflow.dflow DEBUG: Task 1629 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1629 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1629 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1629 has memoization hash 2b1a0eb0fc0db5e4d6f7c6491deb77b7 parsl.dataflow.memoization INFO: Task 1629 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1629 try 0 launched on executor multi with executor id 1630 parsl.dataflow.dflow INFO: Standard output for task 1629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/455fb03a-3d89-4a67-aa49-54c962b95618_isrForCrosstalkSources_997005_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/455fb03a-3d89-4a67-aa49-54c962b95618_isrForCrosstalkSources_997005_5.stderr parsl.dataflow.dflow DEBUG: Task 1630 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1630 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1630 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1630 has memoization hash 453647e33e868bc421d0727b2b883a54 parsl.dataflow.memoization INFO: Task 1630 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b82c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1630 try 0 launched on executor multi with executor id 1631 parsl.dataflow.dflow INFO: Standard output for task 1630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4964578a-a35a-4c15-b1e1-00a9cacac8a7_isrForCrosstalkSources_996954_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4964578a-a35a-4c15-b1e1-00a9cacac8a7_isrForCrosstalkSources_996954_45.stderr parsl.dataflow.dflow DEBUG: Task 1631 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1631 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1631 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1631 has memoization hash ba27ec4e09ef8e5d3d85ab73b92221ac parsl.dataflow.memoization INFO: Task 1631 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1631 try 0 launched on executor multi with executor id 1632 parsl.dataflow.dflow INFO: Standard output for task 1631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/2bd22fe3-4a33-4d14-880d-e88ca92bc0b9_isrForCrosstalkSources_997009_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/2bd22fe3-4a33-4d14-880d-e88ca92bc0b9_isrForCrosstalkSources_997009_55.stderr parsl.dataflow.dflow DEBUG: Task 1632 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1632 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1632 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1632 has memoization hash 058332e4fafeae593b966d8a593fb20c parsl.dataflow.memoization INFO: Task 1632 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b84a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1632 try 0 launched on executor multi with executor id 1633 parsl.dataflow.dflow INFO: Standard output for task 1632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/31fd8b32-87b2-4428-9041-5c8c79c10815_isrForCrosstalkSources_996973_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/31fd8b32-87b2-4428-9041-5c8c79c10815_isrForCrosstalkSources_996973_14.stderr parsl.dataflow.dflow DEBUG: Task 1633 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1633 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1633 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1633 has memoization hash 003c4c8bbe7858d4a26067c00d6ea209 parsl.dataflow.memoization INFO: Task 1633 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1633 try 0 launched on executor multi with executor id 1634 parsl.dataflow.dflow INFO: Standard output for task 1633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/5bea4852-f4e3-4f43-b6a0-cfe8655c2f3b_isrForCrosstalkSources_997002_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/5bea4852-f4e3-4f43-b6a0-cfe8655c2f3b_isrForCrosstalkSources_997002_31.stderr parsl.dataflow.dflow DEBUG: Task 1634 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1634 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1634 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1634 has memoization hash c9e63eaf8c4e3c7f818ca1435a851555 parsl.dataflow.memoization INFO: Task 1634 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1634 try 0 launched on executor multi with executor id 1635 parsl.dataflow.dflow INFO: Standard output for task 1634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b03b64c6-5088-4b17-b505-9cb5b148508e_isrForCrosstalkSources_997007_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b03b64c6-5088-4b17-b505-9cb5b148508e_isrForCrosstalkSources_997007_56.stderr parsl.dataflow.dflow DEBUG: Task 1635 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1635 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1635 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1635 has memoization hash 239902bdfa9edc9783cbad6f87b10735 parsl.dataflow.memoization INFO: Task 1635 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b85e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1635 try 0 launched on executor multi with executor id 1636 parsl.dataflow.dflow INFO: Standard output for task 1635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/85d09345-5669-4bd8-b619-01392e824a92_isrForCrosstalkSources_997028_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/85d09345-5669-4bd8-b619-01392e824a92_isrForCrosstalkSources_997028_49.stderr parsl.dataflow.dflow DEBUG: Task 1636 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1636 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1636 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1636 has memoization hash fa153dedf5d3321aba918a5698f22b3c parsl.dataflow.memoization INFO: Task 1636 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1636 try 0 launched on executor multi with executor id 1637 parsl.dataflow.dflow INFO: Standard output for task 1636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/cb400988-aaca-4f02-a39f-1b0cec073132_isrForCrosstalkSources_996967_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/cb400988-aaca-4f02-a39f-1b0cec073132_isrForCrosstalkSources_996967_9.stderr parsl.dataflow.dflow DEBUG: Task 1637 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1637 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1637 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1637 has memoization hash ebc29c4c134027bda9cdf521850f08f1 parsl.dataflow.memoization INFO: Task 1637 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b87c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1637 try 0 launched on executor multi with executor id 1638 parsl.dataflow.dflow INFO: Standard output for task 1637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/4754eb95-3137-4190-8dc4-cb5db3681ceb_isrForCrosstalkSources_996991_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/4754eb95-3137-4190-8dc4-cb5db3681ceb_isrForCrosstalkSources_996991_37.stderr parsl.dataflow.dflow DEBUG: Task 1638 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1638 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1638 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1638 has memoization hash 304d73605115156950eb37e0e0c37258 parsl.dataflow.memoization INFO: Task 1638 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1638 try 0 launched on executor multi with executor id 1639 parsl.dataflow.dflow INFO: Standard output for task 1638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f0d85de7-b3c1-449c-be99-0afc6c218461_isrForCrosstalkSources_996986_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f0d85de7-b3c1-449c-be99-0afc6c218461_isrForCrosstalkSources_996986_15.stderr parsl.dataflow.dflow DEBUG: Task 1639 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1639 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1639 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1639 has memoization hash c5bc1d3047f0271d625cfc458492f1e1 parsl.dataflow.memoization INFO: Task 1639 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b89a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1639 try 0 launched on executor multi with executor id 1640 parsl.dataflow.dflow INFO: Standard output for task 1639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4e7d6628-47e8-4e29-978b-b5a4f95d3824_isrForCrosstalkSources_996999_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4e7d6628-47e8-4e29-978b-b5a4f95d3824_isrForCrosstalkSources_996999_30.stderr parsl.dataflow.dflow DEBUG: Task 1640 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1640 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1640 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1640 has memoization hash fa551310c2a7f718947f4d42b309d1a9 parsl.dataflow.memoization INFO: Task 1640 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1640 try 0 launched on executor multi with executor id 1641 parsl.dataflow.dflow INFO: Standard output for task 1640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/4f57e350-6256-4b6d-aef0-5ba06ae44e0a_isrForCrosstalkSources_996982_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/4f57e350-6256-4b6d-aef0-5ba06ae44e0a_isrForCrosstalkSources_996982_6.stderr parsl.dataflow.dflow DEBUG: Task 1641 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1641 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1641 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1641 has memoization hash 3b986805adb9c60da3382308c582becb parsl.dataflow.memoization INFO: Task 1641 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1641 try 0 launched on executor multi with executor id 1642 parsl.dataflow.dflow INFO: Standard output for task 1641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/3e18b4ab-d886-491d-9e64-0cfab1a2e09a_isrForCrosstalkSources_997004_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/3e18b4ab-d886-491d-9e64-0cfab1a2e09a_isrForCrosstalkSources_997004_48.stderr parsl.dataflow.dflow DEBUG: Task 1642 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1642 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1642 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1642 has memoization hash 3c91e2ff132dd33ae325f928b0e746f3 parsl.dataflow.memoization INFO: Task 1642 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1642 try 0 launched on executor multi with executor id 1643 parsl.dataflow.dflow INFO: Standard output for task 1642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6ddf105f-a82b-487f-983f-d5b1fdcfe584_isrForCrosstalkSources_996971_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6ddf105f-a82b-487f-983f-d5b1fdcfe584_isrForCrosstalkSources_996971_32.stderr parsl.dataflow.dflow DEBUG: Task 1643 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1643 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1643 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1643 has memoization hash c44cdd4f9079b55396c75c45e22f3561 parsl.dataflow.memoization INFO: Task 1643 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1643 try 0 launched on executor multi with executor id 1644 parsl.dataflow.dflow INFO: Standard output for task 1643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/53c17f98-0b53-4bfc-ba99-232010f149a3_isrForCrosstalkSources_997006_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/53c17f98-0b53-4bfc-ba99-232010f149a3_isrForCrosstalkSources_997006_56.stderr parsl.dataflow.dflow DEBUG: Task 1644 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1644 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1644 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1644 has memoization hash 60edd8e82f8d5d983ce8b6ddf0a6335b parsl.dataflow.memoization INFO: Task 1644 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1644 try 0 launched on executor multi with executor id 1645 parsl.dataflow.dflow INFO: Standard output for task 1644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/642497be-7a33-4a9d-aed1-8b2fa05d54a0_isrForCrosstalkSources_997011_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/642497be-7a33-4a9d-aed1-8b2fa05d54a0_isrForCrosstalkSources_997011_27.stderr parsl.dataflow.dflow DEBUG: Task 1645 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1645 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1645 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1645 has memoization hash c44c4291783a81a654978a3b96374d52 parsl.dataflow.memoization INFO: Task 1645 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1645 try 0 launched on executor multi with executor id 1646 parsl.dataflow.dflow INFO: Standard output for task 1645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/5600fcb8-8e30-44fb-860c-8b122d1701b6_isrForCrosstalkSources_996990_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/5600fcb8-8e30-44fb-860c-8b122d1701b6_isrForCrosstalkSources_996990_57.stderr parsl.dataflow.dflow DEBUG: Task 1646 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1646 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1646 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1646 has memoization hash bcde8b6e964d168ee3680d7748e9167c parsl.dataflow.memoization INFO: Task 1646 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1646 try 0 launched on executor multi with executor id 1647 parsl.dataflow.dflow INFO: Standard output for task 1646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/793f86ef-4e9d-437c-b137-901967300db0_isrForCrosstalkSources_997019_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/793f86ef-4e9d-437c-b137-901967300db0_isrForCrosstalkSources_997019_49.stderr parsl.dataflow.dflow DEBUG: Task 1647 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1647 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1647 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1647 has memoization hash c8c673778396a5dfb702eda57e64195f parsl.dataflow.memoization INFO: Task 1647 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1647 try 0 launched on executor multi with executor id 1648 parsl.dataflow.dflow INFO: Standard output for task 1647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/14a30738-9525-4585-8443-cc5efb8a7d8c_isrForCrosstalkSources_996977_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/14a30738-9525-4585-8443-cc5efb8a7d8c_isrForCrosstalkSources_996977_30.stderr parsl.dataflow.dflow DEBUG: Task 1648 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1648 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1648 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1648 has memoization hash 2bce0ed17b7ffaab2fdf18965f1f3939 parsl.dataflow.memoization INFO: Task 1648 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1648 try 0 launched on executor multi with executor id 1649 parsl.dataflow.dflow INFO: Standard output for task 1648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ec0d97a8-8ece-4a3c-a0b4-f8e1eec1d79e_isrForCrosstalkSources_997021_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ec0d97a8-8ece-4a3c-a0b4-f8e1eec1d79e_isrForCrosstalkSources_997021_4.stderr parsl.dataflow.dflow DEBUG: Task 1649 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1649 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1649 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1649 has memoization hash 303813636b5461844b7f2aeafffce737 parsl.dataflow.memoization INFO: Task 1649 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1649 try 0 launched on executor multi with executor id 1650 parsl.dataflow.dflow INFO: Standard output for task 1649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b6c25168-e813-44ec-b227-51948e9b0231_isrForCrosstalkSources_996992_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b6c25168-e813-44ec-b227-51948e9b0231_isrForCrosstalkSources_996992_31.stderr parsl.dataflow.dflow DEBUG: Task 1650 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1650 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1650 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1650 has memoization hash 0afb6b992ec1b811fdbb1b5d91265e73 parsl.dataflow.memoization INFO: Task 1650 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1650 try 0 launched on executor multi with executor id 1651 parsl.dataflow.dflow INFO: Standard output for task 1650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2021634d-95a7-4520-9254-0a8f0c3580c9_isrForCrosstalkSources_997002_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2021634d-95a7-4520-9254-0a8f0c3580c9_isrForCrosstalkSources_997002_1.stderr parsl.dataflow.dflow DEBUG: Task 1651 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1651 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1651 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1651 has memoization hash a8c1e76eba45cb1cb98847d71db70090 parsl.dataflow.memoization INFO: Task 1651 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1651 try 0 launched on executor multi with executor id 1652 parsl.dataflow.dflow INFO: Standard output for task 1651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e94f31a2-1652-4e3e-8062-5e763cfe37a8_isrForCrosstalkSources_996993_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e94f31a2-1652-4e3e-8062-5e763cfe37a8_isrForCrosstalkSources_996993_46.stderr parsl.dataflow.dflow DEBUG: Task 1652 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1652 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1652 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1652 has memoization hash 7e90beed6a707f191e3e73a362b54daa parsl.dataflow.memoization INFO: Task 1652 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1652 try 0 launched on executor multi with executor id 1653 parsl.dataflow.dflow INFO: Standard output for task 1652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/4f639df0-6fde-498d-87a6-0fc45266dd61_isrForCrosstalkSources_996978_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/4f639df0-6fde-498d-87a6-0fc45266dd61_isrForCrosstalkSources_996978_43.stderr parsl.dataflow.dflow DEBUG: Task 1653 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1653 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1653 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1653 has memoization hash 6a74a69ed2ed5c21bcd1ed09e8ad0126 parsl.dataflow.memoization INFO: Task 1653 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b91c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1653 try 0 launched on executor multi with executor id 1654 parsl.dataflow.dflow INFO: Standard output for task 1653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/a0ec4fbf-d4e2-41f4-b7ad-340c95071a22_isrForCrosstalkSources_997009_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/a0ec4fbf-d4e2-41f4-b7ad-340c95071a22_isrForCrosstalkSources_997009_35.stderr parsl.dataflow.dflow DEBUG: Task 1654 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1654 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1654 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1654 has memoization hash ff94a1cd60143e19e19a61130b2265f3 parsl.dataflow.memoization INFO: Task 1654 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1654 try 0 launched on executor multi with executor id 1655 parsl.dataflow.dflow INFO: Standard output for task 1654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/2f324fbd-cc58-40c0-8b53-38c76319a3e7_isrForCrosstalkSources_997032_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/2f324fbd-cc58-40c0-8b53-38c76319a3e7_isrForCrosstalkSources_997032_18.stderr parsl.dataflow.dflow DEBUG: Task 1655 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1655 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1655 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1655 has memoization hash 240081eb5021da0e43f9240acdd1f7c0 parsl.dataflow.memoization INFO: Task 1655 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1655 try 0 launched on executor multi with executor id 1656 parsl.dataflow.dflow INFO: Standard output for task 1655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/51d0b279-59e5-47be-a230-b9da48f7ebf7_isrForCrosstalkSources_996989_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/51d0b279-59e5-47be-a230-b9da48f7ebf7_isrForCrosstalkSources_996989_45.stderr parsl.dataflow.dflow DEBUG: Task 1656 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1656 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1656 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1656 has memoization hash 03991dbf6dd815a9757bb5ec25a51fed parsl.dataflow.memoization INFO: Task 1656 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b93a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1656 try 0 launched on executor multi with executor id 1657 parsl.dataflow.dflow INFO: Standard output for task 1656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/48833159-ea8f-43b6-aed5-b35a110d8c60_isrForCrosstalkSources_997030_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/48833159-ea8f-43b6-aed5-b35a110d8c60_isrForCrosstalkSources_997030_55.stderr parsl.dataflow.dflow DEBUG: Task 1657 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1657 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1657 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1657 has memoization hash d5b59c189caecd56f9c9b3748bdd75cf parsl.dataflow.memoization INFO: Task 1657 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1657 try 0 launched on executor multi with executor id 1658 parsl.dataflow.dflow INFO: Standard output for task 1657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/85557699-e752-4b90-b806-5c78ba0bcd8c_isrForCrosstalkSources_996994_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/85557699-e752-4b90-b806-5c78ba0bcd8c_isrForCrosstalkSources_996994_2.stderr parsl.dataflow.dflow DEBUG: Task 1658 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1658 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1658 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1658 has memoization hash 73668641246169bc86985c2719a53fa8 parsl.dataflow.memoization INFO: Task 1658 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b94e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1658 try 0 launched on executor multi with executor id 1659 parsl.dataflow.dflow INFO: Standard output for task 1658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d0c54afe-c1c3-420d-b70a-0444c5ae9996_isrForCrosstalkSources_997004_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d0c54afe-c1c3-420d-b70a-0444c5ae9996_isrForCrosstalkSources_997004_45.stderr parsl.dataflow.dflow DEBUG: Task 1659 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1659 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1659 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1659 has memoization hash 53ac8c37d8b3eaeb34fc95b1ed240796 parsl.dataflow.memoization INFO: Task 1659 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1659 try 0 launched on executor multi with executor id 1660 parsl.dataflow.dflow INFO: Standard output for task 1659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1ba1b0f4-39a1-4d37-a2b7-45ff5717c076_isrForCrosstalkSources_997022_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1ba1b0f4-39a1-4d37-a2b7-45ff5717c076_isrForCrosstalkSources_997022_2.stderr parsl.dataflow.dflow DEBUG: Task 1660 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1660 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1660 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1660 has memoization hash 8d5c9772b1430aa09721655f5d2bd2c5 parsl.dataflow.memoization INFO: Task 1660 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1660 try 0 launched on executor multi with executor id 1661 parsl.dataflow.dflow INFO: Standard output for task 1660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3791e36f-9328-4aab-a2f9-c1af2b4f54d0_isrForCrosstalkSources_997033_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3791e36f-9328-4aab-a2f9-c1af2b4f54d0_isrForCrosstalkSources_997033_55.stderr parsl.dataflow.dflow DEBUG: Task 1661 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1661 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1661 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1661 has memoization hash 50a3d79e7bc3431394ad251d08784b1a parsl.dataflow.memoization INFO: Task 1661 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b96c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1661 try 0 launched on executor multi with executor id 1662 parsl.dataflow.dflow INFO: Standard output for task 1661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e09c2bb2-c1bf-4425-87c9-c3765f9a88e2_isrForCrosstalkSources_997011_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e09c2bb2-c1bf-4425-87c9-c3765f9a88e2_isrForCrosstalkSources_997011_20.stderr parsl.dataflow.dflow DEBUG: Task 1662 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1662 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1662 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1662 has memoization hash d73162e34d64999fdc23f29df5248a05 parsl.dataflow.memoization INFO: Task 1662 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1662 try 0 launched on executor multi with executor id 1663 parsl.dataflow.dflow INFO: Standard output for task 1662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/86c0c54c-b93d-4816-9020-9fab717e6971_isrForCrosstalkSources_997002_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/86c0c54c-b93d-4816-9020-9fab717e6971_isrForCrosstalkSources_997002_14.stderr parsl.dataflow.dflow DEBUG: Task 1663 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1663 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1663 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1663 has memoization hash 46619a4c37d53f35e3c9d811f837b74a parsl.dataflow.memoization INFO: Task 1663 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1663 try 0 launched on executor multi with executor id 1664 parsl.dataflow.dflow INFO: Standard output for task 1663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/7409b6cb-ea34-487f-8271-ae9eb835662f_isrForCrosstalkSources_996999_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/7409b6cb-ea34-487f-8271-ae9eb835662f_isrForCrosstalkSources_996999_3.stderr parsl.dataflow.dflow DEBUG: Task 1664 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1664 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1664 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1664 has memoization hash 31a1343c98361466601451bc0788a746 parsl.dataflow.memoization INFO: Task 1664 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1664 try 0 launched on executor multi with executor id 1665 parsl.dataflow.dflow INFO: Standard output for task 1664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/481e0680-9c91-4108-99e6-80db8c9a21f5_isrForCrosstalkSources_996962_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/481e0680-9c91-4108-99e6-80db8c9a21f5_isrForCrosstalkSources_996962_44.stderr parsl.dataflow.dflow DEBUG: Task 1665 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1665 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1665 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1665 has memoization hash 4c89f98c20e432952797c0408d3927ff parsl.dataflow.memoization INFO: Task 1665 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b98a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1665 try 0 launched on executor multi with executor id 1666 parsl.dataflow.dflow INFO: Standard output for task 1665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/f88632fc-f20b-4f5b-9de4-26efd8a62f30_isrForCrosstalkSources_996992_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/f88632fc-f20b-4f5b-9de4-26efd8a62f30_isrForCrosstalkSources_996992_18.stderr parsl.dataflow.dflow DEBUG: Task 1666 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1666 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1666 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1666 has memoization hash dab3e804cce00d41b3d7b6f2c16a1308 parsl.dataflow.memoization INFO: Task 1666 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b99e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1666 try 0 launched on executor multi with executor id 1667 parsl.dataflow.dflow INFO: Standard output for task 1666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6e0630c-bf8d-4c44-8df0-8409da2cc7ff_isrForCrosstalkSources_996976_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6e0630c-bf8d-4c44-8df0-8409da2cc7ff_isrForCrosstalkSources_996976_1.stderr parsl.dataflow.dflow DEBUG: Task 1667 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1667 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1667 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1667 has memoization hash d989d134e475186dc84bad615379d901 parsl.dataflow.memoization INFO: Task 1667 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1667 try 0 launched on executor multi with executor id 1668 parsl.dataflow.dflow INFO: Standard output for task 1667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/d92ffb1a-644d-444a-b294-ebdc2ccb87cd_isrForCrosstalkSources_997020_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/d92ffb1a-644d-444a-b294-ebdc2ccb87cd_isrForCrosstalkSources_997020_53.stderr parsl.dataflow.dflow DEBUG: Task 1668 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1668 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1668 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1668 has memoization hash 4f4679c930f4ab87ecf6325b382e5abe parsl.dataflow.memoization INFO: Task 1668 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1668 try 0 launched on executor multi with executor id 1669 parsl.dataflow.dflow INFO: Standard output for task 1668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6cb1b78a-cc63-48ba-a0d9-f3d5c241646c_isrForCrosstalkSources_996973_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6cb1b78a-cc63-48ba-a0d9-f3d5c241646c_isrForCrosstalkSources_996973_20.stderr parsl.dataflow.dflow DEBUG: Task 1669 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1669 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1669 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1669 has memoization hash 00cacf57a3ab763002f2d69c6cee9d43 parsl.dataflow.memoization INFO: Task 1669 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1669 try 0 launched on executor multi with executor id 1670 parsl.dataflow.dflow INFO: Standard output for task 1669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f6410546-fa7c-4105-bb4a-c189f334a1f6_isrForCrosstalkSources_996957_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f6410546-fa7c-4105-bb4a-c189f334a1f6_isrForCrosstalkSources_996957_13.stderr parsl.dataflow.dflow DEBUG: Task 1670 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1670 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1670 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1670 has memoization hash e8ac065e0f2b0723b9c8a10e876ebeaf parsl.dataflow.memoization INFO: Task 1670 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1670 try 0 launched on executor multi with executor id 1671 parsl.dataflow.dflow INFO: Standard output for task 1670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/b63862d9-4959-4147-ad3c-ea2c05479ff9_isrForCrosstalkSources_997000_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/b63862d9-4959-4147-ad3c-ea2c05479ff9_isrForCrosstalkSources_997000_31.stderr parsl.dataflow.dflow DEBUG: Task 1671 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1671 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1671 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1671 has memoization hash 116f560cdb507ef30d4ae5d5846e1c23 parsl.dataflow.memoization INFO: Task 1671 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1671 try 0 launched on executor multi with executor id 1672 parsl.dataflow.dflow INFO: Standard output for task 1671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bb504754-cec2-4c0e-838d-19078410c9c5_isrForCrosstalkSources_997027_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bb504754-cec2-4c0e-838d-19078410c9c5_isrForCrosstalkSources_997027_16.stderr parsl.dataflow.dflow DEBUG: Task 1672 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1672 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1672 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1672 has memoization hash c11b3ca8f4d408339f3c75092ff73c84 parsl.dataflow.memoization INFO: Task 1672 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1672 try 0 launched on executor multi with executor id 1673 parsl.dataflow.dflow INFO: Standard output for task 1672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ca771207-41f3-47f6-bf4a-3d8f4457ea83_isrForCrosstalkSources_996988_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ca771207-41f3-47f6-bf4a-3d8f4457ea83_isrForCrosstalkSources_996988_17.stderr parsl.dataflow.dflow DEBUG: Task 1673 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1673 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1673 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1673 has memoization hash 10eb014f8ad01829e4314f54c5cda391 parsl.dataflow.memoization INFO: Task 1673 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1673 try 0 launched on executor multi with executor id 1674 parsl.dataflow.dflow INFO: Standard output for task 1673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/c59202b6-b756-43ed-9bc0-2b86587c0acd_isrForCrosstalkSources_996958_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/c59202b6-b756-43ed-9bc0-2b86587c0acd_isrForCrosstalkSources_996958_7.stderr parsl.dataflow.dflow DEBUG: Task 1674 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1674 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1674 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1674 has memoization hash ab5550f2bec136715370568fae5172bd parsl.dataflow.memoization INFO: Task 1674 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1674 try 0 launched on executor multi with executor id 1675 parsl.dataflow.dflow INFO: Standard output for task 1674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b14db87a-0509-4744-8f93-f4120e0ef01d_isrForCrosstalkSources_997016_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b14db87a-0509-4744-8f93-f4120e0ef01d_isrForCrosstalkSources_997016_51.stderr parsl.dataflow.dflow DEBUG: Task 1675 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1675 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1675 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1675 has memoization hash 1cd8c2e95fdb899a5bcb5dc0ebaf6c4b parsl.dataflow.memoization INFO: Task 1675 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b9ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1675 try 0 launched on executor multi with executor id 1676 parsl.dataflow.dflow INFO: Standard output for task 1675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/dac05893-42da-4a57-b8ad-ebcd4b12577c_isrForCrosstalkSources_996994_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/dac05893-42da-4a57-b8ad-ebcd4b12577c_isrForCrosstalkSources_996994_58.stderr parsl.dataflow.dflow DEBUG: Task 1676 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1676 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1676 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1676 has memoization hash 743754766cefcd22d39d936f96300b69 parsl.dataflow.memoization INFO: Task 1676 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1676 try 0 launched on executor multi with executor id 1677 parsl.dataflow.dflow INFO: Standard output for task 1676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0b952525-7cf5-47d0-8a6d-d4c1099d6bee_isrForCrosstalkSources_996982_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0b952525-7cf5-47d0-8a6d-d4c1099d6bee_isrForCrosstalkSources_996982_7.stderr parsl.dataflow.dflow DEBUG: Task 1677 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1677 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1677 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1677 has memoization hash cc83b869f9fd04020fd55fed63411917 parsl.dataflow.memoization INFO: Task 1677 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1677 try 0 launched on executor multi with executor id 1678 parsl.dataflow.dflow INFO: Standard output for task 1677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/10071cd0-cda5-497f-9bfa-5e7fa5815e7b_isrForCrosstalkSources_996989_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/10071cd0-cda5-497f-9bfa-5e7fa5815e7b_isrForCrosstalkSources_996989_20.stderr parsl.dataflow.dflow DEBUG: Task 1678 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1678 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1678 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1678 has memoization hash aa89d477aea802ba7d008421a89647b5 parsl.dataflow.memoization INFO: Task 1678 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1678 try 0 launched on executor multi with executor id 1679 parsl.dataflow.dflow INFO: Standard output for task 1678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/22e51beb-c8fd-46bd-8654-108a8677749c_isrForCrosstalkSources_996970_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/22e51beb-c8fd-46bd-8654-108a8677749c_isrForCrosstalkSources_996970_17.stderr parsl.dataflow.dflow DEBUG: Task 1679 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1679 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1679 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1679 has memoization hash a870fb8534b300639d9a7e7f50c542b7 parsl.dataflow.memoization INFO: Task 1679 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1679 try 0 launched on executor multi with executor id 1680 parsl.dataflow.dflow INFO: Standard output for task 1679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/da0a31e4-aec7-484f-afa1-f17ffa6de7e2_isrForCrosstalkSources_997029_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/da0a31e4-aec7-484f-afa1-f17ffa6de7e2_isrForCrosstalkSources_997029_46.stderr parsl.dataflow.dflow DEBUG: Task 1680 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1680 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1680 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1680 has memoization hash 039e44b4fd6786ee4a9132666dc339e2 parsl.dataflow.memoization INFO: Task 1680 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1680 try 0 launched on executor multi with executor id 1681 parsl.dataflow.dflow INFO: Standard output for task 1680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/98437ad4-b927-438f-b50c-e20677340dfc_isrForCrosstalkSources_996961_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/98437ad4-b927-438f-b50c-e20677340dfc_isrForCrosstalkSources_996961_39.stderr parsl.dataflow.dflow DEBUG: Task 1681 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1681 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1681 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1681 has memoization hash 35d187688611f76c5004436d83112bcb parsl.dataflow.memoization INFO: Task 1681 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1681 try 0 launched on executor multi with executor id 1682 parsl.dataflow.dflow INFO: Standard output for task 1681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/472ea70f-3e7e-4cd7-a544-590e84a643d2_isrForCrosstalkSources_996968_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/472ea70f-3e7e-4cd7-a544-590e84a643d2_isrForCrosstalkSources_996968_55.stderr parsl.dataflow.dflow DEBUG: Task 1682 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1682 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1682 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1682 has memoization hash 92afca080574bd7b224941d2ea5c278d parsl.dataflow.memoization INFO: Task 1682 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1682 try 0 launched on executor multi with executor id 1683 parsl.dataflow.dflow INFO: Standard output for task 1682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f55fe125-2a6b-48ff-9761-69dc1c94e26c_isrForCrosstalkSources_996986_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f55fe125-2a6b-48ff-9761-69dc1c94e26c_isrForCrosstalkSources_996986_58.stderr parsl.dataflow.dflow DEBUG: Task 1683 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1683 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1683 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1683 has memoization hash ea1c7ce020e2a61451727edd0adb45cf parsl.dataflow.memoization INFO: Task 1683 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1683 try 0 launched on executor multi with executor id 1684 parsl.dataflow.dflow INFO: Standard output for task 1683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/6bcc8dde-8eeb-480b-9438-75383f256a38_isrForCrosstalkSources_997021_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/6bcc8dde-8eeb-480b-9438-75383f256a38_isrForCrosstalkSources_997021_32.stderr parsl.dataflow.dflow DEBUG: Task 1684 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1684 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1684 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1684 has memoization hash 340461f418ec653a6ffbe10edda8da15 parsl.dataflow.memoization INFO: Task 1684 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1684 try 0 launched on executor multi with executor id 1685 parsl.dataflow.dflow INFO: Standard output for task 1684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/cbf41ff3-c23c-4f5a-9c9d-4fa928f72fb2_isrForCrosstalkSources_997019_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/cbf41ff3-c23c-4f5a-9c9d-4fa928f72fb2_isrForCrosstalkSources_997019_1.stderr parsl.dataflow.dflow DEBUG: Task 1685 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1685 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1685 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1685 has memoization hash 07b03b23866d40410fbc6340764c7de0 parsl.dataflow.memoization INFO: Task 1685 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1685 try 0 launched on executor multi with executor id 1686 parsl.dataflow.dflow INFO: Standard output for task 1685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/827942e0-744e-4dd9-b65e-233dd6fc3bda_isrForCrosstalkSources_997003_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/827942e0-744e-4dd9-b65e-233dd6fc3bda_isrForCrosstalkSources_997003_22.stderr parsl.dataflow.dflow DEBUG: Task 1686 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1686 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1686 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1686 has memoization hash 4bfe0242d0fb095cd19a27f87c1e5efa parsl.dataflow.memoization INFO: Task 1686 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1686 try 0 launched on executor multi with executor id 1687 parsl.dataflow.dflow INFO: Standard output for task 1686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/61a59458-51c6-45d4-a240-cf1975b5fc0c_isrForCrosstalkSources_996983_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/61a59458-51c6-45d4-a240-cf1975b5fc0c_isrForCrosstalkSources_996983_18.stderr parsl.dataflow.dflow DEBUG: Task 1687 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1687 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1687 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1687 has memoization hash 810cadc6c141031e27f7a86eaab37d00 parsl.dataflow.memoization INFO: Task 1687 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1687 try 0 launched on executor multi with executor id 1688 parsl.dataflow.dflow INFO: Standard output for task 1687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/77dfd0bc-4731-4a32-97b9-8911b297591b_isrForCrosstalkSources_997007_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/77dfd0bc-4731-4a32-97b9-8911b297591b_isrForCrosstalkSources_997007_58.stderr parsl.dataflow.dflow DEBUG: Task 1688 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1688 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1688 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1688 has memoization hash 2468d98fec2f1b413bd2683a403411ca parsl.dataflow.memoization INFO: Task 1688 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1688 try 0 launched on executor multi with executor id 1689 parsl.dataflow.dflow INFO: Standard output for task 1688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/8c11e034-a27b-40c0-a18b-f071a0f0a4ae_isrForCrosstalkSources_997001_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/8c11e034-a27b-40c0-a18b-f071a0f0a4ae_isrForCrosstalkSources_997001_38.stderr parsl.dataflow.dflow DEBUG: Task 1689 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1689 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1689 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1689 has memoization hash 514ebc57ca9f35c47038d50e164c722a parsl.dataflow.memoization INFO: Task 1689 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1689 try 0 launched on executor multi with executor id 1690 parsl.dataflow.dflow INFO: Standard output for task 1689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b80bd3e9-dfc1-4018-91e2-ec78622229e4_isrForCrosstalkSources_996997_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b80bd3e9-dfc1-4018-91e2-ec78622229e4_isrForCrosstalkSources_996997_5.stderr parsl.dataflow.dflow DEBUG: Task 1690 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1690 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1690 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1690 has memoization hash 8ed0bc702a4c7011441fa887a3941453 parsl.dataflow.memoization INFO: Task 1690 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1690 try 0 launched on executor multi with executor id 1691 parsl.dataflow.dflow INFO: Standard output for task 1690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2c6a3aac-93b8-4cf6-aabf-2ae10f634324_isrForCrosstalkSources_996969_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2c6a3aac-93b8-4cf6-aabf-2ae10f634324_isrForCrosstalkSources_996969_58.stderr parsl.dataflow.dflow DEBUG: Task 1691 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1691 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1691 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1691 has memoization hash 2dc2aca583b2b962c9f0d9d638549a6c parsl.dataflow.memoization INFO: Task 1691 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2ba980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1691 try 0 launched on executor multi with executor id 1692 parsl.dataflow.dflow INFO: Standard output for task 1691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/868c3dd5-1ed2-4fb0-83b2-0b1488edad42_isrForCrosstalkSources_996986_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/868c3dd5-1ed2-4fb0-83b2-0b1488edad42_isrForCrosstalkSources_996986_20.stderr parsl.dataflow.dflow DEBUG: Task 1692 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1692 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1692 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1692 has memoization hash 8733450461e969cc5ae35840ab5e0724 parsl.dataflow.memoization INFO: Task 1692 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2baa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1692 try 0 launched on executor multi with executor id 1693 parsl.dataflow.dflow INFO: Standard output for task 1692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8a1a37e7-7043-43e1-ae16-11df66258bc0_isrForCrosstalkSources_996987_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8a1a37e7-7043-43e1-ae16-11df66258bc0_isrForCrosstalkSources_996987_2.stderr parsl.dataflow.dflow DEBUG: Task 1693 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1693 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1693 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1693 has memoization hash 5525c0381f99334364eec2831f1aea77 parsl.dataflow.memoization INFO: Task 1693 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2baac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1693 try 0 launched on executor multi with executor id 1694 parsl.dataflow.dflow INFO: Standard output for task 1693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5daf36cc-4977-4bf4-881d-5a47fac94b3a_isrForCrosstalkSources_997033_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5daf36cc-4977-4bf4-881d-5a47fac94b3a_isrForCrosstalkSources_997033_9.stderr parsl.dataflow.dflow DEBUG: Task 1694 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1694 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1694 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1694 has memoization hash fd280a6ea39b026b34406efc3e3f82d7 parsl.dataflow.memoization INFO: Task 1694 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1694 try 0 launched on executor multi with executor id 1695 parsl.dataflow.dflow INFO: Standard output for task 1694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/401b7390-e5d6-4a9f-b580-029018efd5ed_isrForCrosstalkSources_996971_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/401b7390-e5d6-4a9f-b580-029018efd5ed_isrForCrosstalkSources_996971_3.stderr parsl.dataflow.dflow DEBUG: Task 1695 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1695 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1695 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1695 has memoization hash fd93cf02a40ce9befbf900ee241eda87 parsl.dataflow.memoization INFO: Task 1695 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:43 proc_lsst.multi:170 [INFO] [multi] job_id 17074863 proc_lsst.multi INFO: [multi] job_id 17074863 2024-03-15 13:49:43 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 21 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 21 2024-03-15 13:49:43 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:43 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074863 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074863 parsl.executors.status_handling DEBUG: Launched block 21 on executor multi with job ID 17074863 parsl.executors.status_handling INFO: Allocated block ID 22 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-22 2024-03-15 13:49:43 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=22 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-22 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=22 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-22 2024-03-15 13:49:43 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=22 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=22 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:43 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=22 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=22 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 21 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 21 2024-03-15 13:49:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:43 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=22 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=22 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 1695 try 0 launched on executor multi with executor id 1696 parsl.dataflow.dflow INFO: Standard output for task 1695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/960843c4-ae9b-494e-924d-350e2c05e060_isrForCrosstalkSources_996952_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/960843c4-ae9b-494e-924d-350e2c05e060_isrForCrosstalkSources_996952_22.stderr parsl.dataflow.dflow DEBUG: Task 1696 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1696 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1696 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1696 has memoization hash b3088a96421b36d558643924b6c789cb parsl.dataflow.memoization INFO: Task 1696 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e203ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1696 try 0 launched on executor multi with executor id 1697 parsl.dataflow.dflow INFO: Standard output for task 1696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/136e5b05-366c-4737-a2cc-6d2dc62bf13e_isrForCrosstalkSources_996980_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/136e5b05-366c-4737-a2cc-6d2dc62bf13e_isrForCrosstalkSources_996980_31.stderr parsl.dataflow.dflow DEBUG: Task 1697 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1697 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1697 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1697 has memoization hash c447c546a9ad34e17c90d10247cc0639 parsl.dataflow.memoization INFO: Task 1697 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1697 try 0 launched on executor multi with executor id 1698 parsl.dataflow.dflow INFO: Standard output for task 1697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/1505a9fb-efa7-4914-aeba-fd170a1d6263_isrForCrosstalkSources_996973_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/1505a9fb-efa7-4914-aeba-fd170a1d6263_isrForCrosstalkSources_996973_35.stderr parsl.dataflow.dflow DEBUG: Task 1698 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1698 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1698 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1698 has memoization hash f67f9182325cc98539d02f53177d905c parsl.dataflow.memoization INFO: Task 1698 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e25cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1698 try 0 launched on executor multi with executor id 1699 parsl.dataflow.dflow INFO: Standard output for task 1698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0fd7a8eb-e057-4c87-ad1c-5c010dc8c6cc_isrForCrosstalkSources_997010_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0fd7a8eb-e057-4c87-ad1c-5c010dc8c6cc_isrForCrosstalkSources_997010_26.stderr parsl.dataflow.dflow DEBUG: Task 1699 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1699 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1699 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1699 has memoization hash 0148cd39b8791ed0e709201d7143fb1e parsl.dataflow.memoization INFO: Task 1699 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2b8040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1699 try 0 launched on executor multi with executor id 1700 parsl.dataflow.dflow INFO: Standard output for task 1699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/69c0bd03-6988-47ad-b107-486dabaeed89_isrForCrosstalkSources_997001_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/69c0bd03-6988-47ad-b107-486dabaeed89_isrForCrosstalkSources_997001_8.stderr parsl.dataflow.dflow DEBUG: Task 1700 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1700 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1700 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1700 has memoization hash 6a5742787c08220004da6d319b6a59fc parsl.dataflow.memoization INFO: Task 1700 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2baca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1700 try 0 launched on executor multi with executor id 1701 parsl.dataflow.dflow INFO: Standard output for task 1700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/94344a04-5805-4c3c-b079-be9ce463c908_isrForCrosstalkSources_996999_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/94344a04-5805-4c3c-b079-be9ce463c908_isrForCrosstalkSources_996999_13.stderr parsl.dataflow.dflow DEBUG: Task 1701 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1701 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1701 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1701 has memoization hash ad6d5be867e6fdf5657dfcd2e9afd5c5 parsl.dataflow.memoization INFO: Task 1701 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1701 try 0 launched on executor multi with executor id 1702 parsl.dataflow.dflow INFO: Standard output for task 1701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/617445d2-f68d-47a0-af82-edcac6f96e01_isrForCrosstalkSources_996962_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/617445d2-f68d-47a0-af82-edcac6f96e01_isrForCrosstalkSources_996962_55.stderr parsl.dataflow.dflow DEBUG: Task 1702 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1702 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1702 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1702 has memoization hash c64ef61de642b17d82c7bf497d43862c parsl.dataflow.memoization INFO: Task 1702 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1702 try 0 launched on executor multi with executor id 1703 parsl.dataflow.dflow INFO: Standard output for task 1702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ee77bfcb-f4b6-4970-9572-44aba1a05f6b_isrForCrosstalkSources_996990_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ee77bfcb-f4b6-4970-9572-44aba1a05f6b_isrForCrosstalkSources_996990_1.stderr parsl.dataflow.dflow DEBUG: Task 1703 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1703 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1703 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1703 has memoization hash 72286d106a95bb320c63a7abbe50f1f0 parsl.dataflow.memoization INFO: Task 1703 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1703 try 0 launched on executor multi with executor id 1704 parsl.dataflow.dflow INFO: Standard output for task 1703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/f50c5322-a8ef-4f37-b890-e57f932cb136_isrForCrosstalkSources_997007_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/f50c5322-a8ef-4f37-b890-e57f932cb136_isrForCrosstalkSources_997007_33.stderr parsl.dataflow.dflow DEBUG: Task 1704 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1704 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1704 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1704 has memoization hash b21d32809735205f4ed3e8a987272fc6 parsl.dataflow.memoization INFO: Task 1704 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1704 try 0 launched on executor multi with executor id 1705 parsl.dataflow.dflow INFO: Standard output for task 1704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/219b0437-8060-4484-8b21-d3a2f4758707_isrForCrosstalkSources_997001_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/219b0437-8060-4484-8b21-d3a2f4758707_isrForCrosstalkSources_997001_21.stderr parsl.dataflow.dflow DEBUG: Task 1705 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1705 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1705 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1705 has memoization hash feb120c90417abc933e5942712016c23 parsl.dataflow.memoization INFO: Task 1705 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1705 try 0 launched on executor multi with executor id 1706 parsl.dataflow.dflow INFO: Standard output for task 1705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/99ee6a54-85ef-4cc3-9e5a-1b04b71bc4fa_isrForCrosstalkSources_996968_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/99ee6a54-85ef-4cc3-9e5a-1b04b71bc4fa_isrForCrosstalkSources_996968_46.stderr parsl.dataflow.dflow DEBUG: Task 1706 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1706 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1706 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1706 has memoization hash 370b7fe770f97bc0e4296022b581dddf parsl.dataflow.memoization INFO: Task 1706 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1706 try 0 launched on executor multi with executor id 1707 parsl.dataflow.dflow INFO: Standard output for task 1706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/20b45e25-2b0e-44b9-962f-82b19b72856d_isrForCrosstalkSources_996960_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/20b45e25-2b0e-44b9-962f-82b19b72856d_isrForCrosstalkSources_996960_50.stderr parsl.dataflow.dflow DEBUG: Task 1707 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1707 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1707 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1707 has memoization hash 4825b491f5477f023cf8ebd41da808bb parsl.dataflow.memoization INFO: Task 1707 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1707 try 0 launched on executor multi with executor id 1708 parsl.dataflow.dflow INFO: Standard output for task 1707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/bcdd8641-90b4-4df7-a767-4d19dca63b58_isrForCrosstalkSources_996953_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/bcdd8641-90b4-4df7-a767-4d19dca63b58_isrForCrosstalkSources_996953_34.stderr parsl.dataflow.dflow DEBUG: Task 1708 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1708 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1708 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1708 has memoization hash 1286f16bb8155342c997d39f83083e09 parsl.dataflow.memoization INFO: Task 1708 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1708 try 0 launched on executor multi with executor id 1709 parsl.dataflow.dflow INFO: Standard output for task 1708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6196fbff-c913-4962-94e8-731815216018_isrForCrosstalkSources_997003_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6196fbff-c913-4962-94e8-731815216018_isrForCrosstalkSources_997003_20.stderr parsl.dataflow.dflow DEBUG: Task 1709 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1709 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1709 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1709 has memoization hash 7fc1626f47c4270eb78ef798c89bbcfe parsl.dataflow.memoization INFO: Task 1709 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1709 try 0 launched on executor multi with executor id 1710 parsl.dataflow.dflow INFO: Standard output for task 1709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f6a1e8ff-1342-488a-9ec0-7008449d66b5_isrForCrosstalkSources_996987_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f6a1e8ff-1342-488a-9ec0-7008449d66b5_isrForCrosstalkSources_996987_20.stderr parsl.dataflow.dflow DEBUG: Task 1710 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1710 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1710 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1710 has memoization hash 3b0a6db3ec670014f1238d70ec0ec1ae parsl.dataflow.memoization INFO: Task 1710 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1710 try 0 launched on executor multi with executor id 1711 parsl.dataflow.dflow INFO: Standard output for task 1710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6d968f59-63c3-4099-89c5-48a217dba668_isrForCrosstalkSources_997003_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6d968f59-63c3-4099-89c5-48a217dba668_isrForCrosstalkSources_997003_30.stderr parsl.dataflow.dflow DEBUG: Task 1711 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1711 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1711 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1711 has memoization hash b8b1364c869ab457a7f4feb2452d7ccf parsl.dataflow.memoization INFO: Task 1711 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1711 try 0 launched on executor multi with executor id 1712 parsl.dataflow.dflow INFO: Standard output for task 1711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/3ddf3d63-c391-4019-b516-6a5a9cfbaa75_isrForCrosstalkSources_997011_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/3ddf3d63-c391-4019-b516-6a5a9cfbaa75_isrForCrosstalkSources_997011_7.stderr parsl.dataflow.dflow DEBUG: Task 1712 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1712 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1712 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1712 has memoization hash 828a15e7c165adc404f7cd9827bce581 parsl.dataflow.memoization INFO: Task 1712 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1712 try 0 launched on executor multi with executor id 1713 parsl.dataflow.dflow INFO: Standard output for task 1712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a4b8b12e-25ce-421c-8846-737f8c1f9a46_isrForCrosstalkSources_997002_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a4b8b12e-25ce-421c-8846-737f8c1f9a46_isrForCrosstalkSources_997002_6.stderr parsl.dataflow.dflow DEBUG: Task 1713 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1713 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1713 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1713 has memoization hash e9168173fa45d2b938a6a8c790249591 parsl.dataflow.memoization INFO: Task 1713 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1713 try 0 launched on executor multi with executor id 1714 parsl.dataflow.dflow INFO: Standard output for task 1713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60775b99-e0c3-4420-b918-8c77fa290a3c_isrForCrosstalkSources_996968_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60775b99-e0c3-4420-b918-8c77fa290a3c_isrForCrosstalkSources_996968_9.stderr parsl.dataflow.dflow DEBUG: Task 1714 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1714 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1714 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1714 has memoization hash 03d40c46c19b2806dd6dfdfdd41aa0b6 parsl.dataflow.memoization INFO: Task 1714 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1714 try 0 launched on executor multi with executor id 1715 parsl.dataflow.dflow INFO: Standard output for task 1714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/fde039f1-2006-4569-af95-297e721e7fd1_isrForCrosstalkSources_997000_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/fde039f1-2006-4569-af95-297e721e7fd1_isrForCrosstalkSources_997000_43.stderr parsl.dataflow.dflow DEBUG: Task 1715 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1715 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1715 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1715 has memoization hash 920d9e11be58c3e91591d40d0f612b5d parsl.dataflow.memoization INFO: Task 1715 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1715 try 0 launched on executor multi with executor id 1716 parsl.dataflow.dflow INFO: Standard output for task 1715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c0e9d19c-71de-45fc-96ac-219547d0fb4b_isrForCrosstalkSources_996960_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c0e9d19c-71de-45fc-96ac-219547d0fb4b_isrForCrosstalkSources_996960_58.stderr parsl.dataflow.dflow DEBUG: Task 1716 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1716 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1716 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1716 has memoization hash 0f973fd7e151d717d7ba7f0de94f9ada parsl.dataflow.memoization INFO: Task 1716 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1716 try 0 launched on executor multi with executor id 1717 parsl.dataflow.dflow INFO: Standard output for task 1716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/888a7e3c-7e52-4fba-bab0-03a977952d6e_isrForCrosstalkSources_996960_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/888a7e3c-7e52-4fba-bab0-03a977952d6e_isrForCrosstalkSources_996960_40.stderr parsl.dataflow.dflow DEBUG: Task 1717 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1717 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1717 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1717 has memoization hash 6dfea09c1692f13f03b5454162061da4 parsl.dataflow.memoization INFO: Task 1717 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1717 try 0 launched on executor multi with executor id 1718 parsl.dataflow.dflow INFO: Standard output for task 1717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/5b773f22-9fa4-4304-a5e4-e877d4f91329_isrForCrosstalkSources_996952_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/5b773f22-9fa4-4304-a5e4-e877d4f91329_isrForCrosstalkSources_996952_27.stderr parsl.dataflow.dflow DEBUG: Task 1718 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1718 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1718 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1718 has memoization hash 7b21b20d8fae3e56a8d8d6f0d6b0969a parsl.dataflow.memoization INFO: Task 1718 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1718 try 0 launched on executor multi with executor id 1719 parsl.dataflow.dflow INFO: Standard output for task 1718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/43edbd5b-bfdb-4d82-a86b-6c64d348bf49_isrForCrosstalkSources_996988_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/43edbd5b-bfdb-4d82-a86b-6c64d348bf49_isrForCrosstalkSources_996988_28.stderr parsl.dataflow.dflow DEBUG: Task 1719 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1719 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1719 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1719 has memoization hash f2d7b62a02c401d9b346995db9e888d4 parsl.dataflow.memoization INFO: Task 1719 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bbba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1719 try 0 launched on executor multi with executor id 1720 parsl.dataflow.dflow INFO: Standard output for task 1719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e6c71203-ee7b-4836-bb6f-381b4ade6989_isrForCrosstalkSources_997008_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e6c71203-ee7b-4836-bb6f-381b4ade6989_isrForCrosstalkSources_997008_12.stderr parsl.dataflow.dflow DEBUG: Task 1720 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1720 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1720 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1720 has memoization hash f63315ea1689442ad0c25bdd1a18896d parsl.dataflow.memoization INFO: Task 1720 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bbb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1720 try 0 launched on executor multi with executor id 1721 parsl.dataflow.dflow INFO: Standard output for task 1720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f19c488f-34aa-44f0-8067-dd8ccd0d03b9_isrForCrosstalkSources_996966_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f19c488f-34aa-44f0-8067-dd8ccd0d03b9_isrForCrosstalkSources_996966_1.stderr parsl.dataflow.dflow DEBUG: Task 1721 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1721 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1721 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1721 has memoization hash a620c32886aed45dfb4e2046915cc8be parsl.dataflow.memoization INFO: Task 1721 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1721 try 0 launched on executor multi with executor id 1722 parsl.dataflow.dflow INFO: Standard output for task 1721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/60501077-3067-4734-8481-726ba65af1c4_isrForCrosstalkSources_997009_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/60501077-3067-4734-8481-726ba65af1c4_isrForCrosstalkSources_997009_14.stderr parsl.dataflow.dflow DEBUG: Task 1722 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1722 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1722 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1722 has memoization hash b44032b9c247cbc2f61193d64a968eab parsl.dataflow.memoization INFO: Task 1722 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bbc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1722 try 0 launched on executor multi with executor id 1723 parsl.dataflow.dflow INFO: Standard output for task 1722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/7b80fd65-3ede-460a-a919-756b70e37b1a_isrForCrosstalkSources_997016_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/7b80fd65-3ede-460a-a919-756b70e37b1a_isrForCrosstalkSources_997016_61.stderr parsl.dataflow.dflow DEBUG: Task 1723 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1723 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1723 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1723 has memoization hash 0ed555fe47a16f0cf479f67fd3f1809f parsl.dataflow.memoization INFO: Task 1723 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bbd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1723 try 0 launched on executor multi with executor id 1724 parsl.dataflow.dflow INFO: Standard output for task 1723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/a1d91784-a046-4760-8000-c3d1d2a17063_isrForCrosstalkSources_997005_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/a1d91784-a046-4760-8000-c3d1d2a17063_isrForCrosstalkSources_997005_29.stderr parsl.dataflow.dflow DEBUG: Task 1724 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1724 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1724 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1724 has memoization hash 69248eed4143628a8707610211894d89 parsl.dataflow.memoization INFO: Task 1724 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bbec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1724 try 0 launched on executor multi with executor id 1725 parsl.dataflow.dflow INFO: Standard output for task 1724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2bd80e4b-a25a-44a5-bdda-0ba14b6a065d_isrForCrosstalkSources_996970_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2bd80e4b-a25a-44a5-bdda-0ba14b6a065d_isrForCrosstalkSources_996970_35.stderr parsl.dataflow.dflow DEBUG: Task 1725 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1725 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1725 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1725 has memoization hash 68d345baba49ed4f203e72c7d50aaa49 parsl.dataflow.memoization INFO: Task 1725 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bbe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1725 try 0 launched on executor multi with executor id 1726 parsl.dataflow.dflow INFO: Standard output for task 1725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ee6408b0-fa59-4ad1-a11b-f6ec10f554cf_isrForCrosstalkSources_997005_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ee6408b0-fa59-4ad1-a11b-f6ec10f554cf_isrForCrosstalkSources_997005_52.stderr parsl.dataflow.dflow DEBUG: Task 1726 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1726 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1726 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1726 has memoization hash de92f561fb389cb0a6d7b2aaf1a9c8d0 parsl.dataflow.memoization INFO: Task 1726 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bbce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1726 try 0 launched on executor multi with executor id 1727 parsl.dataflow.dflow INFO: Standard output for task 1726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/d7c16d33-24eb-46d6-b7fc-d215083874b9_isrForCrosstalkSources_996976_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/d7c16d33-24eb-46d6-b7fc-d215083874b9_isrForCrosstalkSources_996976_52.stderr parsl.dataflow.dflow DEBUG: Task 1727 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1727 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1727 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1727 has memoization hash b86ac6b419eece5f702cc99c7de2d40b parsl.dataflow.memoization INFO: Task 1727 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bbf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1727 try 0 launched on executor multi with executor id 1728 parsl.dataflow.dflow INFO: Standard output for task 1727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a29fdc8d-4b04-4712-8388-ae0f19c672f1_isrForCrosstalkSources_997011_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a29fdc8d-4b04-4712-8388-ae0f19c672f1_isrForCrosstalkSources_997011_1.stderr parsl.dataflow.dflow DEBUG: Task 1728 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1728 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1728 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1728 has memoization hash 0e56b808b2ebfbb90c6bb49e676b8db2 parsl.dataflow.memoization INFO: Task 1728 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bb060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1728 try 0 launched on executor multi with executor id 1729 parsl.dataflow.dflow INFO: Standard output for task 1728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/dba2d302-ce67-4a28-90ba-e8c002fe5a39_isrForCrosstalkSources_997003_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/dba2d302-ce67-4a28-90ba-e8c002fe5a39_isrForCrosstalkSources_997003_5.stderr parsl.dataflow.dflow DEBUG: Task 1729 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1729 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1729 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1729 has memoization hash 2c1f120f87b91c6384fcf8138fa15000 parsl.dataflow.memoization INFO: Task 1729 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd23e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1729 try 0 launched on executor multi with executor id 1730 parsl.dataflow.dflow INFO: Standard output for task 1729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/414aa1a4-9c73-4992-9405-705591d1df11_isrForCrosstalkSources_996966_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/414aa1a4-9c73-4992-9405-705591d1df11_isrForCrosstalkSources_996966_6.stderr parsl.dataflow.dflow DEBUG: Task 1730 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1730 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1730 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1730 has memoization hash 7ff0218a6fb2140193ca17f4758a4593 parsl.dataflow.memoization INFO: Task 1730 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1730 try 0 launched on executor multi with executor id 1731 parsl.dataflow.dflow INFO: Standard output for task 1730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/24858623-9d18-489a-b201-fe1e4a3fe6e8_isrForCrosstalkSources_996965_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/24858623-9d18-489a-b201-fe1e4a3fe6e8_isrForCrosstalkSources_996965_18.stderr parsl.dataflow.dflow DEBUG: Task 1731 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1731 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1731 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1731 has memoization hash 959bfb52256fcc79e844b1103ed3b6cb parsl.dataflow.memoization INFO: Task 1731 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd22a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1731 try 0 launched on executor multi with executor id 1732 parsl.dataflow.dflow INFO: Standard output for task 1731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/2591a22e-5455-428c-85b8-d1aad9ba975a_isrForCrosstalkSources_997023_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/2591a22e-5455-428c-85b8-d1aad9ba975a_isrForCrosstalkSources_997023_15.stderr parsl.dataflow.dflow DEBUG: Task 1732 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1732 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1732 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1732 has memoization hash 6211d29f8d7831e024a774af1b126a7d parsl.dataflow.memoization INFO: Task 1732 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1732 try 0 launched on executor multi with executor id 1733 parsl.dataflow.dflow INFO: Standard output for task 1732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e3ba8461-7129-4134-8392-bfb27b20a1ec_isrForCrosstalkSources_996963_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e3ba8461-7129-4134-8392-bfb27b20a1ec_isrForCrosstalkSources_996963_26.stderr parsl.dataflow.dflow DEBUG: Task 1733 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1733 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1733 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1733 has memoization hash a4dfcf4c0e920f25d20d2419bb340319 parsl.dataflow.memoization INFO: Task 1733 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd00e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1733 try 0 launched on executor multi with executor id 1734 parsl.dataflow.dflow INFO: Standard output for task 1733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1925ca27-b4f8-465b-bc34-eb6f19d3f8e6_isrForCrosstalkSources_997000_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1925ca27-b4f8-465b-bc34-eb6f19d3f8e6_isrForCrosstalkSources_997000_38.stderr parsl.dataflow.dflow DEBUG: Task 1734 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1734 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1734 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1734 has memoization hash 130ee89069c811738f44f0db7e2a3c45 parsl.dataflow.memoization INFO: Task 1734 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1734 try 0 launched on executor multi with executor id 1735 parsl.dataflow.dflow INFO: Standard output for task 1734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/8cb01edf-2f3a-4e6d-8fcf-f0dd811c3841_isrForCrosstalkSources_996999_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/8cb01edf-2f3a-4e6d-8fcf-f0dd811c3841_isrForCrosstalkSources_996999_37.stderr parsl.dataflow.dflow DEBUG: Task 1735 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1735 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1735 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1735 has memoization hash efe9fd87822789875b3b6b3ebb3c7085 parsl.dataflow.memoization INFO: Task 1735 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1735 try 0 launched on executor multi with executor id 1736 parsl.dataflow.dflow INFO: Standard output for task 1735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ecdf1f1b-9271-4ab6-97c9-bf8780e055e2_isrForCrosstalkSources_996972_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ecdf1f1b-9271-4ab6-97c9-bf8780e055e2_isrForCrosstalkSources_996972_43.stderr parsl.dataflow.dflow DEBUG: Task 1736 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1736 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1736 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1736 has memoization hash 561dcc08b163130d12bed4515ae68101 parsl.dataflow.memoization INFO: Task 1736 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1736 try 0 launched on executor multi with executor id 1737 parsl.dataflow.dflow INFO: Standard output for task 1736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/439a29ce-aba1-4dba-8fc2-4a564505be02_isrForCrosstalkSources_997002_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/439a29ce-aba1-4dba-8fc2-4a564505be02_isrForCrosstalkSources_997002_32.stderr parsl.dataflow.dflow DEBUG: Task 1737 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1737 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1737 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1737 has memoization hash 9d531923f7c2fe178313c03df97afc0b parsl.dataflow.memoization INFO: Task 1737 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1737 try 0 launched on executor multi with executor id 1738 parsl.dataflow.dflow INFO: Standard output for task 1737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1fd822fb-c7c6-4457-b051-02051af48927_isrForCrosstalkSources_997022_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1fd822fb-c7c6-4457-b051-02051af48927_isrForCrosstalkSources_997022_61.stderr parsl.dataflow.dflow DEBUG: Task 1738 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1738 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1738 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1738 has memoization hash fe38395108e12adc726651ac2a2240f6 parsl.dataflow.memoization INFO: Task 1738 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1738 try 0 launched on executor multi with executor id 1739 parsl.dataflow.dflow INFO: Standard output for task 1738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/3c8477df-e281-4e35-8787-9d137012ba34_isrForCrosstalkSources_996953_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/3c8477df-e281-4e35-8787-9d137012ba34_isrForCrosstalkSources_996953_60.stderr parsl.dataflow.dflow DEBUG: Task 1739 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1739 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1739 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1739 has memoization hash 7990a6a0537e5e82a1ef42548d76b391 parsl.dataflow.memoization INFO: Task 1739 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd04a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1739 try 0 launched on executor multi with executor id 1740 parsl.dataflow.dflow INFO: Standard output for task 1739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/960f70ea-f6a2-4efb-9a61-aba8f585b3f2_isrForCrosstalkSources_996980_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/960f70ea-f6a2-4efb-9a61-aba8f585b3f2_isrForCrosstalkSources_996980_7.stderr parsl.dataflow.dflow DEBUG: Task 1740 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1740 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1740 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1740 has memoization hash f0fba61a07feea919b04a8feda12fb21 parsl.dataflow.memoization INFO: Task 1740 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1740 try 0 launched on executor multi with executor id 1741 parsl.dataflow.dflow INFO: Standard output for task 1740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/de31690c-0bc5-483d-9524-bcf7fde969d1_isrForCrosstalkSources_996962_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/de31690c-0bc5-483d-9524-bcf7fde969d1_isrForCrosstalkSources_996962_52.stderr parsl.dataflow.dflow DEBUG: Task 1741 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1741 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1741 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1741 has memoization hash d74bbef760d9c4314aefd289536a09ab parsl.dataflow.memoization INFO: Task 1741 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1741 try 0 launched on executor multi with executor id 1742 parsl.dataflow.dflow INFO: Standard output for task 1741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/faee15b9-9c04-452b-abb9-123132c1025d_isrForCrosstalkSources_996970_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/faee15b9-9c04-452b-abb9-123132c1025d_isrForCrosstalkSources_996970_19.stderr parsl.dataflow.dflow DEBUG: Task 1742 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1742 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1742 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1742 has memoization hash 3963a1211e02be501229cbce1d7443d2 parsl.dataflow.memoization INFO: Task 1742 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd05e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1742 try 0 launched on executor multi with executor id 1743 parsl.dataflow.dflow INFO: Standard output for task 1742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/30e9c844-bbb8-459a-b488-248d89617f19_isrForCrosstalkSources_997018_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/30e9c844-bbb8-459a-b488-248d89617f19_isrForCrosstalkSources_997018_44.stderr parsl.dataflow.dflow DEBUG: Task 1743 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1743 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1743 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1743 has memoization hash 50fbdb7e96a8b3e057f68525aa763207 parsl.dataflow.memoization INFO: Task 1743 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd07c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1743 try 0 launched on executor multi with executor id 1744 parsl.dataflow.dflow INFO: Standard output for task 1743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/5a140e14-68b7-4d96-ae33-fd0349c43f3d_isrForCrosstalkSources_996992_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/5a140e14-68b7-4d96-ae33-fd0349c43f3d_isrForCrosstalkSources_996992_39.stderr parsl.dataflow.dflow DEBUG: Task 1744 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1744 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1744 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1744 has memoization hash 8649cd199ef764c058a011656bdd8dd2 parsl.dataflow.memoization INFO: Task 1744 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1744 try 0 launched on executor multi with executor id 1745 parsl.dataflow.dflow INFO: Standard output for task 1744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/89518e0a-1cca-49fa-aa54-46d08bc5dc1a_isrForCrosstalkSources_996982_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/89518e0a-1cca-49fa-aa54-46d08bc5dc1a_isrForCrosstalkSources_996982_53.stderr parsl.dataflow.dflow DEBUG: Task 1745 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1745 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1745 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1745 has memoization hash 4225165a5c55110bcb3ff56f19d2fdf6 parsl.dataflow.memoization INFO: Task 1745 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1745 try 0 launched on executor multi with executor id 1746 parsl.dataflow.dflow INFO: Standard output for task 1745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/da97f656-a547-48d3-a5ef-031dcaade546_isrForCrosstalkSources_997009_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/da97f656-a547-48d3-a5ef-031dcaade546_isrForCrosstalkSources_997009_42.stderr parsl.dataflow.dflow DEBUG: Task 1746 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1746 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1746 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1746 has memoization hash 7e0de1ac5ea8a251f961dc92044273d3 parsl.dataflow.memoization INFO: Task 1746 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd09a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1746 try 0 launched on executor multi with executor id 1747 parsl.dataflow.dflow INFO: Standard output for task 1746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9387c9e6-9c08-4ed0-a258-f576ffae3570_isrForCrosstalkSources_996988_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9387c9e6-9c08-4ed0-a258-f576ffae3570_isrForCrosstalkSources_996988_62.stderr parsl.dataflow.dflow DEBUG: Task 1747 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1747 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1747 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1747 has memoization hash 38338cc82b0a30d6c8267ec36f89735f parsl.dataflow.memoization INFO: Task 1747 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1747 try 0 launched on executor multi with executor id 1748 parsl.dataflow.dflow INFO: Standard output for task 1747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/3e4220b5-c000-48c3-810a-57581ae665f9_isrForCrosstalkSources_997000_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/3e4220b5-c000-48c3-810a-57581ae665f9_isrForCrosstalkSources_997000_34.stderr parsl.dataflow.dflow DEBUG: Task 1748 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1748 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1748 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1748 has memoization hash c4cdfe18c40cc8ec721951544ff8090a parsl.dataflow.memoization INFO: Task 1748 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1748 try 0 launched on executor multi with executor id 1749 parsl.dataflow.dflow INFO: Standard output for task 1748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/65b4af3b-f785-4bbd-a36c-6659b362ce2c_isrForCrosstalkSources_997006_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/65b4af3b-f785-4bbd-a36c-6659b362ce2c_isrForCrosstalkSources_997006_58.stderr parsl.dataflow.dflow DEBUG: Task 1749 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1749 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1749 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1749 has memoization hash 72456bc4f69dade24c588000ecc8e5e3 parsl.dataflow.memoization INFO: Task 1749 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1749 try 0 launched on executor multi with executor id 1750 parsl.dataflow.dflow INFO: Standard output for task 1749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/6893e18b-44ff-44f6-b51c-784f158c0467_isrForCrosstalkSources_996956_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/6893e18b-44ff-44f6-b51c-784f158c0467_isrForCrosstalkSources_996956_30.stderr parsl.dataflow.dflow DEBUG: Task 1750 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1750 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1750 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1750 has memoization hash 2403a9a51ef6f00794b165d88b4b6954 parsl.dataflow.memoization INFO: Task 1750 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1750 try 0 launched on executor multi with executor id 1751 parsl.dataflow.dflow INFO: Standard output for task 1750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/77df117b-0fbe-4aa4-8d5b-bc13d108d6fd_isrForCrosstalkSources_997006_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/77df117b-0fbe-4aa4-8d5b-bc13d108d6fd_isrForCrosstalkSources_997006_39.stderr parsl.dataflow.dflow DEBUG: Task 1751 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1751 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1751 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1751 has memoization hash d0a539db8fb0a436a0e3d0c987fc462b parsl.dataflow.memoization INFO: Task 1751 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1751 try 0 launched on executor multi with executor id 1752 parsl.dataflow.dflow INFO: Standard output for task 1751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/659ed4e0-b11f-4b0c-90ee-715677a653c4_isrForCrosstalkSources_996977_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/659ed4e0-b11f-4b0c-90ee-715677a653c4_isrForCrosstalkSources_996977_49.stderr parsl.dataflow.dflow DEBUG: Task 1752 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1752 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1752 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1752 has memoization hash 4aca1c79580546cd83b8e87c19e0fb38 parsl.dataflow.memoization INFO: Task 1752 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1752 try 0 launched on executor multi with executor id 1753 parsl.dataflow.dflow INFO: Standard output for task 1752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/48437d3c-db4c-4f6e-90df-17651f15b0bb_isrForCrosstalkSources_996990_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/48437d3c-db4c-4f6e-90df-17651f15b0bb_isrForCrosstalkSources_996990_8.stderr parsl.dataflow.dflow DEBUG: Task 1753 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1753 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1753 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1753 has memoization hash 7ad2a269d4cc4a9ba986017eeb895371 parsl.dataflow.memoization INFO: Task 1753 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1753 try 0 launched on executor multi with executor id 1754 parsl.dataflow.dflow INFO: Standard output for task 1753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/86e53c76-ac6c-4e53-90d5-a0617cc6b984_isrForCrosstalkSources_997009_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/86e53c76-ac6c-4e53-90d5-a0617cc6b984_isrForCrosstalkSources_997009_24.stderr parsl.dataflow.dflow DEBUG: Task 1754 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1754 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1754 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1754 has memoization hash 472c56406b03f931094ba32f41546fd2 parsl.dataflow.memoization INFO: Task 1754 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1754 try 0 launched on executor multi with executor id 1755 parsl.dataflow.dflow INFO: Standard output for task 1754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0bc11908-d7e6-4afd-9acd-051053d35456_isrForCrosstalkSources_997023_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0bc11908-d7e6-4afd-9acd-051053d35456_isrForCrosstalkSources_997023_10.stderr parsl.dataflow.dflow DEBUG: Task 1755 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1755 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1755 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1755 has memoization hash 2c3d272bf29788e67625a8dfe559491a parsl.dataflow.memoization INFO: Task 1755 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1755 try 0 launched on executor multi with executor id 1756 parsl.dataflow.dflow INFO: Standard output for task 1755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/14daf1c9-e969-4d42-87f7-20c7d2cc423b_isrForCrosstalkSources_997012_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/14daf1c9-e969-4d42-87f7-20c7d2cc423b_isrForCrosstalkSources_997012_61.stderr parsl.dataflow.dflow DEBUG: Task 1756 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1756 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1756 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1756 has memoization hash 94d3e94c4b5a42a4818c246eb48909e1 parsl.dataflow.memoization INFO: Task 1756 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1756 try 0 launched on executor multi with executor id 1757 parsl.dataflow.dflow INFO: Standard output for task 1756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/407086a5-7278-4c06-a738-aea816904c12_isrForCrosstalkSources_996976_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/407086a5-7278-4c06-a738-aea816904c12_isrForCrosstalkSources_996976_42.stderr parsl.dataflow.dflow DEBUG: Task 1757 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1757 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1757 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1757 has memoization hash 71f6e7da597fef7bbbd84058a9bee4a6 parsl.dataflow.memoization INFO: Task 1757 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd20c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1757 try 0 launched on executor multi with executor id 1758 parsl.dataflow.dflow INFO: Standard output for task 1757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a484ff53-92cc-4d29-9d10-a0e598ccc2e5_isrForCrosstalkSources_996983_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a484ff53-92cc-4d29-9d10-a0e598ccc2e5_isrForCrosstalkSources_996983_19.stderr parsl.dataflow.dflow DEBUG: Task 1758 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1758 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1758 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1758 has memoization hash 4f5ba248e4a856ad47ddf879e901d3d9 parsl.dataflow.memoization INFO: Task 1758 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1758 try 0 launched on executor multi with executor id 1759 parsl.dataflow.dflow INFO: Standard output for task 1758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ee87434a-984f-4e28-8695-61ba9fe8badd_isrForCrosstalkSources_996986_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ee87434a-984f-4e28-8695-61ba9fe8badd_isrForCrosstalkSources_996986_44.stderr parsl.dataflow.dflow DEBUG: Task 1759 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1759 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1759 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1759 has memoization hash 2498aff8e50a8cc01507766ba3d7719d parsl.dataflow.memoization INFO: Task 1759 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1759 try 0 launched on executor multi with executor id 1760 parsl.dataflow.dflow INFO: Standard output for task 1759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/c9014848-788d-46f0-b2d3-df714f8374a6_isrForCrosstalkSources_997002_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/c9014848-788d-46f0-b2d3-df714f8374a6_isrForCrosstalkSources_997002_15.stderr parsl.dataflow.dflow DEBUG: Task 1760 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1760 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1760 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1760 has memoization hash ea94c9a2af60a75791a0a046ad2ba3ea parsl.dataflow.memoization INFO: Task 1760 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1760 try 0 launched on executor multi with executor id 1761 parsl.dataflow.dflow INFO: Standard output for task 1760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/98aca171-3a4c-434c-919e-f64be98591e5_isrForCrosstalkSources_997005_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/98aca171-3a4c-434c-919e-f64be98591e5_isrForCrosstalkSources_997005_20.stderr parsl.dataflow.dflow DEBUG: Task 1761 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1761 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1761 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1761 has memoization hash 8f67b7bcd60c65015f2e1ed920bc7cea parsl.dataflow.memoization INFO: Task 1761 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1761 try 0 launched on executor multi with executor id 1762 parsl.dataflow.dflow INFO: Standard output for task 1761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/1999cf19-36ba-46db-ae8b-eab251f2f591_isrForCrosstalkSources_996980_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/1999cf19-36ba-46db-ae8b-eab251f2f591_isrForCrosstalkSources_996980_51.stderr parsl.dataflow.dflow DEBUG: Task 1762 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1762 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1762 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1762 has memoization hash 656797d7f2c46de98d1083cc285203f7 parsl.dataflow.memoization INFO: Task 1762 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1762 try 0 launched on executor multi with executor id 1763 parsl.dataflow.dflow INFO: Standard output for task 1762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/9a5a38b0-d7dc-4ea8-b363-cd326c7f371b_isrForCrosstalkSources_997029_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/9a5a38b0-d7dc-4ea8-b363-cd326c7f371b_isrForCrosstalkSources_997029_13.stderr parsl.dataflow.dflow DEBUG: Task 1763 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1763 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1763 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1763 has memoization hash 2d4c749f01c42518701a32606e6b030c parsl.dataflow.memoization INFO: Task 1763 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1763 try 0 launched on executor multi with executor id 1764 parsl.dataflow.dflow INFO: Standard output for task 1763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/dfc67618-b543-4173-988f-20f5d60d20b4_isrForCrosstalkSources_996978_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/dfc67618-b543-4173-988f-20f5d60d20b4_isrForCrosstalkSources_996978_32.stderr parsl.dataflow.dflow DEBUG: Task 1764 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1764 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1764 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1764 has memoization hash 789844f324c7ac3649e6484b0f20dff2 parsl.dataflow.memoization INFO: Task 1764 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1764 try 0 launched on executor multi with executor id 1765 parsl.dataflow.dflow INFO: Standard output for task 1764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f12b2c49-e199-48fc-95ba-4bbafb45646d_isrForCrosstalkSources_997026_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f12b2c49-e199-48fc-95ba-4bbafb45646d_isrForCrosstalkSources_997026_20.stderr parsl.dataflow.dflow DEBUG: Task 1765 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1765 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1765 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1765 has memoization hash d0b0e4338f61a440787b06f4b4e8adcb parsl.dataflow.memoization INFO: Task 1765 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1765 try 0 launched on executor multi with executor id 1766 parsl.dataflow.dflow INFO: Standard output for task 1765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e893d531-6ab0-4d54-abd6-fd869e3cc1b9_isrForCrosstalkSources_997021_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e893d531-6ab0-4d54-abd6-fd869e3cc1b9_isrForCrosstalkSources_997021_56.stderr parsl.dataflow.dflow DEBUG: Task 1766 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1766 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1766 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1766 has memoization hash 74b3d06697ddeeeb70ea794ee6557bed parsl.dataflow.memoization INFO: Task 1766 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1766 try 0 launched on executor multi with executor id 1767 parsl.dataflow.dflow INFO: Standard output for task 1766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bbf2808d-9a3f-49ea-a513-9c4098c35d19_isrForCrosstalkSources_996992_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bbf2808d-9a3f-49ea-a513-9c4098c35d19_isrForCrosstalkSources_996992_35.stderr parsl.dataflow.dflow DEBUG: Task 1767 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1767 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1767 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1767 has memoization hash 7303f8715e22a602f07fe4cf0a8b19e1 parsl.dataflow.memoization INFO: Task 1767 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1767 try 0 launched on executor multi with executor id 1768 parsl.dataflow.dflow INFO: Standard output for task 1767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/25ed6db6-f8c5-401d-82b0-daac52482ba8_isrForCrosstalkSources_996964_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/25ed6db6-f8c5-401d-82b0-daac52482ba8_isrForCrosstalkSources_996964_14.stderr parsl.dataflow.dflow DEBUG: Task 1768 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1768 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1768 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1768 has memoization hash 11bc7f782c3013b83089e29e0634d83c parsl.dataflow.memoization INFO: Task 1768 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd19e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1768 try 0 launched on executor multi with executor id 1769 parsl.dataflow.dflow INFO: Standard output for task 1768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8c16230b-4c05-4b22-9416-2bcc395bd36c_isrForCrosstalkSources_996960_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8c16230b-4c05-4b22-9416-2bcc395bd36c_isrForCrosstalkSources_996960_47.stderr parsl.dataflow.dflow DEBUG: Task 1769 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1769 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1769 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1769 has memoization hash 1778c9a5e6b027669931eb070b7a962f parsl.dataflow.memoization INFO: Task 1769 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd18a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1769 try 0 launched on executor multi with executor id 1770 parsl.dataflow.dflow INFO: Standard output for task 1769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/ea9dc2ba-7a25-40f5-b167-81095769a4bc_isrForCrosstalkSources_996965_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/ea9dc2ba-7a25-40f5-b167-81095769a4bc_isrForCrosstalkSources_996965_5.stderr parsl.dataflow.dflow DEBUG: Task 1770 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1770 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1770 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1770 has memoization hash 5b39b7ac1df8bda967067fc6f0dc0af5 parsl.dataflow.memoization INFO: Task 1770 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1770 try 0 launched on executor multi with executor id 1771 parsl.dataflow.dflow INFO: Standard output for task 1770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8068d567-8da1-403f-8445-448481792a91_isrForCrosstalkSources_997013_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8068d567-8da1-403f-8445-448481792a91_isrForCrosstalkSources_997013_54.stderr parsl.dataflow.dflow DEBUG: Task 1771 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1771 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1771 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1771 has memoization hash 20ec6f84893fd3aec5d43fbaa59e0683 parsl.dataflow.memoization INFO: Task 1771 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1771 try 0 launched on executor multi with executor id 1772 parsl.dataflow.dflow INFO: Standard output for task 1771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/20cde41d-aabe-4fee-9a8d-9f142170ad33_isrForCrosstalkSources_996984_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/20cde41d-aabe-4fee-9a8d-9f142170ad33_isrForCrosstalkSources_996984_33.stderr parsl.dataflow.dflow DEBUG: Task 1772 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1772 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1772 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1772 has memoization hash ae661d3be4e694686b24cae040342a0d parsl.dataflow.memoization INFO: Task 1772 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1772 try 0 launched on executor multi with executor id 1773 parsl.dataflow.dflow INFO: Standard output for task 1772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/6e0830a0-8260-4a65-888c-fd3862105969_isrForCrosstalkSources_996989_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/6e0830a0-8260-4a65-888c-fd3862105969_isrForCrosstalkSources_996989_26.stderr parsl.dataflow.dflow DEBUG: Task 1773 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1773 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1773 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1773 has memoization hash 411b135e7d9af9ea24ac8506047bf9a6 parsl.dataflow.memoization INFO: Task 1773 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd16c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1773 try 0 launched on executor multi with executor id 1774 parsl.dataflow.dflow INFO: Standard output for task 1773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7e817931-e103-4866-af12-7f5a8bb6e963_isrForCrosstalkSources_997006_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7e817931-e103-4866-af12-7f5a8bb6e963_isrForCrosstalkSources_997006_38.stderr parsl.dataflow.dflow DEBUG: Task 1774 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1774 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1774 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1774 has memoization hash 85cdae7df5a3f5382c2c4b949e3ba2ee parsl.dataflow.memoization INFO: Task 1774 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1774 try 0 launched on executor multi with executor id 1775 parsl.dataflow.dflow INFO: Standard output for task 1774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1044d54b-74d4-41d5-87e0-09f24a0c9e08_isrForCrosstalkSources_997027_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1044d54b-74d4-41d5-87e0-09f24a0c9e08_isrForCrosstalkSources_997027_59.stderr parsl.dataflow.dflow DEBUG: Task 1775 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1775 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1775 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1775 has memoization hash e3e440a9fbe798f3d544472c4c340def parsl.dataflow.memoization INFO: Task 1775 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1775 try 0 launched on executor multi with executor id 1776 parsl.dataflow.dflow INFO: Standard output for task 1775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7d18e1b6-df43-4a11-955e-38864971dc14_isrForCrosstalkSources_996986_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7d18e1b6-df43-4a11-955e-38864971dc14_isrForCrosstalkSources_996986_39.stderr parsl.dataflow.dflow DEBUG: Task 1776 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1776 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1776 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1776 has memoization hash a54cfaf37878a9ba4df77f8051f57769 parsl.dataflow.memoization INFO: Task 1776 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd14e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1776 try 0 launched on executor multi with executor id 1777 parsl.dataflow.dflow INFO: Standard output for task 1776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/387a27c2-41da-4830-b142-c1b375f76b3a_isrForCrosstalkSources_996998_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/387a27c2-41da-4830-b142-c1b375f76b3a_isrForCrosstalkSources_996998_27.stderr parsl.dataflow.dflow DEBUG: Task 1777 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1777 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1777 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1777 has memoization hash 8821bd520b3f7b4c8291d70ab2ec523b parsl.dataflow.memoization INFO: Task 1777 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1777 try 0 launched on executor multi with executor id 1778 parsl.dataflow.dflow INFO: Standard output for task 1777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/94e02588-d9fc-4809-ae4b-0d5914dbec5c_isrForCrosstalkSources_997008_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/94e02588-d9fc-4809-ae4b-0d5914dbec5c_isrForCrosstalkSources_997008_49.stderr parsl.dataflow.dflow DEBUG: Task 1778 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1778 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1778 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1778 has memoization hash cedc8670103f6bdfabc1235fcefd48ea parsl.dataflow.memoization INFO: Task 1778 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd13a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1778 try 0 launched on executor multi with executor id 1779 parsl.dataflow.dflow INFO: Standard output for task 1778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/f336555e-c4bb-4daf-b2f8-fdc1b60dbf20_isrForCrosstalkSources_997022_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/f336555e-c4bb-4daf-b2f8-fdc1b60dbf20_isrForCrosstalkSources_997022_13.stderr parsl.dataflow.dflow DEBUG: Task 1779 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1779 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1779 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1779 has memoization hash feef489ef20151adbce016db2769001a parsl.dataflow.memoization INFO: Task 1779 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1779 try 0 launched on executor multi with executor id 1780 parsl.dataflow.dflow INFO: Standard output for task 1779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d9e3ac89-7cea-4e72-a6cb-008001fbb912_isrForCrosstalkSources_996958_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d9e3ac89-7cea-4e72-a6cb-008001fbb912_isrForCrosstalkSources_996958_50.stderr parsl.dataflow.dflow DEBUG: Task 1780 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1780 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1780 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1780 has memoization hash 1df48f301bb3f64e01f5ea19dff984db parsl.dataflow.memoization INFO: Task 1780 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1780 try 0 launched on executor multi with executor id 1781 parsl.dataflow.dflow INFO: Standard output for task 1780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6008842e-04d9-4fb7-b408-913b641f2f0b_isrForCrosstalkSources_997030_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6008842e-04d9-4fb7-b408-913b641f2f0b_isrForCrosstalkSources_997030_58.stderr parsl.dataflow.dflow DEBUG: Task 1781 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1781 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1781 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1781 has memoization hash 8491abe83c8d7bacf598d1d0efcef2f4 parsl.dataflow.memoization INFO: Task 1781 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd11c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1781 try 0 launched on executor multi with executor id 1782 parsl.dataflow.dflow INFO: Standard output for task 1781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/c1af2275-244d-4ccb-b1bc-94a06891465e_isrForCrosstalkSources_997027_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/c1af2275-244d-4ccb-b1bc-94a06891465e_isrForCrosstalkSources_997027_5.stderr parsl.dataflow.dflow DEBUG: Task 1782 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1782 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1782 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1782 has memoization hash 575b8ec282fe5b2c2ba9293edc34ac7e parsl.dataflow.memoization INFO: Task 1782 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1782 try 0 launched on executor multi with executor id 1783 parsl.dataflow.dflow INFO: Standard output for task 1782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/9a44e585-0b62-4db7-bf4c-618874073f07_isrForCrosstalkSources_996953_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/9a44e585-0b62-4db7-bf4c-618874073f07_isrForCrosstalkSources_996953_21.stderr parsl.dataflow.dflow DEBUG: Task 1783 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1783 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1783 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1783 has memoization hash e860a959e4ac20ab69b3f8d0588252e6 parsl.dataflow.memoization INFO: Task 1783 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd1080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1783 try 0 launched on executor multi with executor id 1784 parsl.dataflow.dflow INFO: Standard output for task 1783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/fd2fbf35-04a2-4ef0-a4da-ff19ecdab52b_isrForCrosstalkSources_996988_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/fd2fbf35-04a2-4ef0-a4da-ff19ecdab52b_isrForCrosstalkSources_996988_3.stderr parsl.dataflow.dflow DEBUG: Task 1784 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1784 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1784 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1784 has memoization hash 05c3996fe9745745bc6fbab24ae4885c parsl.dataflow.memoization INFO: Task 1784 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1784 try 0 launched on executor multi with executor id 1785 parsl.dataflow.dflow INFO: Standard output for task 1784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7fce77a5-07bb-4f2a-9320-8f17daaa338c_isrForCrosstalkSources_996953_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7fce77a5-07bb-4f2a-9320-8f17daaa338c_isrForCrosstalkSources_996953_14.stderr parsl.dataflow.dflow DEBUG: Task 1785 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1785 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1785 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1785 has memoization hash dd0b14ca52ee61fcce7ef6d1302433d2 parsl.dataflow.memoization INFO: Task 1785 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd0f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1785 try 0 launched on executor multi with executor id 1786 parsl.dataflow.dflow INFO: Standard output for task 1785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/575c26a5-d7f2-4ce4-9d33-50900947c5cd_isrForCrosstalkSources_996987_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/575c26a5-d7f2-4ce4-9d33-50900947c5cd_isrForCrosstalkSources_996987_1.stderr parsl.dataflow.dflow DEBUG: Task 1786 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1786 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1786 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1786 has memoization hash 3ed83deea2410b9d5955cad38203008e parsl.dataflow.memoization INFO: Task 1786 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1786 try 0 launched on executor multi with executor id 1787 parsl.dataflow.dflow INFO: Standard output for task 1786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/62f36b7b-aa65-4e72-a336-8def265b395e_isrForCrosstalkSources_996982_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/62f36b7b-aa65-4e72-a336-8def265b395e_isrForCrosstalkSources_996982_20.stderr parsl.dataflow.dflow DEBUG: Task 1787 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1787 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1787 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1787 has memoization hash a1beb3f3479fa4dc96181e25f8921794 parsl.dataflow.memoization INFO: Task 1787 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd25c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1787 try 0 launched on executor multi with executor id 1788 parsl.dataflow.dflow INFO: Standard output for task 1787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/2b264544-2551-4a7c-a3b5-48eefaf65923_isrForCrosstalkSources_997020_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/2b264544-2551-4a7c-a3b5-48eefaf65923_isrForCrosstalkSources_997020_5.stderr parsl.dataflow.dflow DEBUG: Task 1788 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1788 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1788 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1788 has memoization hash 4e44fc6c3d1587e71b4e3a3a39356193 parsl.dataflow.memoization INFO: Task 1788 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1788 try 0 launched on executor multi with executor id 1789 parsl.dataflow.dflow INFO: Standard output for task 1788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/983bd7ae-d370-4c6f-bd4a-c0844676c972_isrForCrosstalkSources_997013_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/983bd7ae-d370-4c6f-bd4a-c0844676c972_isrForCrosstalkSources_997013_19.stderr parsl.dataflow.dflow DEBUG: Task 1789 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1789 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1789 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1789 has memoization hash 9f518364bc23e3ba0883ba5ebaf0cf51 parsl.dataflow.memoization INFO: Task 1789 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd27a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1789 try 0 launched on executor multi with executor id 1790 parsl.dataflow.dflow INFO: Standard output for task 1789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/3b537229-b774-4434-a0be-c226d9798b64_isrForCrosstalkSources_997004_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/3b537229-b774-4434-a0be-c226d9798b64_isrForCrosstalkSources_997004_31.stderr parsl.dataflow.dflow DEBUG: Task 1790 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1790 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1790 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1790 has memoization hash d4dee72b023711911a38373c5ca297f5 parsl.dataflow.memoization INFO: Task 1790 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1790 try 0 launched on executor multi with executor id 1791 parsl.dataflow.dflow INFO: Standard output for task 1790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/dfdff0b5-b6f8-41c7-b61d-d81458141ead_isrForCrosstalkSources_997007_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/dfdff0b5-b6f8-41c7-b61d-d81458141ead_isrForCrosstalkSources_997007_28.stderr parsl.dataflow.dflow DEBUG: Task 1791 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1791 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1791 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1791 has memoization hash f00a4db55fbaa701f4c976e6b32e3336 parsl.dataflow.memoization INFO: Task 1791 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1791 try 0 launched on executor multi with executor id 1792 parsl.dataflow.dflow INFO: Standard output for task 1791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/6d737b6d-f7e7-45aa-bd4c-b2bbec06ffbf_isrForCrosstalkSources_996995_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/6d737b6d-f7e7-45aa-bd4c-b2bbec06ffbf_isrForCrosstalkSources_996995_21.stderr parsl.dataflow.dflow DEBUG: Task 1792 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1792 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1792 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1792 has memoization hash 69da28f3ee739e2c8b55992c1292c081 parsl.dataflow.memoization INFO: Task 1792 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd28e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1792 try 0 launched on executor multi with executor id 1793 parsl.dataflow.dflow INFO: Standard output for task 1792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/366d2304-f930-4c5c-83ba-de1d7500ca54_isrForCrosstalkSources_996981_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/366d2304-f930-4c5c-83ba-de1d7500ca54_isrForCrosstalkSources_996981_56.stderr parsl.dataflow.dflow DEBUG: Task 1793 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1793 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1793 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1793 has memoization hash 813f534f6e8ddc3bb0377c52db0c94e4 parsl.dataflow.memoization INFO: Task 1793 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1793 try 0 launched on executor multi with executor id 1794 parsl.dataflow.dflow INFO: Standard output for task 1793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/95d70b1b-a951-4641-b18c-87468f1bfb18_isrForCrosstalkSources_996994_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/95d70b1b-a951-4641-b18c-87468f1bfb18_isrForCrosstalkSources_996994_42.stderr parsl.dataflow.dflow DEBUG: Task 1794 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1794 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1794 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1794 has memoization hash e2546cb888278cdb560a5e70d830a205 parsl.dataflow.memoization INFO: Task 1794 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1794 try 0 launched on executor multi with executor id 1795 parsl.dataflow.dflow INFO: Standard output for task 1794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/35c282d3-fb7c-44d4-8090-dd86832086a5_isrForCrosstalkSources_996990_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/35c282d3-fb7c-44d4-8090-dd86832086a5_isrForCrosstalkSources_996990_34.stderr parsl.dataflow.dflow DEBUG: Task 1795 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1795 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1795 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1795 has memoization hash 0fe60dd3ff256b3e41ca0523e2946973 parsl.dataflow.memoization INFO: Task 1795 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1795 try 0 launched on executor multi with executor id 1796 parsl.dataflow.dflow INFO: Standard output for task 1795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/976acb37-ec98-47b0-8ff8-d273fe65efb0_isrForCrosstalkSources_997025_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/976acb37-ec98-47b0-8ff8-d273fe65efb0_isrForCrosstalkSources_997025_39.stderr parsl.dataflow.dflow DEBUG: Task 1796 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1796 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1796 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1796 has memoization hash 79cdcd0fc1e72f03023eaed83b79a744 parsl.dataflow.memoization INFO: Task 1796 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1796 try 0 launched on executor multi with executor id 1797 parsl.dataflow.dflow INFO: Standard output for task 1796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/524fa598-d155-4572-9ac9-8cfa4d7b01bd_isrForCrosstalkSources_996998_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/524fa598-d155-4572-9ac9-8cfa4d7b01bd_isrForCrosstalkSources_996998_56.stderr parsl.dataflow.dflow DEBUG: Task 1797 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1797 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1797 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1797 has memoization hash 59fb7258e36f4ebcf38679c8b174b4c8 parsl.dataflow.memoization INFO: Task 1797 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1797 try 0 launched on executor multi with executor id 1798 parsl.dataflow.dflow INFO: Standard output for task 1797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6c96910-0077-4c04-8bd2-37abd3c0cd1a_isrForCrosstalkSources_996976_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6c96910-0077-4c04-8bd2-37abd3c0cd1a_isrForCrosstalkSources_996976_35.stderr parsl.dataflow.dflow DEBUG: Task 1798 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1798 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1798 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1798 has memoization hash ec2c2927905d1db5fce75dca304159f6 parsl.dataflow.memoization INFO: Task 1798 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1798 try 0 launched on executor multi with executor id 1799 parsl.dataflow.dflow INFO: Standard output for task 1798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ac135897-d8bc-48b0-aa6b-d43b397a67be_isrForCrosstalkSources_996954_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ac135897-d8bc-48b0-aa6b-d43b397a67be_isrForCrosstalkSources_996954_54.stderr parsl.dataflow.dflow DEBUG: Task 1799 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1799 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1799 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1799 has memoization hash 3c45ae8ba887332b057b63fb023d088e parsl.dataflow.memoization INFO: Task 1799 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1799 try 0 launched on executor multi with executor id 1800 parsl.dataflow.dflow INFO: Standard output for task 1799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/ecde18dd-4109-43a9-bcdc-97c055548fee_isrForCrosstalkSources_996953_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/ecde18dd-4109-43a9-bcdc-97c055548fee_isrForCrosstalkSources_996953_33.stderr parsl.dataflow.dflow DEBUG: Task 1800 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1800 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1800 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1800 has memoization hash 8593807dc5989565bd7008eb8ac19c57 parsl.dataflow.memoization INFO: Task 1800 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1800 try 0 launched on executor multi with executor id 1801 parsl.dataflow.dflow INFO: Standard output for task 1800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/81da5f35-7794-4ef3-bb5d-5b92fec235bd_isrForCrosstalkSources_996964_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/81da5f35-7794-4ef3-bb5d-5b92fec235bd_isrForCrosstalkSources_996964_13.stderr parsl.dataflow.dflow DEBUG: Task 1801 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1801 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1801 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1801 has memoization hash 1bf500d84283ba650accc6d14c5b46ec parsl.dataflow.memoization INFO: Task 1801 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1801 try 0 launched on executor multi with executor id 1802 parsl.dataflow.dflow INFO: Standard output for task 1801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/178e3b30-ac17-4b97-a000-6b13e79550be_isrForCrosstalkSources_997005_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/178e3b30-ac17-4b97-a000-6b13e79550be_isrForCrosstalkSources_997005_27.stderr parsl.dataflow.dflow DEBUG: Task 1802 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1802 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1802 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1802 has memoization hash 0200ee92f120c501c27c5b78a83ac1eb parsl.dataflow.memoization INFO: Task 1802 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1802 try 0 launched on executor multi with executor id 1803 parsl.dataflow.dflow INFO: Standard output for task 1802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/9a4141ca-c897-4027-b751-5a7136d4ac24_isrForCrosstalkSources_997033_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/9a4141ca-c897-4027-b751-5a7136d4ac24_isrForCrosstalkSources_997033_11.stderr parsl.dataflow.dflow DEBUG: Task 1803 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1803 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1803 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1803 has memoization hash e9e29dbb65f0164e402a2deef7588e35 parsl.dataflow.memoization INFO: Task 1803 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1803 try 0 launched on executor multi with executor id 1804 parsl.dataflow.dflow INFO: Standard output for task 1803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/4f38a6b8-f6eb-40fc-8e47-8711c658a5a1_isrForCrosstalkSources_996996_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/4f38a6b8-f6eb-40fc-8e47-8711c658a5a1_isrForCrosstalkSources_996996_47.stderr parsl.dataflow.dflow DEBUG: Task 1804 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1804 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1804 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1804 has memoization hash 1a4095720be8b8109034c3bbd8dbf1e8 parsl.dataflow.memoization INFO: Task 1804 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1804 try 0 launched on executor multi with executor id 1805 parsl.dataflow.dflow INFO: Standard output for task 1804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/51f3547d-8eb5-47a5-b8c7-aeb3799ef058_isrForCrosstalkSources_997019_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/51f3547d-8eb5-47a5-b8c7-aeb3799ef058_isrForCrosstalkSources_997019_54.stderr parsl.dataflow.dflow DEBUG: Task 1805 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1805 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1805 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1805 has memoization hash 80d389f3a91b802b2906f88868cc3958 parsl.dataflow.memoization INFO: Task 1805 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1805 try 0 launched on executor multi with executor id 1806 parsl.dataflow.dflow INFO: Standard output for task 1805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d1032567-6fd2-4ee8-ab2a-97b63c42b622_isrForCrosstalkSources_996972_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d1032567-6fd2-4ee8-ab2a-97b63c42b622_isrForCrosstalkSources_996972_58.stderr parsl.dataflow.dflow DEBUG: Task 1806 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1806 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1806 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1806 has memoization hash a584a77191bd74bb8dfdd2f230245199 parsl.dataflow.memoization INFO: Task 1806 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd31a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1806 try 0 launched on executor multi with executor id 1807 parsl.dataflow.dflow INFO: Standard output for task 1806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0556cb73-0553-4b10-93cd-07eb3d6cd51f_isrForCrosstalkSources_996975_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0556cb73-0553-4b10-93cd-07eb3d6cd51f_isrForCrosstalkSources_996975_29.stderr parsl.dataflow.dflow DEBUG: Task 1807 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1807 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1807 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1807 has memoization hash 63c19320ddb133930ac4dc2794c927b0 parsl.dataflow.memoization INFO: Task 1807 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1807 try 0 launched on executor multi with executor id 1808 parsl.dataflow.dflow INFO: Standard output for task 1807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/aa4b87bd-f4e4-478a-8ec6-9f27e53cf12b_isrForCrosstalkSources_996975_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/aa4b87bd-f4e4-478a-8ec6-9f27e53cf12b_isrForCrosstalkSources_996975_25.stderr parsl.dataflow.dflow DEBUG: Task 1808 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1808 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1808 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1808 has memoization hash 2622283cb272a0bea1904a7288225c84 parsl.dataflow.memoization INFO: Task 1808 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd32e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1808 try 0 launched on executor multi with executor id 1809 parsl.dataflow.dflow INFO: Standard output for task 1808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2e889981-edba-4ffc-99c0-32a3e8e4adc0_isrForCrosstalkSources_996995_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2e889981-edba-4ffc-99c0-32a3e8e4adc0_isrForCrosstalkSources_996995_10.stderr parsl.dataflow.dflow DEBUG: Task 1809 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1809 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1809 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1809 has memoization hash acb5f5a62c81fb8cbd98e6c8352edf57 parsl.dataflow.memoization INFO: Task 1809 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1809 try 0 launched on executor multi with executor id 1810 parsl.dataflow.dflow INFO: Standard output for task 1809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/0e8a9c4b-bf23-450a-afe0-7256b978055a_isrForCrosstalkSources_996957_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/0e8a9c4b-bf23-450a-afe0-7256b978055a_isrForCrosstalkSources_996957_19.stderr parsl.dataflow.dflow DEBUG: Task 1810 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1810 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1810 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1810 has memoization hash 04d7f10ceddce129351922b203bc59df parsl.dataflow.memoization INFO: Task 1810 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1810 try 0 launched on executor multi with executor id 1811 parsl.dataflow.dflow INFO: Standard output for task 1810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/71271bdf-4e0b-42fd-ae58-5fa2453c95ad_isrForCrosstalkSources_996956_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/71271bdf-4e0b-42fd-ae58-5fa2453c95ad_isrForCrosstalkSources_996956_32.stderr parsl.dataflow.dflow DEBUG: Task 1811 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1811 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1811 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1811 has memoization hash 26d36c5a3ef14bfaff19f2f978baaae8 parsl.dataflow.memoization INFO: Task 1811 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd34c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1811 try 0 launched on executor multi with executor id 1812 parsl.dataflow.dflow INFO: Standard output for task 1811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/aee9cbff-20a1-45b8-8904-32c9e5b1ffca_isrForCrosstalkSources_996992_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/aee9cbff-20a1-45b8-8904-32c9e5b1ffca_isrForCrosstalkSources_996992_13.stderr parsl.dataflow.dflow DEBUG: Task 1812 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1812 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1812 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1812 has memoization hash f1ca4e4489c3985f37ceb95f7d63fc1e parsl.dataflow.memoization INFO: Task 1812 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1812 try 0 launched on executor multi with executor id 1813 parsl.dataflow.dflow INFO: Standard output for task 1812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/65a47739-011e-4b3d-92fa-207934f5d3b5_isrForCrosstalkSources_997013_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/65a47739-011e-4b3d-92fa-207934f5d3b5_isrForCrosstalkSources_997013_24.stderr parsl.dataflow.dflow DEBUG: Task 1813 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1813 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1813 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1813 has memoization hash 3c5b88ebcdb4a3554cc506d7bb66208a parsl.dataflow.memoization INFO: Task 1813 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1813 try 0 launched on executor multi with executor id 1814 parsl.dataflow.dflow INFO: Standard output for task 1813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/e3fde26a-c4a8-4f26-8281-dc324acb3452_isrForCrosstalkSources_996962_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/e3fde26a-c4a8-4f26-8281-dc324acb3452_isrForCrosstalkSources_996962_33.stderr parsl.dataflow.dflow DEBUG: Task 1814 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1814 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1814 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1814 has memoization hash bf8d97eb34cd0fc9168d819cef5ca07a parsl.dataflow.memoization INFO: Task 1814 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1814 try 0 launched on executor multi with executor id 1815 parsl.dataflow.dflow INFO: Standard output for task 1814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7e873933-9df4-4fd3-b05c-2e2ea791af05_isrForCrosstalkSources_997029_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7e873933-9df4-4fd3-b05c-2e2ea791af05_isrForCrosstalkSources_997029_40.stderr parsl.dataflow.dflow DEBUG: Task 1815 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1815 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1815 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1815 has memoization hash fe9838de99b96e9f4d501a401fdd03dd parsl.dataflow.memoization INFO: Task 1815 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd36a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1815 try 0 launched on executor multi with executor id 1816 parsl.dataflow.dflow INFO: Standard output for task 1815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e02b1e02-ad7e-43a3-b53b-fa25bc04906f_isrForCrosstalkSources_997016_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e02b1e02-ad7e-43a3-b53b-fa25bc04906f_isrForCrosstalkSources_997016_28.stderr parsl.dataflow.dflow DEBUG: Task 1816 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1816 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1816 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1816 has memoization hash a5eae9db39d8bf618caba6686b2bb400 parsl.dataflow.memoization INFO: Task 1816 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd37e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1816 try 0 launched on executor multi with executor id 1817 parsl.dataflow.dflow INFO: Standard output for task 1816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/97ac01ba-bdcc-4267-82f6-5f1e1f2f1a67_isrForCrosstalkSources_996973_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/97ac01ba-bdcc-4267-82f6-5f1e1f2f1a67_isrForCrosstalkSources_996973_22.stderr parsl.dataflow.dflow DEBUG: Task 1817 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1817 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1817 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1817 has memoization hash 2e0bb106f77ff3e2814669c8b233f203 parsl.dataflow.memoization INFO: Task 1817 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1817 try 0 launched on executor multi with executor id 1818 parsl.dataflow.dflow INFO: Standard output for task 1817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/0b1bdef3-29b5-435d-80e6-c6f4631d12dd_isrForCrosstalkSources_997024_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/0b1bdef3-29b5-435d-80e6-c6f4631d12dd_isrForCrosstalkSources_997024_21.stderr parsl.dataflow.dflow DEBUG: Task 1818 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1818 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1818 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1818 has memoization hash 853726e27453b020153416d89918d69d parsl.dataflow.memoization INFO: Task 1818 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1818 try 0 launched on executor multi with executor id 1819 parsl.dataflow.dflow INFO: Standard output for task 1818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7ad802ec-214f-4f62-a838-f55c0007b2ab_isrForCrosstalkSources_996986_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7ad802ec-214f-4f62-a838-f55c0007b2ab_isrForCrosstalkSources_996986_55.stderr parsl.dataflow.dflow DEBUG: Task 1819 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1819 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1819 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1819 has memoization hash a7ba272b448aab4112fbf6e95ea5c007 parsl.dataflow.memoization INFO: Task 1819 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:44 proc_lsst.multi:170 [INFO] [multi] job_id 17074865 proc_lsst.multi INFO: [multi] job_id 17074865 2024-03-15 13:49:44 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 22 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 22 2024-03-15 13:49:44 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 parsl.dataflow.dflow INFO: Parsl task 1819 try 0 launched on executor multi with executor id 1820 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:44 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074865 parsl.dataflow.dflow INFO: Standard output for task 1819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/84560068-762c-4b7a-a7ed-638ccc1251ec_isrForCrosstalkSources_996986_57.stdout proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074865 parsl.dataflow.dflow INFO: Standard error for task 1819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/84560068-762c-4b7a-a7ed-638ccc1251ec_isrForCrosstalkSources_996986_57.stderr parsl.executors.status_handling DEBUG: Launched block 22 on executor multi with job ID 17074865 parsl.dataflow.dflow DEBUG: Task 1820 will be sent to executor multi parsl.executors.status_handling INFO: Allocated block ID 23 parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-23 2024-03-15 13:49:44 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=23 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-23 parsl.dataflow.dflow INFO: Task 1820 submitted for App isrForCrosstalkSources, not waiting on any dependency proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=23 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-23 2024-03-15 13:49:44 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=23 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 parsl.dataflow.dflow DEBUG: Task 1820 set to pending state with AppFuture: proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=23 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:44 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=23 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=23 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 22 parsl.dataflow.memoization DEBUG: Task 1820 has memoization hash 983abeb9282b8ef3f4d270561b12ce42 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 22 2024-03-15 13:49:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 parsl.dataflow.memoization INFO: Task 1820 had no result in cache proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:44 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=23 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a78139d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=23 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1820 try 0 launched on executor multi with executor id 1821 parsl.dataflow.dflow INFO: Standard output for task 1820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/ba7bebd4-c6e4-4865-b705-e9cd7fe2ea98_isrForCrosstalkSources_996997_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/ba7bebd4-c6e4-4865-b705-e9cd7fe2ea98_isrForCrosstalkSources_996997_29.stderr parsl.dataflow.dflow DEBUG: Task 1821 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1821 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1821 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1821 has memoization hash 5f5d5dc8d5a795a0cd2148fcc2b7a5af parsl.dataflow.memoization INFO: Task 1821 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780f66f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1821 try 0 launched on executor multi with executor id 1822 parsl.dataflow.dflow INFO: Standard output for task 1821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0d723bfd-ac80-4087-a278-7a614dd729bb_isrForCrosstalkSources_996956_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0d723bfd-ac80-4087-a278-7a614dd729bb_isrForCrosstalkSources_996956_52.stderr parsl.dataflow.dflow DEBUG: Task 1822 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1822 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1822 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1822 has memoization hash 28a718eba1f82c97f22de8ef67e0acc2 parsl.dataflow.memoization INFO: Task 1822 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2baf20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1822 try 0 launched on executor multi with executor id 1823 parsl.dataflow.dflow INFO: Standard output for task 1822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/10a85776-b4a2-4724-a7ba-c1723aee74e0_isrForCrosstalkSources_996995_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/10a85776-b4a2-4724-a7ba-c1723aee74e0_isrForCrosstalkSources_996995_36.stderr parsl.dataflow.dflow DEBUG: Task 1823 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1823 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1823 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1823 has memoization hash cf2a597906554184ce75e8f426dbffe5 parsl.dataflow.memoization INFO: Task 1823 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e200040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1823 try 0 launched on executor multi with executor id 1824 parsl.dataflow.dflow INFO: Standard output for task 1823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/29a08579-8a0b-45f6-a512-1471c8fafe4c_isrForCrosstalkSources_996987_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/29a08579-8a0b-45f6-a512-1471c8fafe4c_isrForCrosstalkSources_996987_24.stderr parsl.dataflow.dflow DEBUG: Task 1824 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1824 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1824 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1824 has memoization hash d7dc9620a28693be9aff06a9180933ed parsl.dataflow.memoization INFO: Task 1824 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1824 try 0 launched on executor multi with executor id 1825 parsl.dataflow.dflow INFO: Standard output for task 1824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d6a21b4c-856a-4b78-931b-9ea25bb1d219_isrForCrosstalkSources_996982_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d6a21b4c-856a-4b78-931b-9ea25bb1d219_isrForCrosstalkSources_996982_29.stderr parsl.dataflow.dflow DEBUG: Task 1825 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1825 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1825 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1825 has memoization hash fdba8ae85f5edc57146ae9dd2ba387d3 parsl.dataflow.memoization INFO: Task 1825 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1825 try 0 launched on executor multi with executor id 1826 parsl.dataflow.dflow INFO: Standard output for task 1825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fc5be51f-df12-4092-be5d-9aa02e23293f_isrForCrosstalkSources_997001_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fc5be51f-df12-4092-be5d-9aa02e23293f_isrForCrosstalkSources_997001_28.stderr parsl.dataflow.dflow DEBUG: Task 1826 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1826 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1826 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1826 has memoization hash 58daffeaee3476712954a7f4a7e1aed8 parsl.dataflow.memoization INFO: Task 1826 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1826 try 0 launched on executor multi with executor id 1827 parsl.dataflow.dflow INFO: Standard output for task 1826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fc5844cd-3043-4794-b568-0dfdb3a77149_isrForCrosstalkSources_996961_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fc5844cd-3043-4794-b568-0dfdb3a77149_isrForCrosstalkSources_996961_2.stderr parsl.dataflow.dflow DEBUG: Task 1827 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1827 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1827 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1827 has memoization hash f709bf1317dc3d91ce6c32ea3a59907e parsl.dataflow.memoization INFO: Task 1827 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1827 try 0 launched on executor multi with executor id 1828 parsl.dataflow.dflow INFO: Standard output for task 1827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9898abcb-de8e-4e5d-a32d-1ace028abb33_isrForCrosstalkSources_997022_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9898abcb-de8e-4e5d-a32d-1ace028abb33_isrForCrosstalkSources_997022_1.stderr parsl.dataflow.dflow DEBUG: Task 1828 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1828 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1828 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1828 has memoization hash acf7f2af9cf071c29890226f4ce0e09c parsl.dataflow.memoization INFO: Task 1828 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1828 try 0 launched on executor multi with executor id 1829 parsl.dataflow.dflow INFO: Standard output for task 1828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/8c114687-ce02-4bd0-8b0a-2d1b67b2cfa0_isrForCrosstalkSources_997002_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/8c114687-ce02-4bd0-8b0a-2d1b67b2cfa0_isrForCrosstalkSources_997002_40.stderr parsl.dataflow.dflow DEBUG: Task 1829 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1829 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1829 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1829 has memoization hash ea7111eb069d99c36bb37d52ff972dc4 parsl.dataflow.memoization INFO: Task 1829 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1829 try 0 launched on executor multi with executor id 1830 parsl.dataflow.dflow INFO: Standard output for task 1829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8523c039-71db-47e6-ad12-f18b90cbeef7_isrForCrosstalkSources_996963_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8523c039-71db-47e6-ad12-f18b90cbeef7_isrForCrosstalkSources_996963_40.stderr parsl.dataflow.dflow DEBUG: Task 1830 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1830 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1830 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1830 has memoization hash 33abb696e209b1e06986d7eb82a67cd7 parsl.dataflow.memoization INFO: Task 1830 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3ae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1830 try 0 launched on executor multi with executor id 1831 parsl.dataflow.dflow INFO: Standard output for task 1830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/2aaf9b69-227a-4fa4-a4ce-0f62c3ad3128_isrForCrosstalkSources_997025_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/2aaf9b69-227a-4fa4-a4ce-0f62c3ad3128_isrForCrosstalkSources_997025_28.stderr parsl.dataflow.dflow DEBUG: Task 1831 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1831 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1831 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1831 has memoization hash eae7551bc7f6b20f3a63c76f5ce710a4 parsl.dataflow.memoization INFO: Task 1831 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd380e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1831 try 0 launched on executor multi with executor id 1832 parsl.dataflow.dflow INFO: Standard output for task 1831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/af95b851-b00f-4320-86ca-3dba64f1152b_isrForCrosstalkSources_996984_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/af95b851-b00f-4320-86ca-3dba64f1152b_isrForCrosstalkSources_996984_37.stderr parsl.dataflow.dflow DEBUG: Task 1832 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1832 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1832 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1832 has memoization hash 7df1aae886796c115d56618b404cce1b parsl.dataflow.memoization INFO: Task 1832 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1832 try 0 launched on executor multi with executor id 1833 parsl.dataflow.dflow INFO: Standard output for task 1832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3cae46ab-7460-4246-8c79-120314ff9c1c_isrForCrosstalkSources_996968_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3cae46ab-7460-4246-8c79-120314ff9c1c_isrForCrosstalkSources_996968_30.stderr parsl.dataflow.dflow DEBUG: Task 1833 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1833 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1833 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1833 has memoization hash a5896ffd8501247cf68a6da4565e8946 parsl.dataflow.memoization INFO: Task 1833 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd382c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1833 try 0 launched on executor multi with executor id 1834 parsl.dataflow.dflow INFO: Standard output for task 1833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/772900ec-28f6-4210-ab44-b5faa2229dc8_isrForCrosstalkSources_996966_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/772900ec-28f6-4210-ab44-b5faa2229dc8_isrForCrosstalkSources_996966_60.stderr parsl.dataflow.dflow DEBUG: Task 1834 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1834 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1834 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1834 has memoization hash 110aec82af3a30799bc3713412d9a657 parsl.dataflow.memoization INFO: Task 1834 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1834 try 0 launched on executor multi with executor id 1835 parsl.dataflow.dflow INFO: Standard output for task 1834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fd92eb1e-036f-4629-ae38-91a23def7ce2_isrForCrosstalkSources_996968_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fd92eb1e-036f-4629-ae38-91a23def7ce2_isrForCrosstalkSources_996968_48.stderr parsl.dataflow.dflow DEBUG: Task 1835 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1835 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1835 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1835 has memoization hash efcb24f4c8fc73783941d4e14479854e parsl.dataflow.memoization INFO: Task 1835 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1835 try 0 launched on executor multi with executor id 1836 parsl.dataflow.dflow INFO: Standard output for task 1835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f3267e7d-eb0b-4206-a687-d96a90441dfd_isrForCrosstalkSources_996960_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f3267e7d-eb0b-4206-a687-d96a90441dfd_isrForCrosstalkSources_996960_43.stderr parsl.dataflow.dflow DEBUG: Task 1836 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1836 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1836 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1836 has memoization hash 1ebe9179ce5d61aae37a27111cc9665c parsl.dataflow.memoization INFO: Task 1836 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1836 try 0 launched on executor multi with executor id 1837 parsl.dataflow.dflow INFO: Standard output for task 1836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/627de1a7-5ed8-43fd-a6d0-89a15bf08e73_isrForCrosstalkSources_996985_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/627de1a7-5ed8-43fd-a6d0-89a15bf08e73_isrForCrosstalkSources_996985_33.stderr parsl.dataflow.dflow DEBUG: Task 1837 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1837 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1837 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1837 has memoization hash e7715fde651d85f0be014ef28cc3b20d parsl.dataflow.memoization INFO: Task 1837 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd384a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1837 try 0 launched on executor multi with executor id 1838 parsl.dataflow.dflow INFO: Standard output for task 1837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/792f8fd2-a2e3-4017-bb0d-bb939d3d2da6_isrForCrosstalkSources_997031_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/792f8fd2-a2e3-4017-bb0d-bb939d3d2da6_isrForCrosstalkSources_997031_27.stderr parsl.dataflow.dflow DEBUG: Task 1838 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1838 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1838 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1838 has memoization hash f69cae33e50e03af0e29b985a2c5d68d parsl.dataflow.memoization INFO: Task 1838 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1838 try 0 launched on executor multi with executor id 1839 parsl.dataflow.dflow INFO: Standard output for task 1838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9e06a674-19c9-480b-930e-c6507ad86e10_isrForCrosstalkSources_997030_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9e06a674-19c9-480b-930e-c6507ad86e10_isrForCrosstalkSources_997030_34.stderr parsl.dataflow.dflow DEBUG: Task 1839 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1839 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1839 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1839 has memoization hash 3ad1eb1a23c91d69daffd3ff6b041e4c parsl.dataflow.memoization INFO: Task 1839 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1839 try 0 launched on executor multi with executor id 1840 parsl.dataflow.dflow INFO: Standard output for task 1839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/202a2949-7366-48de-82f1-0b4e909fe798_isrForCrosstalkSources_996974_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/202a2949-7366-48de-82f1-0b4e909fe798_isrForCrosstalkSources_996974_50.stderr parsl.dataflow.dflow DEBUG: Task 1840 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1840 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1840 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1840 has memoization hash 0272ceb417a5374f29530ae048e06d3a parsl.dataflow.memoization INFO: Task 1840 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1840 try 0 launched on executor multi with executor id 1841 parsl.dataflow.dflow INFO: Standard output for task 1840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4ce1da69-01bf-4f0d-9a16-7dbf9053b61b_isrForCrosstalkSources_997030_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4ce1da69-01bf-4f0d-9a16-7dbf9053b61b_isrForCrosstalkSources_997030_48.stderr parsl.dataflow.dflow DEBUG: Task 1841 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1841 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1841 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1841 has memoization hash d86b03e317a95cc0415904231fa4d1a0 parsl.dataflow.memoization INFO: Task 1841 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd387c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1841 try 0 launched on executor multi with executor id 1842 parsl.dataflow.dflow INFO: Standard output for task 1841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c9dab5ab-4a6a-419d-ba02-e36aa1d9a17d_isrForCrosstalkSources_997012_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c9dab5ab-4a6a-419d-ba02-e36aa1d9a17d_isrForCrosstalkSources_997012_25.stderr parsl.dataflow.dflow DEBUG: Task 1842 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1842 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1842 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1842 has memoization hash 2a81e2077f5094ffd8d29537d1da2801 parsl.dataflow.memoization INFO: Task 1842 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd385e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1842 try 0 launched on executor multi with executor id 1843 parsl.dataflow.dflow INFO: Standard output for task 1842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/659f08a5-2414-405e-a4e7-cba49ca3ab9a_isrForCrosstalkSources_996957_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/659f08a5-2414-405e-a4e7-cba49ca3ab9a_isrForCrosstalkSources_996957_46.stderr parsl.dataflow.dflow DEBUG: Task 1843 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1843 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1843 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1843 has memoization hash abc3894afb4b2ea6cd7021cf2a0065e5 parsl.dataflow.memoization INFO: Task 1843 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1843 try 0 launched on executor multi with executor id 1844 parsl.dataflow.dflow INFO: Standard output for task 1843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1efe983d-15e4-4314-bca8-80cabe07960c_isrForCrosstalkSources_997027_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1efe983d-15e4-4314-bca8-80cabe07960c_isrForCrosstalkSources_997027_37.stderr parsl.dataflow.dflow DEBUG: Task 1844 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1844 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1844 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1844 has memoization hash 94b26ba2627d14d628d1f156b4f65a71 parsl.dataflow.memoization INFO: Task 1844 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1844 try 0 launched on executor multi with executor id 1845 parsl.dataflow.dflow INFO: Standard output for task 1844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/9c7898c0-8e5d-4cdf-a0f5-d4929fe9c79a_isrForCrosstalkSources_996982_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/9c7898c0-8e5d-4cdf-a0f5-d4929fe9c79a_isrForCrosstalkSources_996982_16.stderr parsl.dataflow.dflow DEBUG: Task 1845 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1845 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1845 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1845 has memoization hash 6ff19eddce2e3d59d1f99ee4cf6d883c parsl.dataflow.memoization INFO: Task 1845 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd389a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1845 try 0 launched on executor multi with executor id 1846 parsl.dataflow.dflow INFO: Standard output for task 1845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/4e31aec4-8fb2-4e72-844d-441c97ef3b1a_isrForCrosstalkSources_996955_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/4e31aec4-8fb2-4e72-844d-441c97ef3b1a_isrForCrosstalkSources_996955_54.stderr parsl.dataflow.dflow DEBUG: Task 1846 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1846 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1846 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1846 has memoization hash 88685816f54f03ed87cfa5fcabd5f6be parsl.dataflow.memoization INFO: Task 1846 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1846 try 0 launched on executor multi with executor id 1847 parsl.dataflow.dflow INFO: Standard output for task 1846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0ce80222-7095-4a9b-b68e-6b6da0cc59f8_isrForCrosstalkSources_996953_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0ce80222-7095-4a9b-b68e-6b6da0cc59f8_isrForCrosstalkSources_996953_38.stderr parsl.dataflow.dflow DEBUG: Task 1847 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1847 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1847 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1847 has memoization hash cd0eee8caaa4000c08f925b9b95053ad parsl.dataflow.memoization INFO: Task 1847 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1847 try 0 launched on executor multi with executor id 1848 parsl.dataflow.dflow INFO: Standard output for task 1847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/cd8930a3-172c-456b-af0f-dd18a05c0351_isrForCrosstalkSources_997022_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/cd8930a3-172c-456b-af0f-dd18a05c0351_isrForCrosstalkSources_997022_32.stderr parsl.dataflow.dflow DEBUG: Task 1848 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1848 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1848 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1848 has memoization hash 71e626c1a46b7933be9b81565b9df8e3 parsl.dataflow.memoization INFO: Task 1848 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1848 try 0 launched on executor multi with executor id 1849 parsl.dataflow.dflow INFO: Standard output for task 1848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ee36ed81-1522-45f0-b0fb-33a957a7ff26_isrForCrosstalkSources_996982_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ee36ed81-1522-45f0-b0fb-33a957a7ff26_isrForCrosstalkSources_996982_30.stderr parsl.dataflow.dflow DEBUG: Task 1849 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1849 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1849 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1849 has memoization hash 46ca145f7e515d52b56d8a78ba6e5deb parsl.dataflow.memoization INFO: Task 1849 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1849 try 0 launched on executor multi with executor id 1850 parsl.dataflow.dflow INFO: Standard output for task 1849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b10d63fb-f296-4370-a822-ee2d9f3b1338_isrForCrosstalkSources_996999_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b10d63fb-f296-4370-a822-ee2d9f3b1338_isrForCrosstalkSources_996999_7.stderr parsl.dataflow.dflow DEBUG: Task 1850 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1850 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1850 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1850 has memoization hash 76ec24e8f64343e6f83e8a3f65e4e1bb parsl.dataflow.memoization INFO: Task 1850 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1850 try 0 launched on executor multi with executor id 1851 parsl.dataflow.dflow INFO: Standard output for task 1850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/9b548712-e53e-4ac3-8691-1209ae89d8a8_isrForCrosstalkSources_997025_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/9b548712-e53e-4ac3-8691-1209ae89d8a8_isrForCrosstalkSources_997025_24.stderr parsl.dataflow.dflow DEBUG: Task 1851 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1851 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1851 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1851 has memoization hash d3b22408712a3eb374e749a8cf1d5e47 parsl.dataflow.memoization INFO: Task 1851 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1851 try 0 launched on executor multi with executor id 1852 parsl.dataflow.dflow INFO: Standard output for task 1851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/d3b01399-6d96-4adc-bcc0-b11b11a357d6_isrForCrosstalkSources_996967_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/d3b01399-6d96-4adc-bcc0-b11b11a357d6_isrForCrosstalkSources_996967_40.stderr parsl.dataflow.dflow DEBUG: Task 1852 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1852 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1852 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1852 has memoization hash 23fbaa24e23a5a51a602f3d5ee44f695 parsl.dataflow.memoization INFO: Task 1852 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1852 try 0 launched on executor multi with executor id 1853 parsl.dataflow.dflow INFO: Standard output for task 1852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/ffd0f37c-8572-4daa-9c8b-042b823cc15e_isrForCrosstalkSources_997025_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/ffd0f37c-8572-4daa-9c8b-042b823cc15e_isrForCrosstalkSources_997025_52.stderr parsl.dataflow.dflow DEBUG: Task 1853 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1853 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1853 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1853 has memoization hash c98514a897f0202bd037b803c2dda79e parsl.dataflow.memoization INFO: Task 1853 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1853 try 0 launched on executor multi with executor id 1854 parsl.dataflow.dflow INFO: Standard output for task 1853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/08dc15f0-e66d-40ae-96f0-1ed8bbb82981_isrForCrosstalkSources_997010_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/08dc15f0-e66d-40ae-96f0-1ed8bbb82981_isrForCrosstalkSources_997010_31.stderr parsl.dataflow.dflow DEBUG: Task 1854 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1854 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1854 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1854 has memoization hash 1556eed7e90a72f577684746201d15d9 parsl.dataflow.memoization INFO: Task 1854 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1854 try 0 launched on executor multi with executor id 1855 parsl.dataflow.dflow INFO: Standard output for task 1854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c6373780-921d-4b4c-956f-585d5e24546f_isrForCrosstalkSources_997016_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c6373780-921d-4b4c-956f-585d5e24546f_isrForCrosstalkSources_997016_56.stderr parsl.dataflow.dflow DEBUG: Task 1855 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1855 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1855 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1855 has memoization hash 3eb57eccbb1d134346e66f96bf713561 parsl.dataflow.memoization INFO: Task 1855 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1855 try 0 launched on executor multi with executor id 1856 parsl.dataflow.dflow INFO: Standard output for task 1855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/30379af6-8fd9-4498-82f5-89d01b2c0a3d_isrForCrosstalkSources_996991_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/30379af6-8fd9-4498-82f5-89d01b2c0a3d_isrForCrosstalkSources_996991_61.stderr parsl.dataflow.dflow DEBUG: Task 1856 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1856 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1856 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1856 has memoization hash 245c6c906361f1ec9436c05ac48957db parsl.dataflow.memoization INFO: Task 1856 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1856 try 0 launched on executor multi with executor id 1857 parsl.dataflow.dflow INFO: Standard output for task 1856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4cf0accd-97a7-4adf-92f4-eae505a45803_isrForCrosstalkSources_996959_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4cf0accd-97a7-4adf-92f4-eae505a45803_isrForCrosstalkSources_996959_23.stderr parsl.dataflow.dflow DEBUG: Task 1857 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1857 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1857 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1857 has memoization hash 7fe67cf2b6100115ee6b569fd3b1533b parsl.dataflow.memoization INFO: Task 1857 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1857 try 0 launched on executor multi with executor id 1858 parsl.dataflow.dflow INFO: Standard output for task 1857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bc0fc8af-48b5-4e6a-9892-8d62ec5dbb90_isrForCrosstalkSources_996992_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bc0fc8af-48b5-4e6a-9892-8d62ec5dbb90_isrForCrosstalkSources_996992_19.stderr parsl.dataflow.dflow DEBUG: Task 1858 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1858 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1858 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1858 has memoization hash fdb93575081dce81bbb4884926ae3b4e parsl.dataflow.memoization INFO: Task 1858 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1858 try 0 launched on executor multi with executor id 1859 parsl.dataflow.dflow INFO: Standard output for task 1858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d2a3e477-18b7-4555-9ff0-124296260439_isrForCrosstalkSources_997009_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d2a3e477-18b7-4555-9ff0-124296260439_isrForCrosstalkSources_997009_18.stderr parsl.dataflow.dflow DEBUG: Task 1859 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1859 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1859 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1859 has memoization hash e2fc7caac1d878b79dce593066151f61 parsl.dataflow.memoization INFO: Task 1859 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd393a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1859 try 0 launched on executor multi with executor id 1860 parsl.dataflow.dflow INFO: Standard output for task 1859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1f4cc7cb-4189-46d2-a555-287f0053f87e_isrForCrosstalkSources_997004_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1f4cc7cb-4189-46d2-a555-287f0053f87e_isrForCrosstalkSources_997004_53.stderr parsl.dataflow.dflow DEBUG: Task 1860 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1860 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1860 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1860 has memoization hash a1cd557b67ab0fb452217074089f0818 parsl.dataflow.memoization INFO: Task 1860 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd391c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1860 try 0 launched on executor multi with executor id 1861 parsl.dataflow.dflow INFO: Standard output for task 1860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/171893d7-1f1e-444d-9781-b79ab8450cca_isrForCrosstalkSources_996970_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/171893d7-1f1e-444d-9781-b79ab8450cca_isrForCrosstalkSources_996970_45.stderr parsl.dataflow.dflow DEBUG: Task 1861 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1861 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1861 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1861 has memoization hash 3d6cce721df07dbd082aad7a48050c03 parsl.dataflow.memoization INFO: Task 1861 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1861 try 0 launched on executor multi with executor id 1862 parsl.dataflow.dflow INFO: Standard output for task 1861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/fb22ec23-0354-456f-a0f5-e7bd243b0383_isrForCrosstalkSources_997031_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/fb22ec23-0354-456f-a0f5-e7bd243b0383_isrForCrosstalkSources_997031_46.stderr parsl.dataflow.dflow DEBUG: Task 1862 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1862 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1862 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1862 has memoization hash ac02d6f5d6229126fc5eaa213a6bb00b parsl.dataflow.memoization INFO: Task 1862 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1862 try 0 launched on executor multi with executor id 1863 parsl.dataflow.dflow INFO: Standard output for task 1862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/721f558d-83a8-4139-8ff7-998b3bc7a556_isrForCrosstalkSources_996986_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/721f558d-83a8-4139-8ff7-998b3bc7a556_isrForCrosstalkSources_996986_17.stderr parsl.dataflow.dflow DEBUG: Task 1863 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1863 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1863 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1863 has memoization hash eb069741deedf3ad5513b6a26689591a parsl.dataflow.memoization INFO: Task 1863 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1863 try 0 launched on executor multi with executor id 1864 parsl.dataflow.dflow INFO: Standard output for task 1863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a9cd22d3-11bd-4a7f-8495-a95f2f695b1a_isrForCrosstalkSources_997032_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a9cd22d3-11bd-4a7f-8495-a95f2f695b1a_isrForCrosstalkSources_997032_41.stderr parsl.dataflow.dflow DEBUG: Task 1864 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1864 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1864 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1864 has memoization hash 3d3f1d8df22dfcc394dacff407ddb0b4 parsl.dataflow.memoization INFO: Task 1864 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd396c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1864 try 0 launched on executor multi with executor id 1865 parsl.dataflow.dflow INFO: Standard output for task 1864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/79f84e48-d077-4155-abd4-044ac7ec2474_isrForCrosstalkSources_996952_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/79f84e48-d077-4155-abd4-044ac7ec2474_isrForCrosstalkSources_996952_55.stderr parsl.dataflow.dflow DEBUG: Task 1865 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1865 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1865 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1865 has memoization hash da5727b3977bab1702e7b6f0773c92f1 parsl.dataflow.memoization INFO: Task 1865 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1865 try 0 launched on executor multi with executor id 1866 parsl.dataflow.dflow INFO: Standard output for task 1865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/ecd9f573-951e-47b3-ad11-3ac83772716f_isrForCrosstalkSources_997033_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/ecd9f573-951e-47b3-ad11-3ac83772716f_isrForCrosstalkSources_997033_49.stderr parsl.dataflow.dflow DEBUG: Task 1866 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1866 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1866 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1866 has memoization hash fe3f23184172c8299d99f50f454019bf parsl.dataflow.memoization INFO: Task 1866 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd394e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1866 try 0 launched on executor multi with executor id 1867 parsl.dataflow.dflow INFO: Standard output for task 1866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/751fed7d-5280-4cdb-878c-fc2efd124851_isrForCrosstalkSources_997001_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/751fed7d-5280-4cdb-878c-fc2efd124851_isrForCrosstalkSources_997001_16.stderr parsl.dataflow.dflow DEBUG: Task 1867 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1867 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1867 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1867 has memoization hash 0e52e9819bbcf61e8eb48ae70eb62ac7 parsl.dataflow.memoization INFO: Task 1867 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1867 try 0 launched on executor multi with executor id 1868 parsl.dataflow.dflow INFO: Standard output for task 1867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ecc31ca8-81f2-4015-ad7a-3b6fbfbcae62_isrForCrosstalkSources_996962_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ecc31ca8-81f2-4015-ad7a-3b6fbfbcae62_isrForCrosstalkSources_996962_57.stderr parsl.dataflow.dflow DEBUG: Task 1868 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1868 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1868 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1868 has memoization hash 076734f5cbb85619492fbb7fa36d750a parsl.dataflow.memoization INFO: Task 1868 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd398a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1868 try 0 launched on executor multi with executor id 1869 parsl.dataflow.dflow INFO: Standard output for task 1868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/94ee4584-93e2-4799-b758-846602fbf260_isrForCrosstalkSources_996960_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/94ee4584-93e2-4799-b758-846602fbf260_isrForCrosstalkSources_996960_21.stderr parsl.dataflow.dflow DEBUG: Task 1869 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1869 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1869 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1869 has memoization hash 52b86bf7291d3f55c7370f5a9f90c690 parsl.dataflow.memoization INFO: Task 1869 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1869 try 0 launched on executor multi with executor id 1870 parsl.dataflow.dflow INFO: Standard output for task 1869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/566c529a-0d85-433c-bd11-87d717a2955a_isrForCrosstalkSources_997028_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/566c529a-0d85-433c-bd11-87d717a2955a_isrForCrosstalkSources_997028_36.stderr parsl.dataflow.dflow DEBUG: Task 1870 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1870 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1870 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1870 has memoization hash 6b991d3905a0133db50f252688662e15 parsl.dataflow.memoization INFO: Task 1870 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1870 try 0 launched on executor multi with executor id 1871 parsl.dataflow.dflow INFO: Standard output for task 1870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9d51e1dd-a95e-42f4-aefa-79befa309790_isrForCrosstalkSources_996958_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9d51e1dd-a95e-42f4-aefa-79befa309790_isrForCrosstalkSources_996958_12.stderr parsl.dataflow.dflow DEBUG: Task 1871 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1871 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1871 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1871 has memoization hash 1d0fa0fb9e896cdce14b8f0292ee9dfb parsl.dataflow.memoization INFO: Task 1871 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd399e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1871 try 0 launched on executor multi with executor id 1872 parsl.dataflow.dflow INFO: Standard output for task 1871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/2fe8a882-d1e6-429e-8b8e-0f5214bc1a61_isrForCrosstalkSources_997019_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/2fe8a882-d1e6-429e-8b8e-0f5214bc1a61_isrForCrosstalkSources_997019_58.stderr parsl.dataflow.dflow DEBUG: Task 1872 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1872 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1872 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1872 has memoization hash 3b4665150deadbf33b546d45ef2743e3 parsl.dataflow.memoization INFO: Task 1872 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1872 try 0 launched on executor multi with executor id 1873 parsl.dataflow.dflow INFO: Standard output for task 1872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4e00d835-d395-4b33-9391-5de0e77755b7_isrForCrosstalkSources_996997_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4e00d835-d395-4b33-9391-5de0e77755b7_isrForCrosstalkSources_996997_55.stderr parsl.dataflow.dflow DEBUG: Task 1873 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1873 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1873 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1873 has memoization hash 8c0762b6fef6db8778dc526f3f9bb6f3 parsl.dataflow.memoization INFO: Task 1873 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1873 try 0 launched on executor multi with executor id 1874 parsl.dataflow.dflow INFO: Standard output for task 1873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0a3558b9-1eae-4c56-a4b0-dc51718fccb3_isrForCrosstalkSources_996981_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0a3558b9-1eae-4c56-a4b0-dc51718fccb3_isrForCrosstalkSources_996981_55.stderr parsl.dataflow.dflow DEBUG: Task 1874 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1874 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1874 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1874 has memoization hash b346e646d90fd568a265223f830a8152 parsl.dataflow.memoization INFO: Task 1874 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3af20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1874 try 0 launched on executor multi with executor id 1875 parsl.dataflow.dflow INFO: Standard output for task 1874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/deb0f44e-33d0-4950-891a-f27b14a764b2_isrForCrosstalkSources_997017_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/deb0f44e-33d0-4950-891a-f27b14a764b2_isrForCrosstalkSources_997017_44.stderr parsl.dataflow.dflow DEBUG: Task 1875 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1875 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1875 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1875 has memoization hash eb01f6fe0a711046801d847f1f446699 parsl.dataflow.memoization INFO: Task 1875 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3afc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1875 try 0 launched on executor multi with executor id 1876 parsl.dataflow.dflow INFO: Standard output for task 1875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/85773acf-bed2-4166-a80c-6f3b67ffa419_isrForCrosstalkSources_996959_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/85773acf-bed2-4166-a80c-6f3b67ffa419_isrForCrosstalkSources_996959_33.stderr parsl.dataflow.dflow DEBUG: Task 1876 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1876 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1876 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1876 has memoization hash e3ca8d936d614cd45d5e4be87b482aff parsl.dataflow.memoization INFO: Task 1876 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3ade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1876 try 0 launched on executor multi with executor id 1877 parsl.dataflow.dflow INFO: Standard output for task 1876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7a040c32-a320-4d90-9f02-f768172642cd_isrForCrosstalkSources_997011_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7a040c32-a320-4d90-9f02-f768172642cd_isrForCrosstalkSources_997011_51.stderr parsl.dataflow.dflow DEBUG: Task 1877 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1877 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1877 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1877 has memoization hash 2269c74382b4534fb3a52baa88d33da1 parsl.dataflow.memoization INFO: Task 1877 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3ad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1877 try 0 launched on executor multi with executor id 1878 parsl.dataflow.dflow INFO: Standard output for task 1877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/7d8e637a-f91d-4fd7-8645-36f37fc0b09b_isrForCrosstalkSources_996959_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/7d8e637a-f91d-4fd7-8645-36f37fc0b09b_isrForCrosstalkSources_996959_58.stderr parsl.dataflow.dflow DEBUG: Task 1878 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1878 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1878 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1878 has memoization hash 412de130da7c03a66e2f9ecc5804cb01 parsl.dataflow.memoization INFO: Task 1878 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3aca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1878 try 0 launched on executor multi with executor id 1879 parsl.dataflow.dflow INFO: Standard output for task 1878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4ecaf6c7-98a9-4b96-a6d5-8ba044bad2b5_isrForCrosstalkSources_996997_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4ecaf6c7-98a9-4b96-a6d5-8ba044bad2b5_isrForCrosstalkSources_996997_26.stderr parsl.dataflow.dflow DEBUG: Task 1879 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1879 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1879 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1879 has memoization hash 0497ccb607d27b992ce2763b95a0b331 parsl.dataflow.memoization INFO: Task 1879 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3ab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1879 try 0 launched on executor multi with executor id 1880 parsl.dataflow.dflow INFO: Standard output for task 1879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/79f07025-f157-4aba-b9ea-a3004a1a598c_isrForCrosstalkSources_996957_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/79f07025-f157-4aba-b9ea-a3004a1a598c_isrForCrosstalkSources_996957_53.stderr parsl.dataflow.dflow DEBUG: Task 1880 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1880 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1880 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1880 has memoization hash 75f9778a2329cf1069887ef44bb1dcd9 parsl.dataflow.memoization INFO: Task 1880 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3ac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1880 try 0 launched on executor multi with executor id 1881 parsl.dataflow.dflow INFO: Standard output for task 1880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ca1295f8-a353-4766-ba56-e06ace7ef419_isrForCrosstalkSources_997005_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ca1295f8-a353-4766-ba56-e06ace7ef419_isrForCrosstalkSources_997005_25.stderr parsl.dataflow.dflow DEBUG: Task 1881 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1881 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1881 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1881 has memoization hash 1efb4572a697cc405fe1d3d773b4af91 parsl.dataflow.memoization INFO: Task 1881 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3aac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1881 try 0 launched on executor multi with executor id 1882 parsl.dataflow.dflow INFO: Standard output for task 1881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a5715e6f-1dc3-4dfa-8702-1fe9ca793f84_isrForCrosstalkSources_997018_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a5715e6f-1dc3-4dfa-8702-1fe9ca793f84_isrForCrosstalkSources_997018_62.stderr parsl.dataflow.dflow DEBUG: Task 1882 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1882 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1882 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1882 has memoization hash 7076040cdf96b1d9ee0898d2930e77c2 parsl.dataflow.memoization INFO: Task 1882 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3aa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1882 try 0 launched on executor multi with executor id 1883 parsl.dataflow.dflow INFO: Standard output for task 1882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/386fbc25-d6fc-4bd2-b884-170b6ca850ec_isrForCrosstalkSources_996983_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/386fbc25-d6fc-4bd2-b884-170b6ca850ec_isrForCrosstalkSources_996983_46.stderr parsl.dataflow.dflow DEBUG: Task 1883 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1883 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1883 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1883 has memoization hash 3ba393a7dc33328364d1a11b9ac11459 parsl.dataflow.memoization INFO: Task 1883 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1883 try 0 launched on executor multi with executor id 1884 parsl.dataflow.dflow INFO: Standard output for task 1883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/14fe085c-47de-4956-a0ac-1261f8d464dc_isrForCrosstalkSources_996984_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/14fe085c-47de-4956-a0ac-1261f8d464dc_isrForCrosstalkSources_996984_29.stderr parsl.dataflow.dflow DEBUG: Task 1884 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1884 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1884 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1884 has memoization hash 32f88bd7334b024d91a6e3e0bf1963a8 parsl.dataflow.memoization INFO: Task 1884 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1884 try 0 launched on executor multi with executor id 1885 parsl.dataflow.dflow INFO: Standard output for task 1884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/d8475ca3-2302-40e4-bf0b-865b5938a842_isrForCrosstalkSources_996978_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/d8475ca3-2302-40e4-bf0b-865b5938a842_isrForCrosstalkSources_996978_50.stderr parsl.dataflow.dflow DEBUG: Task 1885 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1885 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1885 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1885 has memoization hash f0770ac7f0ac85402760ca00928f12ff parsl.dataflow.memoization INFO: Task 1885 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1885 try 0 launched on executor multi with executor id 1886 parsl.dataflow.dflow INFO: Standard output for task 1885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/95959b3d-002a-450e-90a3-77cef5228713_isrForCrosstalkSources_997015_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/95959b3d-002a-450e-90a3-77cef5228713_isrForCrosstalkSources_997015_55.stderr parsl.dataflow.dflow DEBUG: Task 1886 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1886 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1886 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1886 has memoization hash abaccdb05bc9b705ce74d5bea1088c2e parsl.dataflow.memoization INFO: Task 1886 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1886 try 0 launched on executor multi with executor id 1887 parsl.dataflow.dflow INFO: Standard output for task 1886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a29ca2f4-2c7b-4136-b637-b2a7b89ca5ce_isrForCrosstalkSources_996991_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a29ca2f4-2c7b-4136-b637-b2a7b89ca5ce_isrForCrosstalkSources_996991_14.stderr parsl.dataflow.dflow DEBUG: Task 1887 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1887 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1887 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1887 has memoization hash 2c8d8725a959e478214a739a77c90ad6 parsl.dataflow.memoization INFO: Task 1887 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1887 try 0 launched on executor multi with executor id 1888 parsl.dataflow.dflow INFO: Standard output for task 1887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/296c4755-5f0d-4a35-9628-e5dc63ed5dfb_isrForCrosstalkSources_996987_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/296c4755-5f0d-4a35-9628-e5dc63ed5dfb_isrForCrosstalkSources_996987_41.stderr parsl.dataflow.dflow DEBUG: Task 1888 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1888 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1888 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1888 has memoization hash 4899e1078081c6af603d9f92503a389c parsl.dataflow.memoization INFO: Task 1888 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1888 try 0 launched on executor multi with executor id 1889 parsl.dataflow.dflow INFO: Standard output for task 1888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5ed3c35d-88a2-4dad-b84a-6706159fc22f_isrForCrosstalkSources_997033_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5ed3c35d-88a2-4dad-b84a-6706159fc22f_isrForCrosstalkSources_997033_21.stderr parsl.dataflow.dflow DEBUG: Task 1889 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1889 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1889 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1889 has memoization hash ef443df86f77613611ebb4563070c054 parsl.dataflow.memoization INFO: Task 1889 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1889 try 0 launched on executor multi with executor id 1890 parsl.dataflow.dflow INFO: Standard output for task 1889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/392dfb93-d7a7-414b-9c78-a9e44948254c_isrForCrosstalkSources_996956_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/392dfb93-d7a7-414b-9c78-a9e44948254c_isrForCrosstalkSources_996956_61.stderr parsl.dataflow.dflow DEBUG: Task 1890 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1890 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1890 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1890 has memoization hash b14cd942b10e016767a1e07bc81bb6fb parsl.dataflow.memoization INFO: Task 1890 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1890 try 0 launched on executor multi with executor id 1891 parsl.dataflow.dflow INFO: Standard output for task 1890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/758810b4-1616-49ea-ba79-c4b32a4f6047_isrForCrosstalkSources_997027_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/758810b4-1616-49ea-ba79-c4b32a4f6047_isrForCrosstalkSources_997027_23.stderr parsl.dataflow.dflow DEBUG: Task 1891 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1891 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1891 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1891 has memoization hash 25bc6dcd448624e6e0a099e52825959b parsl.dataflow.memoization INFO: Task 1891 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1891 try 0 launched on executor multi with executor id 1892 parsl.dataflow.dflow INFO: Standard output for task 1891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e1f128fa-f4c5-456f-b8ab-209c199c8405_isrForCrosstalkSources_997005_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e1f128fa-f4c5-456f-b8ab-209c199c8405_isrForCrosstalkSources_997005_43.stderr parsl.dataflow.dflow DEBUG: Task 1892 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1892 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1892 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1892 has memoization hash afdcd56c1a69b7645b780a52d71284e2 parsl.dataflow.memoization INFO: Task 1892 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1892 try 0 launched on executor multi with executor id 1893 parsl.dataflow.dflow INFO: Standard output for task 1892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/7b8109a3-8aab-400c-843a-78e27796af39_isrForCrosstalkSources_996995_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/7b8109a3-8aab-400c-843a-78e27796af39_isrForCrosstalkSources_996995_9.stderr parsl.dataflow.dflow DEBUG: Task 1893 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1893 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1893 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1893 has memoization hash 23a86297c40662935f85cd424371cf29 parsl.dataflow.memoization INFO: Task 1893 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1893 try 0 launched on executor multi with executor id 1894 parsl.dataflow.dflow INFO: Standard output for task 1893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/bab0e260-dba4-4abb-a58f-ada1c00859b8_isrForCrosstalkSources_997016_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/bab0e260-dba4-4abb-a58f-ada1c00859b8_isrForCrosstalkSources_997016_11.stderr parsl.dataflow.dflow DEBUG: Task 1894 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1894 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1894 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1894 has memoization hash 8893318748153d266135ef93b3d1dc2a parsl.dataflow.memoization INFO: Task 1894 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1894 try 0 launched on executor multi with executor id 1895 parsl.dataflow.dflow INFO: Standard output for task 1894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/54bf96cf-750e-4721-95cc-840aeb33f6aa_isrForCrosstalkSources_996989_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/54bf96cf-750e-4721-95cc-840aeb33f6aa_isrForCrosstalkSources_996989_28.stderr parsl.dataflow.dflow DEBUG: Task 1895 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1895 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1895 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1895 has memoization hash cedbd2191bae9a85eb009b249857084a parsl.dataflow.memoization INFO: Task 1895 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1895 try 0 launched on executor multi with executor id 1896 parsl.dataflow.dflow INFO: Standard output for task 1895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/990a8800-4d91-43e1-b6d3-db942f2190a2_isrForCrosstalkSources_996978_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/990a8800-4d91-43e1-b6d3-db942f2190a2_isrForCrosstalkSources_996978_37.stderr parsl.dataflow.dflow DEBUG: Task 1896 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1896 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1896 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1896 has memoization hash e339f6d74aa5e13778aa46f1b614f2e8 parsl.dataflow.memoization INFO: Task 1896 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1896 try 0 launched on executor multi with executor id 1897 parsl.dataflow.dflow INFO: Standard output for task 1896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9827589b-66c9-43ff-8cd8-b289be770fbe_isrForCrosstalkSources_996997_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9827589b-66c9-43ff-8cd8-b289be770fbe_isrForCrosstalkSources_996997_35.stderr parsl.dataflow.dflow DEBUG: Task 1897 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1897 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1897 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1897 has memoization hash cec9f6993d85a15420b29ee5e3c42137 parsl.dataflow.memoization INFO: Task 1897 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1897 try 0 launched on executor multi with executor id 1898 parsl.dataflow.dflow INFO: Standard output for task 1897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/6e61471f-a0c7-445c-949e-9910f3358cef_isrForCrosstalkSources_997008_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/6e61471f-a0c7-445c-949e-9910f3358cef_isrForCrosstalkSources_997008_52.stderr parsl.dataflow.dflow DEBUG: Task 1898 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1898 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1898 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1898 has memoization hash 7810beefabc5966c36e2a22a2f6e3ff7 parsl.dataflow.memoization INFO: Task 1898 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3a020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1898 try 0 launched on executor multi with executor id 1899 parsl.dataflow.dflow INFO: Standard output for task 1898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/38c5f7b0-f513-4e81-ac6a-a1fc2478bbe4_isrForCrosstalkSources_996964_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/38c5f7b0-f513-4e81-ac6a-a1fc2478bbe4_isrForCrosstalkSources_996964_17.stderr parsl.dataflow.dflow DEBUG: Task 1899 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1899 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1899 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1899 has memoization hash a2f5ee112ff4a3105e3a9b2ef9affaee parsl.dataflow.memoization INFO: Task 1899 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1899 try 0 launched on executor multi with executor id 1900 parsl.dataflow.dflow INFO: Standard output for task 1899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/26769602-fd21-49b8-890d-1251602be253_isrForCrosstalkSources_997016_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/26769602-fd21-49b8-890d-1251602be253_isrForCrosstalkSources_997016_27.stderr parsl.dataflow.dflow DEBUG: Task 1900 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1900 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1900 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1900 has memoization hash 87e898ba6a481f52ef94c47a803137f0 parsl.dataflow.memoization INFO: Task 1900 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1900 try 0 launched on executor multi with executor id 1901 parsl.dataflow.dflow INFO: Standard output for task 1900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/09344bdc-b576-4f52-8b28-f3a2d8b7b58c_isrForCrosstalkSources_996974_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/09344bdc-b576-4f52-8b28-f3a2d8b7b58c_isrForCrosstalkSources_996974_35.stderr parsl.dataflow.dflow DEBUG: Task 1901 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1901 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1901 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1901 has memoization hash 1bbd76f93d3c8f02a8dc0b951f4032a0 parsl.dataflow.memoization INFO: Task 1901 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1901 try 0 launched on executor multi with executor id 1902 parsl.dataflow.dflow INFO: Standard output for task 1901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c90e1972-4325-4a9e-b4a6-dadde2fddb21_isrForCrosstalkSources_996961_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c90e1972-4325-4a9e-b4a6-dadde2fddb21_isrForCrosstalkSources_996961_15.stderr parsl.dataflow.dflow DEBUG: Task 1902 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1902 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1902 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1902 has memoization hash 71bd0eed1556d63671d395a9ef22ccb7 parsl.dataflow.memoization INFO: Task 1902 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1902 try 0 launched on executor multi with executor id 1903 parsl.dataflow.dflow INFO: Standard output for task 1902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/368aeeb0-270a-4129-83fd-62a797ad26f2_isrForCrosstalkSources_997027_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/368aeeb0-270a-4129-83fd-62a797ad26f2_isrForCrosstalkSources_997027_40.stderr parsl.dataflow.dflow DEBUG: Task 1903 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1903 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1903 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1903 has memoization hash f9b654c1863ff30fe5beea635500845a parsl.dataflow.memoization INFO: Task 1903 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1903 try 0 launched on executor multi with executor id 1904 parsl.dataflow.dflow INFO: Standard output for task 1903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d91ddd3d-0fee-4fa0-8d9f-4c23384b331f_isrForCrosstalkSources_996992_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d91ddd3d-0fee-4fa0-8d9f-4c23384b331f_isrForCrosstalkSources_996992_47.stderr parsl.dataflow.dflow DEBUG: Task 1904 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1904 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1904 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1904 has memoization hash 678dd7898fc8e87cdf1e1571c4cd264d parsl.dataflow.memoization INFO: Task 1904 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1904 try 0 launched on executor multi with executor id 1905 parsl.dataflow.dflow INFO: Standard output for task 1904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/171564fe-81c9-41a8-8144-a4c20e692319_isrForCrosstalkSources_996971_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/171564fe-81c9-41a8-8144-a4c20e692319_isrForCrosstalkSources_996971_41.stderr parsl.dataflow.dflow DEBUG: Task 1905 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1905 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1905 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1905 has memoization hash 7e020020ce4429b2eb057401cef833d1 parsl.dataflow.memoization INFO: Task 1905 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd39c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1905 try 0 launched on executor multi with executor id 1906 parsl.dataflow.dflow INFO: Standard output for task 1905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/53916fb4-1abb-4bd8-93a9-699da9f9518f_isrForCrosstalkSources_997032_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/53916fb4-1abb-4bd8-93a9-699da9f9518f_isrForCrosstalkSources_997032_25.stderr parsl.dataflow.dflow DEBUG: Task 1906 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1906 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1906 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1906 has memoization hash b2a297183c1e81f26cce8a0613b261c2 parsl.dataflow.memoization INFO: Task 1906 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1906 try 0 launched on executor multi with executor id 1907 parsl.dataflow.dflow INFO: Standard output for task 1906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e52251dc-e63e-4222-acf1-dc81a4749f1e_isrForCrosstalkSources_997023_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e52251dc-e63e-4222-acf1-dc81a4749f1e_isrForCrosstalkSources_997023_13.stderr parsl.dataflow.dflow DEBUG: Task 1907 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1907 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1907 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1907 has memoization hash e742092c76f3deaf7b5fbc30b3c8c113 parsl.dataflow.memoization INFO: Task 1907 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1907 try 0 launched on executor multi with executor id 1908 parsl.dataflow.dflow INFO: Standard output for task 1907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4cc0b197-05bc-43b4-9a82-2b434f8412ba_isrForCrosstalkSources_997018_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4cc0b197-05bc-43b4-9a82-2b434f8412ba_isrForCrosstalkSources_997018_57.stderr parsl.dataflow.dflow DEBUG: Task 1908 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1908 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1908 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1908 has memoization hash b0a8be92334ba2bb37ded9eda4e1aa14 parsl.dataflow.memoization INFO: Task 1908 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1908 try 0 launched on executor multi with executor id 1909 parsl.dataflow.dflow INFO: Standard output for task 1908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f18b5bec-22f7-4c30-9ef3-fecd32474ebd_isrForCrosstalkSources_996975_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f18b5bec-22f7-4c30-9ef3-fecd32474ebd_isrForCrosstalkSources_996975_62.stderr parsl.dataflow.dflow DEBUG: Task 1909 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1909 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1909 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1909 has memoization hash 6f4d82376edcf17bcf44e36ae2ef6245 parsl.dataflow.memoization INFO: Task 1909 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1909 try 0 launched on executor multi with executor id 1910 parsl.dataflow.dflow INFO: Standard output for task 1909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/af5dfca7-6735-4125-99c9-d6852d122741_isrForCrosstalkSources_997020_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/af5dfca7-6735-4125-99c9-d6852d122741_isrForCrosstalkSources_997020_10.stderr parsl.dataflow.dflow DEBUG: Task 1910 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1910 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1910 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1910 has memoization hash f1d1bb4af524d89585264082d27427da parsl.dataflow.memoization INFO: Task 1910 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1910 try 0 launched on executor multi with executor id 1911 parsl.dataflow.dflow INFO: Standard output for task 1910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/6ba26f90-ad6b-4706-92b5-2710854af85c_isrForCrosstalkSources_997004_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/6ba26f90-ad6b-4706-92b5-2710854af85c_isrForCrosstalkSources_997004_8.stderr parsl.dataflow.dflow DEBUG: Task 1911 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1911 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1911 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1911 has memoization hash ce07bacabbf84ac11c3abaccfb80661f parsl.dataflow.memoization INFO: Task 1911 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1911 try 0 launched on executor multi with executor id 1912 parsl.dataflow.dflow INFO: Standard output for task 1911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/09da4a60-f491-46a1-a53a-a0fee2e77baa_isrForCrosstalkSources_997013_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/09da4a60-f491-46a1-a53a-a0fee2e77baa_isrForCrosstalkSources_997013_6.stderr parsl.dataflow.dflow DEBUG: Task 1912 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1912 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1912 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1912 has memoization hash 5f290412e002bdda28b6269bbf0d2449 parsl.dataflow.memoization INFO: Task 1912 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1912 try 0 launched on executor multi with executor id 1913 parsl.dataflow.dflow INFO: Standard output for task 1912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/826af4f3-7443-41fa-893c-bb048dc120c5_isrForCrosstalkSources_996993_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/826af4f3-7443-41fa-893c-bb048dc120c5_isrForCrosstalkSources_996993_57.stderr parsl.dataflow.dflow DEBUG: Task 1913 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1913 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1913 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1913 has memoization hash 27b9f1ebbbc83b8662be2e6d50876c00 parsl.dataflow.memoization INFO: Task 1913 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1913 try 0 launched on executor multi with executor id 1914 parsl.dataflow.dflow INFO: Standard output for task 1913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/16525e9d-6ec8-4fa7-bf32-9afbe246b379_isrForCrosstalkSources_996975_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/16525e9d-6ec8-4fa7-bf32-9afbe246b379_isrForCrosstalkSources_996975_58.stderr parsl.dataflow.dflow DEBUG: Task 1914 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1914 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1914 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1914 has memoization hash ab2067bcd30738447856379e57314c9d parsl.dataflow.memoization INFO: Task 1914 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1914 try 0 launched on executor multi with executor id 1915 parsl.dataflow.dflow INFO: Standard output for task 1914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/6a8f20bb-8189-4644-b306-89772e23fe85_isrForCrosstalkSources_996993_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/6a8f20bb-8189-4644-b306-89772e23fe85_isrForCrosstalkSources_996993_48.stderr parsl.dataflow.dflow DEBUG: Task 1915 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1915 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1915 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1915 has memoization hash 82b1d5886f4ec359b174e0e47c12102c parsl.dataflow.memoization INFO: Task 1915 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1915 try 0 launched on executor multi with executor id 1916 parsl.dataflow.dflow INFO: Standard output for task 1915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3e72f7e8-2527-45b9-bd2e-9256568982ed_isrForCrosstalkSources_996968_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3e72f7e8-2527-45b9-bd2e-9256568982ed_isrForCrosstalkSources_996968_21.stderr parsl.dataflow.dflow DEBUG: Task 1916 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1916 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1916 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1916 has memoization hash 4f0dd3b14b07a0f446949af3c44319e0 parsl.dataflow.memoization INFO: Task 1916 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1916 try 0 launched on executor multi with executor id 1917 parsl.dataflow.dflow INFO: Standard output for task 1916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ad1504e0-b499-4584-af4e-3d6b0be0047d_isrForCrosstalkSources_996975_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ad1504e0-b499-4584-af4e-3d6b0be0047d_isrForCrosstalkSources_996975_44.stderr parsl.dataflow.dflow DEBUG: Task 1917 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1917 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1917 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1917 has memoization hash a6a5a84c5a02b08828d149b144b035e6 parsl.dataflow.memoization INFO: Task 1917 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1917 try 0 launched on executor multi with executor id 1918 parsl.dataflow.dflow INFO: Standard output for task 1917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f33b7348-7908-404f-ad59-6aee4c382e18_isrForCrosstalkSources_996978_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f33b7348-7908-404f-ad59-6aee4c382e18_isrForCrosstalkSources_996978_51.stderr parsl.dataflow.dflow DEBUG: Task 1918 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1918 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1918 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1918 has memoization hash 6faf264598dcd1bababdb53099f6cb93 parsl.dataflow.memoization INFO: Task 1918 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1918 try 0 launched on executor multi with executor id 1919 parsl.dataflow.dflow INFO: Standard output for task 1918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/24417ec7-de99-497f-8c7d-af09cd40e4d2_isrForCrosstalkSources_996987_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/24417ec7-de99-497f-8c7d-af09cd40e4d2_isrForCrosstalkSources_996987_58.stderr parsl.dataflow.dflow DEBUG: Task 1919 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1919 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1919 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1919 has memoization hash ce9b68522175b6993b9c4649cd14cfa0 parsl.dataflow.memoization INFO: Task 1919 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1919 try 0 launched on executor multi with executor id 1920 parsl.dataflow.dflow INFO: Standard output for task 1919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9189a285-2adf-4650-ac23-371b93a67e31_isrForCrosstalkSources_996958_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9189a285-2adf-4650-ac23-371b93a67e31_isrForCrosstalkSources_996958_59.stderr parsl.dataflow.dflow DEBUG: Task 1920 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1920 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1920 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1920 has memoization hash c045ef6fbfd2bfc6380f388447750d1f parsl.dataflow.memoization INFO: Task 1920 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1920 try 0 launched on executor multi with executor id 1921 parsl.dataflow.dflow INFO: Standard output for task 1920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a3f2812a-6433-4927-ad0c-5918373e6774_isrForCrosstalkSources_997032_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a3f2812a-6433-4927-ad0c-5918373e6774_isrForCrosstalkSources_997032_11.stderr parsl.dataflow.dflow DEBUG: Task 1921 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1921 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1921 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1921 has memoization hash 83f3dad69f46b9e88229534ce10f9524 parsl.dataflow.memoization INFO: Task 1921 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3b9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1921 try 0 launched on executor multi with executor id 1922 parsl.dataflow.dflow INFO: Standard output for task 1921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/790f4585-2811-4947-9d50-2c9e0c176682_isrForCrosstalkSources_996962_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/790f4585-2811-4947-9d50-2c9e0c176682_isrForCrosstalkSources_996962_13.stderr parsl.dataflow.dflow DEBUG: Task 1922 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1922 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1922 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1922 has memoization hash 307edeaf1936a4983b9fa840e219eb3b parsl.dataflow.memoization INFO: Task 1922 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3ba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1922 try 0 launched on executor multi with executor id 1923 parsl.dataflow.dflow INFO: Standard output for task 1922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/a177a73c-111a-4229-9d63-1d3be60d1f6a_isrForCrosstalkSources_997009_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/a177a73c-111a-4229-9d63-1d3be60d1f6a_isrForCrosstalkSources_997009_1.stderr parsl.dataflow.dflow DEBUG: Task 1923 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1923 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1923 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1923 has memoization hash b35bc40b4b965bb0b50f6285a779bd50 parsl.dataflow.memoization INFO: Task 1923 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3bb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1923 try 0 launched on executor multi with executor id 1924 parsl.dataflow.dflow INFO: Standard output for task 1923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/503e2989-b8af-432b-aab7-abab7c37176f_isrForCrosstalkSources_996994_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/503e2989-b8af-432b-aab7-abab7c37176f_isrForCrosstalkSources_996994_11.stderr parsl.dataflow.dflow DEBUG: Task 1924 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1924 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1924 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1924 has memoization hash 8f37e6c25dfd37fb1ef6313b5da94c59 parsl.dataflow.memoization INFO: Task 1924 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3bc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1924 try 0 launched on executor multi with executor id 1925 parsl.dataflow.dflow INFO: Standard output for task 1924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cad7590b-277b-4491-98a2-e2468fc58c28_isrForCrosstalkSources_996988_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cad7590b-277b-4491-98a2-e2468fc58c28_isrForCrosstalkSources_996988_50.stderr parsl.dataflow.dflow DEBUG: Task 1925 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1925 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1925 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1925 has memoization hash 6e6104f85df31788440990b9a9f8acc7 parsl.dataflow.memoization INFO: Task 1925 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3bba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1925 try 0 launched on executor multi with executor id 1926 parsl.dataflow.dflow INFO: Standard output for task 1925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/146cbfc7-c37a-4642-88fe-8988ece87ed4_isrForCrosstalkSources_997018_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/146cbfc7-c37a-4642-88fe-8988ece87ed4_isrForCrosstalkSources_997018_35.stderr parsl.dataflow.dflow DEBUG: Task 1926 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1926 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1926 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1926 has memoization hash bb32600500668adb12a2ed4486a57edf parsl.dataflow.memoization INFO: Task 1926 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3bce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1926 try 0 launched on executor multi with executor id 1927 parsl.dataflow.dflow INFO: Standard output for task 1926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/cf111af4-7b60-4ede-b3f9-dc07a221b9c6_isrForCrosstalkSources_997028_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/cf111af4-7b60-4ede-b3f9-dc07a221b9c6_isrForCrosstalkSources_997028_59.stderr parsl.dataflow.dflow DEBUG: Task 1927 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1927 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1927 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1927 has memoization hash 412b807e6174df0840d54f451cd0847d parsl.dataflow.memoization INFO: Task 1927 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3bd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1927 try 0 launched on executor multi with executor id 1928 parsl.dataflow.dflow INFO: Standard output for task 1927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0ca4b4a2-4c3e-410f-860c-6d7b4b2da874_isrForCrosstalkSources_996997_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0ca4b4a2-4c3e-410f-860c-6d7b4b2da874_isrForCrosstalkSources_996997_8.stderr parsl.dataflow.dflow DEBUG: Task 1928 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1928 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1928 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1928 has memoization hash 4fece7fa9233e69a6fad1ff15334714e parsl.dataflow.memoization INFO: Task 1928 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3be20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1928 try 0 launched on executor multi with executor id 1929 parsl.dataflow.dflow INFO: Standard output for task 1928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/566ebdd4-ce5a-4528-9866-287f2aa1079e_isrForCrosstalkSources_996973_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/566ebdd4-ce5a-4528-9866-287f2aa1079e_isrForCrosstalkSources_996973_12.stderr parsl.dataflow.dflow DEBUG: Task 1929 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1929 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1929 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1929 has memoization hash 7a96ed8f3fef590472d91d67317b776f parsl.dataflow.memoization INFO: Task 1929 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3bf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1929 try 0 launched on executor multi with executor id 1930 parsl.dataflow.dflow INFO: Standard output for task 1929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/bb1996e1-01ce-42ae-956b-10d0cf0f9b66_isrForCrosstalkSources_996993_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/bb1996e1-01ce-42ae-956b-10d0cf0f9b66_isrForCrosstalkSources_996993_28.stderr parsl.dataflow.dflow DEBUG: Task 1930 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1930 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1930 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1930 has memoization hash 3c94e6a33469a06f0ee781a4f84cb437 parsl.dataflow.memoization INFO: Task 1930 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd3bec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1930 try 0 launched on executor multi with executor id 1931 parsl.dataflow.dflow INFO: Standard output for task 1930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/2215079b-7c23-4a7d-b83c-d365139b5d1b_isrForCrosstalkSources_996979_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/2215079b-7c23-4a7d-b83c-d365139b5d1b_isrForCrosstalkSources_996979_5.stderr parsl.dataflow.dflow DEBUG: Task 1931 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1931 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1931 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1931 has memoization hash d28ff653411408ce36791917b892d5d5 parsl.dataflow.memoization INFO: Task 1931 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cd38040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1931 try 0 launched on executor multi with executor id 1932 parsl.dataflow.dflow INFO: Standard output for task 1931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6cfd5861-c76d-460c-946e-9fc6a3d10094_isrForCrosstalkSources_996974_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6cfd5861-c76d-460c-946e-9fc6a3d10094_isrForCrosstalkSources_996974_32.stderr parsl.dataflow.dflow DEBUG: Task 1932 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1932 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1932 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1932 has memoization hash c1c9c3295c461aca36fed72b2498ec4e parsl.dataflow.memoization INFO: Task 1932 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba00e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1932 try 0 launched on executor multi with executor id 1933 parsl.dataflow.dflow INFO: Standard output for task 1932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/72731792-bebb-4402-a625-44476d6690e7_isrForCrosstalkSources_996997_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/72731792-bebb-4402-a625-44476d6690e7_isrForCrosstalkSources_996997_10.stderr parsl.dataflow.dflow DEBUG: Task 1933 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1933 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1933 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1933 has memoization hash 86a0f8944cb9f58ebf8ed70e852eae1f parsl.dataflow.memoization INFO: Task 1933 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1933 try 0 launched on executor multi with executor id 1934 parsl.dataflow.dflow INFO: Standard output for task 1933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/7a02722f-aa62-45bd-b3e5-ad1e1748030e_isrForCrosstalkSources_996966_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/7a02722f-aa62-45bd-b3e5-ad1e1748030e_isrForCrosstalkSources_996966_41.stderr parsl.dataflow.dflow DEBUG: Task 1934 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1934 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1934 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1934 has memoization hash a7766d4bbdd31cd49ba135eedfdaae0e parsl.dataflow.memoization INFO: Task 1934 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1934 try 0 launched on executor multi with executor id 1935 parsl.dataflow.dflow INFO: Standard output for task 1934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/0e6e877d-221d-4f03-9465-337c186cb900_isrForCrosstalkSources_996992_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/0e6e877d-221d-4f03-9465-337c186cb900_isrForCrosstalkSources_996992_46.stderr parsl.dataflow.dflow DEBUG: Task 1935 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1935 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1935 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1935 has memoization hash 0ecdc9375d15e680fc76b770788f2226 parsl.dataflow.memoization INFO: Task 1935 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1935 try 0 launched on executor multi with executor id 1936 parsl.dataflow.dflow INFO: Standard output for task 1935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/96583a66-fe7b-44fe-8ad4-ecf578c5890f_isrForCrosstalkSources_996976_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/96583a66-fe7b-44fe-8ad4-ecf578c5890f_isrForCrosstalkSources_996976_50.stderr parsl.dataflow.dflow DEBUG: Task 1936 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1936 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1936 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1936 has memoization hash 4b2657bf9244d6051f51d1cf62aa82f2 parsl.dataflow.memoization INFO: Task 1936 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1936 try 0 launched on executor multi with executor id 1937 parsl.dataflow.dflow INFO: Standard output for task 1936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/18d25c7d-0e89-44af-8c1a-c00468e78ba6_isrForCrosstalkSources_996963_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/18d25c7d-0e89-44af-8c1a-c00468e78ba6_isrForCrosstalkSources_996963_9.stderr parsl.dataflow.dflow DEBUG: Task 1937 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1937 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1937 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1937 has memoization hash f2864ca242bf8a75dbb57d105fe73e94 parsl.dataflow.memoization INFO: Task 1937 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1937 try 0 launched on executor multi with executor id 1938 parsl.dataflow.dflow INFO: Standard output for task 1937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/099a5e55-d343-4f4d-9094-1761d28a36c6_isrForCrosstalkSources_996979_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/099a5e55-d343-4f4d-9094-1761d28a36c6_isrForCrosstalkSources_996979_35.stderr parsl.dataflow.dflow DEBUG: Task 1938 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1938 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1938 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1938 has memoization hash d7f90912f207f623341879ee75ea3939 parsl.dataflow.memoization INFO: Task 1938 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba04a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1938 try 0 launched on executor multi with executor id 1939 parsl.dataflow.dflow INFO: Standard output for task 1938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/3bfd3b52-ed10-46bf-812d-df43927e7b2d_isrForCrosstalkSources_996956_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/3bfd3b52-ed10-46bf-812d-df43927e7b2d_isrForCrosstalkSources_996956_22.stderr parsl.dataflow.dflow DEBUG: Task 1939 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1939 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1939 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1939 has memoization hash 6b26b65170baf7a37cb1aa4c5ca1038c parsl.dataflow.memoization INFO: Task 1939 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba05e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1939 try 0 launched on executor multi with executor id 1940 parsl.dataflow.dflow INFO: Standard output for task 1939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/57cdf6cc-998c-462e-8b79-f60399feffaf_isrForCrosstalkSources_997021_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/57cdf6cc-998c-462e-8b79-f60399feffaf_isrForCrosstalkSources_997021_53.stderr parsl.dataflow.dflow DEBUG: Task 1940 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1940 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1940 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1940 has memoization hash bc5b52f909fb16aa4eb8239ca05c11e3 parsl.dataflow.memoization INFO: Task 1940 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1940 try 0 launched on executor multi with executor id 1941 parsl.dataflow.dflow INFO: Standard output for task 1940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a9a0a067-3249-4105-904d-97ce975ea8ec_isrForCrosstalkSources_996995_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a9a0a067-3249-4105-904d-97ce975ea8ec_isrForCrosstalkSources_996995_40.stderr parsl.dataflow.dflow DEBUG: Task 1941 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1941 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1941 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1941 has memoization hash bc95c8d2b2d14bd6ce8f2407d271330f parsl.dataflow.memoization INFO: Task 1941 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1941 try 0 launched on executor multi with executor id 1942 parsl.dataflow.dflow INFO: Standard output for task 1941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e5998e2e-45e9-4868-b84d-f3de17996dc8_isrForCrosstalkSources_996954_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e5998e2e-45e9-4868-b84d-f3de17996dc8_isrForCrosstalkSources_996954_6.stderr parsl.dataflow.dflow DEBUG: Task 1942 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1942 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1942 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1942 has memoization hash 90243d312824d085324613974348893f parsl.dataflow.memoization INFO: Task 1942 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1942 try 0 launched on executor multi with executor id 1943 parsl.dataflow.dflow INFO: Standard output for task 1942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8f6118b7-3635-4387-a9be-051a69f294f9_isrForCrosstalkSources_997017_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8f6118b7-3635-4387-a9be-051a69f294f9_isrForCrosstalkSources_997017_10.stderr parsl.dataflow.dflow DEBUG: Task 1943 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1943 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1943 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1943 has memoization hash 5822d8621f599d3ab4f62a154f11945b parsl.dataflow.memoization INFO: Task 1943 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba07c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1943 try 0 launched on executor multi with executor id 1944 parsl.dataflow.dflow INFO: Standard output for task 1943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9b3e5fec-1228-492f-a120-fdb946a467bc_isrForCrosstalkSources_996968_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9b3e5fec-1228-492f-a120-fdb946a467bc_isrForCrosstalkSources_996968_26.stderr parsl.dataflow.dflow DEBUG: Task 1944 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1944 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1944 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1944 has memoization hash aa7e1daa9f01b565a15e36594504bb5f parsl.dataflow.memoization INFO: Task 1944 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:45 proc_lsst.multi:170 [INFO] [multi] job_id 17074869 proc_lsst.multi INFO: [multi] job_id 17074869 2024-03-15 13:49:45 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 23 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 23 2024-03-15 13:49:45 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:45 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074869 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074869 parsl.executors.status_handling DEBUG: Launched block 23 on executor multi with job ID 17074869 parsl.executors.status_handling INFO: Allocated block ID 24 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-24 2024-03-15 13:49:45 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=24 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-24 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=24 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-24 2024-03-15 13:49:45 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=24 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=24 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:45 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=24 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=24 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 23 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 23 2024-03-15 13:49:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:45 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=24 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=24 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 1944 try 0 launched on executor multi with executor id 1945 parsl.dataflow.dflow INFO: Standard output for task 1944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6cfbe654-cc11-4e5e-a904-afdd392ebe52_isrForCrosstalkSources_997015_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6cfbe654-cc11-4e5e-a904-afdd392ebe52_isrForCrosstalkSources_997015_50.stderr parsl.dataflow.dflow DEBUG: Task 1945 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1945 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1945 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1945 has memoization hash c45be3b6ef304a5faf8cbb6adea0fe41 parsl.dataflow.memoization INFO: Task 1945 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd39c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1945 try 0 launched on executor multi with executor id 1946 parsl.dataflow.dflow INFO: Standard output for task 1945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/3bd540ce-ff6d-49b7-ad02-14958449e397_isrForCrosstalkSources_997024_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/3bd540ce-ff6d-49b7-ad02-14958449e397_isrForCrosstalkSources_997024_6.stderr parsl.dataflow.dflow DEBUG: Task 1946 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1946 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1946 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1946 has memoization hash 2cca58b3854115a5a58ad8c143b47876 parsl.dataflow.memoization INFO: Task 1946 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd2160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1946 try 0 launched on executor multi with executor id 1947 parsl.dataflow.dflow INFO: Standard output for task 1946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/75e088b5-1c0d-4dae-8b83-2a8d623cc3d4_isrForCrosstalkSources_997024_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/75e088b5-1c0d-4dae-8b83-2a8d623cc3d4_isrForCrosstalkSources_997024_13.stderr parsl.dataflow.dflow DEBUG: Task 1947 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1947 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1947 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1947 has memoization hash 9f0cd342f2e25429575235693fd2fbee parsl.dataflow.memoization INFO: Task 1947 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1947 try 0 launched on executor multi with executor id 1948 parsl.dataflow.dflow INFO: Standard output for task 1947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ddb04963-3e6b-46c9-a9fe-fa3ac925fae2_isrForCrosstalkSources_996976_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ddb04963-3e6b-46c9-a9fe-fa3ac925fae2_isrForCrosstalkSources_996976_11.stderr parsl.dataflow.dflow DEBUG: Task 1948 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1948 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1948 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1948 has memoization hash c7cd25e6722483550588da0ff14dde1a parsl.dataflow.memoization INFO: Task 1948 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba09a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1948 try 0 launched on executor multi with executor id 1949 parsl.dataflow.dflow INFO: Standard output for task 1948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7a182a1c-e1d6-48e4-82de-affc1d80ea2a_isrForCrosstalkSources_996953_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7a182a1c-e1d6-48e4-82de-affc1d80ea2a_isrForCrosstalkSources_996953_40.stderr parsl.dataflow.dflow DEBUG: Task 1949 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1949 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1949 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1949 has memoization hash 58d8b9d225485e1273defab5562632e4 parsl.dataflow.memoization INFO: Task 1949 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1949 try 0 launched on executor multi with executor id 1950 parsl.dataflow.dflow INFO: Standard output for task 1949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/76d8428f-59d9-4e8c-9e2e-2204c0573cf3_isrForCrosstalkSources_996961_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/76d8428f-59d9-4e8c-9e2e-2204c0573cf3_isrForCrosstalkSources_996961_44.stderr parsl.dataflow.dflow DEBUG: Task 1950 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1950 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1950 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1950 has memoization hash 7a05c12340dc887cbcd86684b969417a parsl.dataflow.memoization INFO: Task 1950 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 1950 try 0 launched on executor multi with executor id 1951 parsl.dataflow.dflow INFO: Standard output for task 1950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4abe25be-86dc-4ce3-9921-4b467aa7ca56_isrForCrosstalkSources_997030_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4abe25be-86dc-4ce3-9921-4b467aa7ca56_isrForCrosstalkSources_997030_22.stderr parsl.dataflow.dflow DEBUG: Task 1951 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1951 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1951 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1951 has memoization hash fd0666a55de3adb7d8bdab191046db08 parsl.dataflow.memoization INFO: Task 1951 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1951 try 0 launched on executor multi with executor id 1952 parsl.dataflow.dflow INFO: Standard output for task 1951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/29e928a1-cee3-4574-9f98-533ac6e205d8_isrForCrosstalkSources_997009_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/29e928a1-cee3-4574-9f98-533ac6e205d8_isrForCrosstalkSources_997009_17.stderr parsl.dataflow.dflow DEBUG: Task 1952 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1952 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1952 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1952 has memoization hash d4a83dfcccac4a6a55becc1408f5a1fe parsl.dataflow.memoization INFO: Task 1952 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1952 try 0 launched on executor multi with executor id 1953 parsl.dataflow.dflow INFO: Standard output for task 1952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/c9ac5096-a490-4731-b552-b20e4f7f77c8_isrForCrosstalkSources_996981_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/c9ac5096-a490-4731-b552-b20e4f7f77c8_isrForCrosstalkSources_996981_25.stderr parsl.dataflow.dflow DEBUG: Task 1953 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1953 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1953 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1953 has memoization hash 51abb4779e106073afafeb7fbafd99d3 parsl.dataflow.memoization INFO: Task 1953 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1953 try 0 launched on executor multi with executor id 1954 parsl.dataflow.dflow INFO: Standard output for task 1953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/001b3917-9f06-4bfa-9350-68fd68f4d1c6_isrForCrosstalkSources_997002_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/001b3917-9f06-4bfa-9350-68fd68f4d1c6_isrForCrosstalkSources_997002_50.stderr parsl.dataflow.dflow DEBUG: Task 1954 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1954 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1954 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1954 has memoization hash 137f4ff2fe158187a59ffd079a517c76 parsl.dataflow.memoization INFO: Task 1954 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1954 try 0 launched on executor multi with executor id 1955 parsl.dataflow.dflow INFO: Standard output for task 1954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3ee71b28-0e42-4ddc-b99b-651e22d4c2ad_isrForCrosstalkSources_996959_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3ee71b28-0e42-4ddc-b99b-651e22d4c2ad_isrForCrosstalkSources_996959_27.stderr parsl.dataflow.dflow DEBUG: Task 1955 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1955 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1955 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1955 has memoization hash f7e568ea28f76de218c6e55189f40eb1 parsl.dataflow.memoization INFO: Task 1955 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1955 try 0 launched on executor multi with executor id 1956 parsl.dataflow.dflow INFO: Standard output for task 1955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/05a33231-f2a8-4c84-b779-871b992ecd92_isrForCrosstalkSources_997032_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/05a33231-f2a8-4c84-b779-871b992ecd92_isrForCrosstalkSources_997032_30.stderr parsl.dataflow.dflow DEBUG: Task 1956 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1956 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1956 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1956 has memoization hash acf536ad9b7d0de5784d05589e013b96 parsl.dataflow.memoization INFO: Task 1956 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1956 try 0 launched on executor multi with executor id 1957 parsl.dataflow.dflow INFO: Standard output for task 1956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/450cef44-2e6c-47ff-890c-5f4964f5f1ac_isrForCrosstalkSources_996952_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/450cef44-2e6c-47ff-890c-5f4964f5f1ac_isrForCrosstalkSources_996952_52.stderr parsl.dataflow.dflow DEBUG: Task 1957 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1957 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1957 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1957 has memoization hash 9d198cff70135819bce2627aeaaa9674 parsl.dataflow.memoization INFO: Task 1957 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1957 try 0 launched on executor multi with executor id 1958 parsl.dataflow.dflow INFO: Standard output for task 1957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6acee158-4e74-483d-9dd1-a0943f666fb2_isrForCrosstalkSources_996974_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6acee158-4e74-483d-9dd1-a0943f666fb2_isrForCrosstalkSources_996974_53.stderr parsl.dataflow.dflow DEBUG: Task 1958 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1958 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1958 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1958 has memoization hash 33eed3bd556cce33befd7787acf684be parsl.dataflow.memoization INFO: Task 1958 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1958 try 0 launched on executor multi with executor id 1959 parsl.dataflow.dflow INFO: Standard output for task 1958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/51712fe1-f8a5-4b4c-9ec5-c0b34468574b_isrForCrosstalkSources_997012_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/51712fe1-f8a5-4b4c-9ec5-c0b34468574b_isrForCrosstalkSources_997012_48.stderr parsl.dataflow.dflow DEBUG: Task 1959 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1959 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1959 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1959 has memoization hash 12fe04c81d27829e58092196b69c4dfb parsl.dataflow.memoization INFO: Task 1959 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1959 try 0 launched on executor multi with executor id 1960 parsl.dataflow.dflow INFO: Standard output for task 1959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/42d0b89d-2c2f-4eae-8b8d-9a4a29babc40_isrForCrosstalkSources_996961_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/42d0b89d-2c2f-4eae-8b8d-9a4a29babc40_isrForCrosstalkSources_996961_23.stderr parsl.dataflow.dflow DEBUG: Task 1960 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1960 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1960 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1960 has memoization hash 31b2ff3d4885e550d018d0c834444990 parsl.dataflow.memoization INFO: Task 1960 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1960 try 0 launched on executor multi with executor id 1961 parsl.dataflow.dflow INFO: Standard output for task 1960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/61d41d83-e8b7-4ee6-82ee-d437db62ba82_isrForCrosstalkSources_996973_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/61d41d83-e8b7-4ee6-82ee-d437db62ba82_isrForCrosstalkSources_996973_40.stderr parsl.dataflow.dflow DEBUG: Task 1961 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1961 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1961 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1961 has memoization hash 548b552d634343a857609dbb8211ab6b parsl.dataflow.memoization INFO: Task 1961 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba11c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1961 try 0 launched on executor multi with executor id 1962 parsl.dataflow.dflow INFO: Standard output for task 1961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/64cbd154-4a52-43fa-87a5-bcebba7f11ee_isrForCrosstalkSources_997017_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/64cbd154-4a52-43fa-87a5-bcebba7f11ee_isrForCrosstalkSources_997017_49.stderr parsl.dataflow.dflow DEBUG: Task 1962 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1962 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1962 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1962 has memoization hash a84f9bc9bc7ecb972871ab1c769b949b parsl.dataflow.memoization INFO: Task 1962 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba13a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1962 try 0 launched on executor multi with executor id 1963 parsl.dataflow.dflow INFO: Standard output for task 1962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e500c71e-fd51-43af-adbf-dc81fec59429_isrForCrosstalkSources_997031_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e500c71e-fd51-43af-adbf-dc81fec59429_isrForCrosstalkSources_997031_61.stderr parsl.dataflow.dflow DEBUG: Task 1963 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1963 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1963 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1963 has memoization hash 9e33bb6143952f0655975315fe187f44 parsl.dataflow.memoization INFO: Task 1963 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1963 try 0 launched on executor multi with executor id 1964 parsl.dataflow.dflow INFO: Standard output for task 1963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/6084eeee-fd04-404c-a442-20a144eeaf63_isrForCrosstalkSources_996990_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/6084eeee-fd04-404c-a442-20a144eeaf63_isrForCrosstalkSources_996990_58.stderr parsl.dataflow.dflow DEBUG: Task 1964 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1964 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1964 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1964 has memoization hash 83efd2d4140f48b149aaf711029c54c7 parsl.dataflow.memoization INFO: Task 1964 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1964 try 0 launched on executor multi with executor id 1965 parsl.dataflow.dflow INFO: Standard output for task 1964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c19ba78a-34cf-42ee-a030-f4fc05f01a52_isrForCrosstalkSources_997016_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c19ba78a-34cf-42ee-a030-f4fc05f01a52_isrForCrosstalkSources_997016_42.stderr parsl.dataflow.dflow DEBUG: Task 1965 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1965 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1965 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1965 has memoization hash 43abec267c2db6c86fdda7feaf46f1ec parsl.dataflow.memoization INFO: Task 1965 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1965 try 0 launched on executor multi with executor id 1966 parsl.dataflow.dflow INFO: Standard output for task 1965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/64740a62-edce-4137-971d-b277165fa822_isrForCrosstalkSources_997002_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/64740a62-edce-4137-971d-b277165fa822_isrForCrosstalkSources_997002_52.stderr parsl.dataflow.dflow DEBUG: Task 1966 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1966 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1966 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1966 has memoization hash 59c7e7ff4e76dca7f7c373e852877362 parsl.dataflow.memoization INFO: Task 1966 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba14e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1966 try 0 launched on executor multi with executor id 1967 parsl.dataflow.dflow INFO: Standard output for task 1966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/2127d0b1-e721-4338-9d63-9647b790182f_isrForCrosstalkSources_997020_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/2127d0b1-e721-4338-9d63-9647b790182f_isrForCrosstalkSources_997020_30.stderr parsl.dataflow.dflow DEBUG: Task 1967 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1967 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1967 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1967 has memoization hash 2147889e83da841d7e394aaa858d0867 parsl.dataflow.memoization INFO: Task 1967 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba16c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1967 try 0 launched on executor multi with executor id 1968 parsl.dataflow.dflow INFO: Standard output for task 1967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/190f9e86-9beb-41f9-8797-cd8ba495a22b_isrForCrosstalkSources_997004_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/190f9e86-9beb-41f9-8797-cd8ba495a22b_isrForCrosstalkSources_997004_47.stderr parsl.dataflow.dflow DEBUG: Task 1968 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1968 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1968 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1968 has memoization hash 369316592fbe9e96d183b6d09d23d8b7 parsl.dataflow.memoization INFO: Task 1968 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1968 try 0 launched on executor multi with executor id 1969 parsl.dataflow.dflow INFO: Standard output for task 1968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9b3757e7-f376-404e-a6f4-3552b2baaf62_isrForCrosstalkSources_996960_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9b3757e7-f376-404e-a6f4-3552b2baaf62_isrForCrosstalkSources_996960_17.stderr parsl.dataflow.dflow DEBUG: Task 1969 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1969 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1969 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1969 has memoization hash 7682a978409222ef52c888b6722f3323 parsl.dataflow.memoization INFO: Task 1969 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1969 try 0 launched on executor multi with executor id 1970 parsl.dataflow.dflow INFO: Standard output for task 1969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/3b603dc8-122b-47e6-bf74-5afcd9d5f2b9_isrForCrosstalkSources_996993_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/3b603dc8-122b-47e6-bf74-5afcd9d5f2b9_isrForCrosstalkSources_996993_38.stderr parsl.dataflow.dflow DEBUG: Task 1970 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1970 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1970 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1970 has memoization hash a10c55c366eb43596d1aea7501969232 parsl.dataflow.memoization INFO: Task 1970 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba18a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1970 try 0 launched on executor multi with executor id 1971 parsl.dataflow.dflow INFO: Standard output for task 1970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/1c5f1024-71bc-487d-9dee-7dc0a86f6e15_isrForCrosstalkSources_997015_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/1c5f1024-71bc-487d-9dee-7dc0a86f6e15_isrForCrosstalkSources_997015_45.stderr parsl.dataflow.dflow DEBUG: Task 1971 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1971 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1971 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1971 has memoization hash 664b3793ce57babceb206c2d6b430aa3 parsl.dataflow.memoization INFO: Task 1971 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1971 try 0 launched on executor multi with executor id 1972 parsl.dataflow.dflow INFO: Standard output for task 1971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ec172865-72b8-4da3-9875-46c50c9a44b6_isrForCrosstalkSources_997001_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ec172865-72b8-4da3-9875-46c50c9a44b6_isrForCrosstalkSources_997001_34.stderr parsl.dataflow.dflow DEBUG: Task 1972 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1972 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1972 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1972 has memoization hash 8d0257ff195850e58ce55e707609a3b0 parsl.dataflow.memoization INFO: Task 1972 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba19e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1972 try 0 launched on executor multi with executor id 1973 parsl.dataflow.dflow INFO: Standard output for task 1972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1788145a-2b51-4699-b25c-b6c605547d3b_isrForCrosstalkSources_997027_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1788145a-2b51-4699-b25c-b6c605547d3b_isrForCrosstalkSources_997027_29.stderr parsl.dataflow.dflow DEBUG: Task 1973 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1973 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1973 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1973 has memoization hash f7e74d82b0624817093dcfd1b75be132 parsl.dataflow.memoization INFO: Task 1973 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1973 try 0 launched on executor multi with executor id 1974 parsl.dataflow.dflow INFO: Standard output for task 1973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ca504218-7396-4930-9ece-ca947907685d_isrForCrosstalkSources_996980_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ca504218-7396-4930-9ece-ca947907685d_isrForCrosstalkSources_996980_40.stderr parsl.dataflow.dflow DEBUG: Task 1974 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1974 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1974 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1974 has memoization hash b8a28c1749d241c1da0480022a1739b1 parsl.dataflow.memoization INFO: Task 1974 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1974 try 0 launched on executor multi with executor id 1975 parsl.dataflow.dflow INFO: Standard output for task 1974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a0a798cf-81f9-4ad3-92f9-924c07cadb52_isrForCrosstalkSources_996972_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a0a798cf-81f9-4ad3-92f9-924c07cadb52_isrForCrosstalkSources_996972_39.stderr parsl.dataflow.dflow DEBUG: Task 1975 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1975 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1975 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1975 has memoization hash 0106cba6a393b10afcbd2c8563e3d5f7 parsl.dataflow.memoization INFO: Task 1975 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1975 try 0 launched on executor multi with executor id 1976 parsl.dataflow.dflow INFO: Standard output for task 1975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bed4f0fe-1b8c-447d-ac89-5267b5690dfb_isrForCrosstalkSources_997027_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bed4f0fe-1b8c-447d-ac89-5267b5690dfb_isrForCrosstalkSources_997027_14.stderr parsl.dataflow.dflow DEBUG: Task 1976 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1976 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1976 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1976 has memoization hash 8c071acf07f9831c7b132c1586df470c parsl.dataflow.memoization INFO: Task 1976 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1976 try 0 launched on executor multi with executor id 1977 parsl.dataflow.dflow INFO: Standard output for task 1976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/94b1bd81-8c82-4792-a0f3-59f384fe755f_isrForCrosstalkSources_997022_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/94b1bd81-8c82-4792-a0f3-59f384fe755f_isrForCrosstalkSources_997022_42.stderr parsl.dataflow.dflow DEBUG: Task 1977 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1977 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1977 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1977 has memoization hash 9c87e7e3dfeb13d4b029e00296e453bb parsl.dataflow.memoization INFO: Task 1977 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1977 try 0 launched on executor multi with executor id 1978 parsl.dataflow.dflow INFO: Standard output for task 1977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/718f4033-221f-4e85-b833-652bdfaebdd3_isrForCrosstalkSources_997021_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/718f4033-221f-4e85-b833-652bdfaebdd3_isrForCrosstalkSources_997021_17.stderr parsl.dataflow.dflow DEBUG: Task 1978 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1978 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1978 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1978 has memoization hash 184383a0320823f658c4dc2d375acbd2 parsl.dataflow.memoization INFO: Task 1978 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1978 try 0 launched on executor multi with executor id 1979 parsl.dataflow.dflow INFO: Standard output for task 1978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/752535a3-569d-470c-b4d5-06c286baf99f_isrForCrosstalkSources_996976_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/752535a3-569d-470c-b4d5-06c286baf99f_isrForCrosstalkSources_996976_16.stderr parsl.dataflow.dflow DEBUG: Task 1979 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1979 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1979 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1979 has memoization hash 7abd7593787f140f26892e95ed4106a0 parsl.dataflow.memoization INFO: Task 1979 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1979 try 0 launched on executor multi with executor id 1980 parsl.dataflow.dflow INFO: Standard output for task 1979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4ab92762-2f00-4d69-82f2-52932f3f46c6_isrForCrosstalkSources_996954_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4ab92762-2f00-4d69-82f2-52932f3f46c6_isrForCrosstalkSources_996954_22.stderr parsl.dataflow.dflow DEBUG: Task 1980 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1980 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1980 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1980 has memoization hash 32998d0852cd91bfdd387c01d5d85db2 parsl.dataflow.memoization INFO: Task 1980 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1980 try 0 launched on executor multi with executor id 1981 parsl.dataflow.dflow INFO: Standard output for task 1980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/2718c44f-ef80-41bd-9ef0-87f4260e2430_isrForCrosstalkSources_997001_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/2718c44f-ef80-41bd-9ef0-87f4260e2430_isrForCrosstalkSources_997001_42.stderr parsl.dataflow.dflow DEBUG: Task 1981 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1981 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1981 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1981 has memoization hash 96625f56e787f3d03f9cf1ddd1fe3afc parsl.dataflow.memoization INFO: Task 1981 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1981 try 0 launched on executor multi with executor id 1982 parsl.dataflow.dflow INFO: Standard output for task 1981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/34e1915a-1c0c-4de1-9c4e-982fdd44431b_isrForCrosstalkSources_996969_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/34e1915a-1c0c-4de1-9c4e-982fdd44431b_isrForCrosstalkSources_996969_32.stderr parsl.dataflow.dflow DEBUG: Task 1982 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1982 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1982 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1982 has memoization hash 83ec8922cbf718450aea77a67ffa17a6 parsl.dataflow.memoization INFO: Task 1982 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba1ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1982 try 0 launched on executor multi with executor id 1983 parsl.dataflow.dflow INFO: Standard output for task 1982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/1bfa2bd7-850e-4695-a792-44bbb3dc7d8f_isrForCrosstalkSources_996995_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/1bfa2bd7-850e-4695-a792-44bbb3dc7d8f_isrForCrosstalkSources_996995_47.stderr parsl.dataflow.dflow DEBUG: Task 1983 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1983 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1983 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1983 has memoization hash 74ced5920d50b78cd4ec1815a9d67d3f parsl.dataflow.memoization INFO: Task 1983 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1983 try 0 launched on executor multi with executor id 1984 parsl.dataflow.dflow INFO: Standard output for task 1983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/fed76d96-ec3f-4a71-8b9e-f42020855e8a_isrForCrosstalkSources_996957_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/fed76d96-ec3f-4a71-8b9e-f42020855e8a_isrForCrosstalkSources_996957_26.stderr parsl.dataflow.dflow DEBUG: Task 1984 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1984 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1984 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1984 has memoization hash 7b781f2756daefe6db88355f81f01888 parsl.dataflow.memoization INFO: Task 1984 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1984 try 0 launched on executor multi with executor id 1985 parsl.dataflow.dflow INFO: Standard output for task 1984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/269494aa-afa7-4487-b1f0-2a7e11028e1d_isrForCrosstalkSources_996960_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/269494aa-afa7-4487-b1f0-2a7e11028e1d_isrForCrosstalkSources_996960_57.stderr parsl.dataflow.dflow DEBUG: Task 1985 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1985 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1985 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1985 has memoization hash 913db9cc7ec0c1b9196a58d6528e5394 parsl.dataflow.memoization INFO: Task 1985 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba22a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1985 try 0 launched on executor multi with executor id 1986 parsl.dataflow.dflow INFO: Standard output for task 1985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b668db7a-690b-4077-9d4e-d8b2debf2e7c_isrForCrosstalkSources_996992_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b668db7a-690b-4077-9d4e-d8b2debf2e7c_isrForCrosstalkSources_996992_40.stderr parsl.dataflow.dflow DEBUG: Task 1986 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1986 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1986 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1986 has memoization hash bdb174bf7fe78c9500c1766ba12dbb30 parsl.dataflow.memoization INFO: Task 1986 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba20c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1986 try 0 launched on executor multi with executor id 1987 parsl.dataflow.dflow INFO: Standard output for task 1986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fcaff9e1-37fb-40f5-a371-8b2b74e92a7b_isrForCrosstalkSources_997008_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fcaff9e1-37fb-40f5-a371-8b2b74e92a7b_isrForCrosstalkSources_997008_7.stderr parsl.dataflow.dflow DEBUG: Task 1987 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1987 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1987 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1987 has memoization hash 96e9c1b1f0b3d1e17e623015217030b0 parsl.dataflow.memoization INFO: Task 1987 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1987 try 0 launched on executor multi with executor id 1988 parsl.dataflow.dflow INFO: Standard output for task 1987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dc2d39b7-9d3d-490c-b64a-8f52666769cc_isrForCrosstalkSources_996971_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dc2d39b7-9d3d-490c-b64a-8f52666769cc_isrForCrosstalkSources_996971_12.stderr parsl.dataflow.dflow DEBUG: Task 1988 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1988 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1988 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1988 has memoization hash 12c776fa337bf52afe4619f84e31011f parsl.dataflow.memoization INFO: Task 1988 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba23e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1988 try 0 launched on executor multi with executor id 1989 parsl.dataflow.dflow INFO: Standard output for task 1988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/678e5ff5-f461-4645-bc18-c470d58a0023_isrForCrosstalkSources_997018_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/678e5ff5-f461-4645-bc18-c470d58a0023_isrForCrosstalkSources_997018_5.stderr parsl.dataflow.dflow DEBUG: Task 1989 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1989 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1989 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1989 has memoization hash e33ea605d2852af728ba9477481f1f7e parsl.dataflow.memoization INFO: Task 1989 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1989 try 0 launched on executor multi with executor id 1990 parsl.dataflow.dflow INFO: Standard output for task 1989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/647ae71b-6e8d-44ae-96d4-938b1ff80cdd_isrForCrosstalkSources_996961_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/647ae71b-6e8d-44ae-96d4-938b1ff80cdd_isrForCrosstalkSources_996961_36.stderr parsl.dataflow.dflow DEBUG: Task 1990 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1990 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1990 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1990 has memoization hash d3f85e917539dc16c39f03d88c2c78ab parsl.dataflow.memoization INFO: Task 1990 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba25c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1990 try 0 launched on executor multi with executor id 1991 parsl.dataflow.dflow INFO: Standard output for task 1990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ebccccfd-0442-4ac7-b25d-913bd7005bf5_isrForCrosstalkSources_997020_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ebccccfd-0442-4ac7-b25d-913bd7005bf5_isrForCrosstalkSources_997020_50.stderr parsl.dataflow.dflow DEBUG: Task 1991 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1991 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1991 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1991 has memoization hash 5a3441c6b81a487cc52db8ef59fbe9e2 parsl.dataflow.memoization INFO: Task 1991 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1991 try 0 launched on executor multi with executor id 1992 parsl.dataflow.dflow INFO: Standard output for task 1991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/868e4092-be79-4164-9873-16f4f7d2272b_isrForCrosstalkSources_996993_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/868e4092-be79-4164-9873-16f4f7d2272b_isrForCrosstalkSources_996993_13.stderr parsl.dataflow.dflow DEBUG: Task 1992 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1992 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1992 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1992 has memoization hash 8b8551c02e391e362d30824077de6904 parsl.dataflow.memoization INFO: Task 1992 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1992 try 0 launched on executor multi with executor id 1993 parsl.dataflow.dflow INFO: Standard output for task 1992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/60efebc1-2ea2-4043-a56e-31fce78b303e_isrForCrosstalkSources_996952_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/60efebc1-2ea2-4043-a56e-31fce78b303e_isrForCrosstalkSources_996952_59.stderr parsl.dataflow.dflow DEBUG: Task 1993 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1993 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1993 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1993 has memoization hash cff85a686582e736caba44e1b4a1c139 parsl.dataflow.memoization INFO: Task 1993 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1993 try 0 launched on executor multi with executor id 1994 parsl.dataflow.dflow INFO: Standard output for task 1993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/ad4a0f24-9912-4ba9-8e18-efd2c1f8d501_isrForCrosstalkSources_996967_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/ad4a0f24-9912-4ba9-8e18-efd2c1f8d501_isrForCrosstalkSources_996967_29.stderr parsl.dataflow.dflow DEBUG: Task 1994 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1994 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1994 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1994 has memoization hash 1cce42d492bd803d1abc27e2196c04c3 parsl.dataflow.memoization INFO: Task 1994 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1994 try 0 launched on executor multi with executor id 1995 parsl.dataflow.dflow INFO: Standard output for task 1994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/193c5a8c-766a-43ed-bf31-89d16b321307_isrForCrosstalkSources_996979_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/193c5a8c-766a-43ed-bf31-89d16b321307_isrForCrosstalkSources_996979_55.stderr parsl.dataflow.dflow DEBUG: Task 1995 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1995 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1995 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1995 has memoization hash f10c80772442614fcde013e0866e7fbc parsl.dataflow.memoization INFO: Task 1995 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba28e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1995 try 0 launched on executor multi with executor id 1996 parsl.dataflow.dflow INFO: Standard output for task 1995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/aa5eb64a-14f7-4772-af2c-e91a23f1f6bb_isrForCrosstalkSources_996968_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/aa5eb64a-14f7-4772-af2c-e91a23f1f6bb_isrForCrosstalkSources_996968_62.stderr parsl.dataflow.dflow DEBUG: Task 1996 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1996 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1996 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1996 has memoization hash 088a5ab471f89d5dcf6119a26a14f1b0 parsl.dataflow.memoization INFO: Task 1996 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1996 try 0 launched on executor multi with executor id 1997 parsl.dataflow.dflow INFO: Standard output for task 1996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a7d458bf-faf6-434a-ab20-133cf4c29809_isrForCrosstalkSources_996956_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a7d458bf-faf6-434a-ab20-133cf4c29809_isrForCrosstalkSources_996956_7.stderr parsl.dataflow.dflow DEBUG: Task 1997 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1997 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1997 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1997 has memoization hash 1f350147799400c103a11dba9471d6c6 parsl.dataflow.memoization INFO: Task 1997 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1997 try 0 launched on executor multi with executor id 1998 parsl.dataflow.dflow INFO: Standard output for task 1997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9e091ec3-ba7e-48eb-9930-d3afc669eb08_isrForCrosstalkSources_997000_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9e091ec3-ba7e-48eb-9930-d3afc669eb08_isrForCrosstalkSources_997000_60.stderr parsl.dataflow.dflow DEBUG: Task 1998 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1998 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1998 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1998 has memoization hash 2dc0aff9e0ea7d9924e90e424849cfbe parsl.dataflow.memoization INFO: Task 1998 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba27a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1998 try 0 launched on executor multi with executor id 1999 parsl.dataflow.dflow INFO: Standard output for task 1998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0a824ce0-f8b5-42b7-be73-383357e39ccf_isrForCrosstalkSources_996956_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0a824ce0-f8b5-42b7-be73-383357e39ccf_isrForCrosstalkSources_996956_17.stderr parsl.dataflow.dflow DEBUG: Task 1999 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 1999 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 1999 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 1999 has memoization hash cc7c2b5e71f43126e93fcb75ac7821e3 parsl.dataflow.memoization INFO: Task 1999 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 1999 try 0 launched on executor multi with executor id 2000 parsl.dataflow.dflow INFO: Standard output for task 1999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9ca305cd-194f-49f4-84dd-53ef2358e2d8_isrForCrosstalkSources_997018_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9ca305cd-194f-49f4-84dd-53ef2358e2d8_isrForCrosstalkSources_997018_31.stderr parsl.dataflow.dflow DEBUG: Task 2000 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2000 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2000 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2000 has memoization hash 1fef5fc26f6950e6caa77dab8b3e3309 parsl.dataflow.memoization INFO: Task 2000 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2000 try 0 launched on executor multi with executor id 2001 parsl.dataflow.dflow INFO: Standard output for task 2000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/9264fe03-d8ef-4991-ba65-5d2c654e6600_isrForCrosstalkSources_997005_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/9264fe03-d8ef-4991-ba65-5d2c654e6600_isrForCrosstalkSources_997005_47.stderr parsl.dataflow.dflow DEBUG: Task 2001 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2001 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2001 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2001 has memoization hash d62b3e5f1c25f875e761eafe2e518a1f parsl.dataflow.memoization INFO: Task 2001 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2001 try 0 launched on executor multi with executor id 2002 parsl.dataflow.dflow INFO: Standard output for task 2001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5a592af6-1f72-41af-b956-5266a290a51f_isrForCrosstalkSources_997004_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5a592af6-1f72-41af-b956-5266a290a51f_isrForCrosstalkSources_997004_59.stderr parsl.dataflow.dflow DEBUG: Task 2002 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2002 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2002 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2002 has memoization hash c1278335129227cf2c7d15e9db1c0eab parsl.dataflow.memoization INFO: Task 2002 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2002 try 0 launched on executor multi with executor id 2003 parsl.dataflow.dflow INFO: Standard output for task 2002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/465a3f94-e236-4360-855e-065dc1c397a5_isrForCrosstalkSources_996988_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/465a3f94-e236-4360-855e-065dc1c397a5_isrForCrosstalkSources_996988_12.stderr parsl.dataflow.dflow DEBUG: Task 2003 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2003 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2003 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2003 has memoization hash 8e87d82b52e4f5715ebb23d33a3a74a2 parsl.dataflow.memoization INFO: Task 2003 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2003 try 0 launched on executor multi with executor id 2004 parsl.dataflow.dflow INFO: Standard output for task 2003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/6dccda82-2d61-4ed3-b0b0-3e53679ebc06_isrForCrosstalkSources_996959_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/6dccda82-2d61-4ed3-b0b0-3e53679ebc06_isrForCrosstalkSources_996959_2.stderr parsl.dataflow.dflow DEBUG: Task 2004 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2004 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2004 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2004 has memoization hash 85400d56556fa945051842604ddb4532 parsl.dataflow.memoization INFO: Task 2004 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2004 try 0 launched on executor multi with executor id 2005 parsl.dataflow.dflow INFO: Standard output for task 2004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/dfb18199-4480-4698-9583-83b6f98afdc5_isrForCrosstalkSources_996952_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/dfb18199-4480-4698-9583-83b6f98afdc5_isrForCrosstalkSources_996952_21.stderr parsl.dataflow.dflow DEBUG: Task 2005 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2005 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2005 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2005 has memoization hash afa0ef468a229abcb4482bfdd42fae12 parsl.dataflow.memoization INFO: Task 2005 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2005 try 0 launched on executor multi with executor id 2006 parsl.dataflow.dflow INFO: Standard output for task 2005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/64612e54-3f42-4e26-9787-3c6e56a24176_isrForCrosstalkSources_997028_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/64612e54-3f42-4e26-9787-3c6e56a24176_isrForCrosstalkSources_997028_8.stderr parsl.dataflow.dflow DEBUG: Task 2006 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2006 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2006 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2006 has memoization hash 8f48ed7edf2375fadf72c436007a0c77 parsl.dataflow.memoization INFO: Task 2006 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2006 try 0 launched on executor multi with executor id 2007 parsl.dataflow.dflow INFO: Standard output for task 2006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/6b019dee-bd9a-499a-a477-fb99c02de5d4_isrForCrosstalkSources_997018_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/6b019dee-bd9a-499a-a477-fb99c02de5d4_isrForCrosstalkSources_997018_15.stderr parsl.dataflow.dflow DEBUG: Task 2007 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2007 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2007 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2007 has memoization hash 56921a7418921fb494d83a8fd546cc16 parsl.dataflow.memoization INFO: Task 2007 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2007 try 0 launched on executor multi with executor id 2008 parsl.dataflow.dflow INFO: Standard output for task 2007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/28c71d6f-9bdf-46b9-8a8e-9940fc8afcf0_isrForCrosstalkSources_997014_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/28c71d6f-9bdf-46b9-8a8e-9940fc8afcf0_isrForCrosstalkSources_997014_10.stderr parsl.dataflow.dflow DEBUG: Task 2008 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2008 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2008 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2008 has memoization hash 8ecea3137d8ef70eacfb5237a2bc0d16 parsl.dataflow.memoization INFO: Task 2008 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2008 try 0 launched on executor multi with executor id 2009 parsl.dataflow.dflow INFO: Standard output for task 2008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/71b6728f-63ad-42e8-ade5-5cc0b8a39b6b_isrForCrosstalkSources_996986_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/71b6728f-63ad-42e8-ade5-5cc0b8a39b6b_isrForCrosstalkSources_996986_29.stderr parsl.dataflow.dflow DEBUG: Task 2009 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2009 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2009 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2009 has memoization hash 97cf43957487ef7481b154047b003c48 parsl.dataflow.memoization INFO: Task 2009 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2009 try 0 launched on executor multi with executor id 2010 parsl.dataflow.dflow INFO: Standard output for task 2009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6fe2051a-24e2-46cc-8494-5ac49a517c56_isrForCrosstalkSources_997023_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6fe2051a-24e2-46cc-8494-5ac49a517c56_isrForCrosstalkSources_997023_45.stderr parsl.dataflow.dflow DEBUG: Task 2010 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2010 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2010 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2010 has memoization hash af67f84a3cfaefd97a4e6546cc861271 parsl.dataflow.memoization INFO: Task 2010 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba39c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2010 try 0 launched on executor multi with executor id 2011 parsl.dataflow.dflow INFO: Standard output for task 2010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/88cc38d7-98c0-4ef6-a905-3ce904433aff_isrForCrosstalkSources_997012_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/88cc38d7-98c0-4ef6-a905-3ce904433aff_isrForCrosstalkSources_997012_30.stderr parsl.dataflow.dflow DEBUG: Task 2011 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2011 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2011 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2011 has memoization hash 7663d37b78276cb5c33440bcaa30ae89 parsl.dataflow.memoization INFO: Task 2011 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2011 try 0 launched on executor multi with executor id 2012 parsl.dataflow.dflow INFO: Standard output for task 2011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/8de5960e-799f-4e7b-aa06-38b149d73eea_isrForCrosstalkSources_996991_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/8de5960e-799f-4e7b-aa06-38b149d73eea_isrForCrosstalkSources_996991_26.stderr parsl.dataflow.dflow DEBUG: Task 2012 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2012 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2012 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2012 has memoization hash aad29a4233eed83819742fadba309aec parsl.dataflow.memoization INFO: Task 2012 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2012 try 0 launched on executor multi with executor id 2013 parsl.dataflow.dflow INFO: Standard output for task 2012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/83a84137-71ef-4510-9bca-a08ab10b0a49_isrForCrosstalkSources_997030_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/83a84137-71ef-4510-9bca-a08ab10b0a49_isrForCrosstalkSources_997030_32.stderr parsl.dataflow.dflow DEBUG: Task 2013 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2013 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2013 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2013 has memoization hash 84cc2515caac8825d95331f5ca53a450 parsl.dataflow.memoization INFO: Task 2013 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2013 try 0 launched on executor multi with executor id 2014 parsl.dataflow.dflow INFO: Standard output for task 2013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a27721c2-6dda-43e8-8538-72ec876d9383_isrForCrosstalkSources_996965_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a27721c2-6dda-43e8-8538-72ec876d9383_isrForCrosstalkSources_996965_6.stderr parsl.dataflow.dflow DEBUG: Task 2014 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2014 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2014 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2014 has memoization hash e5dd0e74bb23467cc3cd05db70db09de parsl.dataflow.memoization INFO: Task 2014 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba37e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2014 try 0 launched on executor multi with executor id 2015 parsl.dataflow.dflow INFO: Standard output for task 2014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/440d0992-faa3-4d2e-9c18-ae9de223496f_isrForCrosstalkSources_997002_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/440d0992-faa3-4d2e-9c18-ae9de223496f_isrForCrosstalkSources_997002_27.stderr parsl.dataflow.dflow DEBUG: Task 2015 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2015 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2015 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2015 has memoization hash 3b681a4a617e1fe711fcea1396289fcc parsl.dataflow.memoization INFO: Task 2015 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba36a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2015 try 0 launched on executor multi with executor id 2016 parsl.dataflow.dflow INFO: Standard output for task 2015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9f825bd4-3d5d-4ce9-9a17-97b2c559cb80_isrForCrosstalkSources_996960_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9f825bd4-3d5d-4ce9-9a17-97b2c559cb80_isrForCrosstalkSources_996960_22.stderr parsl.dataflow.dflow DEBUG: Task 2016 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2016 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2016 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2016 has memoization hash 484aa73b174a7ac75f4797e6aa8670fa parsl.dataflow.memoization INFO: Task 2016 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2016 try 0 launched on executor multi with executor id 2017 parsl.dataflow.dflow INFO: Standard output for task 2016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2deb948d-d35d-4f98-8d23-325ecd54b629_isrForCrosstalkSources_997005_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2deb948d-d35d-4f98-8d23-325ecd54b629_isrForCrosstalkSources_997005_13.stderr parsl.dataflow.dflow DEBUG: Task 2017 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2017 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2017 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2017 has memoization hash d1bc242cf71a13db66aa8d5e581184d9 parsl.dataflow.memoization INFO: Task 2017 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2017 try 0 launched on executor multi with executor id 2018 parsl.dataflow.dflow INFO: Standard output for task 2017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/56b151ce-6c0e-40e9-9c92-eff1ad752b97_isrForCrosstalkSources_997017_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/56b151ce-6c0e-40e9-9c92-eff1ad752b97_isrForCrosstalkSources_997017_47.stderr parsl.dataflow.dflow DEBUG: Task 2018 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2018 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2018 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2018 has memoization hash 847cbba6ee76340b7f08afd44ffcab93 parsl.dataflow.memoization INFO: Task 2018 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2018 try 0 launched on executor multi with executor id 2019 parsl.dataflow.dflow INFO: Standard output for task 2018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/14722342-3f8c-440a-bd55-7b46770612a1_isrForCrosstalkSources_997001_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/14722342-3f8c-440a-bd55-7b46770612a1_isrForCrosstalkSources_997001_5.stderr parsl.dataflow.dflow DEBUG: Task 2019 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2019 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2019 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2019 has memoization hash 40651453cdba9e7b17e3513768b0e945 parsl.dataflow.memoization INFO: Task 2019 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba34c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2019 try 0 launched on executor multi with executor id 2020 parsl.dataflow.dflow INFO: Standard output for task 2019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a1a139f7-58e1-41e9-a903-f8df45522387_isrForCrosstalkSources_996952_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a1a139f7-58e1-41e9-a903-f8df45522387_isrForCrosstalkSources_996952_24.stderr parsl.dataflow.dflow DEBUG: Task 2020 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2020 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2020 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2020 has memoization hash 9ba14ac1bd2c0f1eb7b2f0311e3dc025 parsl.dataflow.memoization INFO: Task 2020 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2020 try 0 launched on executor multi with executor id 2021 parsl.dataflow.dflow INFO: Standard output for task 2020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/73070103-ceb2-4a44-988b-59385aee60bb_isrForCrosstalkSources_997000_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/73070103-ceb2-4a44-988b-59385aee60bb_isrForCrosstalkSources_997000_27.stderr parsl.dataflow.dflow DEBUG: Task 2021 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2021 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2021 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2021 has memoization hash e254aca02f91199a114e33d23608b435 parsl.dataflow.memoization INFO: Task 2021 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2021 try 0 launched on executor multi with executor id 2022 parsl.dataflow.dflow INFO: Standard output for task 2021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/1998069b-d873-4442-bd2c-427bbc3210fe_isrForCrosstalkSources_996965_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/1998069b-d873-4442-bd2c-427bbc3210fe_isrForCrosstalkSources_996965_17.stderr parsl.dataflow.dflow DEBUG: Task 2022 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2022 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2022 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2022 has memoization hash 800f50dc1f46fdf48e15d6055985545b parsl.dataflow.memoization INFO: Task 2022 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba32e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2022 try 0 launched on executor multi with executor id 2023 parsl.dataflow.dflow INFO: Standard output for task 2022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d94f2960-4f57-440d-b166-204de7091285_isrForCrosstalkSources_996992_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d94f2960-4f57-440d-b166-204de7091285_isrForCrosstalkSources_996992_33.stderr parsl.dataflow.dflow DEBUG: Task 2023 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2023 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2023 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2023 has memoization hash 8fc1ae8422cf94a570a72248db3c0630 parsl.dataflow.memoization INFO: Task 2023 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2023 try 0 launched on executor multi with executor id 2024 parsl.dataflow.dflow INFO: Standard output for task 2023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b320fdad-5e9f-4e5e-b030-baa43ede4010_isrForCrosstalkSources_997033_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b320fdad-5e9f-4e5e-b030-baa43ede4010_isrForCrosstalkSources_997033_45.stderr parsl.dataflow.dflow DEBUG: Task 2024 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2024 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2024 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2024 has memoization hash 2e6a484e9664238cafe6c7677e37925f parsl.dataflow.memoization INFO: Task 2024 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba31a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2024 try 0 launched on executor multi with executor id 2025 parsl.dataflow.dflow INFO: Standard output for task 2024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/3125989d-c617-4228-bbb6-e3871121a1e6_isrForCrosstalkSources_997032_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/3125989d-c617-4228-bbb6-e3871121a1e6_isrForCrosstalkSources_997032_59.stderr parsl.dataflow.dflow DEBUG: Task 2025 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2025 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2025 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2025 has memoization hash bb6bc82bd1cb8f262161e16b364ee917 parsl.dataflow.memoization INFO: Task 2025 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2025 try 0 launched on executor multi with executor id 2026 parsl.dataflow.dflow INFO: Standard output for task 2025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a938a158-000e-448f-a018-3cedf6080e97_isrForCrosstalkSources_997008_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a938a158-000e-448f-a018-3cedf6080e97_isrForCrosstalkSources_997008_26.stderr parsl.dataflow.dflow DEBUG: Task 2026 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2026 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2026 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2026 has memoization hash 318327419503a6cd9b3e096921c27a45 parsl.dataflow.memoization INFO: Task 2026 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba3100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2026 try 0 launched on executor multi with executor id 2027 parsl.dataflow.dflow INFO: Standard output for task 2026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6d9ed66e-eb3c-4cf3-a424-615294c34b89_isrForCrosstalkSources_996974_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6d9ed66e-eb3c-4cf3-a424-615294c34b89_isrForCrosstalkSources_996974_37.stderr parsl.dataflow.dflow DEBUG: Task 2027 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2027 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2027 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2027 has memoization hash f0c8a1103f61858242ff741959a6ed64 parsl.dataflow.memoization INFO: Task 2027 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2027 try 0 launched on executor multi with executor id 2028 parsl.dataflow.dflow INFO: Standard output for task 2027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/17084782-3885-4c1b-a4db-68b0e1ef5260_isrForCrosstalkSources_996977_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/17084782-3885-4c1b-a4db-68b0e1ef5260_isrForCrosstalkSources_996977_12.stderr parsl.dataflow.dflow DEBUG: Task 2028 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2028 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2028 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2028 has memoization hash dbfdbcb18f2f529694c546ddfae84e67 parsl.dataflow.memoization INFO: Task 2028 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2028 try 0 launched on executor multi with executor id 2029 parsl.dataflow.dflow INFO: Standard output for task 2028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a64fa012-2af4-4f36-aac4-170554613965_isrForCrosstalkSources_997011_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a64fa012-2af4-4f36-aac4-170554613965_isrForCrosstalkSources_997011_43.stderr parsl.dataflow.dflow DEBUG: Task 2029 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2029 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2029 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2029 has memoization hash df3c7f326624abb1bfc23c4e4eeda598 parsl.dataflow.memoization INFO: Task 2029 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2029 try 0 launched on executor multi with executor id 2030 parsl.dataflow.dflow INFO: Standard output for task 2029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/42109cf8-4f6b-4470-89f7-4d2f9d2d0fb0_isrForCrosstalkSources_996972_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/42109cf8-4f6b-4470-89f7-4d2f9d2d0fb0_isrForCrosstalkSources_996972_3.stderr parsl.dataflow.dflow DEBUG: Task 2030 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2030 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2030 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2030 has memoization hash 31ab6667996e5775e2f912ad94ffa4d0 parsl.dataflow.memoization INFO: Task 2030 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2030 try 0 launched on executor multi with executor id 2031 parsl.dataflow.dflow INFO: Standard output for task 2030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2b5492c8-578f-4de5-906a-a3b469e8440d_isrForCrosstalkSources_996956_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2b5492c8-578f-4de5-906a-a3b469e8440d_isrForCrosstalkSources_996956_41.stderr parsl.dataflow.dflow DEBUG: Task 2031 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2031 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2031 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2031 has memoization hash d157e8e41c57b8687ebe2e542a038268 parsl.dataflow.memoization INFO: Task 2031 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2031 try 0 launched on executor multi with executor id 2032 parsl.dataflow.dflow INFO: Standard output for task 2031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/91448bb0-f229-4da6-bbbb-5dfe5a59c5f2_isrForCrosstalkSources_996986_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/91448bb0-f229-4da6-bbbb-5dfe5a59c5f2_isrForCrosstalkSources_996986_14.stderr parsl.dataflow.dflow DEBUG: Task 2032 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2032 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2032 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2032 has memoization hash 26cfeff72b5ea721757f8bde04440e68 parsl.dataflow.memoization INFO: Task 2032 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2032 try 0 launched on executor multi with executor id 2033 parsl.dataflow.dflow INFO: Standard output for task 2032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9f9667b8-aa36-4527-b431-d5d8870c89f1_isrForCrosstalkSources_997023_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9f9667b8-aa36-4527-b431-d5d8870c89f1_isrForCrosstalkSources_997023_34.stderr parsl.dataflow.dflow DEBUG: Task 2033 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2033 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2033 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2033 has memoization hash a2cbacf4802f6c95e61adabeac4cd660 parsl.dataflow.memoization INFO: Task 2033 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2033 try 0 launched on executor multi with executor id 2034 parsl.dataflow.dflow INFO: Standard output for task 2033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/da3f8c0d-73a7-497c-9b9a-109a5b28e740_isrForCrosstalkSources_996970_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/da3f8c0d-73a7-497c-9b9a-109a5b28e740_isrForCrosstalkSources_996970_16.stderr parsl.dataflow.dflow DEBUG: Task 2034 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2034 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2034 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2034 has memoization hash 02bc3c09c89fe4b07a537a70fcb97062 parsl.dataflow.memoization INFO: Task 2034 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2034 try 0 launched on executor multi with executor id 2035 parsl.dataflow.dflow INFO: Standard output for task 2034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ad9cc232-d4e5-4d2e-baef-a726008549c5_isrForCrosstalkSources_996989_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ad9cc232-d4e5-4d2e-baef-a726008549c5_isrForCrosstalkSources_996989_3.stderr parsl.dataflow.dflow DEBUG: Task 2035 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2035 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2035 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2035 has memoization hash dd7aeda87dec5ea4500f835867380609 parsl.dataflow.memoization INFO: Task 2035 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc040e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2035 try 0 launched on executor multi with executor id 2036 parsl.dataflow.dflow INFO: Standard output for task 2035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/12a3432e-8a58-4a55-85ef-2fe2e396e0ba_isrForCrosstalkSources_997009_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/12a3432e-8a58-4a55-85ef-2fe2e396e0ba_isrForCrosstalkSources_997009_47.stderr parsl.dataflow.dflow DEBUG: Task 2036 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2036 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2036 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2036 has memoization hash 6e21aec0c9b076de518876c9244dd064 parsl.dataflow.memoization INFO: Task 2036 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2036 try 0 launched on executor multi with executor id 2037 parsl.dataflow.dflow INFO: Standard output for task 2036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/8f6b3902-0227-4162-a8bc-16966b48a832_isrForCrosstalkSources_997006_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/8f6b3902-0227-4162-a8bc-16966b48a832_isrForCrosstalkSources_997006_19.stderr parsl.dataflow.dflow DEBUG: Task 2037 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2037 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2037 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2037 has memoization hash e25880e77ebb880ef35663ced6877f03 parsl.dataflow.memoization INFO: Task 2037 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc042c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2037 try 0 launched on executor multi with executor id 2038 parsl.dataflow.dflow INFO: Standard output for task 2037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/4f998954-4b75-423d-ab13-40038a482173_isrForCrosstalkSources_996955_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/4f998954-4b75-423d-ab13-40038a482173_isrForCrosstalkSources_996955_9.stderr parsl.dataflow.dflow DEBUG: Task 2038 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2038 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2038 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2038 has memoization hash 3c5b4694e7a37cb86995680ac59a61e4 parsl.dataflow.memoization INFO: Task 2038 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2038 try 0 launched on executor multi with executor id 2039 parsl.dataflow.dflow INFO: Standard output for task 2038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c6a00ffa-44f7-4ede-b498-3a50a4e2bc5b_isrForCrosstalkSources_997000_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c6a00ffa-44f7-4ede-b498-3a50a4e2bc5b_isrForCrosstalkSources_997000_8.stderr parsl.dataflow.dflow DEBUG: Task 2039 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2039 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2039 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2039 has memoization hash 908a36e87c124681fc9f6d04bcb6f928 parsl.dataflow.memoization INFO: Task 2039 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc044a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2039 try 0 launched on executor multi with executor id 2040 parsl.dataflow.dflow INFO: Standard output for task 2039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/0d44eb83-ea90-4ac1-bcb4-1df66d95dd76_isrForCrosstalkSources_996995_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/0d44eb83-ea90-4ac1-bcb4-1df66d95dd76_isrForCrosstalkSources_996995_38.stderr parsl.dataflow.dflow DEBUG: Task 2040 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2040 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2040 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2040 has memoization hash 46cb17e222df259600a74b07e68fe1b9 parsl.dataflow.memoization INFO: Task 2040 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2040 try 0 launched on executor multi with executor id 2041 parsl.dataflow.dflow INFO: Standard output for task 2040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/83988cea-47fc-47a4-9509-fadc9ae77649_isrForCrosstalkSources_997015_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/83988cea-47fc-47a4-9509-fadc9ae77649_isrForCrosstalkSources_997015_62.stderr parsl.dataflow.dflow DEBUG: Task 2041 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2041 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2041 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2041 has memoization hash db7e09c0e8a07109b580571072a0e9bd parsl.dataflow.memoization INFO: Task 2041 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2041 try 0 launched on executor multi with executor id 2042 parsl.dataflow.dflow INFO: Standard output for task 2041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/13f528fc-3c1c-4896-9499-8ed6ef9c6eea_isrForCrosstalkSources_997013_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/13f528fc-3c1c-4896-9499-8ed6ef9c6eea_isrForCrosstalkSources_997013_5.stderr parsl.dataflow.dflow DEBUG: Task 2042 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2042 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2042 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2042 has memoization hash dab47e2c79a1364d915e86f8ea70c755 parsl.dataflow.memoization INFO: Task 2042 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc045e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2042 try 0 launched on executor multi with executor id 2043 parsl.dataflow.dflow INFO: Standard output for task 2042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/21960d2c-e651-47ba-8f85-52e30ed8f5e6_isrForCrosstalkSources_996981_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/21960d2c-e651-47ba-8f85-52e30ed8f5e6_isrForCrosstalkSources_996981_21.stderr parsl.dataflow.dflow DEBUG: Task 2043 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2043 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2043 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2043 has memoization hash d4a8ac1af4443de5a0cf4f7c8d67ec01 parsl.dataflow.memoization INFO: Task 2043 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2043 try 0 launched on executor multi with executor id 2044 parsl.dataflow.dflow INFO: Standard output for task 2043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/2e981cdf-025c-4b87-afea-749a81705787_isrForCrosstalkSources_997000_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/2e981cdf-025c-4b87-afea-749a81705787_isrForCrosstalkSources_997000_32.stderr parsl.dataflow.dflow DEBUG: Task 2044 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2044 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2044 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2044 has memoization hash a699849893e1634b0d16617f75d5282e parsl.dataflow.memoization INFO: Task 2044 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc047c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2044 try 0 launched on executor multi with executor id 2045 parsl.dataflow.dflow INFO: Standard output for task 2044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/deaa0f57-a09e-4205-abaf-e56809b2237c_isrForCrosstalkSources_996999_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/deaa0f57-a09e-4205-abaf-e56809b2237c_isrForCrosstalkSources_996999_54.stderr parsl.dataflow.dflow DEBUG: Task 2045 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2045 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2045 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2045 has memoization hash 42f72061c48af96505dcf772d6b5ee6a parsl.dataflow.memoization INFO: Task 2045 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2045 try 0 launched on executor multi with executor id 2046 parsl.dataflow.dflow INFO: Standard output for task 2045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d193d9a6-7623-401b-b08a-3092f29d32d2_isrForCrosstalkSources_996998_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d193d9a6-7623-401b-b08a-3092f29d32d2_isrForCrosstalkSources_996998_58.stderr parsl.dataflow.dflow DEBUG: Task 2046 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2046 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2046 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2046 has memoization hash 9453beb3557c96aedb84bfaa30288096 parsl.dataflow.memoization INFO: Task 2046 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2046 try 0 launched on executor multi with executor id 2047 parsl.dataflow.dflow INFO: Standard output for task 2046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/f5bc0dbc-73fd-43ca-b0fa-a7e816089556_isrForCrosstalkSources_996979_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/f5bc0dbc-73fd-43ca-b0fa-a7e816089556_isrForCrosstalkSources_996979_14.stderr parsl.dataflow.dflow DEBUG: Task 2047 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2047 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2047 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2047 has memoization hash dbfdb5b60795b849d0849412ec3148c4 parsl.dataflow.memoization INFO: Task 2047 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2047 try 0 launched on executor multi with executor id 2048 parsl.dataflow.dflow INFO: Standard output for task 2047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5132cef3-414c-4319-8e87-5c852b259b12_isrForCrosstalkSources_997003_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5132cef3-414c-4319-8e87-5c852b259b12_isrForCrosstalkSources_997003_49.stderr parsl.dataflow.dflow DEBUG: Task 2048 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2048 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2048 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2048 has memoization hash 81dcdf0ee21d157c086c1ad4f09e1fdb parsl.dataflow.memoization INFO: Task 2048 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc049a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2048 try 0 launched on executor multi with executor id 2049 parsl.dataflow.dflow INFO: Standard output for task 2048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/20f8338b-1db2-40c1-9854-f69c50d62de0_isrForCrosstalkSources_996963_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/20f8338b-1db2-40c1-9854-f69c50d62de0_isrForCrosstalkSources_996963_45.stderr parsl.dataflow.dflow DEBUG: Task 2049 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2049 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2049 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2049 has memoization hash 6bb1d60b5fe040bca8a0e6832b442f99 parsl.dataflow.memoization INFO: Task 2049 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2049 try 0 launched on executor multi with executor id 2050 parsl.dataflow.dflow INFO: Standard output for task 2049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/353e2760-3496-41f6-9aa3-ead0cf21a449_isrForCrosstalkSources_996960_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/353e2760-3496-41f6-9aa3-ead0cf21a449_isrForCrosstalkSources_996960_9.stderr parsl.dataflow.dflow DEBUG: Task 2050 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2050 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2050 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2050 has memoization hash 43f0411ae986017c74e0fd56ebe76917 parsl.dataflow.memoization INFO: Task 2050 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2050 try 0 launched on executor multi with executor id 2051 parsl.dataflow.dflow INFO: Standard output for task 2050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/5fb398ef-80c5-4711-9419-59623d23fb71_isrForCrosstalkSources_996973_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/5fb398ef-80c5-4711-9419-59623d23fb71_isrForCrosstalkSources_996973_5.stderr parsl.dataflow.dflow DEBUG: Task 2051 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2051 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2051 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2051 has memoization hash e948ea80e105eecc47374cb932201187 parsl.dataflow.memoization INFO: Task 2051 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2051 try 0 launched on executor multi with executor id 2052 parsl.dataflow.dflow INFO: Standard output for task 2051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d8ac8d6a-2092-47f6-872d-4e3a77ee742d_isrForCrosstalkSources_996998_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d8ac8d6a-2092-47f6-872d-4e3a77ee742d_isrForCrosstalkSources_996998_2.stderr parsl.dataflow.dflow DEBUG: Task 2052 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2052 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2052 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2052 has memoization hash e63589d48f9088d9b93c478d0795f2b2 parsl.dataflow.memoization INFO: Task 2052 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2052 try 0 launched on executor multi with executor id 2053 parsl.dataflow.dflow INFO: Standard output for task 2052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/4a939833-bc18-4c4f-8fb8-ac724c515195_isrForCrosstalkSources_996996_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/4a939833-bc18-4c4f-8fb8-ac724c515195_isrForCrosstalkSources_996996_20.stderr parsl.dataflow.dflow DEBUG: Task 2053 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2053 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2053 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2053 has memoization hash b294313b8e58fe6cecc951bc8a0e4c8a parsl.dataflow.memoization INFO: Task 2053 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2053 try 0 launched on executor multi with executor id 2054 parsl.dataflow.dflow INFO: Standard output for task 2053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60aa17cb-d23e-46fe-8e28-de1aebc07c1a_isrForCrosstalkSources_996968_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60aa17cb-d23e-46fe-8e28-de1aebc07c1a_isrForCrosstalkSources_996968_12.stderr parsl.dataflow.dflow DEBUG: Task 2054 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2054 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2054 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2054 has memoization hash d281b652cd396d3208a5a2aae7bfd8cf parsl.dataflow.memoization INFO: Task 2054 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2054 try 0 launched on executor multi with executor id 2055 parsl.dataflow.dflow INFO: Standard output for task 2054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/cfca7db3-3479-40ff-906a-317d9173abfb_isrForCrosstalkSources_997032_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/cfca7db3-3479-40ff-906a-317d9173abfb_isrForCrosstalkSources_997032_1.stderr parsl.dataflow.dflow DEBUG: Task 2055 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2055 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2055 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2055 has memoization hash f89427ca32e0635c9619db59be83cf92 parsl.dataflow.memoization INFO: Task 2055 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2055 try 0 launched on executor multi with executor id 2056 parsl.dataflow.dflow INFO: Standard output for task 2055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6852f9d9-4afc-43f5-ad87-5dc81b271465_isrForCrosstalkSources_996970_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6852f9d9-4afc-43f5-ad87-5dc81b271465_isrForCrosstalkSources_996970_27.stderr parsl.dataflow.dflow DEBUG: Task 2056 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2056 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2056 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2056 has memoization hash 330861c64a068b8dc9aca2c23ce3d175 parsl.dataflow.memoization INFO: Task 2056 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2056 try 0 launched on executor multi with executor id 2057 parsl.dataflow.dflow INFO: Standard output for task 2056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/eee86dbe-fd25-4582-97f8-b397ed9087ec_isrForCrosstalkSources_997028_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/eee86dbe-fd25-4582-97f8-b397ed9087ec_isrForCrosstalkSources_997028_34.stderr parsl.dataflow.dflow DEBUG: Task 2057 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2057 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2057 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2057 has memoization hash c3a232e3abbfbbac3b265d1ff6bf3183 parsl.dataflow.memoization INFO: Task 2057 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2057 try 0 launched on executor multi with executor id 2058 parsl.dataflow.dflow INFO: Standard output for task 2057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/b454fc62-57e0-4c08-80cb-78443a9353d0_isrForCrosstalkSources_997027_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/b454fc62-57e0-4c08-80cb-78443a9353d0_isrForCrosstalkSources_997027_4.stderr parsl.dataflow.dflow DEBUG: Task 2058 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2058 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2058 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2058 has memoization hash 7a9b3d264efbdd623808ebd822b52e2e parsl.dataflow.memoization INFO: Task 2058 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2058 try 0 launched on executor multi with executor id 2059 parsl.dataflow.dflow INFO: Standard output for task 2058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e04429c3-07e5-4e0f-9a42-0eeb22b9c391_isrForCrosstalkSources_997010_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e04429c3-07e5-4e0f-9a42-0eeb22b9c391_isrForCrosstalkSources_997010_47.stderr parsl.dataflow.dflow DEBUG: Task 2059 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2059 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2059 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2059 has memoization hash 5357026e9fdd707352be40ec2d31a481 parsl.dataflow.memoization INFO: Task 2059 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2059 try 0 launched on executor multi with executor id 2060 parsl.dataflow.dflow INFO: Standard output for task 2059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/218633cb-c11e-434e-a4c2-7c11230619cd_isrForCrosstalkSources_997013_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/218633cb-c11e-434e-a4c2-7c11230619cd_isrForCrosstalkSources_997013_1.stderr parsl.dataflow.dflow DEBUG: Task 2060 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2060 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2060 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2060 has memoization hash 04477160549c1a30f9aa502983d43093 parsl.dataflow.memoization INFO: Task 2060 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2060 try 0 launched on executor multi with executor id 2061 parsl.dataflow.dflow INFO: Standard output for task 2060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/60a64a85-d167-43db-8382-694d17781557_isrForCrosstalkSources_997008_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/60a64a85-d167-43db-8382-694d17781557_isrForCrosstalkSources_997008_4.stderr parsl.dataflow.dflow DEBUG: Task 2061 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2061 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2061 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2061 has memoization hash e27fe48612f6326e801e65e62b8a418a parsl.dataflow.memoization INFO: Task 2061 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc051c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2061 try 0 launched on executor multi with executor id 2062 parsl.dataflow.dflow INFO: Standard output for task 2061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2b4706d1-1eee-4ce5-bc14-9f49d5cacd14_isrForCrosstalkSources_996969_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2b4706d1-1eee-4ce5-bc14-9f49d5cacd14_isrForCrosstalkSources_996969_22.stderr parsl.dataflow.dflow DEBUG: Task 2062 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2062 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2062 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2062 has memoization hash 6250a43dc709a2fd9ce614f20fe9be4a parsl.dataflow.memoization INFO: Task 2062 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2062 try 0 launched on executor multi with executor id 2063 parsl.dataflow.dflow INFO: Standard output for task 2062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a384147c-573c-4972-99c2-db853ef76891_isrForCrosstalkSources_997018_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a384147c-573c-4972-99c2-db853ef76891_isrForCrosstalkSources_997018_41.stderr parsl.dataflow.dflow DEBUG: Task 2063 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2063 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2063 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2063 has memoization hash 842334b6e02bb9c58d658653747b9f6c parsl.dataflow.memoization INFO: Task 2063 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2063 try 0 launched on executor multi with executor id 2064 parsl.dataflow.dflow INFO: Standard output for task 2063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c2a76dbf-7a60-441c-a6e4-d564ec97133b_isrForCrosstalkSources_997006_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c2a76dbf-7a60-441c-a6e4-d564ec97133b_isrForCrosstalkSources_997006_11.stderr parsl.dataflow.dflow DEBUG: Task 2064 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2064 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2064 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2064 has memoization hash 3174533675ccbedfb7734c6f2e5a3acf parsl.dataflow.memoization INFO: Task 2064 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2064 try 0 launched on executor multi with executor id 2065 parsl.dataflow.dflow INFO: Standard output for task 2064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cd071930-647a-43ae-bfb7-4f77bb3916b3_isrForCrosstalkSources_996987_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cd071930-647a-43ae-bfb7-4f77bb3916b3_isrForCrosstalkSources_996987_54.stderr parsl.dataflow.dflow DEBUG: Task 2065 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2065 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2065 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2065 has memoization hash 4e0698783132ac91b8d13f4268925648 parsl.dataflow.memoization INFO: Task 2065 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc053a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2065 try 0 launched on executor multi with executor id 2066 parsl.dataflow.dflow INFO: Standard output for task 2065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/b550d3aa-2d81-4e2a-a907-46809332c2dc_isrForCrosstalkSources_997009_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/b550d3aa-2d81-4e2a-a907-46809332c2dc_isrForCrosstalkSources_997009_10.stderr parsl.dataflow.dflow DEBUG: Task 2066 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2066 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2066 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2066 has memoization hash d00d832fb30e09e34b3d09d45e94d539 parsl.dataflow.memoization INFO: Task 2066 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc054e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:46 proc_lsst.multi:170 [INFO] [multi] job_id 17074870 proc_lsst.multi INFO: [multi] job_id 17074870 2024-03-15 13:49:46 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 24 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 24 2024-03-15 13:49:46 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:46 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074870 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074870 parsl.executors.status_handling DEBUG: Launched block 24 on executor multi with job ID 17074870 parsl.executors.status_handling INFO: Allocated block ID 25 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-25 2024-03-15 13:49:46 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=25 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-25 parsl.dataflow.dflow INFO: Parsl task 2066 try 0 launched on executor multi with executor id 2067 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=25 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-25 2024-03-15 13:49:46 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=25 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 parsl.dataflow.dflow INFO: Standard output for task 2066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a13e2926-0edc-4419-bfd9-625e6b0f136a_isrForCrosstalkSources_996965_44.stdout proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=25 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 parsl.dataflow.dflow INFO: Standard error for task 2066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a13e2926-0edc-4419-bfd9-625e6b0f136a_isrForCrosstalkSources_996965_44.stderr proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 parsl.dataflow.dflow DEBUG: Task 2067 will be sent to executor multi proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:46 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=25 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn parsl.dataflow.dflow DEBUG: Adding output dependencies proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=25 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 24 parsl.dataflow.dflow INFO: Task 2067 submitted for App isrForCrosstalkSources, not waiting on any dependency proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 24 2024-03-15 13:49:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 parsl.dataflow.dflow DEBUG: Task 2067 set to pending state with AppFuture: proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:46 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=25 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=25 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.memoization DEBUG: Task 2067 has memoization hash f0d2c2fcc9d07250c87c871aeb0d32cc parsl.dataflow.memoization INFO: Task 2067 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bafc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 2067 try 0 launched on executor multi with executor id 2068 parsl.dataflow.dflow INFO: Standard output for task 2067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/1e913048-fceb-44a0-9c7b-baaa901d0f54_isrForCrosstalkSources_996996_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/1e913048-fceb-44a0-9c7b-baaa901d0f54_isrForCrosstalkSources_996996_6.stderr parsl.dataflow.dflow DEBUG: Task 2068 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2068 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2068 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2068 has memoization hash 13b34a27fd342a8e937c103ace46c721 parsl.dataflow.memoization INFO: Task 2068 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2068 try 0 launched on executor multi with executor id 2069 parsl.dataflow.dflow INFO: Standard output for task 2068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eeab50e-82f6-4a78-a076-1a9ed8ca60a3_isrForCrosstalkSources_996985_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eeab50e-82f6-4a78-a076-1a9ed8ca60a3_isrForCrosstalkSources_996985_12.stderr parsl.dataflow.dflow DEBUG: Task 2069 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2069 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2069 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2069 has memoization hash 06aa34e5f2ed84850317ade270d26389 parsl.dataflow.memoization INFO: Task 2069 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77e2bae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2069 try 0 launched on executor multi with executor id 2070 parsl.dataflow.dflow INFO: Standard output for task 2069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8039e204-d190-4f94-9ec6-8c975b98b7ab_isrForCrosstalkSources_996984_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8039e204-d190-4f94-9ec6-8c975b98b7ab_isrForCrosstalkSources_996984_39.stderr parsl.dataflow.dflow DEBUG: Task 2070 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2070 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2070 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2070 has memoization hash e8ae9f32dfa0f1bb756360e99f1f6c26 parsl.dataflow.memoization INFO: Task 2070 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2070 try 0 launched on executor multi with executor id 2071 parsl.dataflow.dflow INFO: Standard output for task 2070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/b23787fd-6c36-4b54-ae4b-89bea30fd99e_isrForCrosstalkSources_997029_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/b23787fd-6c36-4b54-ae4b-89bea30fd99e_isrForCrosstalkSources_997029_14.stderr parsl.dataflow.dflow DEBUG: Task 2071 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2071 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2071 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2071 has memoization hash 12566178ff19fcb54cd880819713f080 parsl.dataflow.memoization INFO: Task 2071 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2071 try 0 launched on executor multi with executor id 2072 parsl.dataflow.dflow INFO: Standard output for task 2071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f0a3fec3-3e87-4a7f-bb88-89159e796040_isrForCrosstalkSources_996957_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f0a3fec3-3e87-4a7f-bb88-89159e796040_isrForCrosstalkSources_996957_25.stderr parsl.dataflow.dflow DEBUG: Task 2072 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2072 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2072 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2072 has memoization hash 3ac063dcafc5fa14f207acc9dbfcc547 parsl.dataflow.memoization INFO: Task 2072 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc059e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2072 try 0 launched on executor multi with executor id 2073 parsl.dataflow.dflow INFO: Standard output for task 2072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/10b72434-a514-455b-b5ab-eb13ba488999_isrForCrosstalkSources_997002_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/10b72434-a514-455b-b5ab-eb13ba488999_isrForCrosstalkSources_997002_19.stderr parsl.dataflow.dflow DEBUG: Task 2073 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2073 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2073 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2073 has memoization hash f6c58dc59b79db053538824668256663 parsl.dataflow.memoization INFO: Task 2073 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2073 try 0 launched on executor multi with executor id 2074 parsl.dataflow.dflow INFO: Standard output for task 2073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/11ba2719-16e3-4062-aeb9-03e0196db163_isrForCrosstalkSources_996966_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/11ba2719-16e3-4062-aeb9-03e0196db163_isrForCrosstalkSources_996966_54.stderr parsl.dataflow.dflow DEBUG: Task 2074 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2074 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2074 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2074 has memoization hash d869d55433a23b01b88f1698a88c5630 parsl.dataflow.memoization INFO: Task 2074 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2074 try 0 launched on executor multi with executor id 2075 parsl.dataflow.dflow INFO: Standard output for task 2074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/9d2c7896-0c38-4e1f-97ca-c242b52e4359_isrForCrosstalkSources_996987_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/9d2c7896-0c38-4e1f-97ca-c242b52e4359_isrForCrosstalkSources_996987_19.stderr parsl.dataflow.dflow DEBUG: Task 2075 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2075 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2075 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2075 has memoization hash 497f6d109cc1c61709f5cf538cc5e7ae parsl.dataflow.memoization INFO: Task 2075 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2075 try 0 launched on executor multi with executor id 2076 parsl.dataflow.dflow INFO: Standard output for task 2075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/1327477d-d419-48e2-aa30-e0008fb2ed6a_isrForCrosstalkSources_996965_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/1327477d-d419-48e2-aa30-e0008fb2ed6a_isrForCrosstalkSources_996965_12.stderr parsl.dataflow.dflow DEBUG: Task 2076 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2076 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2076 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2076 has memoization hash 8454839e3c756c44492d979677c79aa1 parsl.dataflow.memoization INFO: Task 2076 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2076 try 0 launched on executor multi with executor id 2077 parsl.dataflow.dflow INFO: Standard output for task 2076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f4d3b35d-5cdd-42d0-af58-77583e7d1dcc_isrForCrosstalkSources_996957_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f4d3b35d-5cdd-42d0-af58-77583e7d1dcc_isrForCrosstalkSources_996957_2.stderr parsl.dataflow.dflow DEBUG: Task 2077 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2077 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2077 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2077 has memoization hash 1190e6ec21f46989d45a9f6b13f1a7df parsl.dataflow.memoization INFO: Task 2077 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2077 try 0 launched on executor multi with executor id 2078 parsl.dataflow.dflow INFO: Standard output for task 2077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/99582163-b280-48cf-addc-28e931ecedee_isrForCrosstalkSources_997030_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/99582163-b280-48cf-addc-28e931ecedee_isrForCrosstalkSources_997030_45.stderr parsl.dataflow.dflow DEBUG: Task 2078 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2078 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2078 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2078 has memoization hash 7e9bc8440625b0c069645ac39709a889 parsl.dataflow.memoization INFO: Task 2078 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2078 try 0 launched on executor multi with executor id 2079 parsl.dataflow.dflow INFO: Standard output for task 2078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/00234f11-cbed-4d31-b607-498b321fcb04_isrForCrosstalkSources_996965_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/00234f11-cbed-4d31-b607-498b321fcb04_isrForCrosstalkSources_996965_42.stderr parsl.dataflow.dflow DEBUG: Task 2079 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2079 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2079 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2079 has memoization hash f78b7aeee8be5f3f222e38710f96f019 parsl.dataflow.memoization INFO: Task 2079 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2079 try 0 launched on executor multi with executor id 2080 parsl.dataflow.dflow INFO: Standard output for task 2079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b9d01926-e52d-4aa7-98fd-01866023df37_isrForCrosstalkSources_997033_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b9d01926-e52d-4aa7-98fd-01866023df37_isrForCrosstalkSources_997033_35.stderr parsl.dataflow.dflow DEBUG: Task 2080 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2080 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2080 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2080 has memoization hash 8ccc4cc506cb641398cd7c85f2433b66 parsl.dataflow.memoization INFO: Task 2080 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2080 try 0 launched on executor multi with executor id 2081 parsl.dataflow.dflow INFO: Standard output for task 2080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e6623b08-77ec-40db-a367-aafef1b12d8d_isrForCrosstalkSources_997031_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e6623b08-77ec-40db-a367-aafef1b12d8d_isrForCrosstalkSources_997031_24.stderr parsl.dataflow.dflow DEBUG: Task 2081 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2081 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2081 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2081 has memoization hash 7abe43952bd589d80b9c6d1526416f6c parsl.dataflow.memoization INFO: Task 2081 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2081 try 0 launched on executor multi with executor id 2082 parsl.dataflow.dflow INFO: Standard output for task 2081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a3ea756-2f16-4cde-8bcf-9e404a264d8d_isrForCrosstalkSources_996994_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a3ea756-2f16-4cde-8bcf-9e404a264d8d_isrForCrosstalkSources_996994_7.stderr parsl.dataflow.dflow DEBUG: Task 2082 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2082 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2082 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2082 has memoization hash b567174062ff18dc446d6bff07b7f6e6 parsl.dataflow.memoization INFO: Task 2082 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2082 try 0 launched on executor multi with executor id 2083 parsl.dataflow.dflow INFO: Standard output for task 2082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8746edce-d51a-4ea9-a99c-87ed349dbf3f_isrForCrosstalkSources_997018_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8746edce-d51a-4ea9-a99c-87ed349dbf3f_isrForCrosstalkSources_997018_9.stderr parsl.dataflow.dflow DEBUG: Task 2083 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2083 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2083 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2083 has memoization hash e21e93226d5c127c68ac4d80a113bdd5 parsl.dataflow.memoization INFO: Task 2083 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2083 try 0 launched on executor multi with executor id 2084 parsl.dataflow.dflow INFO: Standard output for task 2083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b51a0ab0-f2f1-448d-965d-6748457e1016_isrForCrosstalkSources_997013_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b51a0ab0-f2f1-448d-965d-6748457e1016_isrForCrosstalkSources_997013_12.stderr parsl.dataflow.dflow DEBUG: Task 2084 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2084 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2084 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2084 has memoization hash 9fa4df54ed0ffb3adb1064722832474c parsl.dataflow.memoization INFO: Task 2084 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2084 try 0 launched on executor multi with executor id 2085 parsl.dataflow.dflow INFO: Standard output for task 2084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e9977d60-d4b1-4019-b174-ae2ca8cab3ad_isrForCrosstalkSources_996997_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e9977d60-d4b1-4019-b174-ae2ca8cab3ad_isrForCrosstalkSources_996997_19.stderr parsl.dataflow.dflow DEBUG: Task 2085 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2085 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2085 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2085 has memoization hash 85b472d194630daf4862ff1400ccf8b2 parsl.dataflow.memoization INFO: Task 2085 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc060c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2085 try 0 launched on executor multi with executor id 2086 parsl.dataflow.dflow INFO: Standard output for task 2085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c7975230-a0f3-46e6-aa67-e4f42296fd52_isrForCrosstalkSources_996997_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c7975230-a0f3-46e6-aa67-e4f42296fd52_isrForCrosstalkSources_996997_18.stderr parsl.dataflow.dflow DEBUG: Task 2086 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2086 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2086 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2086 has memoization hash 3120bfb1b49f0b78585a39ca63711ccb parsl.dataflow.memoization INFO: Task 2086 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc062a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2086 try 0 launched on executor multi with executor id 2087 parsl.dataflow.dflow INFO: Standard output for task 2086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/795ef32b-52e0-45dc-a9c8-b311b79e05b6_isrForCrosstalkSources_996975_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/795ef32b-52e0-45dc-a9c8-b311b79e05b6_isrForCrosstalkSources_996975_47.stderr parsl.dataflow.dflow DEBUG: Task 2087 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2087 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2087 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2087 has memoization hash 0fb2d66e328df5325c33c5c3c918b631 parsl.dataflow.memoization INFO: Task 2087 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2087 try 0 launched on executor multi with executor id 2088 parsl.dataflow.dflow INFO: Standard output for task 2087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7cef2bcc-c5e5-4e26-9c0e-63d783b3c368_isrForCrosstalkSources_997013_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7cef2bcc-c5e5-4e26-9c0e-63d783b3c368_isrForCrosstalkSources_997013_27.stderr parsl.dataflow.dflow DEBUG: Task 2088 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2088 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2088 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2088 has memoization hash 83a29af64b0835f8604757aec5fe2841 parsl.dataflow.memoization INFO: Task 2088 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2088 try 0 launched on executor multi with executor id 2089 parsl.dataflow.dflow INFO: Standard output for task 2088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8c2df2c9-d3a3-4f68-be8c-9c292b9c3962_isrForCrosstalkSources_997018_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8c2df2c9-d3a3-4f68-be8c-9c292b9c3962_isrForCrosstalkSources_997018_61.stderr parsl.dataflow.dflow DEBUG: Task 2089 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2089 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2089 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2089 has memoization hash dd0dd222ca7ccfc572f599989f67715d parsl.dataflow.memoization INFO: Task 2089 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2089 try 0 launched on executor multi with executor id 2090 parsl.dataflow.dflow INFO: Standard output for task 2089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/383c304d-7748-4632-9607-f96851408338_isrForCrosstalkSources_996965_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/383c304d-7748-4632-9607-f96851408338_isrForCrosstalkSources_996965_60.stderr parsl.dataflow.dflow DEBUG: Task 2090 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2090 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2090 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2090 has memoization hash 7d1897e7019d13a8dea5e10ef0766fce parsl.dataflow.memoization INFO: Task 2090 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc063e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2090 try 0 launched on executor multi with executor id 2091 parsl.dataflow.dflow INFO: Standard output for task 2090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/a5ec7064-e6fe-4373-ba81-7c7badb81923_isrForCrosstalkSources_997003_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/a5ec7064-e6fe-4373-ba81-7c7badb81923_isrForCrosstalkSources_997003_14.stderr parsl.dataflow.dflow DEBUG: Task 2091 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2091 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2091 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2091 has memoization hash 5a9b407d2cc4bdde331cd573212496e4 parsl.dataflow.memoization INFO: Task 2091 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc065c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2091 try 0 launched on executor multi with executor id 2092 parsl.dataflow.dflow INFO: Standard output for task 2091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/29e602aa-398f-453f-9a38-4da696078aec_isrForCrosstalkSources_996991_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/29e602aa-398f-453f-9a38-4da696078aec_isrForCrosstalkSources_996991_28.stderr parsl.dataflow.dflow DEBUG: Task 2092 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2092 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2092 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2092 has memoization hash 44b04ab420f4b2d92efe8e42a3643377 parsl.dataflow.memoization INFO: Task 2092 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2092 try 0 launched on executor multi with executor id 2093 parsl.dataflow.dflow INFO: Standard output for task 2092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1573376a-060e-4212-9ab8-42171479245a_isrForCrosstalkSources_997003_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1573376a-060e-4212-9ab8-42171479245a_isrForCrosstalkSources_997003_34.stderr parsl.dataflow.dflow DEBUG: Task 2093 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2093 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2093 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2093 has memoization hash d14703ac5232efb3cbd1934c74808a92 parsl.dataflow.memoization INFO: Task 2093 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2093 try 0 launched on executor multi with executor id 2094 parsl.dataflow.dflow INFO: Standard output for task 2093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/95c9bcc8-636d-4aec-8f5d-9b230ef15d6b_isrForCrosstalkSources_996963_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/95c9bcc8-636d-4aec-8f5d-9b230ef15d6b_isrForCrosstalkSources_996963_51.stderr parsl.dataflow.dflow DEBUG: Task 2094 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2094 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2094 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2094 has memoization hash 1dc9d94003dfa8def7b5f1d99d3738a4 parsl.dataflow.memoization INFO: Task 2094 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2094 try 0 launched on executor multi with executor id 2095 parsl.dataflow.dflow INFO: Standard output for task 2094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/0b70d21c-3acc-4e74-8df0-309501504165_isrForCrosstalkSources_997033_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/0b70d21c-3acc-4e74-8df0-309501504165_isrForCrosstalkSources_997033_13.stderr parsl.dataflow.dflow DEBUG: Task 2095 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2095 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2095 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2095 has memoization hash c162afbe2ca6d333915b311cf4524161 parsl.dataflow.memoization INFO: Task 2095 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc067a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2095 try 0 launched on executor multi with executor id 2096 parsl.dataflow.dflow INFO: Standard output for task 2095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/17de7cc2-7b51-4e23-91e1-927b45e9c3f9_isrForCrosstalkSources_997028_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/17de7cc2-7b51-4e23-91e1-927b45e9c3f9_isrForCrosstalkSources_997028_45.stderr parsl.dataflow.dflow DEBUG: Task 2096 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2096 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2096 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2096 has memoization hash 948bf9c131124dba1c0f18963a9f0017 parsl.dataflow.memoization INFO: Task 2096 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2096 try 0 launched on executor multi with executor id 2097 parsl.dataflow.dflow INFO: Standard output for task 2096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/e53e8533-bcc2-48b0-8be9-7682d862ef60_isrForCrosstalkSources_996981_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/e53e8533-bcc2-48b0-8be9-7682d862ef60_isrForCrosstalkSources_996981_19.stderr parsl.dataflow.dflow DEBUG: Task 2097 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2097 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2097 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2097 has memoization hash 39a1648d09cd27f02cc766950b819c0d parsl.dataflow.memoization INFO: Task 2097 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2097 try 0 launched on executor multi with executor id 2098 parsl.dataflow.dflow INFO: Standard output for task 2097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/31d2f3c8-86cd-403c-a460-dec538353331_isrForCrosstalkSources_996993_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/31d2f3c8-86cd-403c-a460-dec538353331_isrForCrosstalkSources_996993_27.stderr parsl.dataflow.dflow DEBUG: Task 2098 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2098 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2098 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2098 has memoization hash ec6b590730c48d3cca754c013d5d45f1 parsl.dataflow.memoization INFO: Task 2098 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc068e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2098 try 0 launched on executor multi with executor id 2099 parsl.dataflow.dflow INFO: Standard output for task 2098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/b372cf59-1da0-49a4-9bf4-1ded02d4e7d3_isrForCrosstalkSources_996968_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/b372cf59-1da0-49a4-9bf4-1ded02d4e7d3_isrForCrosstalkSources_996968_20.stderr parsl.dataflow.dflow DEBUG: Task 2099 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2099 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2099 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2099 has memoization hash 6458a28fc429d7fb3a7270e8a22cc406 parsl.dataflow.memoization INFO: Task 2099 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2099 try 0 launched on executor multi with executor id 2100 parsl.dataflow.dflow INFO: Standard output for task 2099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/aba25683-c5fe-4ad5-872d-ccd4cf17de4e_isrForCrosstalkSources_997010_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/aba25683-c5fe-4ad5-872d-ccd4cf17de4e_isrForCrosstalkSources_997010_24.stderr parsl.dataflow.dflow DEBUG: Task 2100 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2100 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2100 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2100 has memoization hash 86ab75496b71eb0e8d1e8ee8cb6cfd82 parsl.dataflow.memoization INFO: Task 2100 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2100 try 0 launched on executor multi with executor id 2101 parsl.dataflow.dflow INFO: Standard output for task 2100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/44ae0ece-daaa-4d9a-a771-12c0cfaf4840_isrForCrosstalkSources_997021_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/44ae0ece-daaa-4d9a-a771-12c0cfaf4840_isrForCrosstalkSources_997021_57.stderr parsl.dataflow.dflow DEBUG: Task 2101 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2101 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2101 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2101 has memoization hash 779873a430146843fda34c90a47875e7 parsl.dataflow.memoization INFO: Task 2101 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2101 try 0 launched on executor multi with executor id 2102 parsl.dataflow.dflow INFO: Standard output for task 2101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6622f1a9-cbb3-4e8a-b5a3-31e956f0c0c6_isrForCrosstalkSources_997003_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6622f1a9-cbb3-4e8a-b5a3-31e956f0c0c6_isrForCrosstalkSources_997003_54.stderr parsl.dataflow.dflow DEBUG: Task 2102 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2102 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2102 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2102 has memoization hash 04472939d075bf34ada51ccc498ce0c7 parsl.dataflow.memoization INFO: Task 2102 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2102 try 0 launched on executor multi with executor id 2103 parsl.dataflow.dflow INFO: Standard output for task 2102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/134964ff-76e8-4501-be0c-29d2000d0d29_isrForCrosstalkSources_997004_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/134964ff-76e8-4501-be0c-29d2000d0d29_isrForCrosstalkSources_997004_58.stderr parsl.dataflow.dflow DEBUG: Task 2103 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2103 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2103 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2103 has memoization hash 4046446424005210e33ddde2dd4d72dd parsl.dataflow.memoization INFO: Task 2103 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2103 try 0 launched on executor multi with executor id 2104 parsl.dataflow.dflow INFO: Standard output for task 2103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/035d01b3-c4cd-41e6-95b7-ccd13be9a608_isrForCrosstalkSources_996959_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/035d01b3-c4cd-41e6-95b7-ccd13be9a608_isrForCrosstalkSources_996959_39.stderr parsl.dataflow.dflow DEBUG: Task 2104 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2104 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2104 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2104 has memoization hash 5e3086ce81b47ef20837a5fdc6f0b491 parsl.dataflow.memoization INFO: Task 2104 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2104 try 0 launched on executor multi with executor id 2105 parsl.dataflow.dflow INFO: Standard output for task 2104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b74af192-0fa8-4ac2-ab5c-29b91db5deff_isrForCrosstalkSources_996992_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b74af192-0fa8-4ac2-ab5c-29b91db5deff_isrForCrosstalkSources_996992_32.stderr parsl.dataflow.dflow DEBUG: Task 2105 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2105 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2105 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2105 has memoization hash df598dd034375c7fdf9661c31cf4bd73 parsl.dataflow.memoization INFO: Task 2105 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2105 try 0 launched on executor multi with executor id 2106 parsl.dataflow.dflow INFO: Standard output for task 2105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/77633662-67ee-41cf-b187-fdd63a3a26a8_isrForCrosstalkSources_996985_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/77633662-67ee-41cf-b187-fdd63a3a26a8_isrForCrosstalkSources_996985_5.stderr parsl.dataflow.dflow DEBUG: Task 2106 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2106 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2106 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2106 has memoization hash 2d4c1bae2ac45425478ef71b48868cd3 parsl.dataflow.memoization INFO: Task 2106 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2106 try 0 launched on executor multi with executor id 2107 parsl.dataflow.dflow INFO: Standard output for task 2106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/840482be-b198-4b32-b82f-852c54f7db90_isrForCrosstalkSources_996953_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/840482be-b198-4b32-b82f-852c54f7db90_isrForCrosstalkSources_996953_9.stderr parsl.dataflow.dflow DEBUG: Task 2107 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2107 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2107 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2107 has memoization hash 7446cde50c010adca252d5db5c668578 parsl.dataflow.memoization INFO: Task 2107 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2107 try 0 launched on executor multi with executor id 2108 parsl.dataflow.dflow INFO: Standard output for task 2107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5ed3c61c-02d9-42a7-a586-fa1c7e509ff8_isrForCrosstalkSources_996955_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5ed3c61c-02d9-42a7-a586-fa1c7e509ff8_isrForCrosstalkSources_996955_47.stderr parsl.dataflow.dflow DEBUG: Task 2108 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2108 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2108 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2108 has memoization hash b170325de9bb7ae35aa0d3d0386a27c3 parsl.dataflow.memoization INFO: Task 2108 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc06e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2108 try 0 launched on executor multi with executor id 2109 parsl.dataflow.dflow INFO: Standard output for task 2108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c5963900-7749-4a1e-997f-15784dc26432_isrForCrosstalkSources_996962_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c5963900-7749-4a1e-997f-15784dc26432_isrForCrosstalkSources_996962_38.stderr parsl.dataflow.dflow DEBUG: Task 2109 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2109 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2109 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2109 has memoization hash daa3f03fad58f7ed895739f5133bf1e1 parsl.dataflow.memoization INFO: Task 2109 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc071a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2109 try 0 launched on executor multi with executor id 2110 parsl.dataflow.dflow INFO: Standard output for task 2109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c05b2e8f-88d8-4d43-81ad-25862f71cb14_isrForCrosstalkSources_997003_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c05b2e8f-88d8-4d43-81ad-25862f71cb14_isrForCrosstalkSources_997003_27.stderr parsl.dataflow.dflow DEBUG: Task 2110 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2110 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2110 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2110 has memoization hash 50dfecb1ae57672a8831052dd208a0b7 parsl.dataflow.memoization INFO: Task 2110 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2110 try 0 launched on executor multi with executor id 2111 parsl.dataflow.dflow INFO: Standard output for task 2110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/b24fc94b-9752-4e70-b288-06fad4f6c1c1_isrForCrosstalkSources_996990_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/b24fc94b-9752-4e70-b288-06fad4f6c1c1_isrForCrosstalkSources_996990_56.stderr parsl.dataflow.dflow DEBUG: Task 2111 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2111 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2111 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2111 has memoization hash 27170c60d23c6f4b657a1fb53aa18aa1 parsl.dataflow.memoization INFO: Task 2111 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2111 try 0 launched on executor multi with executor id 2112 parsl.dataflow.dflow INFO: Standard output for task 2111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/adc6ffbf-0757-4042-b90d-009831bb3333_isrForCrosstalkSources_996957_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/adc6ffbf-0757-4042-b90d-009831bb3333_isrForCrosstalkSources_996957_54.stderr parsl.dataflow.dflow DEBUG: Task 2112 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2112 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2112 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2112 has memoization hash 2428998caeec4cd6d028d2b094ebd9f7 parsl.dataflow.memoization INFO: Task 2112 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2112 try 0 launched on executor multi with executor id 2113 parsl.dataflow.dflow INFO: Standard output for task 2112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a5cd789e-00a3-42c8-926a-741dfe6e1f0a_isrForCrosstalkSources_997027_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a5cd789e-00a3-42c8-926a-741dfe6e1f0a_isrForCrosstalkSources_997027_7.stderr parsl.dataflow.dflow DEBUG: Task 2113 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2113 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2113 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2113 has memoization hash b7098313913473cccd5965e830ed3036 parsl.dataflow.memoization INFO: Task 2113 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2113 try 0 launched on executor multi with executor id 2114 parsl.dataflow.dflow INFO: Standard output for task 2113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/687b9fea-c9cf-46f0-9f2b-60366f337de8_isrForCrosstalkSources_996955_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/687b9fea-c9cf-46f0-9f2b-60366f337de8_isrForCrosstalkSources_996955_38.stderr parsl.dataflow.dflow DEBUG: Task 2114 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2114 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2114 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2114 has memoization hash e412d9f364a37541098c0bd2db1c636f parsl.dataflow.memoization INFO: Task 2114 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc074c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2114 try 0 launched on executor multi with executor id 2115 parsl.dataflow.dflow INFO: Standard output for task 2114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/14baf94a-4f96-482d-8526-5647d45ff41d_isrForCrosstalkSources_997012_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/14baf94a-4f96-482d-8526-5647d45ff41d_isrForCrosstalkSources_997012_27.stderr parsl.dataflow.dflow DEBUG: Task 2115 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2115 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2115 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2115 has memoization hash 042ba42001f3f3ae8cfc4ed7d118902d parsl.dataflow.memoization INFO: Task 2115 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2115 try 0 launched on executor multi with executor id 2116 parsl.dataflow.dflow INFO: Standard output for task 2115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8cb23279-0ca3-45bc-b96a-0ce5fbe30793_isrForCrosstalkSources_997018_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8cb23279-0ca3-45bc-b96a-0ce5fbe30793_isrForCrosstalkSources_997018_40.stderr parsl.dataflow.dflow DEBUG: Task 2116 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2116 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2116 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2116 has memoization hash 1ff92624180d1434c28fd67164ccbbb8 parsl.dataflow.memoization INFO: Task 2116 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc072e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2116 try 0 launched on executor multi with executor id 2117 parsl.dataflow.dflow INFO: Standard output for task 2116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/312e381f-27df-410a-bd43-a05eb854ad3a_isrForCrosstalkSources_996972_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/312e381f-27df-410a-bd43-a05eb854ad3a_isrForCrosstalkSources_996972_32.stderr parsl.dataflow.dflow DEBUG: Task 2117 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2117 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2117 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2117 has memoization hash 5841ee83905c5d049c47496d77bae99b parsl.dataflow.memoization INFO: Task 2117 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2117 try 0 launched on executor multi with executor id 2118 parsl.dataflow.dflow INFO: Standard output for task 2117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/8754712c-4e41-4500-8697-6a2bee5c8d79_isrForCrosstalkSources_997031_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/8754712c-4e41-4500-8697-6a2bee5c8d79_isrForCrosstalkSources_997031_32.stderr parsl.dataflow.dflow DEBUG: Task 2118 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2118 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2118 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2118 has memoization hash 68b0df7921f4c5f786ba8a8e799ff372 parsl.dataflow.memoization INFO: Task 2118 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2118 try 0 launched on executor multi with executor id 2119 parsl.dataflow.dflow INFO: Standard output for task 2118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cc660a0c-c3cd-4945-be5f-79cee0696b44_isrForCrosstalkSources_996983_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cc660a0c-c3cd-4945-be5f-79cee0696b44_isrForCrosstalkSources_996983_48.stderr parsl.dataflow.dflow DEBUG: Task 2119 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2119 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2119 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2119 has memoization hash b9206d75b5bb6c53eab0d06d819b4eb4 parsl.dataflow.memoization INFO: Task 2119 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2119 try 0 launched on executor multi with executor id 2120 parsl.dataflow.dflow INFO: Standard output for task 2119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b79e850b-b9ea-4ac4-96f6-5ddb8588df6b_isrForCrosstalkSources_996963_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b79e850b-b9ea-4ac4-96f6-5ddb8588df6b_isrForCrosstalkSources_996963_7.stderr parsl.dataflow.dflow DEBUG: Task 2120 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2120 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2120 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2120 has memoization hash 5a753d5261fad4a62a74059bbcd245b0 parsl.dataflow.memoization INFO: Task 2120 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2120 try 0 launched on executor multi with executor id 2121 parsl.dataflow.dflow INFO: Standard output for task 2120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d18f913c-86a9-4908-9d14-56819c4fed45_isrForCrosstalkSources_996988_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d18f913c-86a9-4908-9d14-56819c4fed45_isrForCrosstalkSources_996988_55.stderr parsl.dataflow.dflow DEBUG: Task 2121 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2121 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2121 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2121 has memoization hash 9a59279dd131766c10c5d88a1c612958 parsl.dataflow.memoization INFO: Task 2121 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2121 try 0 launched on executor multi with executor id 2122 parsl.dataflow.dflow INFO: Standard output for task 2121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/18529793-fec3-454a-9b1e-61a87e198b68_isrForCrosstalkSources_997006_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/18529793-fec3-454a-9b1e-61a87e198b68_isrForCrosstalkSources_997006_59.stderr parsl.dataflow.dflow DEBUG: Task 2122 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2122 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2122 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2122 has memoization hash 03abaeeec3e9f6c46ee4b7d90c53aece parsl.dataflow.memoization INFO: Task 2122 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2122 try 0 launched on executor multi with executor id 2123 parsl.dataflow.dflow INFO: Standard output for task 2122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9cc69a81-4dba-4a6c-9848-bd418fe76240_isrForCrosstalkSources_996971_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9cc69a81-4dba-4a6c-9848-bd418fe76240_isrForCrosstalkSources_996971_8.stderr parsl.dataflow.dflow DEBUG: Task 2123 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2123 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2123 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2123 has memoization hash 8ec421073f0fbeb86363f5209c56f146 parsl.dataflow.memoization INFO: Task 2123 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2123 try 0 launched on executor multi with executor id 2124 parsl.dataflow.dflow INFO: Standard output for task 2123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/7cc35d3a-e44a-4296-8e69-19e1e22aa0fd_isrForCrosstalkSources_997020_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/7cc35d3a-e44a-4296-8e69-19e1e22aa0fd_isrForCrosstalkSources_997020_37.stderr parsl.dataflow.dflow DEBUG: Task 2124 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2124 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2124 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2124 has memoization hash 64ff4a005831b3ca70de02093967506d parsl.dataflow.memoization INFO: Task 2124 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2124 try 0 launched on executor multi with executor id 2125 parsl.dataflow.dflow INFO: Standard output for task 2124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e5a40d57-7657-490a-8316-21f365596d4d_isrForCrosstalkSources_996989_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e5a40d57-7657-490a-8316-21f365596d4d_isrForCrosstalkSources_996989_53.stderr parsl.dataflow.dflow DEBUG: Task 2125 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2125 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2125 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2125 has memoization hash 33b29cd51d2457dfa758e0e31e6c7a0a parsl.dataflow.memoization INFO: Task 2125 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2125 try 0 launched on executor multi with executor id 2126 parsl.dataflow.dflow INFO: Standard output for task 2125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/79c027be-4d97-4ca3-bfdd-059fecd950a9_isrForCrosstalkSources_996964_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/79c027be-4d97-4ca3-bfdd-059fecd950a9_isrForCrosstalkSources_996964_24.stderr parsl.dataflow.dflow DEBUG: Task 2126 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2126 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2126 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2126 has memoization hash d54bb87c73e904498830fbbe185848f8 parsl.dataflow.memoization INFO: Task 2126 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2126 try 0 launched on executor multi with executor id 2127 parsl.dataflow.dflow INFO: Standard output for task 2126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e89bd236-0292-46d5-b5b1-857eaefe7f08_isrForCrosstalkSources_996952_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e89bd236-0292-46d5-b5b1-857eaefe7f08_isrForCrosstalkSources_996952_12.stderr parsl.dataflow.dflow DEBUG: Task 2127 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2127 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2127 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2127 has memoization hash 1c366d2022d349bff334a80df55e6bef parsl.dataflow.memoization INFO: Task 2127 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2127 try 0 launched on executor multi with executor id 2128 parsl.dataflow.dflow INFO: Standard output for task 2127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b34a2079-9be2-4e6b-b9ef-65945d874266_isrForCrosstalkSources_996998_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b34a2079-9be2-4e6b-b9ef-65945d874266_isrForCrosstalkSources_996998_26.stderr parsl.dataflow.dflow DEBUG: Task 2128 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2128 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2128 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2128 has memoization hash b10a64c2fb0330060bf8b5a5701f420f parsl.dataflow.memoization INFO: Task 2128 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2128 try 0 launched on executor multi with executor id 2129 parsl.dataflow.dflow INFO: Standard output for task 2128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/65dcf141-e3f9-46f0-9f37-70190fe67e54_isrForCrosstalkSources_997007_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/65dcf141-e3f9-46f0-9f37-70190fe67e54_isrForCrosstalkSources_997007_51.stderr parsl.dataflow.dflow DEBUG: Task 2129 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2129 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2129 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2129 has memoization hash 4c059a48516dd682f92537c4017ec2d5 parsl.dataflow.memoization INFO: Task 2129 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2129 try 0 launched on executor multi with executor id 2130 parsl.dataflow.dflow INFO: Standard output for task 2129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/0455eab6-59e6-478c-ab4c-e14d90e17990_isrForCrosstalkSources_996969_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/0455eab6-59e6-478c-ab4c-e14d90e17990_isrForCrosstalkSources_996969_19.stderr parsl.dataflow.dflow DEBUG: Task 2130 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2130 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2130 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2130 has memoization hash 6763ae7ea6baa7ed010a0a85683184fe parsl.dataflow.memoization INFO: Task 2130 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc079c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2130 try 0 launched on executor multi with executor id 2131 parsl.dataflow.dflow INFO: Standard output for task 2130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b32e5af8-b67f-4643-a7a2-8ae1b65703ac_isrForCrosstalkSources_996963_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b32e5af8-b67f-4643-a7a2-8ae1b65703ac_isrForCrosstalkSources_996963_32.stderr parsl.dataflow.dflow DEBUG: Task 2131 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2131 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2131 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2131 has memoization hash 612a21bb2080b52fc17292966d093460 parsl.dataflow.memoization INFO: Task 2131 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc07880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2131 try 0 launched on executor multi with executor id 2132 parsl.dataflow.dflow INFO: Standard output for task 2131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/afe28813-1fb9-4275-b269-13024403a21f_isrForCrosstalkSources_997001_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/afe28813-1fb9-4275-b269-13024403a21f_isrForCrosstalkSources_997001_9.stderr parsl.dataflow.dflow DEBUG: Task 2132 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2132 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2132 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2132 has memoization hash 502e1dc6af29d441820aa34dbc870251 parsl.dataflow.memoization INFO: Task 2132 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc077e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2132 try 0 launched on executor multi with executor id 2133 parsl.dataflow.dflow INFO: Standard output for task 2132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a3a2d3b2-2682-4abd-ab69-892619efdfe6_isrForCrosstalkSources_997001_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a3a2d3b2-2682-4abd-ab69-892619efdfe6_isrForCrosstalkSources_997001_24.stderr parsl.dataflow.dflow DEBUG: Task 2133 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2133 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2133 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2133 has memoization hash 0f87661a5ddb0791ac334fadf6deda1c parsl.dataflow.memoization INFO: Task 2133 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc076a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2133 try 0 launched on executor multi with executor id 2134 parsl.dataflow.dflow INFO: Standard output for task 2133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9be69c0e-f115-4132-884f-5e0007942c53_isrForCrosstalkSources_996988_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9be69c0e-f115-4132-884f-5e0007942c53_isrForCrosstalkSources_996988_39.stderr parsl.dataflow.dflow DEBUG: Task 2134 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2134 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2134 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2134 has memoization hash 25534cecb0a6da34e44c0ba481681264 parsl.dataflow.memoization INFO: Task 2134 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc058a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2134 try 0 launched on executor multi with executor id 2135 parsl.dataflow.dflow INFO: Standard output for task 2134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a3bff4f6-d131-46dd-8e41-74a0d2b7fefa_isrForCrosstalkSources_996991_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a3bff4f6-d131-46dd-8e41-74a0d2b7fefa_isrForCrosstalkSources_996991_41.stderr parsl.dataflow.dflow DEBUG: Task 2135 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2135 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2135 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2135 has memoization hash 6773c85e02beae9ac4a5445b9ac74d7f parsl.dataflow.memoization INFO: Task 2135 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2135 try 0 launched on executor multi with executor id 2136 parsl.dataflow.dflow INFO: Standard output for task 2135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a0a27864-897a-4887-8d68-1209e51771ab_isrForCrosstalkSources_997021_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a0a27864-897a-4887-8d68-1209e51771ab_isrForCrosstalkSources_997021_37.stderr parsl.dataflow.dflow DEBUG: Task 2136 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2136 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2136 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2136 has memoization hash 8168c36f7777956594ea40e283c4c375 parsl.dataflow.memoization INFO: Task 2136 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2136 try 0 launched on executor multi with executor id 2137 parsl.dataflow.dflow INFO: Standard output for task 2136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1fb17c38-d822-4ba2-aac1-544eab1858fa_isrForCrosstalkSources_997019_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1fb17c38-d822-4ba2-aac1-544eab1858fa_isrForCrosstalkSources_997019_43.stderr parsl.dataflow.dflow DEBUG: Task 2137 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2137 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2137 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2137 has memoization hash cb8668b4e6825ca9f4c1965384798a70 parsl.dataflow.memoization INFO: Task 2137 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2137 try 0 launched on executor multi with executor id 2138 parsl.dataflow.dflow INFO: Standard output for task 2137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/6d4f6ec3-f404-425b-98e7-dee6eac8a322_isrForCrosstalkSources_996965_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/6d4f6ec3-f404-425b-98e7-dee6eac8a322_isrForCrosstalkSources_996965_2.stderr parsl.dataflow.dflow DEBUG: Task 2138 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2138 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2138 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2138 has memoization hash 5dd9d7e371bde867b2614fff63b1b2ce parsl.dataflow.memoization INFO: Task 2138 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2138 try 0 launched on executor multi with executor id 2139 parsl.dataflow.dflow INFO: Standard output for task 2138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d0d2a3a2-b9dc-4636-b1b7-e3015dab8433_isrForCrosstalkSources_997025_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d0d2a3a2-b9dc-4636-b1b7-e3015dab8433_isrForCrosstalkSources_997025_50.stderr parsl.dataflow.dflow DEBUG: Task 2139 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2139 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2139 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2139 has memoization hash 989c3cdda2d06578e4f32d16c6c3cda4 parsl.dataflow.memoization INFO: Task 2139 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2139 try 0 launched on executor multi with executor id 2140 parsl.dataflow.dflow INFO: Standard output for task 2139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a7b195f9-3bfb-45bc-89b2-0a2297e6080e_isrForCrosstalkSources_997007_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a7b195f9-3bfb-45bc-89b2-0a2297e6080e_isrForCrosstalkSources_997007_46.stderr parsl.dataflow.dflow DEBUG: Task 2140 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2140 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2140 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2140 has memoization hash cd1eafdbd78b4a4b707f7ac23e67ed37 parsl.dataflow.memoization INFO: Task 2140 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2140 try 0 launched on executor multi with executor id 2141 parsl.dataflow.dflow INFO: Standard output for task 2140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6e46ca47-035e-41a9-a516-5f0214dca8c1_isrForCrosstalkSources_996971_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6e46ca47-035e-41a9-a516-5f0214dca8c1_isrForCrosstalkSources_996971_61.stderr parsl.dataflow.dflow DEBUG: Task 2141 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2141 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2141 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2141 has memoization hash d33bbeec8e2b42893e9937ed3365d1ea parsl.dataflow.memoization INFO: Task 2141 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2141 try 0 launched on executor multi with executor id 2142 parsl.dataflow.dflow INFO: Standard output for task 2141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/3d4da5ba-f534-4b7b-ab09-cc35d56dda45_isrForCrosstalkSources_997020_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/3d4da5ba-f534-4b7b-ab09-cc35d56dda45_isrForCrosstalkSources_997020_12.stderr parsl.dataflow.dflow DEBUG: Task 2142 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2142 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2142 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2142 has memoization hash 11e54406481ef0f5989f10ed2eda25c8 parsl.dataflow.memoization INFO: Task 2142 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2142 try 0 launched on executor multi with executor id 2143 parsl.dataflow.dflow INFO: Standard output for task 2142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/f37e1de5-8b18-429c-8085-b34b5a8c2113_isrForCrosstalkSources_997029_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/f37e1de5-8b18-429c-8085-b34b5a8c2113_isrForCrosstalkSources_997029_10.stderr parsl.dataflow.dflow DEBUG: Task 2143 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2143 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2143 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2143 has memoization hash a8ac2248649de22c4f6687b6647c412b parsl.dataflow.memoization INFO: Task 2143 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2143 try 0 launched on executor multi with executor id 2144 parsl.dataflow.dflow INFO: Standard output for task 2143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/683395e9-6c88-414b-be83-244a57cdabbb_isrForCrosstalkSources_997019_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/683395e9-6c88-414b-be83-244a57cdabbb_isrForCrosstalkSources_997019_36.stderr parsl.dataflow.dflow DEBUG: Task 2144 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2144 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2144 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2144 has memoization hash 8542f617280ffcd58ade0d0a4169c15a parsl.dataflow.memoization INFO: Task 2144 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2144 try 0 launched on executor multi with executor id 2145 parsl.dataflow.dflow INFO: Standard output for task 2144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/f78cea55-81f2-4cf5-b04c-d449daf0ce52_isrForCrosstalkSources_996969_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/f78cea55-81f2-4cf5-b04c-d449daf0ce52_isrForCrosstalkSources_996969_24.stderr parsl.dataflow.dflow DEBUG: Task 2145 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2145 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2145 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2145 has memoization hash ed84be43cad7a36a0e27f248754f6081 parsl.dataflow.memoization INFO: Task 2145 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2145 try 0 launched on executor multi with executor id 2146 parsl.dataflow.dflow INFO: Standard output for task 2145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f2141cb5-2b6e-47f5-9663-840f60117665_isrForCrosstalkSources_997017_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f2141cb5-2b6e-47f5-9663-840f60117665_isrForCrosstalkSources_997017_59.stderr parsl.dataflow.dflow DEBUG: Task 2146 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2146 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2146 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2146 has memoization hash 915f60ed797b7fe9fc83fa41d8feaf22 parsl.dataflow.memoization INFO: Task 2146 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2146 try 0 launched on executor multi with executor id 2147 parsl.dataflow.dflow INFO: Standard output for task 2146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/822daef6-7645-43a3-8e47-e570e7c59916_isrForCrosstalkSources_997020_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/822daef6-7645-43a3-8e47-e570e7c59916_isrForCrosstalkSources_997020_59.stderr parsl.dataflow.dflow DEBUG: Task 2147 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2147 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2147 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2147 has memoization hash 641d8d80a7a3b751f084016672f409ea parsl.dataflow.memoization INFO: Task 2147 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2147 try 0 launched on executor multi with executor id 2148 parsl.dataflow.dflow INFO: Standard output for task 2147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/88141355-6069-485d-a329-395cdf2f7356_isrForCrosstalkSources_996973_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/88141355-6069-485d-a329-395cdf2f7356_isrForCrosstalkSources_996973_13.stderr parsl.dataflow.dflow DEBUG: Task 2148 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2148 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2148 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2148 has memoization hash 9236ae806033164002b9fbad6ef9dc28 parsl.dataflow.memoization INFO: Task 2148 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2148 try 0 launched on executor multi with executor id 2149 parsl.dataflow.dflow INFO: Standard output for task 2148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/49565dec-5553-4542-b1c8-321c83b20c46_isrForCrosstalkSources_996998_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/49565dec-5553-4542-b1c8-321c83b20c46_isrForCrosstalkSources_996998_60.stderr parsl.dataflow.dflow DEBUG: Task 2149 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2149 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2149 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2149 has memoization hash 7afc01c014f5a1e124547b0ff4a6117a parsl.dataflow.memoization INFO: Task 2149 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2149 try 0 launched on executor multi with executor id 2150 parsl.dataflow.dflow INFO: Standard output for task 2149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b58ddf98-08bd-4a37-b334-724921c8e486_isrForCrosstalkSources_996969_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b58ddf98-08bd-4a37-b334-724921c8e486_isrForCrosstalkSources_996969_36.stderr parsl.dataflow.dflow DEBUG: Task 2150 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2150 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2150 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2150 has memoization hash bfa987d8f436f56981b446427d74c9aa parsl.dataflow.memoization INFO: Task 2150 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2150 try 0 launched on executor multi with executor id 2151 parsl.dataflow.dflow INFO: Standard output for task 2150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/03c7466f-09b1-4bc2-a9a7-a13b72f0585f_isrForCrosstalkSources_997007_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/03c7466f-09b1-4bc2-a9a7-a13b72f0585f_isrForCrosstalkSources_997007_21.stderr parsl.dataflow.dflow DEBUG: Task 2151 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2151 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2151 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2151 has memoization hash 2c2819a246b6775e20dcfe4087652942 parsl.dataflow.memoization INFO: Task 2151 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2151 try 0 launched on executor multi with executor id 2152 parsl.dataflow.dflow INFO: Standard output for task 2151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/82fcd16d-3479-47e7-aa9d-8a099c12f722_isrForCrosstalkSources_996969_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/82fcd16d-3479-47e7-aa9d-8a099c12f722_isrForCrosstalkSources_996969_30.stderr parsl.dataflow.dflow DEBUG: Task 2152 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2152 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2152 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2152 has memoization hash e36187ef85854a8946b81bf3f2375b30 parsl.dataflow.memoization INFO: Task 2152 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2152 try 0 launched on executor multi with executor id 2153 parsl.dataflow.dflow INFO: Standard output for task 2152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3129e9e4-0eb1-4e15-bad5-b0e03bf845c7_isrForCrosstalkSources_996984_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3129e9e4-0eb1-4e15-bad5-b0e03bf845c7_isrForCrosstalkSources_996984_24.stderr parsl.dataflow.dflow DEBUG: Task 2153 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2153 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2153 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2153 has memoization hash 03ed02fc4ae591ee0cf97fbb9428caf8 parsl.dataflow.memoization INFO: Task 2153 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2153 try 0 launched on executor multi with executor id 2154 parsl.dataflow.dflow INFO: Standard output for task 2153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/57fdc729-8b0e-4f85-9282-5b8eb434ba3e_isrForCrosstalkSources_997007_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/57fdc729-8b0e-4f85-9282-5b8eb434ba3e_isrForCrosstalkSources_997007_11.stderr parsl.dataflow.dflow DEBUG: Task 2154 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2154 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2154 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2154 has memoization hash 1e47d45efe1c627ce1f976d0b27e6e1e parsl.dataflow.memoization INFO: Task 2154 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2154 try 0 launched on executor multi with executor id 2155 parsl.dataflow.dflow INFO: Standard output for task 2154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/36c00b91-5bb1-41e1-94f7-981bf7d4276b_isrForCrosstalkSources_997023_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/36c00b91-5bb1-41e1-94f7-981bf7d4276b_isrForCrosstalkSources_997023_11.stderr parsl.dataflow.dflow DEBUG: Task 2155 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2155 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2155 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2155 has memoization hash 8c057b48e34a28f17950cb530b7f6bc3 parsl.dataflow.memoization INFO: Task 2155 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2155 try 0 launched on executor multi with executor id 2156 parsl.dataflow.dflow INFO: Standard output for task 2155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/6bf944b6-2ed9-4d43-9038-62e0fec84c59_isrForCrosstalkSources_997024_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/6bf944b6-2ed9-4d43-9038-62e0fec84c59_isrForCrosstalkSources_997024_52.stderr parsl.dataflow.dflow DEBUG: Task 2156 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2156 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2156 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2156 has memoization hash a2c20a31c73143b45810b069dc698985 parsl.dataflow.memoization INFO: Task 2156 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2156 try 0 launched on executor multi with executor id 2157 parsl.dataflow.dflow INFO: Standard output for task 2156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/752dd4d5-4cf7-46a9-ace3-83e130580128_isrForCrosstalkSources_996960_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/752dd4d5-4cf7-46a9-ace3-83e130580128_isrForCrosstalkSources_996960_34.stderr parsl.dataflow.dflow DEBUG: Task 2157 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2157 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2157 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2157 has memoization hash 783c8cf300f7979e4770d2a37c1d339e parsl.dataflow.memoization INFO: Task 2157 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2157 try 0 launched on executor multi with executor id 2158 parsl.dataflow.dflow INFO: Standard output for task 2157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/3e0a74ee-d6e8-46ec-87bf-6b8f862cff27_isrForCrosstalkSources_996977_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/3e0a74ee-d6e8-46ec-87bf-6b8f862cff27_isrForCrosstalkSources_996977_44.stderr parsl.dataflow.dflow DEBUG: Task 2158 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2158 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2158 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2158 has memoization hash c7d19dd99f55eb16a5fa1816eff3b971 parsl.dataflow.memoization INFO: Task 2158 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2158 try 0 launched on executor multi with executor id 2159 parsl.dataflow.dflow INFO: Standard output for task 2158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/de047ed7-2711-494f-905a-f7b2537f6bef_isrForCrosstalkSources_996958_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/de047ed7-2711-494f-905a-f7b2537f6bef_isrForCrosstalkSources_996958_56.stderr parsl.dataflow.dflow DEBUG: Task 2159 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2159 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2159 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2159 has memoization hash c2cd04693147b965aebe12bbf48e6bda parsl.dataflow.memoization INFO: Task 2159 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2159 try 0 launched on executor multi with executor id 2160 parsl.dataflow.dflow INFO: Standard output for task 2159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/fbb35f0e-b7fb-4392-89c7-165b843f99d1_isrForCrosstalkSources_996969_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/fbb35f0e-b7fb-4392-89c7-165b843f99d1_isrForCrosstalkSources_996969_39.stderr parsl.dataflow.dflow DEBUG: Task 2160 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2160 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2160 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2160 has memoization hash 19f0b95f459d858daef026dc7a918f77 parsl.dataflow.memoization INFO: Task 2160 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2160 try 0 launched on executor multi with executor id 2161 parsl.dataflow.dflow INFO: Standard output for task 2160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a8569c92-2ddd-421b-a332-ffefb337a59e_isrForCrosstalkSources_997021_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a8569c92-2ddd-421b-a332-ffefb337a59e_isrForCrosstalkSources_997021_44.stderr parsl.dataflow.dflow DEBUG: Task 2161 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2161 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2161 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2161 has memoization hash 684f9008be593ed0ebd59ce08a722ab8 parsl.dataflow.memoization INFO: Task 2161 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2161 try 0 launched on executor multi with executor id 2162 parsl.dataflow.dflow INFO: Standard output for task 2161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/70fc038e-5e2d-4eb1-ab69-b35f0165c0fe_isrForCrosstalkSources_996988_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/70fc038e-5e2d-4eb1-ab69-b35f0165c0fe_isrForCrosstalkSources_996988_59.stderr parsl.dataflow.dflow DEBUG: Task 2162 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2162 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2162 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2162 has memoization hash 5297ef727c935d5406aebd67ca9dd3d0 parsl.dataflow.memoization INFO: Task 2162 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:47 proc_lsst.multi:170 [INFO] [multi] job_id 17074871 proc_lsst.multi INFO: [multi] job_id 17074871 2024-03-15 13:49:47 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 25 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 25 2024-03-15 13:49:47 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:47 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074871 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074871 parsl.executors.status_handling DEBUG: Launched block 25 on executor multi with job ID 17074871 parsl.executors.status_handling INFO: Allocated block ID 26 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-26 2024-03-15 13:49:47 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=26 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-26 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=26 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-26 2024-03-15 13:49:47 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=26 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=26 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:47 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=26 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=26 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 25 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 25 2024-03-15 13:49:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:47 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=26 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=26 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 2162 try 0 launched on executor multi with executor id 2163 parsl.dataflow.dflow INFO: Standard output for task 2162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3cf3efc0-0990-4a3f-aed8-0a0453c264a5_isrForCrosstalkSources_996969_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3cf3efc0-0990-4a3f-aed8-0a0453c264a5_isrForCrosstalkSources_996969_28.stderr parsl.dataflow.dflow DEBUG: Task 2163 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2163 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2163 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2163 has memoization hash 4fd553b98a34634bfbd507235a152d0d parsl.dataflow.memoization INFO: Task 2163 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc056c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2163 try 0 launched on executor multi with executor id 2164 parsl.dataflow.dflow INFO: Standard output for task 2163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/919faa1c-2320-4919-b929-6b05ad40edc0_isrForCrosstalkSources_996991_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/919faa1c-2320-4919-b929-6b05ad40edc0_isrForCrosstalkSources_996991_34.stderr parsl.dataflow.dflow DEBUG: Task 2164 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2164 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2164 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2164 has memoization hash 2b528d019cdf349525a306a0fe6069a9 parsl.dataflow.memoization INFO: Task 2164 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc05760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2164 try 0 launched on executor multi with executor id 2165 parsl.dataflow.dflow INFO: Standard output for task 2164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/348d8b58-7246-485d-98ce-b90353acf3b9_isrForCrosstalkSources_997001_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/348d8b58-7246-485d-98ce-b90353acf3b9_isrForCrosstalkSources_997001_15.stderr parsl.dataflow.dflow DEBUG: Task 2165 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2165 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2165 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2165 has memoization hash e34c0d4cfc4cfa93c66135e8d795917f parsl.dataflow.memoization INFO: Task 2165 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc04040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2165 try 0 launched on executor multi with executor id 2166 parsl.dataflow.dflow INFO: Standard output for task 2165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/855c876b-b3b6-4659-a39b-d078358683ca_isrForCrosstalkSources_996997_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/855c876b-b3b6-4659-a39b-d078358683ca_isrForCrosstalkSources_996997_38.stderr parsl.dataflow.dflow DEBUG: Task 2166 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2166 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2166 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2166 has memoization hash 3733234f69321e24bb4d92511a26948a parsl.dataflow.memoization INFO: Task 2166 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2166 try 0 launched on executor multi with executor id 2167 parsl.dataflow.dflow INFO: Standard output for task 2166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c4d82630-e51b-4a80-9346-4ace8dc26297_isrForCrosstalkSources_997012_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c4d82630-e51b-4a80-9346-4ace8dc26297_isrForCrosstalkSources_997012_4.stderr parsl.dataflow.dflow DEBUG: Task 2167 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2167 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2167 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2167 has memoization hash 3939ed4942f20f21b8d616b633ba94e7 parsl.dataflow.memoization INFO: Task 2167 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2167 try 0 launched on executor multi with executor id 2168 parsl.dataflow.dflow INFO: Standard output for task 2167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/a53c51b2-d912-4136-802b-6ff8eb53acfb_isrForCrosstalkSources_997005_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/a53c51b2-d912-4136-802b-6ff8eb53acfb_isrForCrosstalkSources_997005_39.stderr parsl.dataflow.dflow DEBUG: Task 2168 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2168 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2168 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2168 has memoization hash 892b5d5a2f90f7699225b194b7a07d0a parsl.dataflow.memoization INFO: Task 2168 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 2168 try 0 launched on executor multi with executor id 2169 parsl.dataflow.dflow INFO: Standard output for task 2168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fb3930ba-c0e4-4592-93cb-8a3615170a09_isrForCrosstalkSources_996955_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fb3930ba-c0e4-4592-93cb-8a3615170a09_isrForCrosstalkSources_996955_26.stderr parsl.dataflow.dflow DEBUG: Task 2169 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2169 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2169 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2169 has memoization hash 8c042369bbcbe459ba526e7924468a72 parsl.dataflow.memoization INFO: Task 2169 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2169 try 0 launched on executor multi with executor id 2170 parsl.dataflow.dflow INFO: Standard output for task 2169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/a38780e4-f193-4677-a8c7-d4e05aee82f8_isrForCrosstalkSources_996997_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/a38780e4-f193-4677-a8c7-d4e05aee82f8_isrForCrosstalkSources_996997_36.stderr parsl.dataflow.dflow DEBUG: Task 2170 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2170 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2170 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2170 has memoization hash 1224a3d3a04e3f908caad7650dad9f61 parsl.dataflow.memoization INFO: Task 2170 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2170 try 0 launched on executor multi with executor id 2171 parsl.dataflow.dflow INFO: Standard output for task 2170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/b2a69ac7-ec16-47f2-9f89-23000a43f722_isrForCrosstalkSources_997026_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/b2a69ac7-ec16-47f2-9f89-23000a43f722_isrForCrosstalkSources_997026_1.stderr parsl.dataflow.dflow DEBUG: Task 2171 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2171 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2171 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2171 has memoization hash 3e69fc5f633e0edfcb554615d547c911 parsl.dataflow.memoization INFO: Task 2171 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2171 try 0 launched on executor multi with executor id 2172 parsl.dataflow.dflow INFO: Standard output for task 2171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/85816360-4aa9-4ff4-abee-d41cd6eab71d_isrForCrosstalkSources_996972_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/85816360-4aa9-4ff4-abee-d41cd6eab71d_isrForCrosstalkSources_996972_30.stderr parsl.dataflow.dflow DEBUG: Task 2172 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2172 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2172 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2172 has memoization hash e914ef080b2cdd9991c0f5d0043f783a parsl.dataflow.memoization INFO: Task 2172 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2172 try 0 launched on executor multi with executor id 2173 parsl.dataflow.dflow INFO: Standard output for task 2172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6034ec1f-aac4-4ab8-b157-5e3a1236e11b_isrForCrosstalkSources_996957_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6034ec1f-aac4-4ab8-b157-5e3a1236e11b_isrForCrosstalkSources_996957_31.stderr parsl.dataflow.dflow DEBUG: Task 2173 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2173 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2173 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2173 has memoization hash 27f01a7adcd4c57b1bc3eca8d975ed6e parsl.dataflow.memoization INFO: Task 2173 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2173 try 0 launched on executor multi with executor id 2174 parsl.dataflow.dflow INFO: Standard output for task 2173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e179da26-36b4-4129-8094-1502fa442d88_isrForCrosstalkSources_996970_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e179da26-36b4-4129-8094-1502fa442d88_isrForCrosstalkSources_996970_21.stderr parsl.dataflow.dflow DEBUG: Task 2174 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2174 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2174 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2174 has memoization hash 6693d8689c8f5a768dc375ffc5eb8974 parsl.dataflow.memoization INFO: Task 2174 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2174 try 0 launched on executor multi with executor id 2175 parsl.dataflow.dflow INFO: Standard output for task 2174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/57215ccf-ec7b-496b-b665-439be1576b99_isrForCrosstalkSources_996970_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/57215ccf-ec7b-496b-b665-439be1576b99_isrForCrosstalkSources_996970_59.stderr parsl.dataflow.dflow DEBUG: Task 2175 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2175 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2175 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2175 has memoization hash 5967c2fc6936dee4ddeffa229361e6f1 parsl.dataflow.memoization INFO: Task 2175 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2175 try 0 launched on executor multi with executor id 2176 parsl.dataflow.dflow INFO: Standard output for task 2175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ec0da860-9176-4aeb-afb7-e375ca3c5f74_isrForCrosstalkSources_997028_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ec0da860-9176-4aeb-afb7-e375ca3c5f74_isrForCrosstalkSources_997028_62.stderr parsl.dataflow.dflow DEBUG: Task 2176 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2176 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2176 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2176 has memoization hash b9133befea5be8299691b8f75736b988 parsl.dataflow.memoization INFO: Task 2176 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2176 try 0 launched on executor multi with executor id 2177 parsl.dataflow.dflow INFO: Standard output for task 2176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/fd74c6bd-67c3-4b10-910a-d472e6215e4f_isrForCrosstalkSources_996958_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/fd74c6bd-67c3-4b10-910a-d472e6215e4f_isrForCrosstalkSources_996958_62.stderr parsl.dataflow.dflow DEBUG: Task 2177 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2177 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2177 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2177 has memoization hash 2cc1e88d2125b3e9cea07ed74c68bc6d parsl.dataflow.memoization INFO: Task 2177 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2177 try 0 launched on executor multi with executor id 2178 parsl.dataflow.dflow INFO: Standard output for task 2177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2354f6b4-2336-4bf1-8b65-e9cc05bcd780_isrForCrosstalkSources_996963_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2354f6b4-2336-4bf1-8b65-e9cc05bcd780_isrForCrosstalkSources_996963_30.stderr parsl.dataflow.dflow DEBUG: Task 2178 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2178 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2178 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2178 has memoization hash c9ab3537568b822f72243ef3196b9650 parsl.dataflow.memoization INFO: Task 2178 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2178 try 0 launched on executor multi with executor id 2179 parsl.dataflow.dflow INFO: Standard output for task 2178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c2ceda99-d840-4a06-8b22-a7c07ddbe6b6_isrForCrosstalkSources_997007_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c2ceda99-d840-4a06-8b22-a7c07ddbe6b6_isrForCrosstalkSources_997007_37.stderr parsl.dataflow.dflow DEBUG: Task 2179 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2179 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2179 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2179 has memoization hash 5e89952a3f5119412fb2921d5fa7757d parsl.dataflow.memoization INFO: Task 2179 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2179 try 0 launched on executor multi with executor id 2180 parsl.dataflow.dflow INFO: Standard output for task 2179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4c9388f9-2867-439a-8d75-fef7dfbe1a34_isrForCrosstalkSources_997033_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4c9388f9-2867-439a-8d75-fef7dfbe1a34_isrForCrosstalkSources_997033_54.stderr parsl.dataflow.dflow DEBUG: Task 2180 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2180 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2180 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2180 has memoization hash e84b790c84f9cec3d5c85c0db43fbe16 parsl.dataflow.memoization INFO: Task 2180 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2180 try 0 launched on executor multi with executor id 2181 parsl.dataflow.dflow INFO: Standard output for task 2180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/132c1dde-40e8-4f4d-9dce-4b2cadbfbcb8_isrForCrosstalkSources_996974_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/132c1dde-40e8-4f4d-9dce-4b2cadbfbcb8_isrForCrosstalkSources_996974_55.stderr parsl.dataflow.dflow DEBUG: Task 2181 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2181 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2181 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2181 has memoization hash d2802afdedbd0dea4a84a91b7aa4339a parsl.dataflow.memoization INFO: Task 2181 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2181 try 0 launched on executor multi with executor id 2182 parsl.dataflow.dflow INFO: Standard output for task 2181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/236371bb-cf82-4335-895c-13a2305c9d74_isrForCrosstalkSources_997020_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/236371bb-cf82-4335-895c-13a2305c9d74_isrForCrosstalkSources_997020_56.stderr parsl.dataflow.dflow DEBUG: Task 2182 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2182 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2182 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2182 has memoization hash edc14a4f78c8eb24091182abee2e116a parsl.dataflow.memoization INFO: Task 2182 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2182 try 0 launched on executor multi with executor id 2183 parsl.dataflow.dflow INFO: Standard output for task 2182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/9b46810e-7781-47ae-a1da-5b9005a4f994_isrForCrosstalkSources_997027_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/9b46810e-7781-47ae-a1da-5b9005a4f994_isrForCrosstalkSources_997027_2.stderr parsl.dataflow.dflow DEBUG: Task 2183 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2183 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2183 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2183 has memoization hash 512bf1a8847131b83f281b7ae2b0d7c9 parsl.dataflow.memoization INFO: Task 2183 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2183 try 0 launched on executor multi with executor id 2184 parsl.dataflow.dflow INFO: Standard output for task 2183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b16a7e0d-978c-42ad-bc30-b3561edb2197_isrForCrosstalkSources_997014_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b16a7e0d-978c-42ad-bc30-b3561edb2197_isrForCrosstalkSources_997014_50.stderr parsl.dataflow.dflow DEBUG: Task 2184 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2184 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2184 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2184 has memoization hash 6c3a1c32c1beb67185fa180dc0a1fd37 parsl.dataflow.memoization INFO: Task 2184 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2184 try 0 launched on executor multi with executor id 2185 parsl.dataflow.dflow INFO: Standard output for task 2184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/961cb588-97a2-48ce-b9eb-73868949f096_isrForCrosstalkSources_996980_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/961cb588-97a2-48ce-b9eb-73868949f096_isrForCrosstalkSources_996980_29.stderr parsl.dataflow.dflow DEBUG: Task 2185 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2185 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2185 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2185 has memoization hash 24f9e48ce9f8d32b3e036236cebc5123 parsl.dataflow.memoization INFO: Task 2185 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2185 try 0 launched on executor multi with executor id 2186 parsl.dataflow.dflow INFO: Standard output for task 2185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/9dd4f557-6afc-4b9b-9e65-64e0eedbda30_isrForCrosstalkSources_997026_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/9dd4f557-6afc-4b9b-9e65-64e0eedbda30_isrForCrosstalkSources_997026_56.stderr parsl.dataflow.dflow DEBUG: Task 2186 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2186 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2186 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2186 has memoization hash eba831ab14afe0a88d24a5ed52f4d3d1 parsl.dataflow.memoization INFO: Task 2186 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2186 try 0 launched on executor multi with executor id 2187 parsl.dataflow.dflow INFO: Standard output for task 2186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/9c733c52-040c-4186-8536-99569fac2ade_isrForCrosstalkSources_996981_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/9c733c52-040c-4186-8536-99569fac2ade_isrForCrosstalkSources_996981_37.stderr parsl.dataflow.dflow DEBUG: Task 2187 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2187 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2187 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2187 has memoization hash 7e5051203a406fb4e4e174d4e20e2834 parsl.dataflow.memoization INFO: Task 2187 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2187 try 0 launched on executor multi with executor id 2188 parsl.dataflow.dflow INFO: Standard output for task 2187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/04be391b-083b-4419-9add-585067b01671_isrForCrosstalkSources_997015_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/04be391b-083b-4419-9add-585067b01671_isrForCrosstalkSources_997015_36.stderr parsl.dataflow.dflow DEBUG: Task 2188 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2188 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2188 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2188 has memoization hash a286e3211ff3cacd0803fb8d7c45a4fc parsl.dataflow.memoization INFO: Task 2188 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2188 try 0 launched on executor multi with executor id 2189 parsl.dataflow.dflow INFO: Standard output for task 2188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2b610c61-0b56-4562-8f26-ba65a5bc8c24_isrForCrosstalkSources_996963_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2b610c61-0b56-4562-8f26-ba65a5bc8c24_isrForCrosstalkSources_996963_33.stderr parsl.dataflow.dflow DEBUG: Task 2189 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2189 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2189 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2189 has memoization hash 34d067f2ca95ae4db0a384adadd7e427 parsl.dataflow.memoization INFO: Task 2189 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2189 try 0 launched on executor multi with executor id 2190 parsl.dataflow.dflow INFO: Standard output for task 2189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6607ac2a-330c-4a33-a6da-84955a25658d_isrForCrosstalkSources_997030_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6607ac2a-330c-4a33-a6da-84955a25658d_isrForCrosstalkSources_997030_38.stderr parsl.dataflow.dflow DEBUG: Task 2190 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2190 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2190 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2190 has memoization hash 1d32308ce280075501fd483fee33faa0 parsl.dataflow.memoization INFO: Task 2190 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2190 try 0 launched on executor multi with executor id 2191 parsl.dataflow.dflow INFO: Standard output for task 2190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/2c62b367-d5c6-4dc3-a409-6e7301224925_isrForCrosstalkSources_996976_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/2c62b367-d5c6-4dc3-a409-6e7301224925_isrForCrosstalkSources_996976_44.stderr parsl.dataflow.dflow DEBUG: Task 2191 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2191 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2191 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2191 has memoization hash 4b9bdae34b6270b5e3e9df688c56135a parsl.dataflow.memoization INFO: Task 2191 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2191 try 0 launched on executor multi with executor id 2192 parsl.dataflow.dflow INFO: Standard output for task 2191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e715981b-51fd-4960-9bba-706c599e6c14_isrForCrosstalkSources_996992_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e715981b-51fd-4960-9bba-706c599e6c14_isrForCrosstalkSources_996992_62.stderr parsl.dataflow.dflow DEBUG: Task 2192 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2192 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2192 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2192 has memoization hash 9b6df7469be09575c22b17416b64dc78 parsl.dataflow.memoization INFO: Task 2192 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2192 try 0 launched on executor multi with executor id 2193 parsl.dataflow.dflow INFO: Standard output for task 2192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cf266be9-c504-4c36-9208-dce6d35ed572_isrForCrosstalkSources_996983_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cf266be9-c504-4c36-9208-dce6d35ed572_isrForCrosstalkSources_996983_10.stderr parsl.dataflow.dflow DEBUG: Task 2193 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2193 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2193 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2193 has memoization hash 9cb75a158c72cac297822419b8ed8702 parsl.dataflow.memoization INFO: Task 2193 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2193 try 0 launched on executor multi with executor id 2194 parsl.dataflow.dflow INFO: Standard output for task 2193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/94897950-2944-4302-bb5f-a620443f9f23_isrForCrosstalkSources_996988_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/94897950-2944-4302-bb5f-a620443f9f23_isrForCrosstalkSources_996988_52.stderr parsl.dataflow.dflow DEBUG: Task 2194 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2194 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2194 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2194 has memoization hash aa1312379d3d9b64d6b359023b61856b parsl.dataflow.memoization INFO: Task 2194 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2194 try 0 launched on executor multi with executor id 2195 parsl.dataflow.dflow INFO: Standard output for task 2194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bdcd8e20-3bee-411c-aff2-a3248adeae4e_isrForCrosstalkSources_996960_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bdcd8e20-3bee-411c-aff2-a3248adeae4e_isrForCrosstalkSources_996960_24.stderr parsl.dataflow.dflow DEBUG: Task 2195 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2195 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2195 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2195 has memoization hash d80fd866518608e5b6cdaf65815f42bb parsl.dataflow.memoization INFO: Task 2195 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2195 try 0 launched on executor multi with executor id 2196 parsl.dataflow.dflow INFO: Standard output for task 2195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/8866391a-cf3d-4632-9056-53376d8385b6_isrForCrosstalkSources_996959_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/8866391a-cf3d-4632-9056-53376d8385b6_isrForCrosstalkSources_996959_45.stderr parsl.dataflow.dflow DEBUG: Task 2196 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2196 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2196 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2196 has memoization hash 913e5374030d1b87cc57b5047815a6f9 parsl.dataflow.memoization INFO: Task 2196 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2196 try 0 launched on executor multi with executor id 2197 parsl.dataflow.dflow INFO: Standard output for task 2196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/8e0fe56e-f477-4eeb-9f47-a00bf9c5bfec_isrForCrosstalkSources_997024_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/8e0fe56e-f477-4eeb-9f47-a00bf9c5bfec_isrForCrosstalkSources_997024_25.stderr parsl.dataflow.dflow DEBUG: Task 2197 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2197 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2197 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2197 has memoization hash 30a40f47089d01cf2bfa312a38120c44 parsl.dataflow.memoization INFO: Task 2197 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2197 try 0 launched on executor multi with executor id 2198 parsl.dataflow.dflow INFO: Standard output for task 2197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5b5bd38f-8bbc-4630-901a-88c9f76d05d2_isrForCrosstalkSources_997010_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5b5bd38f-8bbc-4630-901a-88c9f76d05d2_isrForCrosstalkSources_997010_30.stderr parsl.dataflow.dflow DEBUG: Task 2198 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2198 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2198 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2198 has memoization hash ddc4c49f3828f9abc980ac4352107777 parsl.dataflow.memoization INFO: Task 2198 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2198 try 0 launched on executor multi with executor id 2199 parsl.dataflow.dflow INFO: Standard output for task 2198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/72498c58-80a4-435a-86e4-ad3d97a11c7e_isrForCrosstalkSources_997031_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/72498c58-80a4-435a-86e4-ad3d97a11c7e_isrForCrosstalkSources_997031_11.stderr parsl.dataflow.dflow DEBUG: Task 2199 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2199 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2199 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2199 has memoization hash ced0c664e12ae9cd37368306e1f41504 parsl.dataflow.memoization INFO: Task 2199 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2199 try 0 launched on executor multi with executor id 2200 parsl.dataflow.dflow INFO: Standard output for task 2199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/dbfdcc6a-e68a-458a-bdd5-cf6189168ae3_isrForCrosstalkSources_996972_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/dbfdcc6a-e68a-458a-bdd5-cf6189168ae3_isrForCrosstalkSources_996972_48.stderr parsl.dataflow.dflow DEBUG: Task 2200 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2200 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2200 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2200 has memoization hash f96b207778e71a0d3cdd186580fa76a7 parsl.dataflow.memoization INFO: Task 2200 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2200 try 0 launched on executor multi with executor id 2201 parsl.dataflow.dflow INFO: Standard output for task 2200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/b0424264-8dcc-4b00-ab82-174676435f42_isrForCrosstalkSources_996971_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/b0424264-8dcc-4b00-ab82-174676435f42_isrForCrosstalkSources_996971_9.stderr parsl.dataflow.dflow DEBUG: Task 2201 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2201 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2201 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2201 has memoization hash abfcb54f47858fc9185fc4066ebcb726 parsl.dataflow.memoization INFO: Task 2201 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2201 try 0 launched on executor multi with executor id 2202 parsl.dataflow.dflow INFO: Standard output for task 2201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/97d284a0-ffc1-47b8-84af-d46a0cb0f98b_isrForCrosstalkSources_997014_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/97d284a0-ffc1-47b8-84af-d46a0cb0f98b_isrForCrosstalkSources_997014_21.stderr parsl.dataflow.dflow DEBUG: Task 2202 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2202 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2202 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2202 has memoization hash e9dd7ebc4c4678a7a209c3c0133966e0 parsl.dataflow.memoization INFO: Task 2202 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2202 try 0 launched on executor multi with executor id 2203 parsl.dataflow.dflow INFO: Standard output for task 2202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1af38e32-764b-49b6-af0a-eee4fd3469c8_isrForCrosstalkSources_996983_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1af38e32-764b-49b6-af0a-eee4fd3469c8_isrForCrosstalkSources_996983_43.stderr parsl.dataflow.dflow DEBUG: Task 2203 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2203 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2203 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2203 has memoization hash ab07a0183a37675cefc8130e7820077a parsl.dataflow.memoization INFO: Task 2203 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2203 try 0 launched on executor multi with executor id 2204 parsl.dataflow.dflow INFO: Standard output for task 2203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/570be287-6a04-43a0-8949-234732709a6b_isrForCrosstalkSources_996968_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/570be287-6a04-43a0-8949-234732709a6b_isrForCrosstalkSources_996968_28.stderr parsl.dataflow.dflow DEBUG: Task 2204 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2204 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2204 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2204 has memoization hash 902d227ca71337cb4f36134133467ff9 parsl.dataflow.memoization INFO: Task 2204 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2204 try 0 launched on executor multi with executor id 2205 parsl.dataflow.dflow INFO: Standard output for task 2204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/6af65e2c-80f1-4d6c-ab72-70b1f11e4c06_isrForCrosstalkSources_996986_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/6af65e2c-80f1-4d6c-ab72-70b1f11e4c06_isrForCrosstalkSources_996986_19.stderr parsl.dataflow.dflow DEBUG: Task 2205 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2205 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2205 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2205 has memoization hash c7dc116ec08e5dbb8c3ccec79b6e2c62 parsl.dataflow.memoization INFO: Task 2205 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2205 try 0 launched on executor multi with executor id 2206 parsl.dataflow.dflow INFO: Standard output for task 2205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c435e237-f8ac-47f3-89ac-739e3f74f7be_isrForCrosstalkSources_996969_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c435e237-f8ac-47f3-89ac-739e3f74f7be_isrForCrosstalkSources_996969_44.stderr parsl.dataflow.dflow DEBUG: Task 2206 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2206 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2206 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2206 has memoization hash 78d9a0fde7c1bfa0104d783967f89ccb parsl.dataflow.memoization INFO: Task 2206 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2206 try 0 launched on executor multi with executor id 2207 parsl.dataflow.dflow INFO: Standard output for task 2206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/3ee155c4-e026-4b67-bd69-f46c173edc47_isrForCrosstalkSources_996987_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/3ee155c4-e026-4b67-bd69-f46c173edc47_isrForCrosstalkSources_996987_36.stderr parsl.dataflow.dflow DEBUG: Task 2207 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2207 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2207 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2207 has memoization hash 22c25543a617a5f455a743cc0146ad5e parsl.dataflow.memoization INFO: Task 2207 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2207 try 0 launched on executor multi with executor id 2208 parsl.dataflow.dflow INFO: Standard output for task 2207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5067265b-cb64-49e0-9da0-563c24396884_isrForCrosstalkSources_997015_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5067265b-cb64-49e0-9da0-563c24396884_isrForCrosstalkSources_997015_34.stderr parsl.dataflow.dflow DEBUG: Task 2208 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2208 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2208 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2208 has memoization hash 6c541b13d6b01be1980ae3d72e705863 parsl.dataflow.memoization INFO: Task 2208 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2208 try 0 launched on executor multi with executor id 2209 parsl.dataflow.dflow INFO: Standard output for task 2208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6131745f-dbbe-4d77-b53c-80e7b1c04bc8_isrForCrosstalkSources_996983_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6131745f-dbbe-4d77-b53c-80e7b1c04bc8_isrForCrosstalkSources_996983_53.stderr parsl.dataflow.dflow DEBUG: Task 2209 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2209 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2209 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2209 has memoization hash ff44083b784ad50f6bd120598c8db4ed parsl.dataflow.memoization INFO: Task 2209 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2209 try 0 launched on executor multi with executor id 2210 parsl.dataflow.dflow INFO: Standard output for task 2209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/36a33ced-531a-458d-9ba0-4c4d6a348b26_isrForCrosstalkSources_997015_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/36a33ced-531a-458d-9ba0-4c4d6a348b26_isrForCrosstalkSources_997015_2.stderr parsl.dataflow.dflow DEBUG: Task 2210 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2210 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2210 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2210 has memoization hash dd4cf7a5ffbee33da9a301798ecb3e0f parsl.dataflow.memoization INFO: Task 2210 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2210 try 0 launched on executor multi with executor id 2211 parsl.dataflow.dflow INFO: Standard output for task 2210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/df5b9bab-6ddf-45ed-b8d2-96ad3bd287fc_isrForCrosstalkSources_996963_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/df5b9bab-6ddf-45ed-b8d2-96ad3bd287fc_isrForCrosstalkSources_996963_29.stderr parsl.dataflow.dflow DEBUG: Task 2211 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2211 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2211 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2211 has memoization hash 3df1915586ab1209c86eed7acdb0bdf5 parsl.dataflow.memoization INFO: Task 2211 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2211 try 0 launched on executor multi with executor id 2212 parsl.dataflow.dflow INFO: Standard output for task 2211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a77fbafb-e6d1-45cf-849c-20bdc9570903_isrForCrosstalkSources_996996_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a77fbafb-e6d1-45cf-849c-20bdc9570903_isrForCrosstalkSources_996996_17.stderr parsl.dataflow.dflow DEBUG: Task 2212 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2212 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2212 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2212 has memoization hash 464036a1b427dded3e06f52acfcb8191 parsl.dataflow.memoization INFO: Task 2212 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2212 try 0 launched on executor multi with executor id 2213 parsl.dataflow.dflow INFO: Standard output for task 2212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b2790512-f656-4bce-9192-01f2ab27590d_isrForCrosstalkSources_997022_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b2790512-f656-4bce-9192-01f2ab27590d_isrForCrosstalkSources_997022_26.stderr parsl.dataflow.dflow DEBUG: Task 2213 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2213 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2213 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2213 has memoization hash 8fb45dd6633fea77c3824cd17e86d573 parsl.dataflow.memoization INFO: Task 2213 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2213 try 0 launched on executor multi with executor id 2214 parsl.dataflow.dflow INFO: Standard output for task 2213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/65f23818-6214-4824-bd6a-e835f2b9f671_isrForCrosstalkSources_997024_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/65f23818-6214-4824-bd6a-e835f2b9f671_isrForCrosstalkSources_997024_36.stderr parsl.dataflow.dflow DEBUG: Task 2214 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2214 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2214 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2214 has memoization hash 51ba607fb759688021b3cfdd9ebaaafa parsl.dataflow.memoization INFO: Task 2214 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2214 try 0 launched on executor multi with executor id 2215 parsl.dataflow.dflow INFO: Standard output for task 2214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/4718103f-3f0b-4cec-9e7d-237a52035040_isrForCrosstalkSources_996975_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/4718103f-3f0b-4cec-9e7d-237a52035040_isrForCrosstalkSources_996975_6.stderr parsl.dataflow.dflow DEBUG: Task 2215 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2215 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2215 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2215 has memoization hash 884dcfc4796cc082bebae855d9d10be5 parsl.dataflow.memoization INFO: Task 2215 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2215 try 0 launched on executor multi with executor id 2216 parsl.dataflow.dflow INFO: Standard output for task 2215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/79ce3c28-f7aa-4dfb-827e-d5b0570bc7be_isrForCrosstalkSources_996997_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/79ce3c28-f7aa-4dfb-827e-d5b0570bc7be_isrForCrosstalkSources_996997_51.stderr parsl.dataflow.dflow DEBUG: Task 2216 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2216 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2216 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2216 has memoization hash 904f8cafe06f6fcfb3ce4b6b0a206ba3 parsl.dataflow.memoization INFO: Task 2216 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2216 try 0 launched on executor multi with executor id 2217 parsl.dataflow.dflow INFO: Standard output for task 2216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1be9b606-3223-4120-a79e-21633901aead_isrForCrosstalkSources_997019_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1be9b606-3223-4120-a79e-21633901aead_isrForCrosstalkSources_997019_4.stderr parsl.dataflow.dflow DEBUG: Task 2217 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2217 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2217 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2217 has memoization hash 11542e1b67002cede5caf5483fdd1879 parsl.dataflow.memoization INFO: Task 2217 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2217 try 0 launched on executor multi with executor id 2218 parsl.dataflow.dflow INFO: Standard output for task 2217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/af3600c4-a1a0-40a5-940c-d774d8db326d_isrForCrosstalkSources_996995_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/af3600c4-a1a0-40a5-940c-d774d8db326d_isrForCrosstalkSources_996995_23.stderr parsl.dataflow.dflow DEBUG: Task 2218 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2218 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2218 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2218 has memoization hash 84c0e3fefd2343bca72bb4564159e10f parsl.dataflow.memoization INFO: Task 2218 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2218 try 0 launched on executor multi with executor id 2219 parsl.dataflow.dflow INFO: Standard output for task 2218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/cb706d19-cb8b-44d2-b923-539fc10a1108_isrForCrosstalkSources_996970_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/cb706d19-cb8b-44d2-b923-539fc10a1108_isrForCrosstalkSources_996970_26.stderr parsl.dataflow.dflow DEBUG: Task 2219 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2219 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2219 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2219 has memoization hash a6326ac01b40fdae15f6e7779ca87da4 parsl.dataflow.memoization INFO: Task 2219 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2219 try 0 launched on executor multi with executor id 2220 parsl.dataflow.dflow INFO: Standard output for task 2219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5ae23cf7-e2e8-4e48-a5cd-33e49a4f457d_isrForCrosstalkSources_996985_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5ae23cf7-e2e8-4e48-a5cd-33e49a4f457d_isrForCrosstalkSources_996985_17.stderr parsl.dataflow.dflow DEBUG: Task 2220 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2220 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2220 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2220 has memoization hash 3a0c20ee1b6d3ee6a741f9ce360c0ba5 parsl.dataflow.memoization INFO: Task 2220 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2220 try 0 launched on executor multi with executor id 2221 parsl.dataflow.dflow INFO: Standard output for task 2220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/c32dc25c-babb-42c1-8473-8dbd897aa1a0_isrForCrosstalkSources_997031_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/c32dc25c-babb-42c1-8473-8dbd897aa1a0_isrForCrosstalkSources_997031_30.stderr parsl.dataflow.dflow DEBUG: Task 2221 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2221 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2221 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2221 has memoization hash c31e0735aca35d7ac2ebbe8218812478 parsl.dataflow.memoization INFO: Task 2221 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2221 try 0 launched on executor multi with executor id 2222 parsl.dataflow.dflow INFO: Standard output for task 2221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/306c81fd-9d4f-4d2b-b504-9f3c04704e83_isrForCrosstalkSources_997020_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/306c81fd-9d4f-4d2b-b504-9f3c04704e83_isrForCrosstalkSources_997020_60.stderr parsl.dataflow.dflow DEBUG: Task 2222 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2222 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2222 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2222 has memoization hash 7e97a56f7c386cda6219b8c2c48e22e3 parsl.dataflow.memoization INFO: Task 2222 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2222 try 0 launched on executor multi with executor id 2223 parsl.dataflow.dflow INFO: Standard output for task 2222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/13193e84-3894-4f9d-98fb-6432f1c33afb_isrForCrosstalkSources_997004_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/13193e84-3894-4f9d-98fb-6432f1c33afb_isrForCrosstalkSources_997004_22.stderr parsl.dataflow.dflow DEBUG: Task 2223 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2223 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2223 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2223 has memoization hash 26858646df75ea88286d1e543702d60d parsl.dataflow.memoization INFO: Task 2223 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2223 try 0 launched on executor multi with executor id 2224 parsl.dataflow.dflow INFO: Standard output for task 2223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/56849b40-4da7-47d9-bc79-1443bbb6790c_isrForCrosstalkSources_996979_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/56849b40-4da7-47d9-bc79-1443bbb6790c_isrForCrosstalkSources_996979_28.stderr parsl.dataflow.dflow DEBUG: Task 2224 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2224 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2224 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2224 has memoization hash 15a4db629fc599ac85efffb23f1cd8ed parsl.dataflow.memoization INFO: Task 2224 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2224 try 0 launched on executor multi with executor id 2225 parsl.dataflow.dflow INFO: Standard output for task 2224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/aa1623fb-0f6c-465c-be5d-c548730c04ea_isrForCrosstalkSources_997023_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/aa1623fb-0f6c-465c-be5d-c548730c04ea_isrForCrosstalkSources_997023_28.stderr parsl.dataflow.dflow DEBUG: Task 2225 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2225 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2225 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2225 has memoization hash 1b7497b8b7fe114d3d355f9cb448fd57 parsl.dataflow.memoization INFO: Task 2225 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2225 try 0 launched on executor multi with executor id 2226 parsl.dataflow.dflow INFO: Standard output for task 2225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a94b204e-8b5b-404f-9bfd-dc0ffda93573_isrForCrosstalkSources_997008_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a94b204e-8b5b-404f-9bfd-dc0ffda93573_isrForCrosstalkSources_997008_38.stderr parsl.dataflow.dflow DEBUG: Task 2226 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2226 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2226 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2226 has memoization hash d8966f398fe305dfc3fb42be46ca4022 parsl.dataflow.memoization INFO: Task 2226 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2226 try 0 launched on executor multi with executor id 2227 parsl.dataflow.dflow INFO: Standard output for task 2226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/be524e8c-c3b2-47bb-9dd4-dfb54482fa11_isrForCrosstalkSources_996959_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/be524e8c-c3b2-47bb-9dd4-dfb54482fa11_isrForCrosstalkSources_996959_5.stderr parsl.dataflow.dflow DEBUG: Task 2227 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2227 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2227 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2227 has memoization hash 2d8020cf0df4e68f6806cfdeecd80024 parsl.dataflow.memoization INFO: Task 2227 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2227 try 0 launched on executor multi with executor id 2228 parsl.dataflow.dflow INFO: Standard output for task 2227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c78c546c-99ba-4ba7-b616-e80b3b535ab7_isrForCrosstalkSources_996961_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c78c546c-99ba-4ba7-b616-e80b3b535ab7_isrForCrosstalkSources_996961_42.stderr parsl.dataflow.dflow DEBUG: Task 2228 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2228 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2228 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2228 has memoization hash 9f4cc17b636fa1dd1007c5425d5c81c1 parsl.dataflow.memoization INFO: Task 2228 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2228 try 0 launched on executor multi with executor id 2229 parsl.dataflow.dflow INFO: Standard output for task 2228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/3453907b-acef-413d-bb02-0e0f501aae31_isrForCrosstalkSources_996971_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/3453907b-acef-413d-bb02-0e0f501aae31_isrForCrosstalkSources_996971_22.stderr parsl.dataflow.dflow DEBUG: Task 2229 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2229 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2229 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2229 has memoization hash 247a3f7edd181d918d7db084c7138091 parsl.dataflow.memoization INFO: Task 2229 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2229 try 0 launched on executor multi with executor id 2230 parsl.dataflow.dflow INFO: Standard output for task 2229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/17073cc0-b66b-4d26-acf4-9b0d4684cef3_isrForCrosstalkSources_996999_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/17073cc0-b66b-4d26-acf4-9b0d4684cef3_isrForCrosstalkSources_996999_12.stderr parsl.dataflow.dflow DEBUG: Task 2230 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2230 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2230 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2230 has memoization hash 9a6614dc8ff742f28ea11491016038ea parsl.dataflow.memoization INFO: Task 2230 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2230 try 0 launched on executor multi with executor id 2231 parsl.dataflow.dflow INFO: Standard output for task 2230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b7bfecac-8df2-48bb-a746-db1b7fc6f22e_isrForCrosstalkSources_996963_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b7bfecac-8df2-48bb-a746-db1b7fc6f22e_isrForCrosstalkSources_996963_38.stderr parsl.dataflow.dflow DEBUG: Task 2231 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2231 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2231 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2231 has memoization hash f5b50305ab3941acc8e0f1d254f12ace parsl.dataflow.memoization INFO: Task 2231 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2231 try 0 launched on executor multi with executor id 2232 parsl.dataflow.dflow INFO: Standard output for task 2231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9660d3e1-70dd-4683-8460-68e8d8f783fa_isrForCrosstalkSources_997022_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9660d3e1-70dd-4683-8460-68e8d8f783fa_isrForCrosstalkSources_997022_57.stderr parsl.dataflow.dflow DEBUG: Task 2232 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2232 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2232 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2232 has memoization hash 7721bae2141e789f97f9b08b44972c7e parsl.dataflow.memoization INFO: Task 2232 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2232 try 0 launched on executor multi with executor id 2233 parsl.dataflow.dflow INFO: Standard output for task 2232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/fd5c429c-6a61-45ea-8d72-cb8a0b1708aa_isrForCrosstalkSources_997006_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/fd5c429c-6a61-45ea-8d72-cb8a0b1708aa_isrForCrosstalkSources_997006_25.stderr parsl.dataflow.dflow DEBUG: Task 2233 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2233 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2233 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2233 has memoization hash c211390e0cdcc1941fdd99f21a8b8494 parsl.dataflow.memoization INFO: Task 2233 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2233 try 0 launched on executor multi with executor id 2234 parsl.dataflow.dflow INFO: Standard output for task 2233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ec3ce184-1335-4803-82ed-a6ad37b4f7e7_isrForCrosstalkSources_996992_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ec3ce184-1335-4803-82ed-a6ad37b4f7e7_isrForCrosstalkSources_996992_48.stderr parsl.dataflow.dflow DEBUG: Task 2234 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2234 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2234 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2234 has memoization hash ce8d45a47bb0a33c543931c4f3c2d649 parsl.dataflow.memoization INFO: Task 2234 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2234 try 0 launched on executor multi with executor id 2235 parsl.dataflow.dflow INFO: Standard output for task 2234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/fc8456ee-e393-4a65-ab4b-dc64bba0e4d2_isrForCrosstalkSources_996957_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/fc8456ee-e393-4a65-ab4b-dc64bba0e4d2_isrForCrosstalkSources_996957_35.stderr parsl.dataflow.dflow DEBUG: Task 2235 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2235 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2235 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2235 has memoization hash 68f0f7f31d49b45bb21557c25c790265 parsl.dataflow.memoization INFO: Task 2235 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2235 try 0 launched on executor multi with executor id 2236 parsl.dataflow.dflow INFO: Standard output for task 2235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/f9417734-5ffa-4ab8-a437-51acfa45623e_isrForCrosstalkSources_997007_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/f9417734-5ffa-4ab8-a437-51acfa45623e_isrForCrosstalkSources_997007_59.stderr parsl.dataflow.dflow DEBUG: Task 2236 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2236 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2236 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2236 has memoization hash 530995d7f17a82633cb082e799868f5d parsl.dataflow.memoization INFO: Task 2236 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2236 try 0 launched on executor multi with executor id 2237 parsl.dataflow.dflow INFO: Standard output for task 2236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d3bc9205-9f96-4a28-ac31-4a082cb6e6d8_isrForCrosstalkSources_997023_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d3bc9205-9f96-4a28-ac31-4a082cb6e6d8_isrForCrosstalkSources_997023_1.stderr parsl.dataflow.dflow DEBUG: Task 2237 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2237 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2237 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2237 has memoization hash 8b97ef19cfa9837aa9c7101f2ed7ab0a parsl.dataflow.memoization INFO: Task 2237 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2237 try 0 launched on executor multi with executor id 2238 parsl.dataflow.dflow INFO: Standard output for task 2237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/2b3c44b4-76f4-4410-9dd7-e6ae019217dd_isrForCrosstalkSources_997013_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/2b3c44b4-76f4-4410-9dd7-e6ae019217dd_isrForCrosstalkSources_997013_25.stderr parsl.dataflow.dflow DEBUG: Task 2238 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2238 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2238 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2238 has memoization hash 34d5eda9b16422cc6291f3bb95e63eea parsl.dataflow.memoization INFO: Task 2238 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac00e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2238 try 0 launched on executor multi with executor id 2239 parsl.dataflow.dflow INFO: Standard output for task 2238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6f48b8b1-4d14-4d68-9b6e-d9586c104d0e_isrForCrosstalkSources_997030_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6f48b8b1-4d14-4d68-9b6e-d9586c104d0e_isrForCrosstalkSources_997030_31.stderr parsl.dataflow.dflow DEBUG: Task 2239 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2239 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2239 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2239 has memoization hash e00a1a44d33dff9e688ccaaa5e17d854 parsl.dataflow.memoization INFO: Task 2239 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2239 try 0 launched on executor multi with executor id 2240 parsl.dataflow.dflow INFO: Standard output for task 2239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c841a656-38fc-4322-a0ba-d0e8dd939f2f_isrForCrosstalkSources_997013_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c841a656-38fc-4322-a0ba-d0e8dd939f2f_isrForCrosstalkSources_997013_62.stderr parsl.dataflow.dflow DEBUG: Task 2240 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2240 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2240 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2240 has memoization hash ef704453814d879da8f9f39525ba3827 parsl.dataflow.memoization INFO: Task 2240 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2240 try 0 launched on executor multi with executor id 2241 parsl.dataflow.dflow INFO: Standard output for task 2240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/59cd39ce-842f-4523-87f9-2649af3d8f48_isrForCrosstalkSources_996975_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/59cd39ce-842f-4523-87f9-2649af3d8f48_isrForCrosstalkSources_996975_43.stderr parsl.dataflow.dflow DEBUG: Task 2241 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2241 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2241 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2241 has memoization hash 33d0327b34efa59afa31b5127fc0d7bb parsl.dataflow.memoization INFO: Task 2241 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2241 try 0 launched on executor multi with executor id 2242 parsl.dataflow.dflow INFO: Standard output for task 2241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/77fac32a-0989-4734-a6e2-6aaaad6f7eb8_isrForCrosstalkSources_997012_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/77fac32a-0989-4734-a6e2-6aaaad6f7eb8_isrForCrosstalkSources_997012_24.stderr parsl.dataflow.dflow DEBUG: Task 2242 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2242 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2242 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2242 has memoization hash 4a324f228acea98b6e40cf6018973335 parsl.dataflow.memoization INFO: Task 2242 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2242 try 0 launched on executor multi with executor id 2243 parsl.dataflow.dflow INFO: Standard output for task 2242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/fc44ebdf-13d2-43a6-9c30-be0c357a9b0b_isrForCrosstalkSources_996965_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/fc44ebdf-13d2-43a6-9c30-be0c357a9b0b_isrForCrosstalkSources_996965_28.stderr parsl.dataflow.dflow DEBUG: Task 2243 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2243 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2243 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2243 has memoization hash da2d4306bac6938282e0d93c42a6282b parsl.dataflow.memoization INFO: Task 2243 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2243 try 0 launched on executor multi with executor id 2244 parsl.dataflow.dflow INFO: Standard output for task 2243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/70c9d390-557c-4f95-a0d8-415807d8bbba_isrForCrosstalkSources_996983_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/70c9d390-557c-4f95-a0d8-415807d8bbba_isrForCrosstalkSources_996983_15.stderr parsl.dataflow.dflow DEBUG: Task 2244 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2244 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2244 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2244 has memoization hash 2b85b3c7ad1f36e953c8a211eedc59fc parsl.dataflow.memoization INFO: Task 2244 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2244 try 0 launched on executor multi with executor id 2245 parsl.dataflow.dflow INFO: Standard output for task 2244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/4eb28046-c471-4e2f-9028-881fc946c6f5_isrForCrosstalkSources_997009_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/4eb28046-c471-4e2f-9028-881fc946c6f5_isrForCrosstalkSources_997009_26.stderr parsl.dataflow.dflow DEBUG: Task 2245 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2245 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2245 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2245 has memoization hash 50548ba1add922d8139a19b982615071 parsl.dataflow.memoization INFO: Task 2245 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac04a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2245 try 0 launched on executor multi with executor id 2246 parsl.dataflow.dflow INFO: Standard output for task 2245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/4f498bec-6512-44af-b1fd-61ab059a92fa_isrForCrosstalkSources_996992_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/4f498bec-6512-44af-b1fd-61ab059a92fa_isrForCrosstalkSources_996992_60.stderr parsl.dataflow.dflow DEBUG: Task 2246 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2246 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2246 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2246 has memoization hash 1444ce5b5e6853576765fdbcd66de94f parsl.dataflow.memoization INFO: Task 2246 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac05e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2246 try 0 launched on executor multi with executor id 2247 parsl.dataflow.dflow INFO: Standard output for task 2246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6391be9d-8fd4-4454-ba2a-654f112095ce_isrForCrosstalkSources_997015_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6391be9d-8fd4-4454-ba2a-654f112095ce_isrForCrosstalkSources_997015_42.stderr parsl.dataflow.dflow DEBUG: Task 2247 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2247 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2247 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2247 has memoization hash 82187c5f9e869a814cf396755d4e00af parsl.dataflow.memoization INFO: Task 2247 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2247 try 0 launched on executor multi with executor id 2248 parsl.dataflow.dflow INFO: Standard output for task 2247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/610e36a4-80fd-468b-8bc1-b86c68f10419_isrForCrosstalkSources_996998_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/610e36a4-80fd-468b-8bc1-b86c68f10419_isrForCrosstalkSources_996998_47.stderr parsl.dataflow.dflow DEBUG: Task 2248 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2248 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2248 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2248 has memoization hash 5006a6696a27e478e756965ebec3db77 parsl.dataflow.memoization INFO: Task 2248 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2248 try 0 launched on executor multi with executor id 2249 parsl.dataflow.dflow INFO: Standard output for task 2248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/38107634-15e5-49c8-8c4b-6f98d49aea5b_isrForCrosstalkSources_997030_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/38107634-15e5-49c8-8c4b-6f98d49aea5b_isrForCrosstalkSources_997030_41.stderr parsl.dataflow.dflow DEBUG: Task 2249 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2249 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2249 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2249 has memoization hash fcf91f70ca5e3c267ce96038aced4fcc parsl.dataflow.memoization INFO: Task 2249 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2249 try 0 launched on executor multi with executor id 2250 parsl.dataflow.dflow INFO: Standard output for task 2249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4f079502-8718-4562-a799-1dce8edc8d8c_isrForCrosstalkSources_996987_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4f079502-8718-4562-a799-1dce8edc8d8c_isrForCrosstalkSources_996987_56.stderr parsl.dataflow.dflow DEBUG: Task 2250 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2250 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2250 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2250 has memoization hash e0d0e5ceecf0aa426397a72d09d14556 parsl.dataflow.memoization INFO: Task 2250 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac07c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2250 try 0 launched on executor multi with executor id 2251 parsl.dataflow.dflow INFO: Standard output for task 2250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c957badd-2c85-4981-ba8e-d2fe52826c75_isrForCrosstalkSources_997020_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c957badd-2c85-4981-ba8e-d2fe52826c75_isrForCrosstalkSources_997020_43.stderr parsl.dataflow.dflow DEBUG: Task 2251 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2251 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2251 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2251 has memoization hash 130cfb5dcd5c750f552a774c6266e6b3 parsl.dataflow.memoization INFO: Task 2251 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2251 try 0 launched on executor multi with executor id 2252 parsl.dataflow.dflow INFO: Standard output for task 2251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/13cb8a34-07d3-43ba-ad9b-eef47bb41aeb_isrForCrosstalkSources_996992_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/13cb8a34-07d3-43ba-ad9b-eef47bb41aeb_isrForCrosstalkSources_996992_44.stderr parsl.dataflow.dflow DEBUG: Task 2252 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2252 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2252 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2252 has memoization hash 68e5d074c3a4095c3026a074e95ae97f parsl.dataflow.memoization INFO: Task 2252 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac09a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2252 try 0 launched on executor multi with executor id 2253 parsl.dataflow.dflow INFO: Standard output for task 2252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/ca0513d6-53c0-41e3-a7a7-75b173e4a97f_isrForCrosstalkSources_996958_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/ca0513d6-53c0-41e3-a7a7-75b173e4a97f_isrForCrosstalkSources_996958_28.stderr parsl.dataflow.dflow DEBUG: Task 2253 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2253 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2253 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2253 has memoization hash 4b538382c43657b0e5a057373c9a9e08 parsl.dataflow.memoization INFO: Task 2253 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2253 try 0 launched on executor multi with executor id 2254 parsl.dataflow.dflow INFO: Standard output for task 2253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/233dcb1a-553e-41ae-9414-edba00dbab55_isrForCrosstalkSources_996998_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/233dcb1a-553e-41ae-9414-edba00dbab55_isrForCrosstalkSources_996998_4.stderr parsl.dataflow.dflow DEBUG: Task 2254 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2254 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2254 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2254 has memoization hash 60fed9f850fdcdef257d2c48408bc30b parsl.dataflow.memoization INFO: Task 2254 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2254 try 0 launched on executor multi with executor id 2255 parsl.dataflow.dflow INFO: Standard output for task 2254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/6ae5303a-657c-46f0-9d54-07b369308139_isrForCrosstalkSources_997026_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/6ae5303a-657c-46f0-9d54-07b369308139_isrForCrosstalkSources_997026_47.stderr parsl.dataflow.dflow DEBUG: Task 2255 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2255 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2255 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2255 has memoization hash f0449b009f6d3b7cdf970609033467ef parsl.dataflow.memoization INFO: Task 2255 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2255 try 0 launched on executor multi with executor id 2256 parsl.dataflow.dflow INFO: Standard output for task 2255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8c7afdb1-f57e-48ff-bd55-76c31b6c70c2_isrForCrosstalkSources_996976_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8c7afdb1-f57e-48ff-bd55-76c31b6c70c2_isrForCrosstalkSources_996976_7.stderr parsl.dataflow.dflow DEBUG: Task 2256 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2256 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2256 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2256 has memoization hash a95861e8fb29f2b1fbe5097037397ff2 parsl.dataflow.memoization INFO: Task 2256 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2256 try 0 launched on executor multi with executor id 2257 parsl.dataflow.dflow INFO: Standard output for task 2256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/99bd8550-0ace-4838-8af9-96fbf1b076fb_isrForCrosstalkSources_996953_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/99bd8550-0ace-4838-8af9-96fbf1b076fb_isrForCrosstalkSources_996953_27.stderr parsl.dataflow.dflow DEBUG: Task 2257 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2257 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2257 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2257 has memoization hash 26577bece696caf3063fdfec93292d73 parsl.dataflow.memoization INFO: Task 2257 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2257 try 0 launched on executor multi with executor id 2258 parsl.dataflow.dflow INFO: Standard output for task 2257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/42f1b564-9f52-4e12-bf48-5e7b68b7483f_isrForCrosstalkSources_996996_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/42f1b564-9f52-4e12-bf48-5e7b68b7483f_isrForCrosstalkSources_996996_16.stderr parsl.dataflow.dflow DEBUG: Task 2258 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2258 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2258 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2258 has memoization hash 86f2a91f76c55928c61ee58083e4d212 parsl.dataflow.memoization INFO: Task 2258 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2258 try 0 launched on executor multi with executor id 2259 parsl.dataflow.dflow INFO: Standard output for task 2258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d375b4c0-859e-437b-abe6-2a112e8c1cd5_isrForCrosstalkSources_996995_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d375b4c0-859e-437b-abe6-2a112e8c1cd5_isrForCrosstalkSources_996995_14.stderr parsl.dataflow.dflow DEBUG: Task 2259 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2259 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2259 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2259 has memoization hash 19fc54bdf126089ecce01b5c1b3ded7d parsl.dataflow.memoization INFO: Task 2259 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2259 try 0 launched on executor multi with executor id 2260 parsl.dataflow.dflow INFO: Standard output for task 2259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/db38e2fe-6f5f-41af-a93b-78c7554cab49_isrForCrosstalkSources_996971_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/db38e2fe-6f5f-41af-a93b-78c7554cab49_isrForCrosstalkSources_996971_1.stderr parsl.dataflow.dflow DEBUG: Task 2260 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2260 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2260 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2260 has memoization hash d90d4bb5fa282f62fe03c5601e715ed6 parsl.dataflow.memoization INFO: Task 2260 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2260 try 0 launched on executor multi with executor id 2261 parsl.dataflow.dflow INFO: Standard output for task 2260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/86846414-849c-4714-8a90-f94442450042_isrForCrosstalkSources_996969_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/86846414-849c-4714-8a90-f94442450042_isrForCrosstalkSources_996969_1.stderr parsl.dataflow.dflow DEBUG: Task 2261 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2261 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2261 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2261 has memoization hash 5f5f9e5f0e0c9b1380a729bb5290b71b parsl.dataflow.memoization INFO: Task 2261 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2261 try 0 launched on executor multi with executor id 2262 parsl.dataflow.dflow INFO: Standard output for task 2261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/a00b213d-64b3-4eff-b311-3ac94c3040e4_isrForCrosstalkSources_996989_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/a00b213d-64b3-4eff-b311-3ac94c3040e4_isrForCrosstalkSources_996989_21.stderr parsl.dataflow.dflow DEBUG: Task 2262 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2262 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2262 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2262 has memoization hash 2e548b8db897a4aaa705e63d2af5786a parsl.dataflow.memoization INFO: Task 2262 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2262 try 0 launched on executor multi with executor id 2263 parsl.dataflow.dflow INFO: Standard output for task 2262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/9638cdc2-c83c-40d1-a2a6-fd267d00b25f_isrForCrosstalkSources_997020_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/9638cdc2-c83c-40d1-a2a6-fd267d00b25f_isrForCrosstalkSources_997020_26.stderr parsl.dataflow.dflow DEBUG: Task 2263 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2263 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2263 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2263 has memoization hash af686d86609a0e8c344852ad6e902fec parsl.dataflow.memoization INFO: Task 2263 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2263 try 0 launched on executor multi with executor id 2264 parsl.dataflow.dflow INFO: Standard output for task 2263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/772ae935-52f3-4095-9185-44be9cdfb449_isrForCrosstalkSources_996963_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/772ae935-52f3-4095-9185-44be9cdfb449_isrForCrosstalkSources_996963_50.stderr parsl.dataflow.dflow DEBUG: Task 2264 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2264 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2264 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2264 has memoization hash bf456a29cf4b31dfb4c742d54241b220 parsl.dataflow.memoization INFO: Task 2264 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac11c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2264 try 0 launched on executor multi with executor id 2265 parsl.dataflow.dflow INFO: Standard output for task 2264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/4239f5b0-82e6-4322-969a-0f635e519ffa_isrForCrosstalkSources_997009_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/4239f5b0-82e6-4322-969a-0f635e519ffa_isrForCrosstalkSources_997009_52.stderr parsl.dataflow.dflow DEBUG: Task 2265 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2265 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2265 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2265 has memoization hash 3d81a3c37c8d6dcefdc1d3a11d3bea98 parsl.dataflow.memoization INFO: Task 2265 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2265 try 0 launched on executor multi with executor id 2266 parsl.dataflow.dflow INFO: Standard output for task 2265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/3132096b-367a-4954-aff2-fa2cce736e71_isrForCrosstalkSources_996987_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/3132096b-367a-4954-aff2-fa2cce736e71_isrForCrosstalkSources_996987_23.stderr parsl.dataflow.dflow DEBUG: Task 2266 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2266 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2266 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2266 has memoization hash 7feaafe87aebb5970f4bf37d13678a19 parsl.dataflow.memoization INFO: Task 2266 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2266 try 0 launched on executor multi with executor id 2267 parsl.dataflow.dflow INFO: Standard output for task 2266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/466a0032-e4d7-4227-9a52-89fa1d35a7d6_isrForCrosstalkSources_996955_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/466a0032-e4d7-4227-9a52-89fa1d35a7d6_isrForCrosstalkSources_996955_25.stderr parsl.dataflow.dflow DEBUG: Task 2267 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2267 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2267 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2267 has memoization hash ca2ca0fcb4734202fa1d783630fb5079 parsl.dataflow.memoization INFO: Task 2267 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2267 try 0 launched on executor multi with executor id 2268 parsl.dataflow.dflow INFO: Standard output for task 2267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/05e0e374-0ff8-4cfd-ae4d-ef8d8e0f5145_isrForCrosstalkSources_997032_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/05e0e374-0ff8-4cfd-ae4d-ef8d8e0f5145_isrForCrosstalkSources_997032_10.stderr parsl.dataflow.dflow DEBUG: Task 2268 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2268 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2268 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2268 has memoization hash a254129a91457c7d6ab01a47878adb94 parsl.dataflow.memoization INFO: Task 2268 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac13a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2268 try 0 launched on executor multi with executor id 2269 parsl.dataflow.dflow INFO: Standard output for task 2268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/038acdd3-55d5-4140-894b-34ce95fc2f30_isrForCrosstalkSources_997030_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/038acdd3-55d5-4140-894b-34ce95fc2f30_isrForCrosstalkSources_997030_10.stderr parsl.dataflow.dflow DEBUG: Task 2269 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2269 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2269 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2269 has memoization hash b0917ce5c10056246ef37c992de0fee9 parsl.dataflow.memoization INFO: Task 2269 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac14e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2269 try 0 launched on executor multi with executor id 2270 parsl.dataflow.dflow INFO: Standard output for task 2269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/7af39a1c-9c87-4404-a5de-0016975e18f9_isrForCrosstalkSources_997000_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/7af39a1c-9c87-4404-a5de-0016975e18f9_isrForCrosstalkSources_997000_29.stderr parsl.dataflow.dflow DEBUG: Task 2270 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2270 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2270 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2270 has memoization hash a70d1162f5cb0bf319445524e2c976b9 parsl.dataflow.memoization INFO: Task 2270 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2270 try 0 launched on executor multi with executor id 2271 parsl.dataflow.dflow INFO: Standard output for task 2270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c28096fc-d850-437e-9c4b-851f34369d19_isrForCrosstalkSources_996975_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c28096fc-d850-437e-9c4b-851f34369d19_isrForCrosstalkSources_996975_10.stderr parsl.dataflow.dflow DEBUG: Task 2271 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2271 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2271 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2271 has memoization hash dda5bd703ead9cc9a0aad47371d88361 parsl.dataflow.memoization INFO: Task 2271 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2271 try 0 launched on executor multi with executor id 2272 parsl.dataflow.dflow INFO: Standard output for task 2271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e38ee920-d98f-4a37-a43e-9afa535be748_isrForCrosstalkSources_996963_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e38ee920-d98f-4a37-a43e-9afa535be748_isrForCrosstalkSources_996963_57.stderr parsl.dataflow.dflow DEBUG: Task 2272 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2272 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2272 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2272 has memoization hash 87c85341614f239082890b9717442ae3 parsl.dataflow.memoization INFO: Task 2272 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2272 try 0 launched on executor multi with executor id 2273 parsl.dataflow.dflow INFO: Standard output for task 2272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/edb9ed54-5eb0-4f46-87cf-1a252a24bed2_isrForCrosstalkSources_996980_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/edb9ed54-5eb0-4f46-87cf-1a252a24bed2_isrForCrosstalkSources_996980_52.stderr parsl.dataflow.dflow DEBUG: Task 2273 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2273 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2273 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2273 has memoization hash e911f646b764318b62f5d1fa2ada9370 parsl.dataflow.memoization INFO: Task 2273 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac16c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2273 try 0 launched on executor multi with executor id 2274 parsl.dataflow.dflow INFO: Standard output for task 2273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/b32be9ec-57c6-42df-915e-8331344c221c_isrForCrosstalkSources_996989_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/b32be9ec-57c6-42df-915e-8331344c221c_isrForCrosstalkSources_996989_19.stderr parsl.dataflow.dflow DEBUG: Task 2274 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2274 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2274 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2274 has memoization hash 3018d0b28f3d1cced6a41f15c67f37d4 parsl.dataflow.memoization INFO: Task 2274 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2274 try 0 launched on executor multi with executor id 2275 parsl.dataflow.dflow INFO: Standard output for task 2274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0338a735-2141-45f4-9981-af42d67d89a4_isrForCrosstalkSources_996958_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0338a735-2141-45f4-9981-af42d67d89a4_isrForCrosstalkSources_996958_13.stderr parsl.dataflow.dflow DEBUG: Task 2275 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2275 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2275 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2275 has memoization hash 2026eb552709d95ad5c5ba9059b393f9 parsl.dataflow.memoization INFO: Task 2275 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2275 try 0 launched on executor multi with executor id 2276 parsl.dataflow.dflow INFO: Standard output for task 2275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a9ac976d-2fca-414d-bb60-fe192e792eb4_isrForCrosstalkSources_996977_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a9ac976d-2fca-414d-bb60-fe192e792eb4_isrForCrosstalkSources_996977_37.stderr parsl.dataflow.dflow DEBUG: Task 2276 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2276 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2276 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2276 has memoization hash bc9850fd80cd14fc6914bc7a38a019b3 parsl.dataflow.memoization INFO: Task 2276 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac18a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2276 try 0 launched on executor multi with executor id 2277 parsl.dataflow.dflow INFO: Standard output for task 2276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/01dbba16-c818-4614-b8b2-a307031088b7_isrForCrosstalkSources_996965_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/01dbba16-c818-4614-b8b2-a307031088b7_isrForCrosstalkSources_996965_11.stderr parsl.dataflow.dflow DEBUG: Task 2277 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2277 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2277 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2277 has memoization hash c787befff00965a824d918b4d7cf96d0 parsl.dataflow.memoization INFO: Task 2277 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2277 try 0 launched on executor multi with executor id 2278 parsl.dataflow.dflow INFO: Standard output for task 2277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/c94c42ce-b86c-48e0-a197-ccded7a7722d_isrForCrosstalkSources_997018_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/c94c42ce-b86c-48e0-a197-ccded7a7722d_isrForCrosstalkSources_997018_11.stderr parsl.dataflow.dflow DEBUG: Task 2278 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2278 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2278 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2278 has memoization hash 56ae3a22b4efb3fb326ea57a98a3dcde parsl.dataflow.memoization INFO: Task 2278 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac19e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2278 try 0 launched on executor multi with executor id 2279 parsl.dataflow.dflow INFO: Standard output for task 2278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/4bd41279-1372-4d0c-919c-17bfc99a50ef_isrForCrosstalkSources_996983_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/4bd41279-1372-4d0c-919c-17bfc99a50ef_isrForCrosstalkSources_996983_52.stderr parsl.dataflow.dflow DEBUG: Task 2279 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2279 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2279 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2279 has memoization hash d160e61ea809342c35d67b5db551151c parsl.dataflow.memoization INFO: Task 2279 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2279 try 0 launched on executor multi with executor id 2280 parsl.dataflow.dflow INFO: Standard output for task 2279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0aedb66b-80b5-41d0-ab54-f6ad70992aeb_isrForCrosstalkSources_996956_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0aedb66b-80b5-41d0-ab54-f6ad70992aeb_isrForCrosstalkSources_996956_59.stderr parsl.dataflow.dflow DEBUG: Task 2280 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2280 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2280 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2280 has memoization hash c2dd2e5d64c9edf359ef4a6fad0f9bd6 parsl.dataflow.memoization INFO: Task 2280 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2280 try 0 launched on executor multi with executor id 2281 parsl.dataflow.dflow INFO: Standard output for task 2280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/12589320-54a6-45f6-aa34-6b707b2889bd_isrForCrosstalkSources_996952_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/12589320-54a6-45f6-aa34-6b707b2889bd_isrForCrosstalkSources_996952_35.stderr parsl.dataflow.dflow DEBUG: Task 2281 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2281 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2281 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2281 has memoization hash b9647359631b66bf056fc6c9117dbe4c parsl.dataflow.memoization INFO: Task 2281 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2281 try 0 launched on executor multi with executor id 2282 parsl.dataflow.dflow INFO: Standard output for task 2281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b458cb13-9de5-4adf-b2ca-ab1845486254_isrForCrosstalkSources_996996_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b458cb13-9de5-4adf-b2ca-ab1845486254_isrForCrosstalkSources_996996_29.stderr parsl.dataflow.dflow DEBUG: Task 2282 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2282 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2282 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2282 has memoization hash a4f28553e0bcc75a0bc05184598683be parsl.dataflow.memoization INFO: Task 2282 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2282 try 0 launched on executor multi with executor id 2283 parsl.dataflow.dflow INFO: Standard output for task 2282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c63af9f5-a364-46b5-a442-c2c8ff6f03a1_isrForCrosstalkSources_996972_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c63af9f5-a364-46b5-a442-c2c8ff6f03a1_isrForCrosstalkSources_996972_12.stderr parsl.dataflow.dflow DEBUG: Task 2283 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2283 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2283 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2283 has memoization hash 1c02a52cc49ffd6e4989d176d13b66c4 parsl.dataflow.memoization INFO: Task 2283 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2283 try 0 launched on executor multi with executor id 2284 parsl.dataflow.dflow INFO: Standard output for task 2283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/c81522dd-cb26-4619-93d8-42d8400b088b_isrForCrosstalkSources_997011_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/c81522dd-cb26-4619-93d8-42d8400b088b_isrForCrosstalkSources_997011_60.stderr parsl.dataflow.dflow DEBUG: Task 2284 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2284 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2284 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2284 has memoization hash 4a07a1b8f046885d3008e5a8a2a1d77b parsl.dataflow.memoization INFO: Task 2284 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2284 try 0 launched on executor multi with executor id 2285 parsl.dataflow.dflow INFO: Standard output for task 2284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7b8b24b4-7655-4878-9b39-e315906475a9_isrForCrosstalkSources_997005_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7b8b24b4-7655-4878-9b39-e315906475a9_isrForCrosstalkSources_997005_10.stderr parsl.dataflow.dflow DEBUG: Task 2285 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2285 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2285 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2285 has memoization hash 46315a5a0ac1973ba56dcb9dd8d98d1a parsl.dataflow.memoization INFO: Task 2285 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2285 try 0 launched on executor multi with executor id 2286 parsl.dataflow.dflow INFO: Standard output for task 2285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/05270c06-1a63-4414-94c5-c0d6bb7718e8_isrForCrosstalkSources_996997_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/05270c06-1a63-4414-94c5-c0d6bb7718e8_isrForCrosstalkSources_996997_57.stderr parsl.dataflow.dflow DEBUG: Task 2286 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2286 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2286 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2286 has memoization hash afefd1f9ea57072f94ed384be36e10fa parsl.dataflow.memoization INFO: Task 2286 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2286 try 0 launched on executor multi with executor id 2287 parsl.dataflow.dflow INFO: Standard output for task 2286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/68de810e-90e2-44fc-b93b-73cc436991af_isrForCrosstalkSources_996976_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/68de810e-90e2-44fc-b93b-73cc436991af_isrForCrosstalkSources_996976_23.stderr parsl.dataflow.dflow DEBUG: Task 2287 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2287 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2287 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2287 has memoization hash 621e1d68b5bd4021283ab79eeb479f70 parsl.dataflow.memoization INFO: Task 2287 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac1f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2287 try 0 launched on executor multi with executor id 2288 parsl.dataflow.dflow INFO: Standard output for task 2287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f3895cb1-0870-4185-8200-714eae2dad49_isrForCrosstalkSources_996955_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f3895cb1-0870-4185-8200-714eae2dad49_isrForCrosstalkSources_996955_52.stderr parsl.dataflow.dflow DEBUG: Task 2288 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2288 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2288 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2288 has memoization hash c703812c55cb35194710b5d359b712f2 parsl.dataflow.memoization INFO: Task 2288 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2288 try 0 launched on executor multi with executor id 2289 parsl.dataflow.dflow INFO: Standard output for task 2288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b292662f-ac07-4c9d-bbd7-da431635549b_isrForCrosstalkSources_996979_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b292662f-ac07-4c9d-bbd7-da431635549b_isrForCrosstalkSources_996979_57.stderr parsl.dataflow.dflow DEBUG: Task 2289 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2289 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2289 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2289 has memoization hash aabb5458773965c8f715fd6d7314948f parsl.dataflow.memoization INFO: Task 2289 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2289 try 0 launched on executor multi with executor id 2290 parsl.dataflow.dflow INFO: Standard output for task 2289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/e3af2b1b-e718-4d3e-8736-ecf47241cf8a_isrForCrosstalkSources_997015_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/e3af2b1b-e718-4d3e-8736-ecf47241cf8a_isrForCrosstalkSources_997015_32.stderr parsl.dataflow.dflow DEBUG: Task 2290 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2290 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2290 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2290 has memoization hash 1f1a0740422c88d731cb597b91ca9c19 parsl.dataflow.memoization INFO: Task 2290 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac20c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2290 try 0 launched on executor multi with executor id 2291 parsl.dataflow.dflow INFO: Standard output for task 2290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7daa3566-174b-4bcc-8810-25445ea85b3c_isrForCrosstalkSources_996953_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7daa3566-174b-4bcc-8810-25445ea85b3c_isrForCrosstalkSources_996953_32.stderr parsl.dataflow.dflow DEBUG: Task 2291 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2291 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2291 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2291 has memoization hash 8595ec834d181da415f4ff8c55154564 parsl.dataflow.memoization INFO: Task 2291 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2291 try 0 launched on executor multi with executor id 2292 parsl.dataflow.dflow INFO: Standard output for task 2291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8472e40c-4f4a-4bb5-bcb2-6f45cf93dbf3_isrForCrosstalkSources_996956_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8472e40c-4f4a-4bb5-bcb2-6f45cf93dbf3_isrForCrosstalkSources_996956_25.stderr parsl.dataflow.dflow DEBUG: Task 2292 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2292 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2292 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2292 has memoization hash 23c0ecc8d60b7674ac8e9f3e9c2bd9c5 parsl.dataflow.memoization INFO: Task 2292 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac22a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2292 try 0 launched on executor multi with executor id 2293 parsl.dataflow.dflow INFO: Standard output for task 2292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f978894b-c412-4e29-a463-847e0203483a_isrForCrosstalkSources_997028_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f978894b-c412-4e29-a463-847e0203483a_isrForCrosstalkSources_997028_33.stderr parsl.dataflow.dflow DEBUG: Task 2293 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2293 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2293 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2293 has memoization hash 8cfc2dbd969ddee15524d3981ba9dbc5 parsl.dataflow.memoization INFO: Task 2293 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2293 try 0 launched on executor multi with executor id 2294 parsl.dataflow.dflow INFO: Standard output for task 2293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/b1559aab-078f-4e4f-84a1-de6bbddd4720_isrForCrosstalkSources_996954_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/b1559aab-078f-4e4f-84a1-de6bbddd4720_isrForCrosstalkSources_996954_1.stderr parsl.dataflow.dflow DEBUG: Task 2294 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2294 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2294 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2294 has memoization hash 93702a911fd512d9974d61ab31b2cd2a parsl.dataflow.memoization INFO: Task 2294 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2294 try 0 launched on executor multi with executor id 2295 parsl.dataflow.dflow INFO: Standard output for task 2294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/02b3a8a3-f33a-4aae-b351-0d37b039e45b_isrForCrosstalkSources_996953_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/02b3a8a3-f33a-4aae-b351-0d37b039e45b_isrForCrosstalkSources_996953_51.stderr parsl.dataflow.dflow DEBUG: Task 2295 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2295 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2295 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2295 has memoization hash 76c9ff24184d26ab2d5a147d3dbb10f6 parsl.dataflow.memoization INFO: Task 2295 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac23e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2295 try 0 launched on executor multi with executor id 2296 parsl.dataflow.dflow INFO: Standard output for task 2295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f6fba253-d409-4da3-95dd-09946f57cc7d_isrForCrosstalkSources_996988_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f6fba253-d409-4da3-95dd-09946f57cc7d_isrForCrosstalkSources_996988_19.stderr parsl.dataflow.dflow DEBUG: Task 2296 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2296 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2296 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2296 has memoization hash 5fca6ff367c5506c79489c9b1e97a686 parsl.dataflow.memoization INFO: Task 2296 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2296 try 0 launched on executor multi with executor id 2297 parsl.dataflow.dflow INFO: Standard output for task 2296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/8626658e-4568-4120-bf54-d591a8db30b3_isrForCrosstalkSources_997031_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/8626658e-4568-4120-bf54-d591a8db30b3_isrForCrosstalkSources_997031_29.stderr parsl.dataflow.dflow DEBUG: Task 2297 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2297 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2297 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2297 has memoization hash 517ea7ca57370e339ce47a9cc7f98ea3 parsl.dataflow.memoization INFO: Task 2297 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac25c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2297 try 0 launched on executor multi with executor id 2298 parsl.dataflow.dflow INFO: Standard output for task 2297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/35167d03-9076-428b-bf4f-77b6c2b3a570_isrForCrosstalkSources_997002_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/35167d03-9076-428b-bf4f-77b6c2b3a570_isrForCrosstalkSources_997002_8.stderr parsl.dataflow.dflow DEBUG: Task 2298 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2298 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2298 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2298 has memoization hash 39ea1864b497bf9f4dcc4151b58cd13f parsl.dataflow.memoization INFO: Task 2298 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2298 try 0 launched on executor multi with executor id 2299 parsl.dataflow.dflow INFO: Standard output for task 2298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc761b97-bc75-4cb6-b95c-e29c5e1553fa_isrForCrosstalkSources_997025_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc761b97-bc75-4cb6-b95c-e29c5e1553fa_isrForCrosstalkSources_997025_22.stderr parsl.dataflow.dflow DEBUG: Task 2299 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2299 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2299 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2299 has memoization hash f489279c1f6ebca5bdd031e7a9394962 parsl.dataflow.memoization INFO: Task 2299 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac27a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2299 try 0 launched on executor multi with executor id 2300 parsl.dataflow.dflow INFO: Standard output for task 2299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/61b50259-ce11-49a8-9f58-e5a2dc76ec81_isrForCrosstalkSources_996958_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/61b50259-ce11-49a8-9f58-e5a2dc76ec81_isrForCrosstalkSources_996958_34.stderr parsl.dataflow.dflow DEBUG: Task 2300 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2300 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2300 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2300 has memoization hash ec5fb436118ff2faa876b682cf1b8b76 parsl.dataflow.memoization INFO: Task 2300 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2300 try 0 launched on executor multi with executor id 2301 parsl.dataflow.dflow INFO: Standard output for task 2300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d3df1fa7-a626-4320-a93f-d15b1aa620ac_isrForCrosstalkSources_996972_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d3df1fa7-a626-4320-a93f-d15b1aa620ac_isrForCrosstalkSources_996972_2.stderr parsl.dataflow.dflow DEBUG: Task 2301 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2301 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2301 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2301 has memoization hash 308830a56d5454f088cf479839f256d8 parsl.dataflow.memoization INFO: Task 2301 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2301 try 0 launched on executor multi with executor id 2302 parsl.dataflow.dflow INFO: Standard output for task 2301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/134fe129-e407-4822-9f00-2115478ff8e2_isrForCrosstalkSources_997000_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/134fe129-e407-4822-9f00-2115478ff8e2_isrForCrosstalkSources_997000_61.stderr parsl.dataflow.dflow DEBUG: Task 2302 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2302 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2302 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2302 has memoization hash eac6ad66baa78512706bc4db84f935e0 parsl.dataflow.memoization INFO: Task 2302 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac28e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2302 try 0 launched on executor multi with executor id 2303 parsl.dataflow.dflow INFO: Standard output for task 2302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b6d5aebd-03d4-4d8e-8fbf-df34cb614d5c_isrForCrosstalkSources_997004_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b6d5aebd-03d4-4d8e-8fbf-df34cb614d5c_isrForCrosstalkSources_997004_39.stderr parsl.dataflow.dflow DEBUG: Task 2303 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2303 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2303 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2303 has memoization hash bd679444d09739899e8d4e81991f7bb5 parsl.dataflow.memoization INFO: Task 2303 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2303 try 0 launched on executor multi with executor id 2304 parsl.dataflow.dflow INFO: Standard output for task 2303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/370cd279-35fb-42cd-bd6e-c373a3a3a387_isrForCrosstalkSources_996958_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/370cd279-35fb-42cd-bd6e-c373a3a3a387_isrForCrosstalkSources_996958_21.stderr parsl.dataflow.dflow DEBUG: Task 2304 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2304 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2304 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2304 has memoization hash 0bb70e4674486597c6b9e4907ed008f8 parsl.dataflow.memoization INFO: Task 2304 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2304 try 0 launched on executor multi with executor id 2305 parsl.dataflow.dflow INFO: Standard output for task 2304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8407d9fc-c0ad-4bbb-83a7-6a048bd9dc08_isrForCrosstalkSources_996971_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8407d9fc-c0ad-4bbb-83a7-6a048bd9dc08_isrForCrosstalkSources_996971_50.stderr parsl.dataflow.dflow DEBUG: Task 2305 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2305 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2305 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2305 has memoization hash ae76bfa2b59a6b03c07f1b1425a8367f parsl.dataflow.memoization INFO: Task 2305 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2305 try 0 launched on executor multi with executor id 2306 parsl.dataflow.dflow INFO: Standard output for task 2305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/dc9f8eb9-1d0e-43a2-b8c7-69b8daa31cbe_isrForCrosstalkSources_996955_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/dc9f8eb9-1d0e-43a2-b8c7-69b8daa31cbe_isrForCrosstalkSources_996955_11.stderr parsl.dataflow.dflow DEBUG: Task 2306 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2306 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2306 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2306 has memoization hash 23ea64fbbf4c4389092e93e7ec4c1437 parsl.dataflow.memoization INFO: Task 2306 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2306 try 0 launched on executor multi with executor id 2307 parsl.dataflow.dflow INFO: Standard output for task 2306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a8abc9e2-c90c-48cc-b245-59407b65737b_isrForCrosstalkSources_997020_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a8abc9e2-c90c-48cc-b245-59407b65737b_isrForCrosstalkSources_997020_47.stderr parsl.dataflow.dflow DEBUG: Task 2307 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2307 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2307 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2307 has memoization hash cc4b7967fd2db112c633949bdc544f92 parsl.dataflow.memoization INFO: Task 2307 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2307 try 0 launched on executor multi with executor id 2308 parsl.dataflow.dflow INFO: Standard output for task 2307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/98e3f3e5-bd41-4c3e-b032-90e775f7d7f3_isrForCrosstalkSources_997021_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/98e3f3e5-bd41-4c3e-b032-90e775f7d7f3_isrForCrosstalkSources_997021_50.stderr parsl.dataflow.dflow DEBUG: Task 2308 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2308 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2308 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2308 has memoization hash 7ed81afed29cf8b09b3b5bde9d0670ba parsl.dataflow.memoization INFO: Task 2308 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2308 try 0 launched on executor multi with executor id 2309 parsl.dataflow.dflow INFO: Standard output for task 2308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b0d5cd1a-76ea-49ed-871b-87dc45786a83_isrForCrosstalkSources_997001_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b0d5cd1a-76ea-49ed-871b-87dc45786a83_isrForCrosstalkSources_997001_22.stderr parsl.dataflow.dflow DEBUG: Task 2309 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2309 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2309 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2309 has memoization hash fd70242675cf1c7bfbf8960e20b36c4c parsl.dataflow.memoization INFO: Task 2309 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2309 try 0 launched on executor multi with executor id 2310 parsl.dataflow.dflow INFO: Standard output for task 2309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a3dcb1e9-2ce1-4ecb-858e-e0d7d91fc3a0_isrForCrosstalkSources_996984_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a3dcb1e9-2ce1-4ecb-858e-e0d7d91fc3a0_isrForCrosstalkSources_996984_61.stderr parsl.dataflow.dflow DEBUG: Task 2310 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2310 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2310 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2310 has memoization hash f8998b99d313fd013dc5738b50bc9f2c parsl.dataflow.memoization INFO: Task 2310 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2310 try 0 launched on executor multi with executor id 2311 parsl.dataflow.dflow INFO: Standard output for task 2310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/9af04eba-280f-4b0c-8a65-8f189ef461a0_isrForCrosstalkSources_996973_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/9af04eba-280f-4b0c-8a65-8f189ef461a0_isrForCrosstalkSources_996973_31.stderr parsl.dataflow.dflow DEBUG: Task 2311 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2311 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2311 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2311 has memoization hash 5e614061aa3104a0886411a2e30788e7 parsl.dataflow.memoization INFO: Task 2311 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2311 try 0 launched on executor multi with executor id 2312 parsl.dataflow.dflow INFO: Standard output for task 2311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f1292f42-c500-44ec-a352-c90173ac1293_isrForCrosstalkSources_996958_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f1292f42-c500-44ec-a352-c90173ac1293_isrForCrosstalkSources_996958_58.stderr parsl.dataflow.dflow DEBUG: Task 2312 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2312 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2312 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2312 has memoization hash 557b8a28098c9ea209529d0aea414a97 parsl.dataflow.memoization INFO: Task 2312 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2312 try 0 launched on executor multi with executor id 2313 parsl.dataflow.dflow INFO: Standard output for task 2312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7610221a-6fb0-4df4-bc92-22b22fb34a73_isrForCrosstalkSources_997033_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7610221a-6fb0-4df4-bc92-22b22fb34a73_isrForCrosstalkSources_997033_30.stderr parsl.dataflow.dflow DEBUG: Task 2313 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2313 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2313 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2313 has memoization hash 35ee51dc92947312d4284bebdabf3a37 parsl.dataflow.memoization INFO: Task 2313 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac2fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2313 try 0 launched on executor multi with executor id 2314 parsl.dataflow.dflow INFO: Standard output for task 2313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5c418e62-1f77-49ff-b579-9e02223b6a26_isrForCrosstalkSources_996972_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5c418e62-1f77-49ff-b579-9e02223b6a26_isrForCrosstalkSources_996972_29.stderr parsl.dataflow.dflow DEBUG: Task 2314 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2314 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2314 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2314 has memoization hash 8b64372c4131e701cf2115edfaa6aceb parsl.dataflow.memoization INFO: Task 2314 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2314 try 0 launched on executor multi with executor id 2315 parsl.dataflow.dflow INFO: Standard output for task 2314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/62f3367a-f01c-49a4-96e4-c02353766d00_isrForCrosstalkSources_996953_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/62f3367a-f01c-49a4-96e4-c02353766d00_isrForCrosstalkSources_996953_7.stderr parsl.dataflow.dflow DEBUG: Task 2315 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2315 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2315 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2315 has memoization hash e7de8c73f90ec207c8c65a95f79003ad parsl.dataflow.memoization INFO: Task 2315 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2315 try 0 launched on executor multi with executor id 2316 parsl.dataflow.dflow INFO: Standard output for task 2315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f19ee6af-3e72-4b94-9534-0e2ca3e9dd60_isrForCrosstalkSources_996956_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f19ee6af-3e72-4b94-9534-0e2ca3e9dd60_isrForCrosstalkSources_996956_5.stderr parsl.dataflow.dflow DEBUG: Task 2316 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2316 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2316 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2316 has memoization hash 8af353d2868ebc1b76768cb134c717b4 parsl.dataflow.memoization INFO: Task 2316 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac31a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2316 try 0 launched on executor multi with executor id 2317 parsl.dataflow.dflow INFO: Standard output for task 2316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/38f3938a-a124-4eab-986f-52dfd63e44d8_isrForCrosstalkSources_996952_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/38f3938a-a124-4eab-986f-52dfd63e44d8_isrForCrosstalkSources_996952_4.stderr parsl.dataflow.dflow DEBUG: Task 2317 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2317 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2317 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2317 has memoization hash df10ff9179afc286cf9a1d46b8130ac9 parsl.dataflow.memoization INFO: Task 2317 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2317 try 0 launched on executor multi with executor id 2318 parsl.dataflow.dflow INFO: Standard output for task 2317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f4e662a3-bcae-4069-ac7f-ec660633b6c3_isrForCrosstalkSources_996975_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f4e662a3-bcae-4069-ac7f-ec660633b6c3_isrForCrosstalkSources_996975_49.stderr parsl.dataflow.dflow DEBUG: Task 2318 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2318 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2318 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2318 has memoization hash a8a5a8143f97236dd8074f342d889f74 parsl.dataflow.memoization INFO: Task 2318 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac32e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2318 try 0 launched on executor multi with executor id 2319 parsl.dataflow.dflow INFO: Standard output for task 2318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1b35cfa5-5166-47ef-a34d-a265584fface_isrForCrosstalkSources_996958_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1b35cfa5-5166-47ef-a34d-a265584fface_isrForCrosstalkSources_996958_49.stderr parsl.dataflow.dflow DEBUG: Task 2319 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2319 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2319 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2319 has memoization hash caf7f59801e30f2d3b647a1ce2ffb774 parsl.dataflow.memoization INFO: Task 2319 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2319 try 0 launched on executor multi with executor id 2320 parsl.dataflow.dflow INFO: Standard output for task 2319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/7e16fa75-1547-41dc-a36f-ec710d302b4f_isrForCrosstalkSources_996955_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/7e16fa75-1547-41dc-a36f-ec710d302b4f_isrForCrosstalkSources_996955_39.stderr parsl.dataflow.dflow DEBUG: Task 2320 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2320 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2320 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2320 has memoization hash 6cf1a30fd2bd1a1fe1a6283e3123bb36 parsl.dataflow.memoization INFO: Task 2320 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2320 try 0 launched on executor multi with executor id 2321 parsl.dataflow.dflow INFO: Standard output for task 2320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6148703e-3260-4dfa-88cc-b056dcd24a7a_isrForCrosstalkSources_996957_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6148703e-3260-4dfa-88cc-b056dcd24a7a_isrForCrosstalkSources_996957_59.stderr parsl.dataflow.dflow DEBUG: Task 2321 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2321 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2321 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2321 has memoization hash adcd7a4f792bc9742ca58e5f4f910fd8 parsl.dataflow.memoization INFO: Task 2321 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac34c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2321 try 0 launched on executor multi with executor id 2322 parsl.dataflow.dflow INFO: Standard output for task 2321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2ea93fa7-11a4-4ec2-b89f-4c870e07444f_isrForCrosstalkSources_997003_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2ea93fa7-11a4-4ec2-b89f-4c870e07444f_isrForCrosstalkSources_997003_21.stderr parsl.dataflow.dflow DEBUG: Task 2322 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2322 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2322 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2322 has memoization hash ef1443d77e3d9ee7856c3328f876d082 parsl.dataflow.memoization INFO: Task 2322 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2322 try 0 launched on executor multi with executor id 2323 parsl.dataflow.dflow INFO: Standard output for task 2322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/49717cbe-fda2-48be-ba7d-4faf2b3af477_isrForCrosstalkSources_997029_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/49717cbe-fda2-48be-ba7d-4faf2b3af477_isrForCrosstalkSources_997029_44.stderr parsl.dataflow.dflow DEBUG: Task 2323 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2323 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2323 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2323 has memoization hash d89bce5aa42abbabc284735f90f8d8e7 parsl.dataflow.memoization INFO: Task 2323 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2323 try 0 launched on executor multi with executor id 2324 parsl.dataflow.dflow INFO: Standard output for task 2323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5eb0b2ed-cf44-43d8-9fa7-3f2d59941ba4_isrForCrosstalkSources_997018_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5eb0b2ed-cf44-43d8-9fa7-3f2d59941ba4_isrForCrosstalkSources_997018_46.stderr parsl.dataflow.dflow DEBUG: Task 2324 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2324 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2324 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2324 has memoization hash 5a993a47afc38f4cd67794fa6dd52103 parsl.dataflow.memoization INFO: Task 2324 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2324 try 0 launched on executor multi with executor id 2325 parsl.dataflow.dflow INFO: Standard output for task 2324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7c517b2a-1633-4e0d-aa36-06a022fac9f5_isrForCrosstalkSources_997033_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7c517b2a-1633-4e0d-aa36-06a022fac9f5_isrForCrosstalkSources_997033_44.stderr parsl.dataflow.dflow DEBUG: Task 2325 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2325 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2325 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2325 has memoization hash 42048a06ddfb7296048137624027e242 parsl.dataflow.memoization INFO: Task 2325 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac36a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2325 try 0 launched on executor multi with executor id 2326 parsl.dataflow.dflow INFO: Standard output for task 2325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ded12831-5a56-4c49-a64a-64db7727d7fd_isrForCrosstalkSources_997020_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ded12831-5a56-4c49-a64a-64db7727d7fd_isrForCrosstalkSources_997020_11.stderr parsl.dataflow.dflow DEBUG: Task 2326 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2326 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2326 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2326 has memoization hash da413637f565cce703cfe68c76c01ee7 parsl.dataflow.memoization INFO: Task 2326 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac37e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2326 try 0 launched on executor multi with executor id 2327 parsl.dataflow.dflow INFO: Standard output for task 2326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/b12f7786-096a-4f76-9bdf-6f56ccd63e9f_isrForCrosstalkSources_996974_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/b12f7786-096a-4f76-9bdf-6f56ccd63e9f_isrForCrosstalkSources_996974_61.stderr parsl.dataflow.dflow DEBUG: Task 2327 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2327 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2327 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2327 has memoization hash 84b605b9d3bbefa383228e9f0e0111cc parsl.dataflow.memoization INFO: Task 2327 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2327 try 0 launched on executor multi with executor id 2328 parsl.dataflow.dflow INFO: Standard output for task 2327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/b393910a-37a5-446f-86b6-514de41396f3_isrForCrosstalkSources_997023_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/b393910a-37a5-446f-86b6-514de41396f3_isrForCrosstalkSources_997023_41.stderr parsl.dataflow.dflow DEBUG: Task 2328 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2328 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2328 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2328 has memoization hash d9865362fe8c01578c7b8ea934237230 parsl.dataflow.memoization INFO: Task 2328 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2328 try 0 launched on executor multi with executor id 2329 parsl.dataflow.dflow INFO: Standard output for task 2328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/322afac2-ec5c-4d41-a621-a8fbf199d08d_isrForCrosstalkSources_996984_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/322afac2-ec5c-4d41-a621-a8fbf199d08d_isrForCrosstalkSources_996984_50.stderr parsl.dataflow.dflow DEBUG: Task 2329 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2329 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2329 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2329 has memoization hash b9356dca9fb2532e98ce424ca54b8772 parsl.dataflow.memoization INFO: Task 2329 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2329 try 0 launched on executor multi with executor id 2330 parsl.dataflow.dflow INFO: Standard output for task 2329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a46628d5-9765-4267-a426-d2a6a8e3a74e_isrForCrosstalkSources_996983_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a46628d5-9765-4267-a426-d2a6a8e3a74e_isrForCrosstalkSources_996983_39.stderr parsl.dataflow.dflow DEBUG: Task 2330 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2330 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2330 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2330 has memoization hash 5ba203a200a9e9d362d8abc6bc89085e parsl.dataflow.memoization INFO: Task 2330 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac39c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:48 proc_lsst.multi:170 [INFO] [multi] job_id 17074872 proc_lsst.multi INFO: [multi] job_id 17074872 2024-03-15 13:49:48 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 26 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 26 2024-03-15 13:49:48 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:48 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074872 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074872 parsl.executors.status_handling DEBUG: Launched block 26 on executor multi with job ID 17074872 parsl.executors.status_handling INFO: Allocated block ID 27 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-27 2024-03-15 13:49:48 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=27 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-27 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=27 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-27 2024-03-15 13:49:48 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=27 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=27 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:48 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=27 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=27 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 26 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 26 2024-03-15 13:49:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:48 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=27 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=27 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 2330 try 0 launched on executor multi with executor id 2331 parsl.dataflow.dflow INFO: Standard output for task 2330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/585c3807-0007-43ac-8b82-dab2edad9d46_isrForCrosstalkSources_997031_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/585c3807-0007-43ac-8b82-dab2edad9d46_isrForCrosstalkSources_997031_54.stderr parsl.dataflow.dflow DEBUG: Task 2331 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2331 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2331 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2331 has memoization hash e44d39183eb9ba22b0ec7bc609f1ce7e parsl.dataflow.memoization INFO: Task 2331 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2331 try 0 launched on executor multi with executor id 2332 parsl.dataflow.dflow INFO: Standard output for task 2331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c5966daf-5fee-4b1d-b7ab-f67a08e2e1ae_isrForCrosstalkSources_997006_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c5966daf-5fee-4b1d-b7ab-f67a08e2e1ae_isrForCrosstalkSources_997006_17.stderr parsl.dataflow.dflow DEBUG: Task 2332 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2332 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2332 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2332 has memoization hash 7545e7d21602261d5c750f3f72ee39b4 parsl.dataflow.memoization INFO: Task 2332 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2332 try 0 launched on executor multi with executor id 2333 parsl.dataflow.dflow INFO: Standard output for task 2332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/13021847-a9c9-4e24-9979-93e1384949ae_isrForCrosstalkSources_996985_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/13021847-a9c9-4e24-9979-93e1384949ae_isrForCrosstalkSources_996985_57.stderr parsl.dataflow.dflow DEBUG: Task 2333 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2333 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2333 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2333 has memoization hash cb139be7dfc4898820446094a3d24930 parsl.dataflow.memoization INFO: Task 2333 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cc5d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 2333 try 0 launched on executor multi with executor id 2334 parsl.dataflow.dflow INFO: Standard output for task 2333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/4fd1d483-7b1b-477e-ab4c-c23b06167ed2_isrForCrosstalkSources_997027_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/4fd1d483-7b1b-477e-ab4c-c23b06167ed2_isrForCrosstalkSources_997027_45.stderr parsl.dataflow.dflow DEBUG: Task 2334 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2334 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2334 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2334 has memoization hash 63ba768684b609e070e508713ccc4fab parsl.dataflow.memoization INFO: Task 2334 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac0040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2334 try 0 launched on executor multi with executor id 2335 parsl.dataflow.dflow INFO: Standard output for task 2334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e9c698c8-8cca-4b19-9460-5d3fea9761d4_isrForCrosstalkSources_996952_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e9c698c8-8cca-4b19-9460-5d3fea9761d4_isrForCrosstalkSources_996952_51.stderr parsl.dataflow.dflow DEBUG: Task 2335 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2335 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2335 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2335 has memoization hash c27747e1d1f16ae150c76a10eb3c05ad parsl.dataflow.memoization INFO: Task 2335 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2335 try 0 launched on executor multi with executor id 2336 parsl.dataflow.dflow INFO: Standard output for task 2335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/95c4b543-aec5-45a8-9c64-3f2d76468d9d_isrForCrosstalkSources_996986_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/95c4b543-aec5-45a8-9c64-3f2d76468d9d_isrForCrosstalkSources_996986_50.stderr parsl.dataflow.dflow DEBUG: Task 2336 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2336 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2336 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2336 has memoization hash 5def5c50252b430300132e5c121ab6b5 parsl.dataflow.memoization INFO: Task 2336 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2336 try 0 launched on executor multi with executor id 2337 parsl.dataflow.dflow INFO: Standard output for task 2336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ce6ad8ca-22f5-4298-b4ca-8d5df929a612_isrForCrosstalkSources_997029_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ce6ad8ca-22f5-4298-b4ca-8d5df929a612_isrForCrosstalkSources_997029_29.stderr parsl.dataflow.dflow DEBUG: Task 2337 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2337 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2337 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2337 has memoization hash d92105bc9c53d88c031778ba0411c98b parsl.dataflow.memoization INFO: Task 2337 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2337 try 0 launched on executor multi with executor id 2338 parsl.dataflow.dflow INFO: Standard output for task 2337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/686f3651-adf3-4ef3-a66f-896f09b34367_isrForCrosstalkSources_996986_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/686f3651-adf3-4ef3-a66f-896f09b34367_isrForCrosstalkSources_996986_28.stderr parsl.dataflow.dflow DEBUG: Task 2338 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2338 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2338 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2338 has memoization hash 8b1b0622996f54cce7f7e995794abb0b parsl.dataflow.memoization INFO: Task 2338 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2338 try 0 launched on executor multi with executor id 2339 parsl.dataflow.dflow INFO: Standard output for task 2338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7f8deade-5b9e-4461-8ee4-a678369317f7_isrForCrosstalkSources_996963_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7f8deade-5b9e-4461-8ee4-a678369317f7_isrForCrosstalkSources_996963_49.stderr parsl.dataflow.dflow DEBUG: Task 2339 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2339 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2339 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2339 has memoization hash beb660800f587a192c173623e97b92b5 parsl.dataflow.memoization INFO: Task 2339 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2339 try 0 launched on executor multi with executor id 2340 parsl.dataflow.dflow INFO: Standard output for task 2339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/7d900203-d89c-4f77-933c-4b5f12403f4b_isrForCrosstalkSources_996966_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/7d900203-d89c-4f77-933c-4b5f12403f4b_isrForCrosstalkSources_996966_46.stderr parsl.dataflow.dflow DEBUG: Task 2340 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2340 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2340 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2340 has memoization hash 75d53baa1c53abb333fc894f70280e0e parsl.dataflow.memoization INFO: Task 2340 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2340 try 0 launched on executor multi with executor id 2341 parsl.dataflow.dflow INFO: Standard output for task 2340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/15766452-75c0-4266-a55e-51ae9d3a27f7_isrForCrosstalkSources_996996_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/15766452-75c0-4266-a55e-51ae9d3a27f7_isrForCrosstalkSources_996996_26.stderr parsl.dataflow.dflow DEBUG: Task 2341 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2341 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2341 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2341 has memoization hash 6f3c60a7dffde3621c498701fcfbd83c parsl.dataflow.memoization INFO: Task 2341 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2341 try 0 launched on executor multi with executor id 2342 parsl.dataflow.dflow INFO: Standard output for task 2341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ac9668e7-52c2-4d98-a6de-353aca3dce4d_isrForCrosstalkSources_996984_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ac9668e7-52c2-4d98-a6de-353aca3dce4d_isrForCrosstalkSources_996984_5.stderr parsl.dataflow.dflow DEBUG: Task 2342 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2342 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2342 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2342 has memoization hash 5f73937dbf80e40dc27b3dd9ae820502 parsl.dataflow.memoization INFO: Task 2342 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2342 try 0 launched on executor multi with executor id 2343 parsl.dataflow.dflow INFO: Standard output for task 2342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4fd2b40e-2f7a-4d3d-98e7-00f0e9b7d140_isrForCrosstalkSources_996962_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4fd2b40e-2f7a-4d3d-98e7-00f0e9b7d140_isrForCrosstalkSources_996962_36.stderr parsl.dataflow.dflow DEBUG: Task 2343 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2343 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2343 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2343 has memoization hash facebf3539f3bd435744a26564ee1f63 parsl.dataflow.memoization INFO: Task 2343 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb182c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2343 try 0 launched on executor multi with executor id 2344 parsl.dataflow.dflow INFO: Standard output for task 2343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/195b8f09-1561-4bb9-8984-c1e710b05d70_isrForCrosstalkSources_997020_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/195b8f09-1561-4bb9-8984-c1e710b05d70_isrForCrosstalkSources_997020_1.stderr parsl.dataflow.dflow DEBUG: Task 2344 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2344 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2344 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2344 has memoization hash b6a96f3636feda4feff149c361ee6189 parsl.dataflow.memoization INFO: Task 2344 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2344 try 0 launched on executor multi with executor id 2345 parsl.dataflow.dflow INFO: Standard output for task 2344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/db941cf0-74db-47bb-a16b-a1c374367d35_isrForCrosstalkSources_997000_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/db941cf0-74db-47bb-a16b-a1c374367d35_isrForCrosstalkSources_997000_59.stderr parsl.dataflow.dflow DEBUG: Task 2345 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2345 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2345 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2345 has memoization hash ad40f45959d4df9f9b888020e12f07bc parsl.dataflow.memoization INFO: Task 2345 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2345 try 0 launched on executor multi with executor id 2346 parsl.dataflow.dflow INFO: Standard output for task 2345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/50aabd07-16ab-4121-ad26-867684a5b22d_isrForCrosstalkSources_996989_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/50aabd07-16ab-4121-ad26-867684a5b22d_isrForCrosstalkSources_996989_29.stderr parsl.dataflow.dflow DEBUG: Task 2346 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2346 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2346 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2346 has memoization hash b056e941ac6aa0c9f5b0b6d53497f457 parsl.dataflow.memoization INFO: Task 2346 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb184a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2346 try 0 launched on executor multi with executor id 2347 parsl.dataflow.dflow INFO: Standard output for task 2346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0baafbfe-a18f-44c6-957e-7181a455151a_isrForCrosstalkSources_997010_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0baafbfe-a18f-44c6-957e-7181a455151a_isrForCrosstalkSources_997010_60.stderr parsl.dataflow.dflow DEBUG: Task 2347 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2347 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2347 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2347 has memoization hash 82e6927c85796de801fbab8253e1c5b8 parsl.dataflow.memoization INFO: Task 2347 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2347 try 0 launched on executor multi with executor id 2348 parsl.dataflow.dflow INFO: Standard output for task 2347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/08ecb673-2cd6-49a1-8bcd-dbd666546e46_isrForCrosstalkSources_997014_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/08ecb673-2cd6-49a1-8bcd-dbd666546e46_isrForCrosstalkSources_997014_52.stderr parsl.dataflow.dflow DEBUG: Task 2348 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2348 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2348 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2348 has memoization hash d682ff1e123ec71be23497986cd8df1c parsl.dataflow.memoization INFO: Task 2348 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb185e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2348 try 0 launched on executor multi with executor id 2349 parsl.dataflow.dflow INFO: Standard output for task 2348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/b969317e-641f-4702-841f-a79d273d0317_isrForCrosstalkSources_997019_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/b969317e-641f-4702-841f-a79d273d0317_isrForCrosstalkSources_997019_41.stderr parsl.dataflow.dflow DEBUG: Task 2349 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2349 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2349 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2349 has memoization hash 64f6fd8e5399be4ce358f8aa21abc4ab parsl.dataflow.memoization INFO: Task 2349 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2349 try 0 launched on executor multi with executor id 2350 parsl.dataflow.dflow INFO: Standard output for task 2349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a3bfede6-ebdb-461d-bb4e-db00b3998a45_isrForCrosstalkSources_996957_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a3bfede6-ebdb-461d-bb4e-db00b3998a45_isrForCrosstalkSources_996957_9.stderr parsl.dataflow.dflow DEBUG: Task 2350 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2350 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2350 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2350 has memoization hash 160c44e1700dedfaeb22d52d098fcbc6 parsl.dataflow.memoization INFO: Task 2350 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2350 try 0 launched on executor multi with executor id 2351 parsl.dataflow.dflow INFO: Standard output for task 2350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0ec4b600-b272-4517-ade4-befa804a129a_isrForCrosstalkSources_996970_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0ec4b600-b272-4517-ade4-befa804a129a_isrForCrosstalkSources_996970_3.stderr parsl.dataflow.dflow DEBUG: Task 2351 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2351 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2351 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2351 has memoization hash 3e3287f0aee99991114aa080992b856d parsl.dataflow.memoization INFO: Task 2351 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb187c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2351 try 0 launched on executor multi with executor id 2352 parsl.dataflow.dflow INFO: Standard output for task 2351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/15c62a46-4a2d-45f1-95ff-2b8fd50b0bd8_isrForCrosstalkSources_997005_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/15c62a46-4a2d-45f1-95ff-2b8fd50b0bd8_isrForCrosstalkSources_997005_24.stderr parsl.dataflow.dflow DEBUG: Task 2352 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2352 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2352 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2352 has memoization hash 02eb689a0071c4e699973e07d939c655 parsl.dataflow.memoization INFO: Task 2352 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2352 try 0 launched on executor multi with executor id 2353 parsl.dataflow.dflow INFO: Standard output for task 2352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a98f46c2-5418-4598-8758-668049960e86_isrForCrosstalkSources_996977_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a98f46c2-5418-4598-8758-668049960e86_isrForCrosstalkSources_996977_27.stderr parsl.dataflow.dflow DEBUG: Task 2353 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2353 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2353 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2353 has memoization hash a00318272b6b8d54a78c9ca92d27f359 parsl.dataflow.memoization INFO: Task 2353 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2353 try 0 launched on executor multi with executor id 2354 parsl.dataflow.dflow INFO: Standard output for task 2353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/400dd857-950d-4c04-8efb-d0b8fe5e5b42_isrForCrosstalkSources_997032_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/400dd857-950d-4c04-8efb-d0b8fe5e5b42_isrForCrosstalkSources_997032_8.stderr parsl.dataflow.dflow DEBUG: Task 2354 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2354 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2354 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2354 has memoization hash da85923ded72fe7c087ce5ecf8411900 parsl.dataflow.memoization INFO: Task 2354 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2354 try 0 launched on executor multi with executor id 2355 parsl.dataflow.dflow INFO: Standard output for task 2354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/ae8c5241-939d-4c59-8b82-0f5993ddd889_isrForCrosstalkSources_996955_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/ae8c5241-939d-4c59-8b82-0f5993ddd889_isrForCrosstalkSources_996955_15.stderr parsl.dataflow.dflow DEBUG: Task 2355 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2355 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2355 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2355 has memoization hash 51149444bd6ef0b49d8f5605da9566b7 parsl.dataflow.memoization INFO: Task 2355 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb189a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 5 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c287647f8836234b28715b4d7f8aa82 with result from task 5 parsl.dataflow.dflow INFO: Parsl task 2355 try 0 launched on executor multi with executor id 2356 parsl.dataflow.dflow INFO: Standard output for task 2355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/103d5168-2eac-4860-97b4-17f165647879_isrForCrosstalkSources_996974_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/103d5168-2eac-4860-97b4-17f165647879_isrForCrosstalkSources_996974_48.stderr parsl.dataflow.dflow DEBUG: Task 2356 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2356 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2356 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2356 has memoization hash 6f3807bea90b2bcc80a8a126e74847c5 parsl.dataflow.memoization INFO: Task 2356 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2356 try 0 launched on executor multi with executor id 2357 parsl.dataflow.dflow INFO: Standard output for task 2356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/e3467e9f-0323-49fa-99d3-b6c502dceb7b_isrForCrosstalkSources_997012_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/e3467e9f-0323-49fa-99d3-b6c502dceb7b_isrForCrosstalkSources_997012_57.stderr parsl.dataflow.dflow DEBUG: Task 2357 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2357 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2357 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2357 has memoization hash bc544781642608ee16f3694ded082fd7 parsl.dataflow.memoization INFO: Task 2357 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 5 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/cbc2ed89-621f-475b-b998-9c02920f4515_isrForCrosstalkSources_996965_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/cbc2ed89-621f-475b-b998-9c02920f4515_isrForCrosstalkSources_996965_46.stderr parsl.dataflow.dflow INFO: Task 4 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3216ca8a9b19aed1201d12bd12c14c8f with result from task 4 parsl.dataflow.dflow INFO: Parsl task 2357 try 0 launched on executor multi with executor id 2358 parsl.dataflow.dflow DEBUG: Task 4 checkpointed parsl.dataflow.dflow INFO: Standard output for task 2357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2be186b5-cd16-4cff-8c0a-4097e1d9d384_isrForCrosstalkSources_996974_31.stdout parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard error for task 2357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2be186b5-cd16-4cff-8c0a-4097e1d9d384_isrForCrosstalkSources_996974_31.stderr parsl.dataflow.dflow INFO: Standard output for task 4 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/91141d3f-a9b7-4a43-a117-94fd85d92ddb_isrForCrosstalkSources_996995_46.stdout parsl.dataflow.dflow DEBUG: Task 2358 will be sent to executor multi parsl.dataflow.dflow INFO: Standard error for task 4 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/91141d3f-a9b7-4a43-a117-94fd85d92ddb_isrForCrosstalkSources_996995_46.stderr parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2358 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2358 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2358 has memoization hash 7a5b0214686010431162432c99d27af5 parsl.dataflow.memoization INFO: Task 2358 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2358 try 0 launched on executor multi with executor id 2359 parsl.dataflow.dflow INFO: Standard output for task 2358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/55165652-a43e-4b59-b606-2e8b750ee0aa_isrForCrosstalkSources_996988_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/55165652-a43e-4b59-b606-2e8b750ee0aa_isrForCrosstalkSources_996988_61.stderr parsl.dataflow.dflow DEBUG: Task 2359 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2359 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2359 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2359 has memoization hash 46d865cc14698a9d4e102363163be648 parsl.dataflow.memoization INFO: Task 2359 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb180e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2359 try 0 launched on executor multi with executor id 2360 parsl.dataflow.dflow INFO: Standard output for task 2359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/88496502-1e01-4dcb-8749-b170b1854d10_isrForCrosstalkSources_996993_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/88496502-1e01-4dcb-8749-b170b1854d10_isrForCrosstalkSources_996993_47.stderr parsl.dataflow.dflow DEBUG: Task 2360 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2360 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2360 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2360 has memoization hash 0f19bfcf2bf65581bc9b3d93808610ed parsl.dataflow.memoization INFO: Task 2360 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2360 try 0 launched on executor multi with executor id 2361 parsl.dataflow.dflow INFO: Standard output for task 2360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/760e0a35-cf73-438b-8a15-b6120b151f19_isrForCrosstalkSources_997019_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/760e0a35-cf73-438b-8a15-b6120b151f19_isrForCrosstalkSources_997019_44.stderr parsl.dataflow.dflow DEBUG: Task 2361 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2361 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2361 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2361 has memoization hash 0c7941d098dd36137cd7574b333ffb4c parsl.dataflow.memoization INFO: Task 2361 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2361 try 0 launched on executor multi with executor id 2362 parsl.dataflow.dflow INFO: Standard output for task 2361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f3b4401c-0211-444a-88e4-6f547e451910_isrForCrosstalkSources_996966_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f3b4401c-0211-444a-88e4-6f547e451910_isrForCrosstalkSources_996966_4.stderr parsl.dataflow.dflow DEBUG: Task 2362 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2362 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2362 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2362 has memoization hash 5d13e6594e3df6c9648663d0f74ec601 parsl.dataflow.memoization INFO: Task 2362 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2362 try 0 launched on executor multi with executor id 2363 parsl.dataflow.dflow INFO: Standard output for task 2362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/95dfffa6-e5e4-40ba-815d-9ea5107735b3_isrForCrosstalkSources_997006_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/95dfffa6-e5e4-40ba-815d-9ea5107735b3_isrForCrosstalkSources_997006_40.stderr parsl.dataflow.dflow DEBUG: Task 2363 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2363 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2363 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2363 has memoization hash 16d94d98edd3274a44354c08b02c246e parsl.dataflow.memoization INFO: Task 2363 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2363 try 0 launched on executor multi with executor id 2364 parsl.dataflow.dflow INFO: Standard output for task 2363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/86eab017-047f-4bae-b26b-5cd283ca5e26_isrForCrosstalkSources_996971_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/86eab017-047f-4bae-b26b-5cd283ca5e26_isrForCrosstalkSources_996971_52.stderr parsl.dataflow.dflow DEBUG: Task 2364 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2364 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2364 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2364 has memoization hash 68b03ff06709dff6edfa583f9b2ae328 parsl.dataflow.memoization INFO: Task 2364 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2364 try 0 launched on executor multi with executor id 2365 parsl.dataflow.dflow INFO: Standard output for task 2364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/cd8a2dcc-0c45-44ed-a552-d8345ea09422_isrForCrosstalkSources_996954_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/cd8a2dcc-0c45-44ed-a552-d8345ea09422_isrForCrosstalkSources_996954_16.stderr parsl.dataflow.dflow DEBUG: Task 2365 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2365 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2365 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2365 has memoization hash cb003ed40a29be64df73b8a0fdd6c2c6 parsl.dataflow.memoization INFO: Task 2365 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2365 try 0 launched on executor multi with executor id 2366 parsl.dataflow.dflow INFO: Standard output for task 2365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a0fba2b3-d9c6-4148-8028-b991df2bb15e_isrForCrosstalkSources_997028_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a0fba2b3-d9c6-4148-8028-b991df2bb15e_isrForCrosstalkSources_997028_60.stderr parsl.dataflow.dflow DEBUG: Task 2366 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2366 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2366 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2366 has memoization hash 43cefc72b2162085f21e65d42ee068ee parsl.dataflow.memoization INFO: Task 2366 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2366 try 0 launched on executor multi with executor id 2367 parsl.dataflow.dflow INFO: Standard output for task 2366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/7f4635cb-c5cc-437a-9cc0-e5dc5e951fd7_isrForCrosstalkSources_997009_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/7f4635cb-c5cc-437a-9cc0-e5dc5e951fd7_isrForCrosstalkSources_997009_57.stderr parsl.dataflow.dflow DEBUG: Task 2367 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2367 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2367 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2367 has memoization hash ed4ff1d596626080f158872c59aa201d parsl.dataflow.memoization INFO: Task 2367 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb191c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2367 try 0 launched on executor multi with executor id 2368 parsl.dataflow.dflow INFO: Standard output for task 2367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a08ed3f2-1b95-4623-a453-c06408898a8b_isrForCrosstalkSources_996957_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a08ed3f2-1b95-4623-a453-c06408898a8b_isrForCrosstalkSources_996957_50.stderr parsl.dataflow.dflow DEBUG: Task 2368 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2368 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2368 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2368 has memoization hash c96a65bf35758cad24eec6db02cf5f8d parsl.dataflow.memoization INFO: Task 2368 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2368 try 0 launched on executor multi with executor id 2369 parsl.dataflow.dflow INFO: Standard output for task 2368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b38a189a-715c-49c2-bdda-1ce6a221ed9b_isrForCrosstalkSources_996979_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b38a189a-715c-49c2-bdda-1ce6a221ed9b_isrForCrosstalkSources_996979_53.stderr parsl.dataflow.dflow DEBUG: Task 2369 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2369 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2369 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2369 has memoization hash cc2e323e1d000cf339bb89b29d017c5e parsl.dataflow.memoization INFO: Task 2369 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb193a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2369 try 0 launched on executor multi with executor id 2370 parsl.dataflow.dflow INFO: Standard output for task 2369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/50a6cf56-b69b-492f-8e6b-bbc57e5b724e_isrForCrosstalkSources_996978_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/50a6cf56-b69b-492f-8e6b-bbc57e5b724e_isrForCrosstalkSources_996978_22.stderr parsl.dataflow.dflow DEBUG: Task 2370 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2370 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2370 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2370 has memoization hash dc29d51a04c8c9a4641afac7d0df4293 parsl.dataflow.memoization INFO: Task 2370 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2370 try 0 launched on executor multi with executor id 2371 parsl.dataflow.dflow INFO: Standard output for task 2370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/27c25d35-9692-406f-b1bc-603452afb85c_isrForCrosstalkSources_997032_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/27c25d35-9692-406f-b1bc-603452afb85c_isrForCrosstalkSources_997032_36.stderr parsl.dataflow.dflow DEBUG: Task 2371 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2371 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2371 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2371 has memoization hash 5152cb63b668ec549c53a64d2ec739e7 parsl.dataflow.memoization INFO: Task 2371 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 6 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8cd92b448567b79e1586dd8bb650a318 with result from task 6 parsl.dataflow.dflow DEBUG: Task 6 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 6 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/170248f1-9a17-471a-a1cf-8412f367c74b_isrForCrosstalkSources_996965_56.stdout parsl.dataflow.dflow INFO: Standard error for task 6 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/170248f1-9a17-471a-a1cf-8412f367c74b_isrForCrosstalkSources_996965_56.stderr parsl.dataflow.dflow INFO: Parsl task 2371 try 0 launched on executor multi with executor id 2372 parsl.dataflow.dflow INFO: Standard output for task 2371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/ef58bdd8-1353-4649-ad5c-1feab72a0960_isrForCrosstalkSources_997031_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/ef58bdd8-1353-4649-ad5c-1feab72a0960_isrForCrosstalkSources_997031_48.stderr parsl.dataflow.dflow DEBUG: Task 2372 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2372 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2372 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2372 has memoization hash 5e43fd98b82c7765282cf86d0a13a58a parsl.dataflow.memoization INFO: Task 2372 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb194e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2372 try 0 launched on executor multi with executor id 2373 parsl.dataflow.dflow INFO: Standard output for task 2372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/0369cc5d-7ed1-4b20-bcd4-2fa42cb9e2e4_isrForCrosstalkSources_996962_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/0369cc5d-7ed1-4b20-bcd4-2fa42cb9e2e4_isrForCrosstalkSources_996962_49.stderr parsl.dataflow.dflow DEBUG: Task 2373 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2373 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2373 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2373 has memoization hash 2343dfaa31186a5a3803cb02a377b906 parsl.dataflow.memoization INFO: Task 2373 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2373 try 0 launched on executor multi with executor id 2374 parsl.dataflow.dflow INFO: Standard output for task 2373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d3dfaea4-0754-412b-be0e-adee9d783261_isrForCrosstalkSources_997001_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d3dfaea4-0754-412b-be0e-adee9d783261_isrForCrosstalkSources_997001_60.stderr parsl.dataflow.dflow DEBUG: Task 2374 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2374 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2374 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2374 has memoization hash 1b0801f4523f08f251b413edd6c2c850 parsl.dataflow.memoization INFO: Task 2374 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb196c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2374 try 0 launched on executor multi with executor id 2375 parsl.dataflow.dflow INFO: Standard output for task 2374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f66a3714-58ae-4a58-89e8-58680541e8f5_isrForCrosstalkSources_996980_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f66a3714-58ae-4a58-89e8-58680541e8f5_isrForCrosstalkSources_996980_10.stderr parsl.dataflow.dflow DEBUG: Task 2375 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2375 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2375 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2375 has memoization hash 66c078e3c054202310c8b2e5fb59a228 parsl.dataflow.memoization INFO: Task 2375 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2375 try 0 launched on executor multi with executor id 2376 parsl.dataflow.dflow INFO: Standard output for task 2375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1d7243f9-c156-4ef2-9423-29d852e2c45d_isrForCrosstalkSources_997005_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1d7243f9-c156-4ef2-9423-29d852e2c45d_isrForCrosstalkSources_997005_1.stderr parsl.dataflow.dflow DEBUG: Task 2376 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2376 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2376 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2376 has memoization hash b5f030dbfc7b78b9891979da4151cc5c parsl.dataflow.memoization INFO: Task 2376 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2376 try 0 launched on executor multi with executor id 2377 parsl.dataflow.dflow INFO: Standard output for task 2376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/78259061-b009-493e-be19-7e7a08107207_isrForCrosstalkSources_996996_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/78259061-b009-493e-be19-7e7a08107207_isrForCrosstalkSources_996996_24.stderr parsl.dataflow.dflow DEBUG: Task 2377 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2377 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2377 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2377 has memoization hash c1f506e359c8757a5fa55f5e325e35aa parsl.dataflow.memoization INFO: Task 2377 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2377 try 0 launched on executor multi with executor id 2378 parsl.dataflow.dflow INFO: Standard output for task 2377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/7899b428-f394-4161-9114-6ce37c1fb0f5_isrForCrosstalkSources_996978_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/7899b428-f394-4161-9114-6ce37c1fb0f5_isrForCrosstalkSources_996978_35.stderr parsl.dataflow.dflow DEBUG: Task 2378 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2378 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2378 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2378 has memoization hash a58f73e89226db50ba87bd34256bb77b parsl.dataflow.memoization INFO: Task 2378 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb198a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2378 try 0 launched on executor multi with executor id 2379 parsl.dataflow.dflow INFO: Standard output for task 2378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8e70616b-b673-44b0-8a69-e37685f7167f_isrForCrosstalkSources_996967_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8e70616b-b673-44b0-8a69-e37685f7167f_isrForCrosstalkSources_996967_2.stderr parsl.dataflow.dflow DEBUG: Task 2379 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2379 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2379 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2379 has memoization hash 3bfa0595c1c7dc229b014d1f3377395a parsl.dataflow.memoization INFO: Task 2379 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb199e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2379 try 0 launched on executor multi with executor id 2380 parsl.dataflow.dflow INFO: Standard output for task 2379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/fc58b97a-ba11-4584-b549-e33455b30fcc_isrForCrosstalkSources_996999_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/fc58b97a-ba11-4584-b549-e33455b30fcc_isrForCrosstalkSources_996999_23.stderr parsl.dataflow.dflow DEBUG: Task 2380 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2380 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2380 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2380 has memoization hash fd111a20541b9b424d48b40cfe9fb5a8 parsl.dataflow.memoization INFO: Task 2380 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2380 try 0 launched on executor multi with executor id 2381 parsl.dataflow.dflow INFO: Standard output for task 2380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/477ef399-4ab9-4d13-b50f-5a175a205387_isrForCrosstalkSources_997010_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/477ef399-4ab9-4d13-b50f-5a175a205387_isrForCrosstalkSources_997010_16.stderr parsl.dataflow.dflow DEBUG: Task 2381 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2381 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2381 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2381 has memoization hash 98f6148ce782f8dcc5bc978da6a1d446 parsl.dataflow.memoization INFO: Task 2381 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2381 try 0 launched on executor multi with executor id 2382 parsl.dataflow.dflow INFO: Standard output for task 2381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/33283a1a-4caf-401c-84df-188c17c0bb4c_isrForCrosstalkSources_996973_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/33283a1a-4caf-401c-84df-188c17c0bb4c_isrForCrosstalkSources_996973_4.stderr parsl.dataflow.dflow DEBUG: Task 2382 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2382 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2382 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2382 has memoization hash 5693d352b3ee1e3eb6fe30690ce811ee parsl.dataflow.memoization INFO: Task 2382 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2382 try 0 launched on executor multi with executor id 2383 parsl.dataflow.dflow INFO: Standard output for task 2382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/218b6921-021f-4185-a321-44762812b00a_isrForCrosstalkSources_996991_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/218b6921-021f-4185-a321-44762812b00a_isrForCrosstalkSources_996991_47.stderr parsl.dataflow.dflow DEBUG: Task 2383 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2383 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2383 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2383 has memoization hash 30d51808976a16436f80966e47f23f66 parsl.dataflow.memoization INFO: Task 2383 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2383 try 0 launched on executor multi with executor id 2384 parsl.dataflow.dflow INFO: Standard output for task 2383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/11d053dc-d160-4344-928b-65da9daaa21a_isrForCrosstalkSources_996975_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/11d053dc-d160-4344-928b-65da9daaa21a_isrForCrosstalkSources_996975_51.stderr parsl.dataflow.dflow DEBUG: Task 2384 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2384 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2384 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2384 has memoization hash 213439039c6db695c33988a93ba576b5 parsl.dataflow.memoization INFO: Task 2384 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2384 try 0 launched on executor multi with executor id 2385 parsl.dataflow.dflow INFO: Standard output for task 2384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/d40fba4e-bb8f-49db-95b0-36ed290ced33_isrForCrosstalkSources_996968_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/d40fba4e-bb8f-49db-95b0-36ed290ced33_isrForCrosstalkSources_996968_16.stderr parsl.dataflow.dflow DEBUG: Task 2385 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2385 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2385 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2385 has memoization hash 746ac9c8a8448419e6077d89f759b7ff parsl.dataflow.memoization INFO: Task 2385 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2385 try 0 launched on executor multi with executor id 2386 parsl.dataflow.dflow INFO: Standard output for task 2385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8c5eb432-7ae8-4d9b-9596-eeec28b7cf4b_isrForCrosstalkSources_997017_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8c5eb432-7ae8-4d9b-9596-eeec28b7cf4b_isrForCrosstalkSources_997017_60.stderr parsl.dataflow.dflow DEBUG: Task 2386 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2386 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2386 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2386 has memoization hash c1a953b576d72588a7cfc3699de36434 parsl.dataflow.memoization INFO: Task 2386 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2386 try 0 launched on executor multi with executor id 2387 parsl.dataflow.dflow INFO: Standard output for task 2386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/41205893-5e13-4f96-ad41-25117b1cbf8e_isrForCrosstalkSources_997006_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/41205893-5e13-4f96-ad41-25117b1cbf8e_isrForCrosstalkSources_997006_49.stderr parsl.dataflow.dflow DEBUG: Task 2387 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2387 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2387 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2387 has memoization hash 5fbd80f4730899a4b44bcdce1b41e133 parsl.dataflow.memoization INFO: Task 2387 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2387 try 0 launched on executor multi with executor id 2388 parsl.dataflow.dflow INFO: Standard output for task 2387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/f61e2d11-fe4c-4231-89c5-2e58d1802a8c_isrForCrosstalkSources_996981_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/f61e2d11-fe4c-4231-89c5-2e58d1802a8c_isrForCrosstalkSources_996981_8.stderr parsl.dataflow.dflow DEBUG: Task 2388 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2388 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2388 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2388 has memoization hash ec4dadad0f5472ae29948335a23a661b parsl.dataflow.memoization INFO: Task 2388 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2388 try 0 launched on executor multi with executor id 2389 parsl.dataflow.dflow INFO: Standard output for task 2388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/38ba85de-2615-4860-81e9-c84d8c7df94a_isrForCrosstalkSources_996994_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/38ba85de-2615-4860-81e9-c84d8c7df94a_isrForCrosstalkSources_996994_6.stderr parsl.dataflow.dflow DEBUG: Task 2389 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2389 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2389 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2389 has memoization hash d2134e4c84abe530721c987109b92a11 parsl.dataflow.memoization INFO: Task 2389 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2389 try 0 launched on executor multi with executor id 2390 parsl.dataflow.dflow INFO: Standard output for task 2389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/737149d2-6be7-4c1b-828c-3f9903ee5b6a_isrForCrosstalkSources_997016_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/737149d2-6be7-4c1b-828c-3f9903ee5b6a_isrForCrosstalkSources_997016_32.stderr parsl.dataflow.dflow DEBUG: Task 2390 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2390 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2390 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2390 has memoization hash c896e1c527288a8d8f33c393c745a100 parsl.dataflow.memoization INFO: Task 2390 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb19f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2390 try 0 launched on executor multi with executor id 2391 parsl.dataflow.dflow INFO: Standard output for task 2390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6f00fc4a-000d-4970-869d-6fa9f7b4f0c4_isrForCrosstalkSources_996982_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6f00fc4a-000d-4970-869d-6fa9f7b4f0c4_isrForCrosstalkSources_996982_13.stderr parsl.dataflow.dflow DEBUG: Task 2391 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2391 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2391 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2391 has memoization hash 2509c009e44339e633b08a423b5ff07d parsl.dataflow.memoization INFO: Task 2391 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2391 try 0 launched on executor multi with executor id 2392 parsl.dataflow.dflow INFO: Standard output for task 2391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bc40cf74-283f-4ce5-b383-000ec9fa5508_isrForCrosstalkSources_997026_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bc40cf74-283f-4ce5-b383-000ec9fa5508_isrForCrosstalkSources_997026_30.stderr parsl.dataflow.dflow DEBUG: Task 2392 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2392 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2392 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2392 has memoization hash 01d7f043e8f6382d87e63dc94d7f5282 parsl.dataflow.memoization INFO: Task 2392 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2392 try 0 launched on executor multi with executor id 2393 parsl.dataflow.dflow INFO: Standard output for task 2392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cc8d876e-1dbf-4a7b-99f5-a4eb752f6858_isrForCrosstalkSources_996956_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cc8d876e-1dbf-4a7b-99f5-a4eb752f6858_isrForCrosstalkSources_996956_27.stderr parsl.dataflow.dflow DEBUG: Task 2393 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2393 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2393 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2393 has memoization hash 0e0c8ad6efdc67f734c6a79f44f7a6c8 parsl.dataflow.memoization INFO: Task 2393 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2393 try 0 launched on executor multi with executor id 2394 parsl.dataflow.dflow INFO: Standard output for task 2393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/49c9839d-b3b0-49ea-a34c-12a152c76d05_isrForCrosstalkSources_997005_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/49c9839d-b3b0-49ea-a34c-12a152c76d05_isrForCrosstalkSources_997005_33.stderr parsl.dataflow.dflow DEBUG: Task 2394 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2394 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2394 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2394 has memoization hash 99d6e3a2f1a6f86d8d630b8838ee42cd parsl.dataflow.memoization INFO: Task 2394 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2394 try 0 launched on executor multi with executor id 2395 parsl.dataflow.dflow INFO: Standard output for task 2394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8459a4f1-ec2c-4512-98a9-1dc0dd6213bf_isrForCrosstalkSources_996992_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8459a4f1-ec2c-4512-98a9-1dc0dd6213bf_isrForCrosstalkSources_996992_38.stderr parsl.dataflow.dflow DEBUG: Task 2395 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2395 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2395 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2395 has memoization hash f29fb9ba819a61df081b7af5c7807fa3 parsl.dataflow.memoization INFO: Task 2395 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2395 try 0 launched on executor multi with executor id 2396 parsl.dataflow.dflow INFO: Standard output for task 2395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/a096cc60-3ce6-4c01-947e-308a1393dc4d_isrForCrosstalkSources_996968_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/a096cc60-3ce6-4c01-947e-308a1393dc4d_isrForCrosstalkSources_996968_11.stderr parsl.dataflow.dflow DEBUG: Task 2396 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2396 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2396 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2396 has memoization hash 288d00c5cf820890304500d731b494c2 parsl.dataflow.memoization INFO: Task 2396 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2396 try 0 launched on executor multi with executor id 2397 parsl.dataflow.dflow INFO: Standard output for task 2396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2689f274-13da-4b87-a203-307f7c3b603d_isrForCrosstalkSources_996981_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2689f274-13da-4b87-a203-307f7c3b603d_isrForCrosstalkSources_996981_54.stderr parsl.dataflow.dflow DEBUG: Task 2397 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2397 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2397 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2397 has memoization hash c96c426b955bbfbb86844d52af09b07e parsl.dataflow.memoization INFO: Task 2397 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2397 try 0 launched on executor multi with executor id 2398 parsl.dataflow.dflow INFO: Standard output for task 2397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/1a69faea-feca-496a-8552-571b2e32712d_isrForCrosstalkSources_996997_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/1a69faea-feca-496a-8552-571b2e32712d_isrForCrosstalkSources_996997_25.stderr parsl.dataflow.dflow DEBUG: Task 2398 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2398 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2398 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2398 has memoization hash 1aec00d0d8d56beaa96edf5d1250cd38 parsl.dataflow.memoization INFO: Task 2398 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2398 try 0 launched on executor multi with executor id 2399 parsl.dataflow.dflow INFO: Standard output for task 2398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/c4a20854-ad56-4e70-9c1b-fb9183a1c8d4_isrForCrosstalkSources_997017_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/c4a20854-ad56-4e70-9c1b-fb9183a1c8d4_isrForCrosstalkSources_997017_41.stderr parsl.dataflow.dflow DEBUG: Task 2399 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2399 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2399 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2399 has memoization hash bb132719983704b9ae65f12b68ac5482 parsl.dataflow.memoization INFO: Task 2399 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2399 try 0 launched on executor multi with executor id 2400 parsl.dataflow.dflow INFO: Standard output for task 2399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/4ab1dfb4-08fa-4ad6-9b92-0edc725765cb_isrForCrosstalkSources_997000_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/4ab1dfb4-08fa-4ad6-9b92-0edc725765cb_isrForCrosstalkSources_997000_62.stderr parsl.dataflow.dflow DEBUG: Task 2400 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2400 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2400 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2400 has memoization hash 2ecb948f8ed7a7b06dded197fcaa28ac parsl.dataflow.memoization INFO: Task 2400 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2400 try 0 launched on executor multi with executor id 2401 parsl.dataflow.dflow INFO: Standard output for task 2400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/72fafcad-3cb8-45d5-8398-ae4cbbe50dd9_isrForCrosstalkSources_996962_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/72fafcad-3cb8-45d5-8398-ae4cbbe50dd9_isrForCrosstalkSources_996962_60.stderr parsl.dataflow.dflow DEBUG: Task 2401 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2401 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2401 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2401 has memoization hash 128a0a9defd27d32da446a3cc8dcac9d parsl.dataflow.memoization INFO: Task 2401 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2401 try 0 launched on executor multi with executor id 2402 parsl.dataflow.dflow INFO: Standard output for task 2401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/05056ad7-6842-4dce-9d2c-401bfd824e34_isrForCrosstalkSources_997028_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/05056ad7-6842-4dce-9d2c-401bfd824e34_isrForCrosstalkSources_997028_9.stderr parsl.dataflow.dflow DEBUG: Task 2402 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2402 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2402 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2402 has memoization hash 10a156c51647f297975c59a85211c4d0 parsl.dataflow.memoization INFO: Task 2402 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2402 try 0 launched on executor multi with executor id 2403 parsl.dataflow.dflow INFO: Standard output for task 2402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c6423196-6aa3-4ec3-894b-d5ec2774c65e_isrForCrosstalkSources_997001_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c6423196-6aa3-4ec3-894b-d5ec2774c65e_isrForCrosstalkSources_997001_13.stderr parsl.dataflow.dflow DEBUG: Task 2403 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2403 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2403 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2403 has memoization hash 97f362183d3038e70817a0a14d9407c9 parsl.dataflow.memoization INFO: Task 2403 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2403 try 0 launched on executor multi with executor id 2404 parsl.dataflow.dflow INFO: Standard output for task 2403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c969a6d8-421c-42ff-983b-888023b6a480_isrForCrosstalkSources_996992_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c969a6d8-421c-42ff-983b-888023b6a480_isrForCrosstalkSources_996992_52.stderr parsl.dataflow.dflow DEBUG: Task 2404 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2404 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2404 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2404 has memoization hash 9ee8c597a297c06df9a70cdff8c8a1d7 parsl.dataflow.memoization INFO: Task 2404 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2404 try 0 launched on executor multi with executor id 2405 parsl.dataflow.dflow INFO: Standard output for task 2404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e78b924c-9b09-400a-aa6f-4411a9c43c28_isrForCrosstalkSources_996996_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e78b924c-9b09-400a-aa6f-4411a9c43c28_isrForCrosstalkSources_996996_55.stderr parsl.dataflow.dflow DEBUG: Task 2405 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2405 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2405 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2405 has memoization hash b2ee1dc5f7f7d2a0fb78159f1723b0cb parsl.dataflow.memoization INFO: Task 2405 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1a8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2405 try 0 launched on executor multi with executor id 2406 parsl.dataflow.dflow INFO: Standard output for task 2405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/860b07e2-e5a2-414e-a678-af4d9be0b237_isrForCrosstalkSources_996999_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/860b07e2-e5a2-414e-a678-af4d9be0b237_isrForCrosstalkSources_996999_9.stderr parsl.dataflow.dflow DEBUG: Task 2406 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2406 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2406 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2406 has memoization hash dabf49e2c8c2e2f4028183aec1ea2ed6 parsl.dataflow.memoization INFO: Task 2406 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1aa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2406 try 0 launched on executor multi with executor id 2407 parsl.dataflow.dflow INFO: Standard output for task 2406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8b1dd569-8556-4de6-8c56-1bde38c24d7e_isrForCrosstalkSources_996973_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8b1dd569-8556-4de6-8c56-1bde38c24d7e_isrForCrosstalkSources_996973_1.stderr parsl.dataflow.dflow DEBUG: Task 2407 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2407 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2407 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2407 has memoization hash 4855e6d51d3ecbff72588898e878fd6c parsl.dataflow.memoization INFO: Task 2407 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1aac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2407 try 0 launched on executor multi with executor id 2408 parsl.dataflow.dflow INFO: Standard output for task 2407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/b637eada-9ee7-4fcf-9ed3-6786d812be2d_isrForCrosstalkSources_996995_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/b637eada-9ee7-4fcf-9ed3-6786d812be2d_isrForCrosstalkSources_996995_19.stderr parsl.dataflow.dflow DEBUG: Task 2408 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2408 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2408 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2408 has memoization hash af1d59bdc526b9e4840a24c2a07e772e parsl.dataflow.memoization INFO: Task 2408 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1ab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2408 try 0 launched on executor multi with executor id 2409 parsl.dataflow.dflow INFO: Standard output for task 2408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e0e4d389-d8d1-45d5-bdb2-82ddbded0df6_isrForCrosstalkSources_997001_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e0e4d389-d8d1-45d5-bdb2-82ddbded0df6_isrForCrosstalkSources_997001_18.stderr parsl.dataflow.dflow DEBUG: Task 2409 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2409 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2409 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2409 has memoization hash a55a6d4ab2529e1d229d18d474da98fa parsl.dataflow.memoization INFO: Task 2409 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1aca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2409 try 0 launched on executor multi with executor id 2410 parsl.dataflow.dflow INFO: Standard output for task 2409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/76e54c44-1c11-4e1e-b3aa-303162554bd4_isrForCrosstalkSources_997029_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/76e54c44-1c11-4e1e-b3aa-303162554bd4_isrForCrosstalkSources_997029_20.stderr parsl.dataflow.dflow DEBUG: Task 2410 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2410 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2410 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2410 has memoization hash 4c25c7c5553129c3c6a1e41fd29cd230 parsl.dataflow.memoization INFO: Task 2410 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1ac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2410 try 0 launched on executor multi with executor id 2411 parsl.dataflow.dflow INFO: Standard output for task 2410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/65ceb63c-17bd-4789-8aeb-5089d31f0083_isrForCrosstalkSources_996966_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/65ceb63c-17bd-4789-8aeb-5089d31f0083_isrForCrosstalkSources_996966_38.stderr parsl.dataflow.dflow DEBUG: Task 2411 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2411 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2411 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2411 has memoization hash d36051d8fd8273010403131bdfa9bce4 parsl.dataflow.memoization INFO: Task 2411 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1ad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2411 try 0 launched on executor multi with executor id 2412 parsl.dataflow.dflow INFO: Standard output for task 2411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/364605aa-ecf8-4ce8-80d7-c3d0ee05d59e_isrForCrosstalkSources_996976_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/364605aa-ecf8-4ce8-80d7-c3d0ee05d59e_isrForCrosstalkSources_996976_26.stderr parsl.dataflow.dflow DEBUG: Task 2412 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2412 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2412 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2412 has memoization hash e8b18914ece0cbd7c56d05ad30887b11 parsl.dataflow.memoization INFO: Task 2412 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1ade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:49 proc_lsst.multi:170 [INFO] [multi] job_id 17074873 proc_lsst.multi INFO: [multi] job_id 17074873 2024-03-15 13:49:49 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 27 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 27 2024-03-15 13:49:49 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:49 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074873 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074873 parsl.executors.status_handling DEBUG: Launched block 27 on executor multi with job ID 17074873 parsl.executors.status_handling INFO: Allocated block ID 28 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-28 2024-03-15 13:49:49 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=28 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-28 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=28 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-28 2024-03-15 13:49:49 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=28 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=28 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:49 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=28 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=28 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 27 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 27 2024-03-15 13:49:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:49 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=28 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=28 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 2412 try 0 launched on executor multi with executor id 2413 parsl.dataflow.dflow INFO: Standard output for task 2412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/14a6f43a-650b-4793-be66-0c1ab2bb8064_isrForCrosstalkSources_996956_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/14a6f43a-650b-4793-be66-0c1ab2bb8064_isrForCrosstalkSources_996956_45.stderr parsl.dataflow.dflow DEBUG: Task 2413 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2413 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2413 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2413 has memoization hash 103c719bd2d876e8bb7318d0cce2d6aa parsl.dataflow.memoization INFO: Task 2413 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2413 try 0 launched on executor multi with executor id 2414 parsl.dataflow.dflow INFO: Standard output for task 2413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/af5fa671-4b50-43ab-b98a-853def799f8c_isrForCrosstalkSources_996994_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/af5fa671-4b50-43ab-b98a-853def799f8c_isrForCrosstalkSources_996994_8.stderr parsl.dataflow.dflow DEBUG: Task 2414 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2414 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2414 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2414 has memoization hash 19af73b078245c4db4df84fae2bd7c6f parsl.dataflow.memoization INFO: Task 2414 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2414 try 0 launched on executor multi with executor id 2415 parsl.dataflow.dflow INFO: Standard output for task 2414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/0c01c62d-d5fc-4baa-a79f-200d0b57dfc1_isrForCrosstalkSources_996999_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/0c01c62d-d5fc-4baa-a79f-200d0b57dfc1_isrForCrosstalkSources_996999_29.stderr parsl.dataflow.dflow DEBUG: Task 2415 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2415 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2415 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2415 has memoization hash 1a629ebe6e047d800a69fcf1b7d7afbd parsl.dataflow.memoization INFO: Task 2415 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cac3d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2415 try 0 launched on executor multi with executor id 2416 parsl.dataflow.dflow INFO: Standard output for task 2415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8bf87953-7879-4593-bba8-0e894197105e_isrForCrosstalkSources_996992_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8bf87953-7879-4593-bba8-0e894197105e_isrForCrosstalkSources_996992_15.stderr parsl.dataflow.dflow DEBUG: Task 2416 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2416 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2416 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2416 has memoization hash 89c767f56dbace3c1196800882428de7 parsl.dataflow.memoization INFO: Task 2416 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1ae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2416 try 0 launched on executor multi with executor id 2417 parsl.dataflow.dflow INFO: Standard output for task 2416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d5ee5dbb-2b87-4deb-88bc-6c6f458e4d8e_isrForCrosstalkSources_996993_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d5ee5dbb-2b87-4deb-88bc-6c6f458e4d8e_isrForCrosstalkSources_996993_25.stderr parsl.dataflow.dflow DEBUG: Task 2417 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2417 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2417 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2417 has memoization hash db67748a48b7a9d4d96c723c6bf1c6eb parsl.dataflow.memoization INFO: Task 2417 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb18cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2417 try 0 launched on executor multi with executor id 2418 parsl.dataflow.dflow INFO: Standard output for task 2417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2a07c6cb-dee7-401b-ab92-b65ed7ca3c4a_isrForCrosstalkSources_997005_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2a07c6cb-dee7-401b-ab92-b65ed7ca3c4a_isrForCrosstalkSources_997005_61.stderr parsl.dataflow.dflow DEBUG: Task 2418 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2418 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2418 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2418 has memoization hash 2be992483963b9305e075ec5d3f5f1fc parsl.dataflow.memoization INFO: Task 2418 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1afc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2418 try 0 launched on executor multi with executor id 2419 parsl.dataflow.dflow INFO: Standard output for task 2418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/645d30c1-d123-48b0-9c4c-4615498a7796_isrForCrosstalkSources_996996_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/645d30c1-d123-48b0-9c4c-4615498a7796_isrForCrosstalkSources_996996_56.stderr parsl.dataflow.dflow DEBUG: Task 2419 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2419 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2419 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2419 has memoization hash 0ecf48d8baf880cd9a10cf1dc4360357 parsl.dataflow.memoization INFO: Task 2419 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 2419 try 0 launched on executor multi with executor id 2420 parsl.dataflow.dflow INFO: Standard output for task 2419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/91492f0d-b716-4f0a-9b72-4da7fcdec4bb_isrForCrosstalkSources_996965_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/91492f0d-b716-4f0a-9b72-4da7fcdec4bb_isrForCrosstalkSources_996965_39.stderr parsl.dataflow.dflow DEBUG: Task 2420 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2420 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2420 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2420 has memoization hash 48824af55f7a3b06218917c32927fa54 parsl.dataflow.memoization INFO: Task 2420 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba2b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2420 try 0 launched on executor multi with executor id 2421 parsl.dataflow.dflow INFO: Standard output for task 2420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4c1109c8-7b9a-4045-8bf2-a921a2d5baf8_isrForCrosstalkSources_997017_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4c1109c8-7b9a-4045-8bf2-a921a2d5baf8_isrForCrosstalkSources_997017_2.stderr parsl.dataflow.dflow DEBUG: Task 2421 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2421 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2421 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2421 has memoization hash 0ee3c796025eaa90d3cc3617651509ac parsl.dataflow.memoization INFO: Task 2421 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2421 try 0 launched on executor multi with executor id 2422 parsl.dataflow.dflow INFO: Standard output for task 2421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7a4331d4-bc2f-4a77-98b5-47abaf00e95f_isrForCrosstalkSources_996974_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7a4331d4-bc2f-4a77-98b5-47abaf00e95f_isrForCrosstalkSources_996974_10.stderr parsl.dataflow.dflow DEBUG: Task 2422 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2422 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2422 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2422 has memoization hash 9b76b22f42b6fe4bff3f711c01bf7ef1 parsl.dataflow.memoization INFO: Task 2422 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2422 try 0 launched on executor multi with executor id 2423 parsl.dataflow.dflow INFO: Standard output for task 2422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/4ef8eded-c7aa-48c7-9864-c05c5fddf6f0_isrForCrosstalkSources_997019_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/4ef8eded-c7aa-48c7-9864-c05c5fddf6f0_isrForCrosstalkSources_997019_11.stderr parsl.dataflow.dflow DEBUG: Task 2423 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2423 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2423 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2423 has memoization hash 2d31f6c4ddafa8244893149d60b2ca23 parsl.dataflow.memoization INFO: Task 2423 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2423 try 0 launched on executor multi with executor id 2424 parsl.dataflow.dflow INFO: Standard output for task 2423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/be141674-e723-4a29-b14a-9f49e9536bc7_isrForCrosstalkSources_996976_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/be141674-e723-4a29-b14a-9f49e9536bc7_isrForCrosstalkSources_996976_25.stderr parsl.dataflow.dflow DEBUG: Task 2424 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2424 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2424 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2424 has memoization hash 2ba3318075ee78147c5e36e9f48a5e2d parsl.dataflow.memoization INFO: Task 2424 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2424 try 0 launched on executor multi with executor id 2425 parsl.dataflow.dflow INFO: Standard output for task 2424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4cbbf5b2-2f05-4abb-8930-a1bc5425e772_isrForCrosstalkSources_996986_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4cbbf5b2-2f05-4abb-8930-a1bc5425e772_isrForCrosstalkSources_996986_22.stderr parsl.dataflow.dflow DEBUG: Task 2425 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2425 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2425 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2425 has memoization hash 5ff560ce536161e6b8592119ac7c8843 parsl.dataflow.memoization INFO: Task 2425 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2425 try 0 launched on executor multi with executor id 2426 parsl.dataflow.dflow INFO: Standard output for task 2425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a1c4edd9-8d28-4d50-826d-c9bbc3bcd2b6_isrForCrosstalkSources_996956_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a1c4edd9-8d28-4d50-826d-c9bbc3bcd2b6_isrForCrosstalkSources_996956_46.stderr parsl.dataflow.dflow DEBUG: Task 2426 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2426 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2426 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2426 has memoization hash 200d00e90bce5f78086555df3ab12116 parsl.dataflow.memoization INFO: Task 2426 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2426 try 0 launched on executor multi with executor id 2427 parsl.dataflow.dflow INFO: Standard output for task 2426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9bd7f08d-987a-4d0c-9f87-16b9198954e8_isrForCrosstalkSources_996964_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9bd7f08d-987a-4d0c-9f87-16b9198954e8_isrForCrosstalkSources_996964_26.stderr parsl.dataflow.dflow DEBUG: Task 2427 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2427 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2427 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2427 has memoization hash 851c0114bdbe8cf85768ea669ec957af parsl.dataflow.memoization INFO: Task 2427 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2427 try 0 launched on executor multi with executor id 2428 parsl.dataflow.dflow INFO: Standard output for task 2427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/1f0fdf87-4dfc-4b02-8dd1-7ef91aa4e55d_isrForCrosstalkSources_996994_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/1f0fdf87-4dfc-4b02-8dd1-7ef91aa4e55d_isrForCrosstalkSources_996994_41.stderr parsl.dataflow.dflow DEBUG: Task 2428 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2428 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2428 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2428 has memoization hash def46531bf389f8c73d090b7d4bb05f1 parsl.dataflow.memoization INFO: Task 2428 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2428 try 0 launched on executor multi with executor id 2429 parsl.dataflow.dflow INFO: Standard output for task 2428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bf149585-6159-493b-845e-d34f2d53f379_isrForCrosstalkSources_996992_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bf149585-6159-493b-845e-d34f2d53f379_isrForCrosstalkSources_996992_26.stderr parsl.dataflow.dflow DEBUG: Task 2429 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2429 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2429 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2429 has memoization hash fabaa299d8727fa3fdc96b4c16fca021 parsl.dataflow.memoization INFO: Task 2429 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2429 try 0 launched on executor multi with executor id 2430 parsl.dataflow.dflow INFO: Standard output for task 2429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4436cd2a-05a5-4391-92be-1758638ea0e9_isrForCrosstalkSources_996963_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4436cd2a-05a5-4391-92be-1758638ea0e9_isrForCrosstalkSources_996963_24.stderr parsl.dataflow.dflow DEBUG: Task 2430 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2430 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2430 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2430 has memoization hash 3b7d07d9d4350bfb97a891885b516c3f parsl.dataflow.memoization INFO: Task 2430 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2430 try 0 launched on executor multi with executor id 2431 parsl.dataflow.dflow INFO: Standard output for task 2430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/65bae40d-8ba1-4d11-a404-fe61f30e54e4_isrForCrosstalkSources_997029_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/65bae40d-8ba1-4d11-a404-fe61f30e54e4_isrForCrosstalkSources_997029_6.stderr parsl.dataflow.dflow DEBUG: Task 2431 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2431 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2431 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2431 has memoization hash 1f8a25c60699202b7cfeb2f6e8c9997f parsl.dataflow.memoization INFO: Task 2431 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2431 try 0 launched on executor multi with executor id 2432 parsl.dataflow.dflow INFO: Standard output for task 2431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1c751a80-1e97-4e29-ac36-1cbd6fbf4c3a_isrForCrosstalkSources_996977_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1c751a80-1e97-4e29-ac36-1cbd6fbf4c3a_isrForCrosstalkSources_996977_59.stderr parsl.dataflow.dflow DEBUG: Task 2432 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2432 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2432 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2432 has memoization hash f186277d561e5064d831c81d77cd0d87 parsl.dataflow.memoization INFO: Task 2432 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2432 try 0 launched on executor multi with executor id 2433 parsl.dataflow.dflow INFO: Standard output for task 2432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c747872f-aae6-4744-8eb1-8a0b3ef70368_isrForCrosstalkSources_996965_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c747872f-aae6-4744-8eb1-8a0b3ef70368_isrForCrosstalkSources_996965_41.stderr parsl.dataflow.dflow DEBUG: Task 2433 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2433 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2433 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2433 has memoization hash 3e4249754c0f655167b07e5513028d99 parsl.dataflow.memoization INFO: Task 2433 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2433 try 0 launched on executor multi with executor id 2434 parsl.dataflow.dflow INFO: Standard output for task 2433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c9096242-5b0a-477e-8a9d-d92701c4019e_isrForCrosstalkSources_997019_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c9096242-5b0a-477e-8a9d-d92701c4019e_isrForCrosstalkSources_997019_31.stderr parsl.dataflow.dflow DEBUG: Task 2434 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2434 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2434 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2434 has memoization hash e87bad6c59b8f052a1567aec70e5bbc7 parsl.dataflow.memoization INFO: Task 2434 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2434 try 0 launched on executor multi with executor id 2435 parsl.dataflow.dflow INFO: Standard output for task 2434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/cb234b67-67d5-49f2-9a46-6574dda2edfe_isrForCrosstalkSources_997031_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/cb234b67-67d5-49f2-9a46-6574dda2edfe_isrForCrosstalkSources_997031_6.stderr parsl.dataflow.dflow DEBUG: Task 2435 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2435 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2435 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2435 has memoization hash f158f19292d8a1d18cff880725cc0775 parsl.dataflow.memoization INFO: Task 2435 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2435 try 0 launched on executor multi with executor id 2436 parsl.dataflow.dflow INFO: Standard output for task 2435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c4ff4b8c-64e6-4f7a-9ad0-a30607d641c4_isrForCrosstalkSources_997015_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c4ff4b8c-64e6-4f7a-9ad0-a30607d641c4_isrForCrosstalkSources_997015_58.stderr parsl.dataflow.dflow DEBUG: Task 2436 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2436 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2436 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2436 has memoization hash 5ad1b96b3e3278399345ac5f6b349af0 parsl.dataflow.memoization INFO: Task 2436 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2436 try 0 launched on executor multi with executor id 2437 parsl.dataflow.dflow INFO: Standard output for task 2436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/49d348af-1fde-44b0-9e15-d555cbb09835_isrForCrosstalkSources_996963_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/49d348af-1fde-44b0-9e15-d555cbb09835_isrForCrosstalkSources_996963_2.stderr parsl.dataflow.dflow DEBUG: Task 2437 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2437 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2437 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2437 has memoization hash f42bc254909fccc437417752d58b5731 parsl.dataflow.memoization INFO: Task 2437 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2437 try 0 launched on executor multi with executor id 2438 parsl.dataflow.dflow INFO: Standard output for task 2437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/7e99dbda-8d2d-4f2d-ad88-73c1cd9ec446_isrForCrosstalkSources_996982_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/7e99dbda-8d2d-4f2d-ad88-73c1cd9ec446_isrForCrosstalkSources_996982_10.stderr parsl.dataflow.dflow DEBUG: Task 2438 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2438 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2438 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2438 has memoization hash 0207b81697b1c8d9333c724e2b6767d0 parsl.dataflow.memoization INFO: Task 2438 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1ba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2438 try 0 launched on executor multi with executor id 2439 parsl.dataflow.dflow INFO: Standard output for task 2438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/610affb9-841b-465c-94e7-daee2c1c8add_isrForCrosstalkSources_997003_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/610affb9-841b-465c-94e7-daee2c1c8add_isrForCrosstalkSources_997003_51.stderr parsl.dataflow.dflow DEBUG: Task 2439 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2439 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2439 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2439 has memoization hash d9319b3019a67ed43445dc3890f7b38f parsl.dataflow.memoization INFO: Task 2439 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2439 try 0 launched on executor multi with executor id 2440 parsl.dataflow.dflow INFO: Standard output for task 2439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c6f82afe-3c56-425c-9839-33bcc9162ac8_isrForCrosstalkSources_996973_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c6f82afe-3c56-425c-9839-33bcc9162ac8_isrForCrosstalkSources_996973_39.stderr parsl.dataflow.dflow DEBUG: Task 2440 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2440 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2440 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2440 has memoization hash e4a833179b2e50ae096b74c3c0a960d0 parsl.dataflow.memoization INFO: Task 2440 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1bba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2440 try 0 launched on executor multi with executor id 2441 parsl.dataflow.dflow INFO: Standard output for task 2440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/fe2b8a88-d89e-47a4-b099-43701377807a_isrForCrosstalkSources_996973_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/fe2b8a88-d89e-47a4-b099-43701377807a_isrForCrosstalkSources_996973_8.stderr parsl.dataflow.dflow DEBUG: Task 2441 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2441 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2441 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2441 has memoization hash f0a3dc4f54b169fa3cb67a19b8d182e9 parsl.dataflow.memoization INFO: Task 2441 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1bb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2441 try 0 launched on executor multi with executor id 2442 parsl.dataflow.dflow INFO: Standard output for task 2441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/e44fe8d3-654f-4413-8507-99d0e8741d13_isrForCrosstalkSources_996966_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/e44fe8d3-654f-4413-8507-99d0e8741d13_isrForCrosstalkSources_996966_25.stderr parsl.dataflow.dflow DEBUG: Task 2442 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2442 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2442 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2442 has memoization hash 7030d124275ee84f8802ed4b99bab688 parsl.dataflow.memoization INFO: Task 2442 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1bc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2442 try 0 launched on executor multi with executor id 2443 parsl.dataflow.dflow INFO: Standard output for task 2442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/d37c86b8-960c-4d87-9611-6c10a86d517c_isrForCrosstalkSources_997015_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/d37c86b8-960c-4d87-9611-6c10a86d517c_isrForCrosstalkSources_997015_40.stderr parsl.dataflow.dflow DEBUG: Task 2443 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2443 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2443 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2443 has memoization hash c6fd650011383e93d7c66ae3df779bce parsl.dataflow.memoization INFO: Task 2443 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1bd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2443 try 0 launched on executor multi with executor id 2444 parsl.dataflow.dflow INFO: Standard output for task 2443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e42547e2-847c-4e83-9451-c6fd89f5499e_isrForCrosstalkSources_996996_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e42547e2-847c-4e83-9451-c6fd89f5499e_isrForCrosstalkSources_996996_60.stderr parsl.dataflow.dflow DEBUG: Task 2444 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2444 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2444 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2444 has memoization hash 0833283e51a80221e832db4cb3164812 parsl.dataflow.memoization INFO: Task 2444 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1be20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 3 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a9a61dd054367440a832a3ad9bca7af5 with result from task 3 parsl.dataflow.dflow INFO: Parsl task 2444 try 0 launched on executor multi with executor id 2445 parsl.dataflow.dflow INFO: Standard output for task 2444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/2729b324-7e87-459a-98c0-8f4edf821115_isrForCrosstalkSources_996952_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/2729b324-7e87-459a-98c0-8f4edf821115_isrForCrosstalkSources_996952_25.stderr parsl.dataflow.dflow DEBUG: Task 2445 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2445 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2445 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2445 has memoization hash 78c60508df45bee4780c06b6a9554296 parsl.dataflow.memoization INFO: Task 2445 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1bce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2445 try 0 launched on executor multi with executor id 2446 parsl.dataflow.dflow INFO: Standard output for task 2445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/8c57645e-6903-4d2d-8290-c0b00925546c_isrForCrosstalkSources_996998_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/8c57645e-6903-4d2d-8290-c0b00925546c_isrForCrosstalkSources_996998_39.stderr parsl.dataflow.dflow DEBUG: Task 2446 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2446 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2446 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2446 has memoization hash 4b1563024c39ec72d8e43ef35c9bf1c0 parsl.dataflow.memoization INFO: Task 2446 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1bf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2446 try 0 launched on executor multi with executor id 2447 parsl.dataflow.dflow INFO: Standard output for task 2446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a0381627-cc6d-42a3-904b-7de3fbbba8b8_isrForCrosstalkSources_997008_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a0381627-cc6d-42a3-904b-7de3fbbba8b8_isrForCrosstalkSources_997008_15.stderr parsl.dataflow.dflow DEBUG: Task 2447 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2447 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2447 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2447 has memoization hash ddd8c04e548c1f6794b4a38f097b8c47 parsl.dataflow.memoization INFO: Task 2447 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 3 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b96ee8cf-2a1f-442f-90ec-4e9916c5de70_isrForCrosstalkSources_997001_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b96ee8cf-2a1f-442f-90ec-4e9916c5de70_isrForCrosstalkSources_997001_35.stderr parsl.dataflow.dflow INFO: Parsl task 2447 try 0 launched on executor multi with executor id 2448 parsl.dataflow.dflow INFO: Standard output for task 2447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/dbe5a498-f1ab-4c05-b938-cc974d35dd9c_isrForCrosstalkSources_996969_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/dbe5a498-f1ab-4c05-b938-cc974d35dd9c_isrForCrosstalkSources_996969_49.stderr parsl.dataflow.dflow DEBUG: Task 2448 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2448 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2448 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2448 has memoization hash 872672f35d81a99de6f7348ab2f12429 parsl.dataflow.memoization INFO: Task 2448 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2448 try 0 launched on executor multi with executor id 2449 parsl.dataflow.dflow INFO: Standard output for task 2448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/3708f8ad-aad2-45db-9273-e0c385798f16_isrForCrosstalkSources_997032_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/3708f8ad-aad2-45db-9273-e0c385798f16_isrForCrosstalkSources_997032_20.stderr parsl.dataflow.dflow DEBUG: Task 2449 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2449 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2449 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2449 has memoization hash d3169f589b290cc5b66554cb2e0d8740 parsl.dataflow.memoization INFO: Task 2449 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2449 try 0 launched on executor multi with executor id 2450 parsl.dataflow.dflow INFO: Standard output for task 2449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a08f790f-0ba9-4bc9-9028-3773def0df93_isrForCrosstalkSources_996961_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a08f790f-0ba9-4bc9-9028-3773def0df93_isrForCrosstalkSources_996961_33.stderr parsl.dataflow.dflow DEBUG: Task 2450 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2450 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2450 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2450 has memoization hash 036250e2c6bbf35b1c2fea34cdca7df1 parsl.dataflow.memoization INFO: Task 2450 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2450 try 0 launched on executor multi with executor id 2451 parsl.dataflow.dflow INFO: Standard output for task 2450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1a985e81-3f63-4ba3-8f8a-28f807b4e0e6_isrForCrosstalkSources_996957_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1a985e81-3f63-4ba3-8f8a-28f807b4e0e6_isrForCrosstalkSources_996957_27.stderr parsl.dataflow.dflow DEBUG: Task 2451 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2451 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2451 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2451 has memoization hash 57f1fd5080077486d600de957266f54e parsl.dataflow.memoization INFO: Task 2451 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2451 try 0 launched on executor multi with executor id 2452 parsl.dataflow.dflow INFO: Standard output for task 2451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/17fd1367-aba9-4b53-a984-b540f70c6369_isrForCrosstalkSources_997009_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/17fd1367-aba9-4b53-a984-b540f70c6369_isrForCrosstalkSources_997009_59.stderr parsl.dataflow.dflow DEBUG: Task 2452 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2452 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2452 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2452 has memoization hash b95587336b3417106b3f1eef9207ca93 parsl.dataflow.memoization INFO: Task 2452 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2452 try 0 launched on executor multi with executor id 2453 parsl.dataflow.dflow INFO: Standard output for task 2452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/579c9296-e156-4ea6-ba6e-4e5a9e8848c4_isrForCrosstalkSources_997001_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/579c9296-e156-4ea6-ba6e-4e5a9e8848c4_isrForCrosstalkSources_997001_31.stderr parsl.dataflow.dflow DEBUG: Task 2453 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2453 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2453 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2453 has memoization hash 1126cedbf377441295bd1a1504bbfd97 parsl.dataflow.memoization INFO: Task 2453 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2453 try 0 launched on executor multi with executor id 2454 parsl.dataflow.dflow INFO: Standard output for task 2453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e867ae5d-2253-4819-a14d-1085ec94fd1b_isrForCrosstalkSources_996984_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e867ae5d-2253-4819-a14d-1085ec94fd1b_isrForCrosstalkSources_996984_59.stderr parsl.dataflow.dflow DEBUG: Task 2454 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2454 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2454 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2454 has memoization hash a876414f11a0961e971e5a27f07a3e26 parsl.dataflow.memoization INFO: Task 2454 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2454 try 0 launched on executor multi with executor id 2455 parsl.dataflow.dflow INFO: Standard output for task 2454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/fc9ca2e1-f850-4551-8228-53d808e69358_isrForCrosstalkSources_997017_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/fc9ca2e1-f850-4551-8228-53d808e69358_isrForCrosstalkSources_997017_38.stderr parsl.dataflow.dflow DEBUG: Task 2455 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2455 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2455 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2455 has memoization hash e54fd4d166d594b302114e00c0b67a43 parsl.dataflow.memoization INFO: Task 2455 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2455 try 0 launched on executor multi with executor id 2456 parsl.dataflow.dflow INFO: Standard output for task 2455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0471d374-d912-4f99-8e28-4aa0d131415a_isrForCrosstalkSources_996966_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0471d374-d912-4f99-8e28-4aa0d131415a_isrForCrosstalkSources_996966_61.stderr parsl.dataflow.dflow DEBUG: Task 2456 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2456 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2456 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2456 has memoization hash 2b7cdd18ccef5f7e97e24bfc0cab4f9d parsl.dataflow.memoization INFO: Task 2456 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2456 try 0 launched on executor multi with executor id 2457 parsl.dataflow.dflow INFO: Standard output for task 2456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f1b9d339-8abc-40bf-9a5e-0d6e1d6f37f1_isrForCrosstalkSources_996971_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f1b9d339-8abc-40bf-9a5e-0d6e1d6f37f1_isrForCrosstalkSources_996971_54.stderr parsl.dataflow.dflow DEBUG: Task 2457 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2457 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2457 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2457 has memoization hash e5f8e01a06aebc6df39ab3adbe227cd3 parsl.dataflow.memoization INFO: Task 2457 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2457 try 0 launched on executor multi with executor id 2458 parsl.dataflow.dflow INFO: Standard output for task 2457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/492263b8-11cc-44eb-8c51-0903220296be_isrForCrosstalkSources_997017_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/492263b8-11cc-44eb-8c51-0903220296be_isrForCrosstalkSources_997017_52.stderr parsl.dataflow.dflow DEBUG: Task 2458 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2458 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2458 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2458 has memoization hash 8708ac4031919e23c3a477caab87d0e7 parsl.dataflow.memoization INFO: Task 2458 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2458 try 0 launched on executor multi with executor id 2459 parsl.dataflow.dflow INFO: Standard output for task 2458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/724c8f7e-96cb-4872-af97-9c022b86fa82_isrForCrosstalkSources_996993_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/724c8f7e-96cb-4872-af97-9c022b86fa82_isrForCrosstalkSources_996993_11.stderr parsl.dataflow.dflow DEBUG: Task 2459 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2459 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2459 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2459 has memoization hash fc9824ac246af854bb61742027a47209 parsl.dataflow.memoization INFO: Task 2459 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2459 try 0 launched on executor multi with executor id 2460 parsl.dataflow.dflow INFO: Standard output for task 2459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f20081f4-fbcd-4799-b97f-62252ce28131_isrForCrosstalkSources_996974_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f20081f4-fbcd-4799-b97f-62252ce28131_isrForCrosstalkSources_996974_36.stderr parsl.dataflow.dflow DEBUG: Task 2460 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2460 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2460 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2460 has memoization hash 68cabebaf32759d49082995f415a039e parsl.dataflow.memoization INFO: Task 2460 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2460 try 0 launched on executor multi with executor id 2461 parsl.dataflow.dflow INFO: Standard output for task 2460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/15f14bc9-f730-4345-b9d6-ea3fe490240b_isrForCrosstalkSources_996972_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/15f14bc9-f730-4345-b9d6-ea3fe490240b_isrForCrosstalkSources_996972_11.stderr parsl.dataflow.dflow DEBUG: Task 2461 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2461 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2461 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2461 has memoization hash f892b67a5ca21d346eb591221a149da1 parsl.dataflow.memoization INFO: Task 2461 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2461 try 0 launched on executor multi with executor id 2462 parsl.dataflow.dflow INFO: Standard output for task 2461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/72ecae83-c79a-4b9c-8613-bb69978ee144_isrForCrosstalkSources_996990_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/72ecae83-c79a-4b9c-8613-bb69978ee144_isrForCrosstalkSources_996990_9.stderr parsl.dataflow.dflow DEBUG: Task 2462 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2462 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2462 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2462 has memoization hash 02ddc54f1072666e22b2ffe185e5d2e3 parsl.dataflow.memoization INFO: Task 2462 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2462 try 0 launched on executor multi with executor id 2463 parsl.dataflow.dflow INFO: Standard output for task 2462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/e607574e-e2a4-465f-8fd5-d3a706f60b10_isrForCrosstalkSources_997029_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/e607574e-e2a4-465f-8fd5-d3a706f60b10_isrForCrosstalkSources_997029_23.stderr parsl.dataflow.dflow DEBUG: Task 2463 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2463 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2463 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2463 has memoization hash 498c76a5ee74c1063cddd4605f5504f9 parsl.dataflow.memoization INFO: Task 2463 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2463 try 0 launched on executor multi with executor id 2464 parsl.dataflow.dflow INFO: Standard output for task 2463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4fdc2511-159d-46ae-88d2-0bccf62888c4_isrForCrosstalkSources_996966_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4fdc2511-159d-46ae-88d2-0bccf62888c4_isrForCrosstalkSources_996966_27.stderr parsl.dataflow.dflow DEBUG: Task 2464 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2464 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2464 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2464 has memoization hash 52bc1bc0d1424a2c2505dc754e55b353 parsl.dataflow.memoization INFO: Task 2464 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2464 try 0 launched on executor multi with executor id 2465 parsl.dataflow.dflow INFO: Standard output for task 2464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9cdebbfb-7ea2-40d1-a1a2-211471c5379b_isrForCrosstalkSources_997022_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9cdebbfb-7ea2-40d1-a1a2-211471c5379b_isrForCrosstalkSources_997022_9.stderr parsl.dataflow.dflow DEBUG: Task 2465 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2465 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2465 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2465 has memoization hash c5bddacd231c835b998d9e42496370e9 parsl.dataflow.memoization INFO: Task 2465 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2465 try 0 launched on executor multi with executor id 2466 parsl.dataflow.dflow INFO: Standard output for task 2465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/484dc16c-5b0d-44f2-94d7-f5e9244f16b1_isrForCrosstalkSources_996999_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/484dc16c-5b0d-44f2-94d7-f5e9244f16b1_isrForCrosstalkSources_996999_1.stderr parsl.dataflow.dflow DEBUG: Task 2466 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2466 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2466 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2466 has memoization hash f98ea358fee44726827be2de0efc19f8 parsl.dataflow.memoization INFO: Task 2466 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2466 try 0 launched on executor multi with executor id 2467 parsl.dataflow.dflow INFO: Standard output for task 2466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/685920a1-6843-4699-a8a8-a01d5cfe8976_isrForCrosstalkSources_997026_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/685920a1-6843-4699-a8a8-a01d5cfe8976_isrForCrosstalkSources_997026_16.stderr parsl.dataflow.dflow DEBUG: Task 2467 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2467 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2467 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2467 has memoization hash 36cb604b22f6f043fbfaddbb72623181 parsl.dataflow.memoization INFO: Task 2467 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2467 try 0 launched on executor multi with executor id 2468 parsl.dataflow.dflow INFO: Standard output for task 2467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/03e5d046-77c3-42f2-a8a2-0aedd569a6c8_isrForCrosstalkSources_996960_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/03e5d046-77c3-42f2-a8a2-0aedd569a6c8_isrForCrosstalkSources_996960_5.stderr parsl.dataflow.dflow DEBUG: Task 2468 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2468 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2468 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2468 has memoization hash 2bd2615bc8c04257998925b0be80f5f6 parsl.dataflow.memoization INFO: Task 2468 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2468 try 0 launched on executor multi with executor id 2469 parsl.dataflow.dflow INFO: Standard output for task 2468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1b7314ed-c185-49dd-b4e8-9cf61969ef6e_isrForCrosstalkSources_997032_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1b7314ed-c185-49dd-b4e8-9cf61969ef6e_isrForCrosstalkSources_997032_19.stderr parsl.dataflow.dflow DEBUG: Task 2469 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2469 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2469 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2469 has memoization hash 562d3f7035bd07f3b8cef91d964facc9 parsl.dataflow.memoization INFO: Task 2469 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2469 try 0 launched on executor multi with executor id 2470 parsl.dataflow.dflow INFO: Standard output for task 2469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/4a8e0011-e1cd-42e7-a81c-02dd1621a718_isrForCrosstalkSources_997026_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/4a8e0011-e1cd-42e7-a81c-02dd1621a718_isrForCrosstalkSources_997026_59.stderr parsl.dataflow.dflow DEBUG: Task 2470 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2470 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2470 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2470 has memoization hash 6c9f9447a4f06cdec92882ad84d4dce1 parsl.dataflow.memoization INFO: Task 2470 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2470 try 0 launched on executor multi with executor id 2471 parsl.dataflow.dflow INFO: Standard output for task 2470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/08921616-d482-4177-8707-1f59c89e32b3_isrForCrosstalkSources_997032_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/08921616-d482-4177-8707-1f59c89e32b3_isrForCrosstalkSources_997032_5.stderr parsl.dataflow.dflow DEBUG: Task 2471 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2471 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2471 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2471 has memoization hash 781b5231ea0a7a00f7e4a6cc34a124ea parsl.dataflow.memoization INFO: Task 2471 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2471 try 0 launched on executor multi with executor id 2472 parsl.dataflow.dflow INFO: Standard output for task 2471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/38f02035-7cc3-4c76-8368-231bce1c5ac3_isrForCrosstalkSources_997029_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/38f02035-7cc3-4c76-8368-231bce1c5ac3_isrForCrosstalkSources_997029_32.stderr parsl.dataflow.dflow DEBUG: Task 2472 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2472 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2472 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2472 has memoization hash d1f5fde84c553d3cd8393fec4e84b331 parsl.dataflow.memoization INFO: Task 2472 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2472 try 0 launched on executor multi with executor id 2473 parsl.dataflow.dflow INFO: Standard output for task 2472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/8ac482ed-aee0-4b71-b9d5-45d3b046e2d7_isrForCrosstalkSources_997015_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/8ac482ed-aee0-4b71-b9d5-45d3b046e2d7_isrForCrosstalkSources_997015_4.stderr parsl.dataflow.dflow DEBUG: Task 2473 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2473 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2473 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2473 has memoization hash a2c3a175d9a6f5a1a8cbb76d3d8f0fe7 parsl.dataflow.memoization INFO: Task 2473 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2473 try 0 launched on executor multi with executor id 2474 parsl.dataflow.dflow INFO: Standard output for task 2473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1c9cd0c9-4744-4539-8ad7-955b52bf14b6_isrForCrosstalkSources_997005_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1c9cd0c9-4744-4539-8ad7-955b52bf14b6_isrForCrosstalkSources_997005_42.stderr parsl.dataflow.dflow DEBUG: Task 2474 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2474 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2474 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2474 has memoization hash efced680dd6914d72f6d2519c467b40a parsl.dataflow.memoization INFO: Task 2474 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2474 try 0 launched on executor multi with executor id 2475 parsl.dataflow.dflow INFO: Standard output for task 2474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e3383d57-ed9b-4120-a091-85e161433571_isrForCrosstalkSources_996964_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e3383d57-ed9b-4120-a091-85e161433571_isrForCrosstalkSources_996964_35.stderr parsl.dataflow.dflow DEBUG: Task 2475 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2475 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2475 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2475 has memoization hash bb686b2aaa37567c09383e1d6d8b93f6 parsl.dataflow.memoization INFO: Task 2475 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2475 try 0 launched on executor multi with executor id 2476 parsl.dataflow.dflow INFO: Standard output for task 2475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/2080cdc4-6ecb-4ee9-8e83-85e04556475b_isrForCrosstalkSources_997029_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/2080cdc4-6ecb-4ee9-8e83-85e04556475b_isrForCrosstalkSources_997029_31.stderr parsl.dataflow.dflow DEBUG: Task 2476 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2476 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2476 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2476 has memoization hash 059601dbdf9c011f4fcaa2381d4c77f2 parsl.dataflow.memoization INFO: Task 2476 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2476 try 0 launched on executor multi with executor id 2477 parsl.dataflow.dflow INFO: Standard output for task 2476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/9ba5ef64-c34b-4780-88a7-6f45bd9e8285_isrForCrosstalkSources_997016_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/9ba5ef64-c34b-4780-88a7-6f45bd9e8285_isrForCrosstalkSources_997016_26.stderr parsl.dataflow.dflow DEBUG: Task 2477 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2477 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2477 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2477 has memoization hash 07e34658a450737cc46fcc3b23336afa parsl.dataflow.memoization INFO: Task 2477 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2477 try 0 launched on executor multi with executor id 2478 parsl.dataflow.dflow INFO: Standard output for task 2477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8ea4466b-04e3-47c5-aa75-4e88c1148067_isrForCrosstalkSources_997027_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8ea4466b-04e3-47c5-aa75-4e88c1148067_isrForCrosstalkSources_997027_3.stderr parsl.dataflow.dflow DEBUG: Task 2478 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2478 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2478 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2478 has memoization hash 2392741d05ff22cdbe754565430daf43 parsl.dataflow.memoization INFO: Task 2478 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2478 try 0 launched on executor multi with executor id 2479 parsl.dataflow.dflow INFO: Standard output for task 2478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/81247663-5844-4260-a9c4-66b5f246ab9c_isrForCrosstalkSources_997007_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/81247663-5844-4260-a9c4-66b5f246ab9c_isrForCrosstalkSources_997007_25.stderr parsl.dataflow.dflow DEBUG: Task 2479 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2479 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2479 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2479 has memoization hash e75bb72910aa7bc5bba8ca9f3683e5ab parsl.dataflow.memoization INFO: Task 2479 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2479 try 0 launched on executor multi with executor id 2480 parsl.dataflow.dflow INFO: Standard output for task 2479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4c2c0b3b-ecaa-4871-8102-6bc9c9e6fb33_isrForCrosstalkSources_997030_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4c2c0b3b-ecaa-4871-8102-6bc9c9e6fb33_isrForCrosstalkSources_997030_54.stderr parsl.dataflow.dflow DEBUG: Task 2480 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2480 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2480 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2480 has memoization hash 8a8b5e819527d534d24c729576827543 parsl.dataflow.memoization INFO: Task 2480 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2480 try 0 launched on executor multi with executor id 2481 parsl.dataflow.dflow INFO: Standard output for task 2480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/9ce3b733-8748-4685-bbbc-4a98009a8a81_isrForCrosstalkSources_996975_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/9ce3b733-8748-4685-bbbc-4a98009a8a81_isrForCrosstalkSources_996975_7.stderr parsl.dataflow.dflow DEBUG: Task 2481 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2481 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2481 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2481 has memoization hash d64f5a5a1beafa3dd84ad6a83489a9f6 parsl.dataflow.memoization INFO: Task 2481 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2481 try 0 launched on executor multi with executor id 2482 parsl.dataflow.dflow INFO: Standard output for task 2481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/143bd40d-6928-4262-92a0-7002f1a0abbd_isrForCrosstalkSources_996997_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/143bd40d-6928-4262-92a0-7002f1a0abbd_isrForCrosstalkSources_996997_49.stderr parsl.dataflow.dflow DEBUG: Task 2482 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2482 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2482 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2482 has memoization hash 9dfc1eb554e4e311cfb982ae742d5d2c parsl.dataflow.memoization INFO: Task 2482 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2482 try 0 launched on executor multi with executor id 2483 parsl.dataflow.dflow INFO: Standard output for task 2482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4a6a226d-cc30-4365-a423-8824a217a3ab_isrForCrosstalkSources_997005_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4a6a226d-cc30-4365-a423-8824a217a3ab_isrForCrosstalkSources_997005_18.stderr parsl.dataflow.dflow DEBUG: Task 2483 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2483 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2483 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2483 has memoization hash 08646ce45971d1711841e7af7d9d5be4 parsl.dataflow.memoization INFO: Task 2483 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2483 try 0 launched on executor multi with executor id 2484 parsl.dataflow.dflow INFO: Standard output for task 2483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b911106d-ee8a-4703-b84d-81a0937a869f_isrForCrosstalkSources_997006_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b911106d-ee8a-4703-b84d-81a0937a869f_isrForCrosstalkSources_997006_15.stderr parsl.dataflow.dflow DEBUG: Task 2484 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2484 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2484 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2484 has memoization hash 17e05cd2f6819ee1f656f035b05e7aeb parsl.dataflow.memoization INFO: Task 2484 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 2 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4387cc0859726aff11e4f26e92e4892 with result from task 2 parsl.dataflow.dflow INFO: Parsl task 2484 try 0 launched on executor multi with executor id 2485 parsl.dataflow.dflow INFO: Standard output for task 2484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/3c368c17-3eec-4892-b776-78198c59fa48_isrForCrosstalkSources_996956_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/3c368c17-3eec-4892-b776-78198c59fa48_isrForCrosstalkSources_996956_31.stderr parsl.dataflow.dflow DEBUG: Task 2485 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2485 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2485 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2485 has memoization hash 89dc1d8819db0bd1f612956ee6a0c190 parsl.dataflow.memoization INFO: Task 2485 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2485 try 0 launched on executor multi with executor id 2486 parsl.dataflow.dflow INFO: Standard output for task 2485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/07b53967-722f-47da-8b96-c776a6a98019_isrForCrosstalkSources_996991_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/07b53967-722f-47da-8b96-c776a6a98019_isrForCrosstalkSources_996991_8.stderr parsl.dataflow.dflow DEBUG: Task 2486 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2486 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2486 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2486 has memoization hash 98e8749d43ce7b2817ebd51b02c4f563 parsl.dataflow.memoization INFO: Task 2486 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2486 try 0 launched on executor multi with executor id 2487 parsl.dataflow.dflow INFO: Standard output for task 2486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/a2a2e00a-1564-490a-8b5e-730478417679_isrForCrosstalkSources_997014_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/a2a2e00a-1564-490a-8b5e-730478417679_isrForCrosstalkSources_997014_23.stderr parsl.dataflow.dflow DEBUG: Task 2487 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2487 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2487 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2487 has memoization hash 1ea46e1dc323ee0ba459732f68441a57 parsl.dataflow.memoization INFO: Task 2487 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2487 try 0 launched on executor multi with executor id 2488 parsl.dataflow.dflow INFO: Standard output for task 2487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/754cf6b7-4e54-4b7f-aeb2-23469d72e2ea_isrForCrosstalkSources_996967_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/754cf6b7-4e54-4b7f-aeb2-23469d72e2ea_isrForCrosstalkSources_996967_24.stderr parsl.dataflow.dflow DEBUG: Task 2488 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2488 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2488 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2488 has memoization hash 557385d7ec7d701bb43162b56b566195 parsl.dataflow.memoization INFO: Task 2488 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 2 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/70f8f7df-bc1d-47fe-aa8d-917f01994001_isrForCrosstalkSources_997032_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/70f8f7df-bc1d-47fe-aa8d-917f01994001_isrForCrosstalkSources_997032_2.stderr parsl.dataflow.dflow INFO: Parsl task 2488 try 0 launched on executor multi with executor id 2489 parsl.dataflow.dflow INFO: Standard output for task 2488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7ad86f58-710a-4c0f-b3f6-23b129cbde74_isrForCrosstalkSources_997008_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7ad86f58-710a-4c0f-b3f6-23b129cbde74_isrForCrosstalkSources_997008_17.stderr parsl.dataflow.dflow DEBUG: Task 2489 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2489 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2489 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2489 has memoization hash e3b2305ca3dadbc497e18ed4303018ee parsl.dataflow.memoization INFO: Task 2489 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2489 try 0 launched on executor multi with executor id 2490 parsl.dataflow.dflow INFO: Standard output for task 2489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5f54fc04-60a6-49ee-b4b8-42d420a5ef84_isrForCrosstalkSources_996954_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5f54fc04-60a6-49ee-b4b8-42d420a5ef84_isrForCrosstalkSources_996954_14.stderr parsl.dataflow.dflow DEBUG: Task 2490 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2490 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2490 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2490 has memoization hash 8ce7ee62d3641dd2192cb02cd3496ab4 parsl.dataflow.memoization INFO: Task 2490 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2490 try 0 launched on executor multi with executor id 2491 parsl.dataflow.dflow INFO: Standard output for task 2490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/650b7206-afc9-4367-b9df-144f24bf23bd_isrForCrosstalkSources_996979_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/650b7206-afc9-4367-b9df-144f24bf23bd_isrForCrosstalkSources_996979_59.stderr parsl.dataflow.dflow DEBUG: Task 2491 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2491 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2491 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2491 has memoization hash 8477bfcc29f56ae1c58e456b82203a3a parsl.dataflow.memoization INFO: Task 2491 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2491 try 0 launched on executor multi with executor id 2492 parsl.dataflow.dflow INFO: Standard output for task 2491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/62cabc1c-21c4-417a-8203-df71e1ba332a_isrForCrosstalkSources_997028_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/62cabc1c-21c4-417a-8203-df71e1ba332a_isrForCrosstalkSources_997028_23.stderr parsl.dataflow.dflow DEBUG: Task 2492 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2492 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2492 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2492 has memoization hash e4ed7ef5ea64020c2f280412e874b652 parsl.dataflow.memoization INFO: Task 2492 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2492 try 0 launched on executor multi with executor id 2493 parsl.dataflow.dflow INFO: Standard output for task 2492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/579844c0-081b-4b15-b94c-2e968bbce65b_isrForCrosstalkSources_996955_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/579844c0-081b-4b15-b94c-2e968bbce65b_isrForCrosstalkSources_996955_6.stderr parsl.dataflow.dflow DEBUG: Task 2493 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2493 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2493 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2493 has memoization hash b03c2f887f68b7b295aafe7bf68841ee parsl.dataflow.memoization INFO: Task 2493 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2493 try 0 launched on executor multi with executor id 2494 parsl.dataflow.dflow INFO: Standard output for task 2493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c5b07f0c-aa3c-462d-ae73-f44b94528d80_isrForCrosstalkSources_996977_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c5b07f0c-aa3c-462d-ae73-f44b94528d80_isrForCrosstalkSources_996977_34.stderr parsl.dataflow.dflow DEBUG: Task 2494 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2494 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2494 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2494 has memoization hash 18d9406b38d8d348243849f7bb4fee0a parsl.dataflow.memoization INFO: Task 2494 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2494 try 0 launched on executor multi with executor id 2495 parsl.dataflow.dflow INFO: Standard output for task 2494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8ca2649b-6d3e-4f40-9270-a4f90741c4f9_isrForCrosstalkSources_996976_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8ca2649b-6d3e-4f40-9270-a4f90741c4f9_isrForCrosstalkSources_996976_51.stderr parsl.dataflow.dflow DEBUG: Task 2495 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2495 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2495 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2495 has memoization hash 2eb3a7561784f9c6b1a06475fd895e65 parsl.dataflow.memoization INFO: Task 2495 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2495 try 0 launched on executor multi with executor id 2496 parsl.dataflow.dflow INFO: Standard output for task 2495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/87fca8ad-ed7f-4ff8-b086-e42d3ac53f38_isrForCrosstalkSources_997013_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/87fca8ad-ed7f-4ff8-b086-e42d3ac53f38_isrForCrosstalkSources_997013_10.stderr parsl.dataflow.dflow DEBUG: Task 2496 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2496 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2496 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2496 has memoization hash ced802752026dcade0d825eec58bdc00 parsl.dataflow.memoization INFO: Task 2496 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2496 try 0 launched on executor multi with executor id 2497 parsl.dataflow.dflow INFO: Standard output for task 2496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e7e16adb-6898-4987-84ce-1ba6bf8772b6_isrForCrosstalkSources_997008_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e7e16adb-6898-4987-84ce-1ba6bf8772b6_isrForCrosstalkSources_997008_9.stderr parsl.dataflow.dflow DEBUG: Task 2497 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2497 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2497 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2497 has memoization hash ec9400d761c239592411ab4ded60f28c parsl.dataflow.memoization INFO: Task 2497 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2497 try 0 launched on executor multi with executor id 2498 parsl.dataflow.dflow INFO: Standard output for task 2497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/fe80ea70-4405-417a-8655-6b8650e43324_isrForCrosstalkSources_996988_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/fe80ea70-4405-417a-8655-6b8650e43324_isrForCrosstalkSources_996988_35.stderr parsl.dataflow.dflow DEBUG: Task 2498 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2498 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2498 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2498 has memoization hash 653f69e8d691e799a634422c555e0bb8 parsl.dataflow.memoization INFO: Task 2498 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2498 try 0 launched on executor multi with executor id 2499 parsl.dataflow.dflow INFO: Standard output for task 2498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/db8f6832-f73c-40ab-872e-d3a12bde39a6_isrForCrosstalkSources_997025_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/db8f6832-f73c-40ab-872e-d3a12bde39a6_isrForCrosstalkSources_997025_55.stderr parsl.dataflow.dflow DEBUG: Task 2499 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2499 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2499 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2499 has memoization hash 6b2a17db9100e636863fac6625bbb1ed parsl.dataflow.memoization INFO: Task 2499 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2499 try 0 launched on executor multi with executor id 2500 parsl.dataflow.dflow INFO: Standard output for task 2499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/73648851-7089-4d14-bb8c-324b585fd67b_isrForCrosstalkSources_996984_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/73648851-7089-4d14-bb8c-324b585fd67b_isrForCrosstalkSources_996984_4.stderr parsl.dataflow.dflow DEBUG: Task 2500 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2500 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2500 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2500 has memoization hash be036de564dd67efdf8428f2b242d8ab parsl.dataflow.memoization INFO: Task 2500 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2500 try 0 launched on executor multi with executor id 2501 parsl.dataflow.dflow INFO: Standard output for task 2500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6ef61b85-fc2e-45cf-b8d0-ba12b77b90b1_isrForCrosstalkSources_997023_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6ef61b85-fc2e-45cf-b8d0-ba12b77b90b1_isrForCrosstalkSources_997023_5.stderr parsl.dataflow.dflow DEBUG: Task 2501 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2501 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2501 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2501 has memoization hash d8f8594cff654b75d55b49845b4f300e parsl.dataflow.memoization INFO: Task 2501 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2501 try 0 launched on executor multi with executor id 2502 parsl.dataflow.dflow INFO: Standard output for task 2501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/b2904d3b-2683-451f-a2d1-fd1275089632_isrForCrosstalkSources_997002_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/b2904d3b-2683-451f-a2d1-fd1275089632_isrForCrosstalkSources_997002_57.stderr parsl.dataflow.dflow DEBUG: Task 2502 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2502 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2502 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2502 has memoization hash e36598c00f2eac1ed89eb0d34a280eaf parsl.dataflow.memoization INFO: Task 2502 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2502 try 0 launched on executor multi with executor id 2503 parsl.dataflow.dflow INFO: Standard output for task 2502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/45d860bc-81d6-4ff7-85be-7fded96dbf90_isrForCrosstalkSources_997027_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/45d860bc-81d6-4ff7-85be-7fded96dbf90_isrForCrosstalkSources_997027_18.stderr parsl.dataflow.dflow DEBUG: Task 2503 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2503 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2503 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2503 has memoization hash 6e643e9be5aba110caeb714f269100c8 parsl.dataflow.memoization INFO: Task 2503 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2503 try 0 launched on executor multi with executor id 2504 parsl.dataflow.dflow INFO: Standard output for task 2503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/76183e15-1ecc-4def-8c97-4c7cecb44fc7_isrForCrosstalkSources_997008_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/76183e15-1ecc-4def-8c97-4c7cecb44fc7_isrForCrosstalkSources_997008_23.stderr parsl.dataflow.dflow DEBUG: Task 2504 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2504 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2504 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2504 has memoization hash 9bc6ef324859262a4d5771c19db253a6 parsl.dataflow.memoization INFO: Task 2504 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2504 try 0 launched on executor multi with executor id 2505 parsl.dataflow.dflow INFO: Standard output for task 2504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/bd71fbae-748d-4b5c-ba5b-b88b238a644c_isrForCrosstalkSources_997017_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/bd71fbae-748d-4b5c-ba5b-b88b238a644c_isrForCrosstalkSources_997017_56.stderr parsl.dataflow.dflow DEBUG: Task 2505 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2505 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2505 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2505 has memoization hash 92d37ca69b9cb157c18612970a9679c1 parsl.dataflow.memoization INFO: Task 2505 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2505 try 0 launched on executor multi with executor id 2506 parsl.dataflow.dflow INFO: Standard output for task 2505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/394682c5-8e1d-4b39-b647-15739f1a658a_isrForCrosstalkSources_996977_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/394682c5-8e1d-4b39-b647-15739f1a658a_isrForCrosstalkSources_996977_55.stderr parsl.dataflow.dflow DEBUG: Task 2506 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2506 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2506 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2506 has memoization hash 4cf3e116837c379716037ec4ffb44458 parsl.dataflow.memoization INFO: Task 2506 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2506 try 0 launched on executor multi with executor id 2507 parsl.dataflow.dflow INFO: Standard output for task 2506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5460541a-e91e-4a97-a592-96d9e131ea99_isrForCrosstalkSources_996981_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5460541a-e91e-4a97-a592-96d9e131ea99_isrForCrosstalkSources_996981_33.stderr parsl.dataflow.dflow DEBUG: Task 2507 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2507 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2507 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2507 has memoization hash e58cf9b9f463fe6f56ed989562b7df0b parsl.dataflow.memoization INFO: Task 2507 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2507 try 0 launched on executor multi with executor id 2508 parsl.dataflow.dflow INFO: Standard output for task 2507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a6192808-b073-4ce9-8dff-8edd986d57fe_isrForCrosstalkSources_997027_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a6192808-b073-4ce9-8dff-8edd986d57fe_isrForCrosstalkSources_997027_17.stderr parsl.dataflow.dflow DEBUG: Task 2508 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2508 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2508 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2508 has memoization hash c2a64ba4d851d76dafae9027295b7dc7 parsl.dataflow.memoization INFO: Task 2508 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2508 try 0 launched on executor multi with executor id 2509 parsl.dataflow.dflow INFO: Standard output for task 2508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/feac3256-5eba-4877-9b2a-c9d123437b7f_isrForCrosstalkSources_996953_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/feac3256-5eba-4877-9b2a-c9d123437b7f_isrForCrosstalkSources_996953_17.stderr parsl.dataflow.dflow DEBUG: Task 2509 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2509 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2509 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2509 has memoization hash 75b3c7237a2402994b659c8a4fc50634 parsl.dataflow.memoization INFO: Task 2509 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2509 try 0 launched on executor multi with executor id 2510 parsl.dataflow.dflow INFO: Standard output for task 2509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/5385e89d-ebf1-44cd-991d-7a40cd57190a_isrForCrosstalkSources_996962_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/5385e89d-ebf1-44cd-991d-7a40cd57190a_isrForCrosstalkSources_996962_59.stderr parsl.dataflow.dflow DEBUG: Task 2510 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2510 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2510 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2510 has memoization hash eec0cdff45e777b69e475d201dc504ba parsl.dataflow.memoization INFO: Task 2510 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2510 try 0 launched on executor multi with executor id 2511 parsl.dataflow.dflow INFO: Standard output for task 2510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/764fe377-13be-424e-a011-5c09bb1e2cfd_isrForCrosstalkSources_996995_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/764fe377-13be-424e-a011-5c09bb1e2cfd_isrForCrosstalkSources_996995_39.stderr parsl.dataflow.dflow DEBUG: Task 2511 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2511 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2511 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2511 has memoization hash cae6e1ab3c6d9127e4b4d7bef75e84a4 parsl.dataflow.memoization INFO: Task 2511 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2511 try 0 launched on executor multi with executor id 2512 parsl.dataflow.dflow INFO: Standard output for task 2511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/03731a2e-3cf8-434b-9fd9-5af968cb2539_isrForCrosstalkSources_996985_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/03731a2e-3cf8-434b-9fd9-5af968cb2539_isrForCrosstalkSources_996985_26.stderr parsl.dataflow.dflow DEBUG: Task 2512 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2512 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2512 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2512 has memoization hash c2604eb843130f9a305135e7e857d049 parsl.dataflow.memoization INFO: Task 2512 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2512 try 0 launched on executor multi with executor id 2513 parsl.dataflow.dflow INFO: Standard output for task 2512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5a96248f-e412-43e3-9b24-77bfcdeda066_isrForCrosstalkSources_996981_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5a96248f-e412-43e3-9b24-77bfcdeda066_isrForCrosstalkSources_996981_60.stderr parsl.dataflow.dflow DEBUG: Task 2513 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2513 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2513 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2513 has memoization hash d437cae2cd533975ca868d365d3e3b5c parsl.dataflow.memoization INFO: Task 2513 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2513 try 0 launched on executor multi with executor id 2514 parsl.dataflow.dflow INFO: Standard output for task 2513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/06b8cd52-0a81-4d95-85ca-9cfe0540d2ed_isrForCrosstalkSources_996980_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/06b8cd52-0a81-4d95-85ca-9cfe0540d2ed_isrForCrosstalkSources_996980_53.stderr parsl.dataflow.dflow DEBUG: Task 2514 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2514 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2514 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2514 has memoization hash 3b7c645ba5c37205373c29d369c1b65e parsl.dataflow.memoization INFO: Task 2514 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2514 try 0 launched on executor multi with executor id 2515 parsl.dataflow.dflow INFO: Standard output for task 2514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/3172333c-75f5-4104-9619-8e8c04c8b768_isrForCrosstalkSources_997018_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/3172333c-75f5-4104-9619-8e8c04c8b768_isrForCrosstalkSources_997018_43.stderr parsl.dataflow.dflow DEBUG: Task 2515 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2515 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2515 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2515 has memoization hash 6817eaac74498ff3348ae30c9273707f parsl.dataflow.memoization INFO: Task 2515 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2515 try 0 launched on executor multi with executor id 2516 parsl.dataflow.dflow INFO: Standard output for task 2515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/59204e0e-5518-4ed2-8029-e0be336d48fe_isrForCrosstalkSources_996991_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/59204e0e-5518-4ed2-8029-e0be336d48fe_isrForCrosstalkSources_996991_56.stderr parsl.dataflow.dflow DEBUG: Task 2516 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2516 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2516 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2516 has memoization hash 729b51b58fca56a820da52eac97c8965 parsl.dataflow.memoization INFO: Task 2516 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2516 try 0 launched on executor multi with executor id 2517 parsl.dataflow.dflow INFO: Standard output for task 2516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6a67ea77-85da-4374-98fb-cc78d10823eb_isrForCrosstalkSources_996996_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6a67ea77-85da-4374-98fb-cc78d10823eb_isrForCrosstalkSources_996996_18.stderr parsl.dataflow.dflow DEBUG: Task 2517 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2517 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2517 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2517 has memoization hash 5aa081cfc8a0cd923c3786e53ffed8af parsl.dataflow.memoization INFO: Task 2517 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2517 try 0 launched on executor multi with executor id 2518 parsl.dataflow.dflow INFO: Standard output for task 2517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c020edf7-b683-4a2a-8f84-a174fa2dee2c_isrForCrosstalkSources_996962_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c020edf7-b683-4a2a-8f84-a174fa2dee2c_isrForCrosstalkSources_996962_48.stderr parsl.dataflow.dflow DEBUG: Task 2518 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2518 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2518 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2518 has memoization hash 15c94904ba770187f7956282605c5691 parsl.dataflow.memoization INFO: Task 2518 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2518 try 0 launched on executor multi with executor id 2519 parsl.dataflow.dflow INFO: Standard output for task 2518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/413ab978-1b98-4b0f-9b9b-fabd1a0f672c_isrForCrosstalkSources_997026_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/413ab978-1b98-4b0f-9b9b-fabd1a0f672c_isrForCrosstalkSources_997026_49.stderr parsl.dataflow.dflow DEBUG: Task 2519 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2519 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2519 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2519 has memoization hash 030a5ab8d8c99932fe07e4e71a6c607f parsl.dataflow.memoization INFO: Task 2519 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2519 try 0 launched on executor multi with executor id 2520 parsl.dataflow.dflow INFO: Standard output for task 2519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/51773e66-985a-44e6-91a2-3792fb879878_isrForCrosstalkSources_996990_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/51773e66-985a-44e6-91a2-3792fb879878_isrForCrosstalkSources_996990_60.stderr parsl.dataflow.dflow DEBUG: Task 2520 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2520 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2520 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2520 has memoization hash 19d0a0fbebcf8f49fc0d701a8f7e8e51 parsl.dataflow.memoization INFO: Task 2520 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2520 try 0 launched on executor multi with executor id 2521 parsl.dataflow.dflow INFO: Standard output for task 2520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/21e0a183-290c-443a-a185-fbcac2759de6_isrForCrosstalkSources_997010_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/21e0a183-290c-443a-a185-fbcac2759de6_isrForCrosstalkSources_997010_49.stderr parsl.dataflow.dflow DEBUG: Task 2521 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2521 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2521 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2521 has memoization hash 08e24e9e6c82d6a39feab5be0543d5f2 parsl.dataflow.memoization INFO: Task 2521 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2521 try 0 launched on executor multi with executor id 2522 parsl.dataflow.dflow INFO: Standard output for task 2521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b2b673d3-e075-45ac-92f9-60bea2581246_isrForCrosstalkSources_996996_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b2b673d3-e075-45ac-92f9-60bea2581246_isrForCrosstalkSources_996996_54.stderr parsl.dataflow.dflow DEBUG: Task 2522 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2522 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2522 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2522 has memoization hash 853e196ca5f09a982e6d37833bb6c816 parsl.dataflow.memoization INFO: Task 2522 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2522 try 0 launched on executor multi with executor id 2523 parsl.dataflow.dflow INFO: Standard output for task 2522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/30b477fa-457c-478d-9ceb-a91664ba836a_isrForCrosstalkSources_997023_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/30b477fa-457c-478d-9ceb-a91664ba836a_isrForCrosstalkSources_997023_35.stderr parsl.dataflow.dflow DEBUG: Task 2523 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2523 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2523 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2523 has memoization hash e357cd7e4622c7a6aebe1561ca828ae6 parsl.dataflow.memoization INFO: Task 2523 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2523 try 0 launched on executor multi with executor id 2524 parsl.dataflow.dflow INFO: Standard output for task 2523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/bc53c342-b5a9-47dc-9c69-3c24a26100d5_isrForCrosstalkSources_997011_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/bc53c342-b5a9-47dc-9c69-3c24a26100d5_isrForCrosstalkSources_997011_22.stderr parsl.dataflow.dflow DEBUG: Task 2524 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2524 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2524 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2524 has memoization hash 849bf1054e542eb98d29921074a9f290 parsl.dataflow.memoization INFO: Task 2524 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2524 try 0 launched on executor multi with executor id 2525 parsl.dataflow.dflow INFO: Standard output for task 2524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/868d37d6-b274-4f24-b22d-8d9e4d8a46a5_isrForCrosstalkSources_997014_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/868d37d6-b274-4f24-b22d-8d9e4d8a46a5_isrForCrosstalkSources_997014_13.stderr parsl.dataflow.dflow DEBUG: Task 2525 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2525 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2525 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2525 has memoization hash c7edad8c4ec3ba5ecc70d1de6762531d parsl.dataflow.memoization INFO: Task 2525 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2525 try 0 launched on executor multi with executor id 2526 parsl.dataflow.dflow INFO: Standard output for task 2525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f9fbd15a-0dc8-4636-94d4-f6a4dfd58dfc_isrForCrosstalkSources_997015_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f9fbd15a-0dc8-4636-94d4-f6a4dfd58dfc_isrForCrosstalkSources_997015_10.stderr parsl.dataflow.dflow DEBUG: Task 2526 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2526 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2526 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2526 has memoization hash 5afe9a79c32a8378c0b023f4cf052b08 parsl.dataflow.memoization INFO: Task 2526 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2526 try 0 launched on executor multi with executor id 2527 parsl.dataflow.dflow INFO: Standard output for task 2526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3fa69f95-c7de-4048-bea3-d270e19a3d9f_isrForCrosstalkSources_997019_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3fa69f95-c7de-4048-bea3-d270e19a3d9f_isrForCrosstalkSources_997019_29.stderr parsl.dataflow.dflow DEBUG: Task 2527 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2527 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2527 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2527 has memoization hash 86a3ae909dea49d8f355f7ad47b32145 parsl.dataflow.memoization INFO: Task 2527 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2527 try 0 launched on executor multi with executor id 2528 parsl.dataflow.dflow INFO: Standard output for task 2527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/3948b96a-b3d8-4690-a8ca-8c79213e0b66_isrForCrosstalkSources_996980_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/3948b96a-b3d8-4690-a8ca-8c79213e0b66_isrForCrosstalkSources_996980_28.stderr parsl.dataflow.dflow DEBUG: Task 2528 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2528 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2528 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2528 has memoization hash 297adee4b391c9e3f7eed228b9b5eb57 parsl.dataflow.memoization INFO: Task 2528 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2528 try 0 launched on executor multi with executor id 2529 parsl.dataflow.dflow INFO: Standard output for task 2528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e0232251-e2f4-4e0b-a157-cb45ccc84f66_isrForCrosstalkSources_997031_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e0232251-e2f4-4e0b-a157-cb45ccc84f66_isrForCrosstalkSources_997031_60.stderr parsl.dataflow.dflow DEBUG: Task 2529 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2529 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2529 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2529 has memoization hash 6484dee228eda5c5d3d2b5fa75af6e3e parsl.dataflow.memoization INFO: Task 2529 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 8 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e1df712d18da8692d9ea9245a6a69b77 with result from task 8 parsl.dataflow.dflow INFO: Parsl task 2529 try 0 launched on executor multi with executor id 2530 parsl.dataflow.dflow INFO: Standard output for task 2529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/98746efa-895b-483c-b3cd-a6c71e697dfd_isrForCrosstalkSources_997029_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/98746efa-895b-483c-b3cd-a6c71e697dfd_isrForCrosstalkSources_997029_53.stderr parsl.dataflow.dflow DEBUG: Task 2530 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2530 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2530 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2530 has memoization hash 7d177dd68143a2ec83baf68c91e94e1d parsl.dataflow.memoization INFO: Task 2530 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2530 try 0 launched on executor multi with executor id 2531 parsl.dataflow.dflow INFO: Standard output for task 2530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6ca57d18-4a67-4396-90d2-64dcd0f5a348_isrForCrosstalkSources_997023_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6ca57d18-4a67-4396-90d2-64dcd0f5a348_isrForCrosstalkSources_997023_16.stderr parsl.dataflow.dflow DEBUG: Task 2531 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2531 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2531 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2531 has memoization hash a67e122a1127943337806e234baf4f03 parsl.dataflow.memoization INFO: Task 2531 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2531 try 0 launched on executor multi with executor id 2532 parsl.dataflow.dflow INFO: Standard output for task 2531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/de156fff-c812-4c97-a8e0-d3a683b0db3f_isrForCrosstalkSources_996974_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/de156fff-c812-4c97-a8e0-d3a683b0db3f_isrForCrosstalkSources_996974_23.stderr parsl.dataflow.dflow DEBUG: Task 2532 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2532 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2532 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2532 has memoization hash 9de0f8d6ff42ca83d4d93f96d252fb61 parsl.dataflow.memoization INFO: Task 2532 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 8 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 8 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/26cfef11-c7f6-4152-b2b1-94753ce475ad_isrForCrosstalkSources_996992_1.stdout parsl.dataflow.dflow INFO: Standard error for task 8 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/26cfef11-c7f6-4152-b2b1-94753ce475ad_isrForCrosstalkSources_996992_1.stderr parsl.dataflow.dflow INFO: Parsl task 2532 try 0 launched on executor multi with executor id 2533 parsl.dataflow.dflow INFO: Standard output for task 2532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/20bd8833-d712-42f6-9a06-50b38fbd287f_isrForCrosstalkSources_997012_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/20bd8833-d712-42f6-9a06-50b38fbd287f_isrForCrosstalkSources_997012_43.stderr parsl.dataflow.dflow DEBUG: Task 2533 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2533 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2533 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2533 has memoization hash 88bb6bd844feef6e01b18c08511851a4 parsl.dataflow.memoization INFO: Task 2533 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2533 try 0 launched on executor multi with executor id 2534 parsl.dataflow.dflow INFO: Standard output for task 2533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/42e8012f-63ac-4590-9217-9e342f4a13a7_isrForCrosstalkSources_997012_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/42e8012f-63ac-4590-9217-9e342f4a13a7_isrForCrosstalkSources_997012_49.stderr parsl.dataflow.dflow DEBUG: Task 2534 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2534 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2534 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2534 has memoization hash 8732bc1e0015fadb18f56dfac6db5155 parsl.dataflow.memoization INFO: Task 2534 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2534 try 0 launched on executor multi with executor id 2535 parsl.dataflow.dflow INFO: Standard output for task 2534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/62f2341b-bcd8-4701-876f-b8ea82df081e_isrForCrosstalkSources_997017_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/62f2341b-bcd8-4701-876f-b8ea82df081e_isrForCrosstalkSources_997017_13.stderr parsl.dataflow.dflow DEBUG: Task 2535 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2535 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2535 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2535 has memoization hash 13ba6241938afd4fea36b9598bbfe47f parsl.dataflow.memoization INFO: Task 2535 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2535 try 0 launched on executor multi with executor id 2536 parsl.dataflow.dflow INFO: Standard output for task 2535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5cf45e66-bdef-466f-a6c5-85d17ff20cb8_isrForCrosstalkSources_996979_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5cf45e66-bdef-466f-a6c5-85d17ff20cb8_isrForCrosstalkSources_996979_62.stderr parsl.dataflow.dflow DEBUG: Task 2536 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2536 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2536 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2536 has memoization hash 38a4b7b3d44845eb5bd470071a4d6a83 parsl.dataflow.memoization INFO: Task 2536 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2536 try 0 launched on executor multi with executor id 2537 parsl.dataflow.dflow INFO: Standard output for task 2536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/1f6a0812-ddbd-426a-a30a-aa6143e52cf7_isrForCrosstalkSources_997033_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/1f6a0812-ddbd-426a-a30a-aa6143e52cf7_isrForCrosstalkSources_997033_8.stderr parsl.dataflow.dflow DEBUG: Task 2537 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2537 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2537 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2537 has memoization hash 44ac4c799bebe3cfad5bbcea73735197 parsl.dataflow.memoization INFO: Task 2537 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2537 try 0 launched on executor multi with executor id 2538 parsl.dataflow.dflow INFO: Standard output for task 2537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b39cff8b-aa1f-4a37-9a5b-f805dbe46e6b_isrForCrosstalkSources_996964_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b39cff8b-aa1f-4a37-9a5b-f805dbe46e6b_isrForCrosstalkSources_996964_59.stderr parsl.dataflow.dflow DEBUG: Task 2538 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2538 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2538 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2538 has memoization hash f347afc6cb5624a868db70c0cb8b0d19 parsl.dataflow.memoization INFO: Task 2538 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2538 try 0 launched on executor multi with executor id 2539 parsl.dataflow.dflow INFO: Standard output for task 2538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/d2680210-80a9-4585-82f7-88a65cdd93f7_isrForCrosstalkSources_996959_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/d2680210-80a9-4585-82f7-88a65cdd93f7_isrForCrosstalkSources_996959_34.stderr parsl.dataflow.dflow DEBUG: Task 2539 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2539 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2539 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2539 has memoization hash 6ea89b6f8cafbcf803abd19eff21ab42 parsl.dataflow.memoization INFO: Task 2539 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2539 try 0 launched on executor multi with executor id 2540 parsl.dataflow.dflow INFO: Standard output for task 2539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ded3e5c8-5caa-4fdd-9e3f-e245a7424138_isrForCrosstalkSources_996986_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ded3e5c8-5caa-4fdd-9e3f-e245a7424138_isrForCrosstalkSources_996986_30.stderr parsl.dataflow.dflow DEBUG: Task 2540 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2540 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2540 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2540 has memoization hash eee50c68eacb25a4e78605dc5e215ec6 parsl.dataflow.memoization INFO: Task 2540 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2540 try 0 launched on executor multi with executor id 2541 parsl.dataflow.dflow INFO: Standard output for task 2540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b1ca13d8-cc56-4693-9dc3-79315d5b8b94_isrForCrosstalkSources_997033_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b1ca13d8-cc56-4693-9dc3-79315d5b8b94_isrForCrosstalkSources_997033_29.stderr parsl.dataflow.dflow DEBUG: Task 2541 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2541 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2541 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2541 has memoization hash 5146f5c498f4e2d6baf3e2feaa74a1dd parsl.dataflow.memoization INFO: Task 2541 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2541 try 0 launched on executor multi with executor id 2542 parsl.dataflow.dflow INFO: Standard output for task 2541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/38fa187a-5416-4344-bd36-98ab40652641_isrForCrosstalkSources_996987_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/38fa187a-5416-4344-bd36-98ab40652641_isrForCrosstalkSources_996987_34.stderr parsl.dataflow.dflow DEBUG: Task 2542 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2542 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2542 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2542 has memoization hash f426943cf9786dee02e9153a49b3a8dc parsl.dataflow.memoization INFO: Task 2542 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2542 try 0 launched on executor multi with executor id 2543 parsl.dataflow.dflow INFO: Standard output for task 2542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a9a002b0-6053-434b-a53b-85d70c1825f3_isrForCrosstalkSources_997028_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a9a002b0-6053-434b-a53b-85d70c1825f3_isrForCrosstalkSources_997028_18.stderr parsl.dataflow.dflow DEBUG: Task 2543 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2543 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2543 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2543 has memoization hash 210e359ee95d7c590380cee79d0cf48a parsl.dataflow.memoization INFO: Task 2543 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb6dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2543 try 0 launched on executor multi with executor id 2544 parsl.dataflow.dflow INFO: Standard output for task 2543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9442ad18-939e-4ee5-a229-91d64b56c929_isrForCrosstalkSources_996979_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9442ad18-939e-4ee5-a229-91d64b56c929_isrForCrosstalkSources_996979_39.stderr parsl.dataflow.dflow DEBUG: Task 2544 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2544 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2544 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2544 has memoization hash a0e4e9895b549704c0075b11bd86894f parsl.dataflow.memoization INFO: Task 2544 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2544 try 0 launched on executor multi with executor id 2545 parsl.dataflow.dflow INFO: Standard output for task 2544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b9f8fc73-64cf-448b-86a4-ea68ad349bfe_isrForCrosstalkSources_997013_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b9f8fc73-64cf-448b-86a4-ea68ad349bfe_isrForCrosstalkSources_997013_43.stderr parsl.dataflow.dflow DEBUG: Task 2545 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2545 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2545 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2545 has memoization hash 5c0e9192849296f7cd50ebc871da7bd4 parsl.dataflow.memoization INFO: Task 2545 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e00e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2545 try 0 launched on executor multi with executor id 2546 parsl.dataflow.dflow INFO: Standard output for task 2545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/da9b95f2-85f4-4049-9123-66b4308abe0b_isrForCrosstalkSources_996955_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/da9b95f2-85f4-4049-9123-66b4308abe0b_isrForCrosstalkSources_996955_5.stderr parsl.dataflow.dflow DEBUG: Task 2546 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2546 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2546 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2546 has memoization hash 96e6df47d14c05d02fb2f4354600556c parsl.dataflow.memoization INFO: Task 2546 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2546 try 0 launched on executor multi with executor id 2547 parsl.dataflow.dflow INFO: Standard output for task 2546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/1a76e343-5c9a-42d9-b71f-b66036f2a9c1_isrForCrosstalkSources_997031_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/1a76e343-5c9a-42d9-b71f-b66036f2a9c1_isrForCrosstalkSources_997031_53.stderr parsl.dataflow.dflow DEBUG: Task 2547 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2547 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2547 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2547 has memoization hash 15896911c0a8c32d74fafffb55b70315 parsl.dataflow.memoization INFO: Task 2547 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2547 try 0 launched on executor multi with executor id 2548 parsl.dataflow.dflow INFO: Standard output for task 2547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/71fd6127-99fa-4335-a4f5-f8404aa626cf_isrForCrosstalkSources_997000_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/71fd6127-99fa-4335-a4f5-f8404aa626cf_isrForCrosstalkSources_997000_56.stderr parsl.dataflow.dflow DEBUG: Task 2548 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2548 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2548 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2548 has memoization hash ef5d9115bb2dd4465f83c9b9c1c9c94d parsl.dataflow.memoization INFO: Task 2548 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2548 try 0 launched on executor multi with executor id 2549 parsl.dataflow.dflow INFO: Standard output for task 2548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/66efa14a-c4e3-4ec7-9dcc-bd9f1e30ca43_isrForCrosstalkSources_997013_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/66efa14a-c4e3-4ec7-9dcc-bd9f1e30ca43_isrForCrosstalkSources_997013_8.stderr parsl.dataflow.dflow DEBUG: Task 2549 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2549 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2549 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2549 has memoization hash 2232c44b540c52dcb1c5df28090bea53 parsl.dataflow.memoization INFO: Task 2549 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e04a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2549 try 0 launched on executor multi with executor id 2550 parsl.dataflow.dflow INFO: Standard output for task 2549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9e59db63-d377-416c-addc-8b51b903b519_isrForCrosstalkSources_996959_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9e59db63-d377-416c-addc-8b51b903b519_isrForCrosstalkSources_996959_31.stderr parsl.dataflow.dflow DEBUG: Task 2550 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2550 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2550 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2550 has memoization hash 6f30b9bf5a4add14b0fa132c39e88842 parsl.dataflow.memoization INFO: Task 2550 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2550 try 0 launched on executor multi with executor id 2551 parsl.dataflow.dflow INFO: Standard output for task 2550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a4fcdb89-f046-42f7-878a-9cdf558f28c5_isrForCrosstalkSources_997002_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a4fcdb89-f046-42f7-878a-9cdf558f28c5_isrForCrosstalkSources_997002_51.stderr parsl.dataflow.dflow DEBUG: Task 2551 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2551 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2551 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2551 has memoization hash 460fcfa92a17ea4fa16ae96f71c35a59 parsl.dataflow.memoization INFO: Task 2551 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2551 try 0 launched on executor multi with executor id 2552 parsl.dataflow.dflow INFO: Standard output for task 2551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ebdf3984-c96e-4a49-b716-635e0d216e26_isrForCrosstalkSources_996978_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ebdf3984-c96e-4a49-b716-635e0d216e26_isrForCrosstalkSources_996978_55.stderr parsl.dataflow.dflow DEBUG: Task 2552 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2552 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2552 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2552 has memoization hash f55bb01db5201bb84ea777d8df7bb250 parsl.dataflow.memoization INFO: Task 2552 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e05e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2552 try 0 launched on executor multi with executor id 2553 parsl.dataflow.dflow INFO: Standard output for task 2552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/dc286c2e-bc72-4c30-8743-9c7fdc105ef8_isrForCrosstalkSources_996972_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/dc286c2e-bc72-4c30-8743-9c7fdc105ef8_isrForCrosstalkSources_996972_25.stderr parsl.dataflow.dflow DEBUG: Task 2553 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2553 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2553 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2553 has memoization hash 2cd567e28d03dcfd35dbc035322f1355 parsl.dataflow.memoization INFO: Task 2553 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2553 try 0 launched on executor multi with executor id 2554 parsl.dataflow.dflow INFO: Standard output for task 2553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c343e056-dbd9-4114-a859-3b99a9e2b7e3_isrForCrosstalkSources_997003_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c343e056-dbd9-4114-a859-3b99a9e2b7e3_isrForCrosstalkSources_997003_37.stderr parsl.dataflow.dflow DEBUG: Task 2554 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2554 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2554 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2554 has memoization hash f2e75a52dbded46f592bae290f15e0e0 parsl.dataflow.memoization INFO: Task 2554 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e07c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2554 try 0 launched on executor multi with executor id 2555 parsl.dataflow.dflow INFO: Standard output for task 2554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/feec21b0-a4fd-4273-8390-95e3c42face1_isrForCrosstalkSources_996965_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/feec21b0-a4fd-4273-8390-95e3c42face1_isrForCrosstalkSources_996965_45.stderr parsl.dataflow.dflow DEBUG: Task 2555 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2555 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2555 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2555 has memoization hash 7e2f0de0cf72e5d5eab479be32e5014c parsl.dataflow.memoization INFO: Task 2555 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2555 try 0 launched on executor multi with executor id 2556 parsl.dataflow.dflow INFO: Standard output for task 2555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3e25aceb-39c0-4872-882a-f26cb0ce7d19_isrForCrosstalkSources_997001_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3e25aceb-39c0-4872-882a-f26cb0ce7d19_isrForCrosstalkSources_997001_29.stderr parsl.dataflow.dflow DEBUG: Task 2556 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2556 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2556 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2556 has memoization hash f2b5bea9ec9a11079ab89a8af8633303 parsl.dataflow.memoization INFO: Task 2556 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2556 try 0 launched on executor multi with executor id 2557 parsl.dataflow.dflow INFO: Standard output for task 2556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a7cc4fc9-8af3-48c9-940a-5bef9ef6fa51_isrForCrosstalkSources_996960_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a7cc4fc9-8af3-48c9-940a-5bef9ef6fa51_isrForCrosstalkSources_996960_18.stderr parsl.dataflow.dflow DEBUG: Task 2557 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2557 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2557 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2557 has memoization hash 045a9bed7561f5dc433d807bfbbfa415 parsl.dataflow.memoization INFO: Task 2557 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2557 try 0 launched on executor multi with executor id 2558 parsl.dataflow.dflow INFO: Standard output for task 2557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/39c452e7-9090-4e8a-b15d-47bea445cc6c_isrForCrosstalkSources_997032_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/39c452e7-9090-4e8a-b15d-47bea445cc6c_isrForCrosstalkSources_997032_21.stderr parsl.dataflow.dflow DEBUG: Task 2558 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2558 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2558 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2558 has memoization hash e7db0e84450a8555b5e4f469ef9a3f2a parsl.dataflow.memoization INFO: Task 2558 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e09a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2558 try 0 launched on executor multi with executor id 2559 parsl.dataflow.dflow INFO: Standard output for task 2558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/10b9d48a-e43b-4e80-a571-322332c74ccb_isrForCrosstalkSources_996983_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/10b9d48a-e43b-4e80-a571-322332c74ccb_isrForCrosstalkSources_996983_58.stderr parsl.dataflow.dflow DEBUG: Task 2559 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2559 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2559 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2559 has memoization hash 042e102cb8a25faca5ff3cad90671602 parsl.dataflow.memoization INFO: Task 2559 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2559 try 0 launched on executor multi with executor id 2560 parsl.dataflow.dflow INFO: Standard output for task 2559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c2abc604-e524-4229-9cf1-a57ccd5aaf61_isrForCrosstalkSources_996991_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c2abc604-e524-4229-9cf1-a57ccd5aaf61_isrForCrosstalkSources_996991_44.stderr parsl.dataflow.dflow DEBUG: Task 2560 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2560 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2560 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2560 has memoization hash 58d88f852f9aaecddfbee7109bfe93f4 parsl.dataflow.memoization INFO: Task 2560 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2560 try 0 launched on executor multi with executor id 2561 parsl.dataflow.dflow INFO: Standard output for task 2560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/1dc1edf6-fdcd-4521-a361-cade7b1e97ce_isrForCrosstalkSources_996981_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/1dc1edf6-fdcd-4521-a361-cade7b1e97ce_isrForCrosstalkSources_996981_11.stderr parsl.dataflow.dflow DEBUG: Task 2561 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2561 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2561 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2561 has memoization hash 9e2bb78cefd7a3a9f4461169155e3ff9 parsl.dataflow.memoization INFO: Task 2561 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2561 try 0 launched on executor multi with executor id 2562 parsl.dataflow.dflow INFO: Standard output for task 2561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6180b11a-9b47-46d9-a18d-beca6b400a07_isrForCrosstalkSources_996974_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6180b11a-9b47-46d9-a18d-beca6b400a07_isrForCrosstalkSources_996974_39.stderr parsl.dataflow.dflow DEBUG: Task 2562 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2562 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2562 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2562 has memoization hash a35b9493466ef4368103693755b34c6f parsl.dataflow.memoization INFO: Task 2562 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2562 try 0 launched on executor multi with executor id 2563 parsl.dataflow.dflow INFO: Standard output for task 2562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/087ff17d-a266-40bf-a9ac-728311263af0_isrForCrosstalkSources_997022_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/087ff17d-a266-40bf-a9ac-728311263af0_isrForCrosstalkSources_997022_52.stderr parsl.dataflow.dflow DEBUG: Task 2563 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2563 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2563 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2563 has memoization hash c55cf614819cf5edee80085c635d17ea parsl.dataflow.memoization INFO: Task 2563 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2563 try 0 launched on executor multi with executor id 2564 parsl.dataflow.dflow INFO: Standard output for task 2563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d1521340-7aa7-4dfd-93ea-27da545e50d3_isrForCrosstalkSources_996954_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d1521340-7aa7-4dfd-93ea-27da545e50d3_isrForCrosstalkSources_996954_20.stderr parsl.dataflow.dflow DEBUG: Task 2564 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2564 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2564 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2564 has memoization hash 8a6650dfc180e232edeba8c0b6a4364c parsl.dataflow.memoization INFO: Task 2564 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2564 try 0 launched on executor multi with executor id 2565 parsl.dataflow.dflow INFO: Standard output for task 2564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/5fdf38de-fb87-40ca-924f-e2cd44eeba8a_isrForCrosstalkSources_996958_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/5fdf38de-fb87-40ca-924f-e2cd44eeba8a_isrForCrosstalkSources_996958_44.stderr parsl.dataflow.dflow DEBUG: Task 2565 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2565 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2565 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2565 has memoization hash 27491767cc1b98ae6c4a789a92fee361 parsl.dataflow.memoization INFO: Task 2565 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2565 try 0 launched on executor multi with executor id 2566 parsl.dataflow.dflow INFO: Standard output for task 2565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c595c899-3311-4727-93b1-197d8103878c_isrForCrosstalkSources_997007_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c595c899-3311-4727-93b1-197d8103878c_isrForCrosstalkSources_997007_52.stderr parsl.dataflow.dflow DEBUG: Task 2566 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2566 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2566 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2566 has memoization hash caa238fa946f2a919515e46237d37c9a parsl.dataflow.memoization INFO: Task 2566 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2566 try 0 launched on executor multi with executor id 2567 parsl.dataflow.dflow INFO: Standard output for task 2566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/fd5e8791-ed28-4a56-834f-c0a50373ee0a_isrForCrosstalkSources_996969_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/fd5e8791-ed28-4a56-834f-c0a50373ee0a_isrForCrosstalkSources_996969_4.stderr parsl.dataflow.dflow DEBUG: Task 2567 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2567 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2567 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2567 has memoization hash e3c43f74f19265673d661a65327fd25e parsl.dataflow.memoization INFO: Task 2567 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2567 try 0 launched on executor multi with executor id 2568 parsl.dataflow.dflow INFO: Standard output for task 2567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/27f805d0-4f12-4356-a945-798ef6e1a64c_isrForCrosstalkSources_996959_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/27f805d0-4f12-4356-a945-798ef6e1a64c_isrForCrosstalkSources_996959_38.stderr parsl.dataflow.dflow DEBUG: Task 2568 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2568 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2568 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2568 has memoization hash ffb9bee71e00014e27d11c66854ef1e2 parsl.dataflow.memoization INFO: Task 2568 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:50 proc_lsst.multi:170 [INFO] [multi] job_id 17074876 proc_lsst.multi INFO: [multi] job_id 17074876 2024-03-15 13:49:50 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 28 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 28 2024-03-15 13:49:50 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:50 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074876 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074876 parsl.executors.status_handling DEBUG: Launched block 28 on executor multi with job ID 17074876 parsl.executors.status_handling INFO: Allocated block ID 29 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-29 2024-03-15 13:49:50 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=29 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-29 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=29 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-29 2024-03-15 13:49:50 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=29 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=29 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:50 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=29 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=29 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 28 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 28 2024-03-15 13:49:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:50 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=29 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=29 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 2568 try 0 launched on executor multi with executor id 2569 parsl.dataflow.dflow INFO: Standard output for task 2568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a5ab9a3c-b4b2-40d3-b424-76cd44e4ba4b_isrForCrosstalkSources_997008_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a5ab9a3c-b4b2-40d3-b424-76cd44e4ba4b_isrForCrosstalkSources_997008_16.stderr parsl.dataflow.dflow DEBUG: Task 2569 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 2569 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2569 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2569 has memoization hash 1f4ee247348b743605a5ac4b57224826 parsl.dataflow.memoization INFO: Task 2569 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1af20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2569 try 0 launched on executor multi with executor id 2570 parsl.dataflow.dflow INFO: Standard output for task 2569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4ca84b95-0cd6-4d90-acff-065f0507d488_isrForCrosstalkSources_996966_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4ca84b95-0cd6-4d90-acff-065f0507d488_isrForCrosstalkSources_996966_23.stderr parsl.dataflow.dflow DEBUG: Task 2570 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2570 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2570 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2570 has memoization hash a485cf90f0c5dc89820f7f463d8721af parsl.dataflow.memoization INFO: Task 2570 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2570 try 0 launched on executor multi with executor id 2571 parsl.dataflow.dflow INFO: Standard output for task 2570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/39872200-09b6-4e2f-87c4-233009294f96_isrForCrosstalkSources_997016_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/39872200-09b6-4e2f-87c4-233009294f96_isrForCrosstalkSources_997016_2.stderr parsl.dataflow.dflow DEBUG: Task 2571 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2571 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2571 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2571 has memoization hash 3a33795a99a40ac2c064fcea1df4db70 parsl.dataflow.memoization INFO: Task 2571 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2571 try 0 launched on executor multi with executor id 2572 parsl.dataflow.dflow INFO: Standard output for task 2571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6e642f81-acb5-4cb0-b552-3b7e4fc65b82_isrForCrosstalkSources_996983_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6e642f81-acb5-4cb0-b552-3b7e4fc65b82_isrForCrosstalkSources_996983_35.stderr parsl.dataflow.dflow DEBUG: Task 2572 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2572 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2572 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2572 has memoization hash 25f2b10b416f6c829ea46ce27acd70b7 parsl.dataflow.memoization INFO: Task 2572 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e11c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2572 try 0 launched on executor multi with executor id 2573 parsl.dataflow.dflow INFO: Standard output for task 2572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/281259b6-c15d-46b7-bb19-e88a26758b6b_isrForCrosstalkSources_996983_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/281259b6-c15d-46b7-bb19-e88a26758b6b_isrForCrosstalkSources_996983_36.stderr parsl.dataflow.dflow DEBUG: Task 2573 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2573 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2573 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2573 has memoization hash 8acada2d4058c4a2142aa8dc4ecba13a parsl.dataflow.memoization INFO: Task 2573 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2573 try 0 launched on executor multi with executor id 2574 parsl.dataflow.dflow INFO: Standard output for task 2573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ac4d8e26-96f9-4c1c-82f9-2586505d6024_isrForCrosstalkSources_997002_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ac4d8e26-96f9-4c1c-82f9-2586505d6024_isrForCrosstalkSources_997002_43.stderr parsl.dataflow.dflow DEBUG: Task 2574 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2574 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2574 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2574 has memoization hash 248e5d3a74969eff2212b90317e405d4 parsl.dataflow.memoization INFO: Task 2574 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2574 try 0 launched on executor multi with executor id 2575 parsl.dataflow.dflow INFO: Standard output for task 2574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2483d421-9186-441f-b13b-cf696ae4202e_isrForCrosstalkSources_997007_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2483d421-9186-441f-b13b-cf696ae4202e_isrForCrosstalkSources_997007_44.stderr parsl.dataflow.dflow DEBUG: Task 2575 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2575 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2575 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2575 has memoization hash 0dbb0b34537f629ab2b165cdeaaaddb2 parsl.dataflow.memoization INFO: Task 2575 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2575 try 0 launched on executor multi with executor id 2576 parsl.dataflow.dflow INFO: Standard output for task 2575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/39d1fc9f-93aa-4602-b8e4-741fa167910e_isrForCrosstalkSources_997008_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/39d1fc9f-93aa-4602-b8e4-741fa167910e_isrForCrosstalkSources_997008_33.stderr parsl.dataflow.dflow DEBUG: Task 2576 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2576 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2576 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2576 has memoization hash d9edb174fc673c7301896495d3476e03 parsl.dataflow.memoization INFO: Task 2576 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2576 try 0 launched on executor multi with executor id 2577 parsl.dataflow.dflow INFO: Standard output for task 2576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/e4ab543e-d052-4430-9c47-48ce79e89f90_isrForCrosstalkSources_996971_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/e4ab543e-d052-4430-9c47-48ce79e89f90_isrForCrosstalkSources_996971_23.stderr parsl.dataflow.dflow DEBUG: Task 2577 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2577 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2577 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2577 has memoization hash 56b8fc92df7c0c3673bad71c23f2b91d parsl.dataflow.memoization INFO: Task 2577 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2577 try 0 launched on executor multi with executor id 2578 parsl.dataflow.dflow INFO: Standard output for task 2577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/071f24d3-e3e3-435e-b623-d19ceddb2658_isrForCrosstalkSources_997025_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/071f24d3-e3e3-435e-b623-d19ceddb2658_isrForCrosstalkSources_997025_5.stderr parsl.dataflow.dflow DEBUG: Task 2578 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2578 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2578 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2578 has memoization hash 17df761c0dfe8bf2ca0d99061f51642a parsl.dataflow.memoization INFO: Task 2578 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e0040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2578 try 0 launched on executor multi with executor id 2579 parsl.dataflow.dflow INFO: Standard output for task 2578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/9a09f7f5-e970-4a9c-b285-d11e90486c53_isrForCrosstalkSources_996970_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/9a09f7f5-e970-4a9c-b285-d11e90486c53_isrForCrosstalkSources_996970_11.stderr parsl.dataflow.dflow DEBUG: Task 2579 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2579 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2579 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2579 has memoization hash 93012d5ad734f5331a363ad458206cb6 parsl.dataflow.memoization INFO: Task 2579 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e39c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2579 try 0 launched on executor multi with executor id 2580 parsl.dataflow.dflow INFO: Standard output for task 2579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/bc4273d0-665e-4e08-9359-edc683cb50fd_isrForCrosstalkSources_996998_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/bc4273d0-665e-4e08-9359-edc683cb50fd_isrForCrosstalkSources_996998_37.stderr parsl.dataflow.dflow DEBUG: Task 2580 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2580 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2580 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2580 has memoization hash 29a4fd3fbab5faebd6534670fb48cb5a parsl.dataflow.memoization INFO: Task 2580 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2580 try 0 launched on executor multi with executor id 2581 parsl.dataflow.dflow INFO: Standard output for task 2580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/8afba572-5a69-417e-96ba-866893c0b569_isrForCrosstalkSources_996964_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/8afba572-5a69-417e-96ba-866893c0b569_isrForCrosstalkSources_996964_45.stderr parsl.dataflow.dflow DEBUG: Task 2581 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2581 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2581 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2581 has memoization hash fe3f11765bae80c49793df5e6ee1f70a parsl.dataflow.memoization INFO: Task 2581 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2581 try 0 launched on executor multi with executor id 2582 parsl.dataflow.dflow INFO: Standard output for task 2581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/eb4f0903-e5dc-4753-a2cd-fa64f31506b3_isrForCrosstalkSources_996978_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/eb4f0903-e5dc-4753-a2cd-fa64f31506b3_isrForCrosstalkSources_996978_2.stderr parsl.dataflow.dflow DEBUG: Task 2582 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2582 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2582 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2582 has memoization hash 3f11df8e6a9613c973d4f7628e12ccf6 parsl.dataflow.memoization INFO: Task 2582 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2582 try 0 launched on executor multi with executor id 2583 parsl.dataflow.dflow INFO: Standard output for task 2582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2fdc53db-d588-4460-af40-81d8dfe71e80_isrForCrosstalkSources_996989_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2fdc53db-d588-4460-af40-81d8dfe71e80_isrForCrosstalkSources_996989_51.stderr parsl.dataflow.dflow DEBUG: Task 2583 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2583 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2583 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2583 has memoization hash 17eed2a8dd1b90f6a72523e2c273f93b parsl.dataflow.memoization INFO: Task 2583 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e13a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2583 try 0 launched on executor multi with executor id 2584 parsl.dataflow.dflow INFO: Standard output for task 2583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7a39fd16-c28a-4088-b688-48844a38bd36_isrForCrosstalkSources_997006_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7a39fd16-c28a-4088-b688-48844a38bd36_isrForCrosstalkSources_997006_44.stderr parsl.dataflow.dflow DEBUG: Task 2584 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2584 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2584 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2584 has memoization hash fe18abbb5ec7ba5bba96fef581680dd9 parsl.dataflow.memoization INFO: Task 2584 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2584 try 0 launched on executor multi with executor id 2585 parsl.dataflow.dflow INFO: Standard output for task 2584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/36808f1e-818d-47bc-9013-4002573683b2_isrForCrosstalkSources_996976_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/36808f1e-818d-47bc-9013-4002573683b2_isrForCrosstalkSources_996976_10.stderr parsl.dataflow.dflow DEBUG: Task 2585 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2585 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2585 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2585 has memoization hash 8f5f7c95241802ea657f286e42eb9b28 parsl.dataflow.memoization INFO: Task 2585 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e14e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2585 try 0 launched on executor multi with executor id 2586 parsl.dataflow.dflow INFO: Standard output for task 2585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c32f6c55-d22f-4c91-abb1-c81bee90dce7_isrForCrosstalkSources_996991_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c32f6c55-d22f-4c91-abb1-c81bee90dce7_isrForCrosstalkSources_996991_42.stderr parsl.dataflow.dflow DEBUG: Task 2586 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2586 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2586 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2586 has memoization hash 16a5ac785396a7b56b19a78a06b91a1f parsl.dataflow.memoization INFO: Task 2586 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2586 try 0 launched on executor multi with executor id 2587 parsl.dataflow.dflow INFO: Standard output for task 2586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0278204a-2dd8-4068-b9d1-5c20fda73e61_isrForCrosstalkSources_996970_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0278204a-2dd8-4068-b9d1-5c20fda73e61_isrForCrosstalkSources_996970_10.stderr parsl.dataflow.dflow DEBUG: Task 2587 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2587 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2587 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2587 has memoization hash b4fe49efd5514a0dda96993361dad6cb parsl.dataflow.memoization INFO: Task 2587 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2587 try 0 launched on executor multi with executor id 2588 parsl.dataflow.dflow INFO: Standard output for task 2587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a80af2af-788f-4b4f-94c7-243a4372e717_isrForCrosstalkSources_996977_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a80af2af-788f-4b4f-94c7-243a4372e717_isrForCrosstalkSources_996977_11.stderr parsl.dataflow.dflow DEBUG: Task 2588 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2588 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2588 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2588 has memoization hash 632350d8bbd3bcdcfa045b48b0a00c05 parsl.dataflow.memoization INFO: Task 2588 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e16c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2588 try 0 launched on executor multi with executor id 2589 parsl.dataflow.dflow INFO: Standard output for task 2588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7445fe4f-e616-4127-90a1-14cbea6e3871_isrForCrosstalkSources_996973_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7445fe4f-e616-4127-90a1-14cbea6e3871_isrForCrosstalkSources_996973_46.stderr parsl.dataflow.dflow DEBUG: Task 2589 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2589 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2589 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2589 has memoization hash 440fa3bcf262879d5aff5fb04a82fd57 parsl.dataflow.memoization INFO: Task 2589 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2589 try 0 launched on executor multi with executor id 2590 parsl.dataflow.dflow INFO: Standard output for task 2589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/acd5be88-6143-47ab-907b-a65d5f788446_isrForCrosstalkSources_996988_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/acd5be88-6143-47ab-907b-a65d5f788446_isrForCrosstalkSources_996988_42.stderr parsl.dataflow.dflow DEBUG: Task 2590 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2590 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2590 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2590 has memoization hash c8d840a9da40449c5f68cf7d2263002b parsl.dataflow.memoization INFO: Task 2590 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2590 try 0 launched on executor multi with executor id 2591 parsl.dataflow.dflow INFO: Standard output for task 2590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/36d2aa9c-6d7d-44e3-85d2-11b2295c9aa6_isrForCrosstalkSources_996966_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/36d2aa9c-6d7d-44e3-85d2-11b2295c9aa6_isrForCrosstalkSources_996966_52.stderr parsl.dataflow.dflow DEBUG: Task 2591 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2591 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2591 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2591 has memoization hash 7e31c550a617881eb906c13fb18e9343 parsl.dataflow.memoization INFO: Task 2591 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2591 try 0 launched on executor multi with executor id 2592 parsl.dataflow.dflow INFO: Standard output for task 2591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/770ec2ca-0647-48a9-b9a5-b8881c727c68_isrForCrosstalkSources_996981_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/770ec2ca-0647-48a9-b9a5-b8881c727c68_isrForCrosstalkSources_996981_44.stderr parsl.dataflow.dflow DEBUG: Task 2592 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2592 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2592 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2592 has memoization hash eb4b489d658818e8df6f9fe2ceb567d7 parsl.dataflow.memoization INFO: Task 2592 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e18a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2592 try 0 launched on executor multi with executor id 2593 parsl.dataflow.dflow INFO: Standard output for task 2592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/83cb04a9-350a-4805-9495-54bc5bfe0187_isrForCrosstalkSources_997003_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/83cb04a9-350a-4805-9495-54bc5bfe0187_isrForCrosstalkSources_997003_19.stderr parsl.dataflow.dflow DEBUG: Task 2593 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2593 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2593 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2593 has memoization hash e3077f9948569c2772e6dcf831a0160e parsl.dataflow.memoization INFO: Task 2593 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2593 try 0 launched on executor multi with executor id 2594 parsl.dataflow.dflow INFO: Standard output for task 2593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b64c3941-b1f7-4c3f-9035-6dd648b1dfaa_isrForCrosstalkSources_997021_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b64c3941-b1f7-4c3f-9035-6dd648b1dfaa_isrForCrosstalkSources_997021_59.stderr parsl.dataflow.dflow DEBUG: Task 2594 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2594 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2594 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2594 has memoization hash 3d54892718bae715d1c4edd878447e10 parsl.dataflow.memoization INFO: Task 2594 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2594 try 0 launched on executor multi with executor id 2595 parsl.dataflow.dflow INFO: Standard output for task 2594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/3492f5ae-37eb-44bd-9b7f-32e994af9bdb_isrForCrosstalkSources_996973_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/3492f5ae-37eb-44bd-9b7f-32e994af9bdb_isrForCrosstalkSources_996973_34.stderr parsl.dataflow.dflow DEBUG: Task 2595 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2595 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2595 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2595 has memoization hash ee5ffb1e89b7edceff25368b3743720d parsl.dataflow.memoization INFO: Task 2595 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2595 try 0 launched on executor multi with executor id 2596 parsl.dataflow.dflow INFO: Standard output for task 2595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/265b42d9-367b-4fd3-9144-711257904332_isrForCrosstalkSources_997005_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/265b42d9-367b-4fd3-9144-711257904332_isrForCrosstalkSources_997005_55.stderr parsl.dataflow.dflow DEBUG: Task 2596 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2596 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2596 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2596 has memoization hash 4022f48289057d48a38b9811bbb7393a parsl.dataflow.memoization INFO: Task 2596 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2596 try 0 launched on executor multi with executor id 2597 parsl.dataflow.dflow INFO: Standard output for task 2596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/7a980967-2f99-41df-84db-d0c43fb332c3_isrForCrosstalkSources_996962_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/7a980967-2f99-41df-84db-d0c43fb332c3_isrForCrosstalkSources_996962_20.stderr parsl.dataflow.dflow DEBUG: Task 2597 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2597 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2597 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2597 has memoization hash fa4e30a0959d95f055287729b66f03e6 parsl.dataflow.memoization INFO: Task 2597 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e19e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2597 try 0 launched on executor multi with executor id 2598 parsl.dataflow.dflow INFO: Standard output for task 2597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7a58cd63-0897-42d2-9cc0-dfabedc02b2b_isrForCrosstalkSources_996984_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7a58cd63-0897-42d2-9cc0-dfabedc02b2b_isrForCrosstalkSources_996984_2.stderr parsl.dataflow.dflow DEBUG: Task 2598 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2598 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2598 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2598 has memoization hash 3fcb3c85a46fba0d67b1def8a35a7155 parsl.dataflow.memoization INFO: Task 2598 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2598 try 0 launched on executor multi with executor id 2599 parsl.dataflow.dflow INFO: Standard output for task 2598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/3916e32b-66fa-47df-81da-7893a58702d4_isrForCrosstalkSources_997027_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/3916e32b-66fa-47df-81da-7893a58702d4_isrForCrosstalkSources_997027_41.stderr parsl.dataflow.dflow DEBUG: Task 2599 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2599 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2599 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2599 has memoization hash a68339f2f6eaad8cab8e4f19e0d73c43 parsl.dataflow.memoization INFO: Task 2599 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2599 try 0 launched on executor multi with executor id 2600 parsl.dataflow.dflow INFO: Standard output for task 2599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/4eb0951c-daf1-4ac1-9dff-f65fe4986bc5_isrForCrosstalkSources_996977_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/4eb0951c-daf1-4ac1-9dff-f65fe4986bc5_isrForCrosstalkSources_996977_40.stderr parsl.dataflow.dflow DEBUG: Task 2600 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2600 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2600 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2600 has memoization hash cad85ef95f4b0fb40440a01405b8bd44 parsl.dataflow.memoization INFO: Task 2600 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2600 try 0 launched on executor multi with executor id 2601 parsl.dataflow.dflow INFO: Standard output for task 2600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/036606b3-a2a2-40d6-b183-15750e3e3b44_isrForCrosstalkSources_996953_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/036606b3-a2a2-40d6-b183-15750e3e3b44_isrForCrosstalkSources_996953_11.stderr parsl.dataflow.dflow DEBUG: Task 2601 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2601 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2601 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2601 has memoization hash 921d1152b417dba4d8865aba29aaa0a8 parsl.dataflow.memoization INFO: Task 2601 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2601 try 0 launched on executor multi with executor id 2602 parsl.dataflow.dflow INFO: Standard output for task 2601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/08a0bd20-d045-454e-a692-a2e5d67e1785_isrForCrosstalkSources_996994_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/08a0bd20-d045-454e-a692-a2e5d67e1785_isrForCrosstalkSources_996994_16.stderr parsl.dataflow.dflow DEBUG: Task 2602 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2602 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2602 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2602 has memoization hash 93ab47e5bfe0ccf5de174bbcfc74d92f parsl.dataflow.memoization INFO: Task 2602 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2602 try 0 launched on executor multi with executor id 2603 parsl.dataflow.dflow INFO: Standard output for task 2602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/dd714b4c-38fb-4c6b-98c3-94b0644a5e6a_isrForCrosstalkSources_997024_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/dd714b4c-38fb-4c6b-98c3-94b0644a5e6a_isrForCrosstalkSources_997024_61.stderr parsl.dataflow.dflow DEBUG: Task 2603 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2603 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2603 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2603 has memoization hash 56aea219653b0999928b73525b5a8773 parsl.dataflow.memoization INFO: Task 2603 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2603 try 0 launched on executor multi with executor id 2604 parsl.dataflow.dflow INFO: Standard output for task 2603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/21bbfa9b-3fef-4be1-9a2a-bef3445f62e9_isrForCrosstalkSources_996978_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/21bbfa9b-3fef-4be1-9a2a-bef3445f62e9_isrForCrosstalkSources_996978_16.stderr parsl.dataflow.dflow DEBUG: Task 2604 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2604 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2604 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2604 has memoization hash fe616df3f411c7a749d758042bc18ac0 parsl.dataflow.memoization INFO: Task 2604 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e1f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 7 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46ed9a2e159e063f3f5254380ee15ad0 with result from task 7 parsl.dataflow.dflow INFO: Parsl task 2604 try 0 launched on executor multi with executor id 2605 parsl.dataflow.dflow INFO: Standard output for task 2604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/6969d4a8-0d67-40ae-a123-46b20f323c78_isrForCrosstalkSources_996972_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/6969d4a8-0d67-40ae-a123-46b20f323c78_isrForCrosstalkSources_996972_13.stderr parsl.dataflow.dflow DEBUG: Task 2605 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2605 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2605 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2605 has memoization hash d6fe9d10617fd76cabd40b52314186b4 parsl.dataflow.memoization INFO: Task 2605 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2605 try 0 launched on executor multi with executor id 2606 parsl.dataflow.dflow INFO: Standard output for task 2605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/557f4f17-703b-4c33-a404-a3681524e05d_isrForCrosstalkSources_996958_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/557f4f17-703b-4c33-a404-a3681524e05d_isrForCrosstalkSources_996958_41.stderr parsl.dataflow.dflow DEBUG: Task 2606 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2606 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2606 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2606 has memoization hash e47f6fb9f62d6c646496af21f6a1bbd0 parsl.dataflow.memoization INFO: Task 2606 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e20c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2606 try 0 launched on executor multi with executor id 2607 parsl.dataflow.dflow INFO: Standard output for task 2606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/35e2ac81-78d7-4373-9a0d-71ad0505a66f_isrForCrosstalkSources_996995_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/35e2ac81-78d7-4373-9a0d-71ad0505a66f_isrForCrosstalkSources_996995_26.stderr parsl.dataflow.dflow DEBUG: Task 2607 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2607 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2607 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2607 has memoization hash 9ce6c129667384003e1c5d5e1f241bdc parsl.dataflow.memoization INFO: Task 2607 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 7 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 7 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5d164c49-eb09-42fa-9ca6-d58f5228685b_isrForCrosstalkSources_997004_54.stdout parsl.dataflow.dflow INFO: Standard error for task 7 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5d164c49-eb09-42fa-9ca6-d58f5228685b_isrForCrosstalkSources_997004_54.stderr parsl.dataflow.dflow INFO: Parsl task 2607 try 0 launched on executor multi with executor id 2608 parsl.dataflow.dflow INFO: Standard output for task 2607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cbe6dc4f-05da-4110-be4c-9278ae59b966_isrForCrosstalkSources_996956_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cbe6dc4f-05da-4110-be4c-9278ae59b966_isrForCrosstalkSources_996956_57.stderr parsl.dataflow.dflow DEBUG: Task 2608 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2608 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2608 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2608 has memoization hash 72356c7d8d61ffd7470978a7ca7d63cf parsl.dataflow.memoization INFO: Task 2608 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2608 try 0 launched on executor multi with executor id 2609 parsl.dataflow.dflow INFO: Standard output for task 2608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7b19e421-56f9-40f6-843e-28e6347a3a56_isrForCrosstalkSources_996991_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7b19e421-56f9-40f6-843e-28e6347a3a56_isrForCrosstalkSources_996991_45.stderr parsl.dataflow.dflow DEBUG: Task 2609 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2609 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2609 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2609 has memoization hash 85b2b11749af0d87df5e08543322ea2a parsl.dataflow.memoization INFO: Task 2609 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e22a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2609 try 0 launched on executor multi with executor id 2610 parsl.dataflow.dflow INFO: Standard output for task 2609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/68c3614e-9d2a-4129-8f3e-bd86e81638d2_isrForCrosstalkSources_997014_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/68c3614e-9d2a-4129-8f3e-bd86e81638d2_isrForCrosstalkSources_997014_12.stderr parsl.dataflow.dflow DEBUG: Task 2610 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2610 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2610 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2610 has memoization hash ac7de8bb475d34519a65ce5c8cdeb9b3 parsl.dataflow.memoization INFO: Task 2610 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2610 try 0 launched on executor multi with executor id 2611 parsl.dataflow.dflow INFO: Standard output for task 2610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/777fa400-579e-4f6e-aa18-94b2a4e046ad_isrForCrosstalkSources_996957_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/777fa400-579e-4f6e-aa18-94b2a4e046ad_isrForCrosstalkSources_996957_23.stderr parsl.dataflow.dflow DEBUG: Task 2611 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2611 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2611 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2611 has memoization hash aed49d9f9853ac6b1839a6d40575fd14 parsl.dataflow.memoization INFO: Task 2611 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e23e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2611 try 0 launched on executor multi with executor id 2612 parsl.dataflow.dflow INFO: Standard output for task 2611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e97d17bb-34ec-4905-8c6e-1cec0f7af9dc_isrForCrosstalkSources_997020_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e97d17bb-34ec-4905-8c6e-1cec0f7af9dc_isrForCrosstalkSources_997020_46.stderr parsl.dataflow.dflow DEBUG: Task 2612 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2612 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2612 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2612 has memoization hash 956b93814802fdb7b3ce0eaec758d77c parsl.dataflow.memoization INFO: Task 2612 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2612 try 0 launched on executor multi with executor id 2613 parsl.dataflow.dflow INFO: Standard output for task 2612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7a53f2f4-e351-4b3d-908e-df81eb086838_isrForCrosstalkSources_997013_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7a53f2f4-e351-4b3d-908e-df81eb086838_isrForCrosstalkSources_997013_38.stderr parsl.dataflow.dflow DEBUG: Task 2613 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2613 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2613 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2613 has memoization hash ef0ec4073c9cbe374f4ec4c7e0d6c748 parsl.dataflow.memoization INFO: Task 2613 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2613 try 0 launched on executor multi with executor id 2614 parsl.dataflow.dflow INFO: Standard output for task 2613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/fe8a36b5-08ab-4e34-9d22-cb734a875064_isrForCrosstalkSources_996993_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/fe8a36b5-08ab-4e34-9d22-cb734a875064_isrForCrosstalkSources_996993_42.stderr parsl.dataflow.dflow DEBUG: Task 2614 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2614 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2614 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2614 has memoization hash 58570e4f8de30f521fafc50fc18887db parsl.dataflow.memoization INFO: Task 2614 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e25c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2614 try 0 launched on executor multi with executor id 2615 parsl.dataflow.dflow INFO: Standard output for task 2614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/645014c2-9766-4167-933c-f7308a25b6f4_isrForCrosstalkSources_996967_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/645014c2-9766-4167-933c-f7308a25b6f4_isrForCrosstalkSources_996967_44.stderr parsl.dataflow.dflow DEBUG: Task 2615 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2615 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2615 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2615 has memoization hash 036c4fc21f3470ffc54da106a2cdfba7 parsl.dataflow.memoization INFO: Task 2615 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2615 try 0 launched on executor multi with executor id 2616 parsl.dataflow.dflow INFO: Standard output for task 2615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dde14437-5b34-4979-af4e-a1fdfa93f649_isrForCrosstalkSources_997001_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dde14437-5b34-4979-af4e-a1fdfa93f649_isrForCrosstalkSources_997001_19.stderr parsl.dataflow.dflow DEBUG: Task 2616 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2616 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2616 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2616 has memoization hash 0972565de7a6ccf1419b9ed80c688b81 parsl.dataflow.memoization INFO: Task 2616 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e27a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2616 try 0 launched on executor multi with executor id 2617 parsl.dataflow.dflow INFO: Standard output for task 2616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d90414c4-8499-4221-90c6-09ad9e85b10e_isrForCrosstalkSources_996964_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d90414c4-8499-4221-90c6-09ad9e85b10e_isrForCrosstalkSources_996964_1.stderr parsl.dataflow.dflow DEBUG: Task 2617 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2617 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2617 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2617 has memoization hash 7160393b534e0d00c057ee98c6ac544e parsl.dataflow.memoization INFO: Task 2617 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2617 try 0 launched on executor multi with executor id 2618 parsl.dataflow.dflow INFO: Standard output for task 2617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/7b3c40c0-e8c0-4307-8633-fb4edfbc29f9_isrForCrosstalkSources_996952_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/7b3c40c0-e8c0-4307-8633-fb4edfbc29f9_isrForCrosstalkSources_996952_28.stderr parsl.dataflow.dflow DEBUG: Task 2618 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2618 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2618 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2618 has memoization hash 9b557fc8434f2adb1a0cc1556c3c0314 parsl.dataflow.memoization INFO: Task 2618 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2618 try 0 launched on executor multi with executor id 2619 parsl.dataflow.dflow INFO: Standard output for task 2618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/71ec74cf-45d9-436b-a681-cbf6b314a6e0_isrForCrosstalkSources_997030_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/71ec74cf-45d9-436b-a681-cbf6b314a6e0_isrForCrosstalkSources_997030_11.stderr parsl.dataflow.dflow DEBUG: Task 2619 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2619 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2619 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2619 has memoization hash 3acdfbcea847a749888ccfc8f1899227 parsl.dataflow.memoization INFO: Task 2619 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2619 try 0 launched on executor multi with executor id 2620 parsl.dataflow.dflow INFO: Standard output for task 2619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/efeb379a-d01a-4841-86fd-46ad34c38012_isrForCrosstalkSources_996965_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/efeb379a-d01a-4841-86fd-46ad34c38012_isrForCrosstalkSources_996965_58.stderr parsl.dataflow.dflow DEBUG: Task 2620 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2620 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2620 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2620 has memoization hash a6784977b84ce6649e50599e3e2b2e0b parsl.dataflow.memoization INFO: Task 2620 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e28e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2620 try 0 launched on executor multi with executor id 2621 parsl.dataflow.dflow INFO: Standard output for task 2620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f53a0e22-2f3a-4bc8-bd18-827255e463e0_isrForCrosstalkSources_996980_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f53a0e22-2f3a-4bc8-bd18-827255e463e0_isrForCrosstalkSources_996980_20.stderr parsl.dataflow.dflow DEBUG: Task 2621 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2621 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2621 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2621 has memoization hash 30f3a62c87c7b9f3064b72e03e2ddc66 parsl.dataflow.memoization INFO: Task 2621 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2621 try 0 launched on executor multi with executor id 2622 parsl.dataflow.dflow INFO: Standard output for task 2621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/22504e2b-9ac3-44ee-af4b-1f6b80649aea_isrForCrosstalkSources_996993_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/22504e2b-9ac3-44ee-af4b-1f6b80649aea_isrForCrosstalkSources_996993_41.stderr parsl.dataflow.dflow DEBUG: Task 2622 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2622 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2622 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2622 has memoization hash 8d3da08888e6b2042b0f918948e02886 parsl.dataflow.memoization INFO: Task 2622 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2622 try 0 launched on executor multi with executor id 2623 parsl.dataflow.dflow INFO: Standard output for task 2622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/2d671726-7e83-45e0-bc24-511e573a135f_isrForCrosstalkSources_996994_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/2d671726-7e83-45e0-bc24-511e573a135f_isrForCrosstalkSources_996994_48.stderr parsl.dataflow.dflow DEBUG: Task 2623 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2623 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2623 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2623 has memoization hash f2dbf207cfac3c3ee40586ddadd72236 parsl.dataflow.memoization INFO: Task 2623 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2623 try 0 launched on executor multi with executor id 2624 parsl.dataflow.dflow INFO: Standard output for task 2623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/9c413603-45f9-4a17-82b0-fd42bdf7f18f_isrForCrosstalkSources_996955_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/9c413603-45f9-4a17-82b0-fd42bdf7f18f_isrForCrosstalkSources_996955_62.stderr parsl.dataflow.dflow DEBUG: Task 2624 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2624 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2624 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2624 has memoization hash 51e15a2269e5778231795f03cbb0b936 parsl.dataflow.memoization INFO: Task 2624 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2624 try 0 launched on executor multi with executor id 2625 parsl.dataflow.dflow INFO: Standard output for task 2624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f79804a0-b1e6-46b1-a14c-1fffcd630cbe_isrForCrosstalkSources_996973_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f79804a0-b1e6-46b1-a14c-1fffcd630cbe_isrForCrosstalkSources_996973_11.stderr parsl.dataflow.dflow DEBUG: Task 2625 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2625 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2625 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2625 has memoization hash 1866f787dc64aa45cf11e63a7985fe1e parsl.dataflow.memoization INFO: Task 2625 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2625 try 0 launched on executor multi with executor id 2626 parsl.dataflow.dflow INFO: Standard output for task 2625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e19ecad4-2960-4595-93c0-7365be2795a5_isrForCrosstalkSources_996995_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e19ecad4-2960-4595-93c0-7365be2795a5_isrForCrosstalkSources_996995_4.stderr parsl.dataflow.dflow DEBUG: Task 2626 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2626 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2626 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2626 has memoization hash 1f508ade7686e353327a83b99fa77048 parsl.dataflow.memoization INFO: Task 2626 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2626 try 0 launched on executor multi with executor id 2627 parsl.dataflow.dflow INFO: Standard output for task 2626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/8ab45938-6b41-4a6f-a61e-23080ef96ef4_isrForCrosstalkSources_996965_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/8ab45938-6b41-4a6f-a61e-23080ef96ef4_isrForCrosstalkSources_996965_9.stderr parsl.dataflow.dflow DEBUG: Task 2627 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2627 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2627 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2627 has memoization hash cf2f7463a019afa3581bddfb9f42aa2e parsl.dataflow.memoization INFO: Task 2627 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2627 try 0 launched on executor multi with executor id 2628 parsl.dataflow.dflow INFO: Standard output for task 2627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/992ef550-3514-42f0-8a8d-0e991a0c2ae9_isrForCrosstalkSources_996963_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/992ef550-3514-42f0-8a8d-0e991a0c2ae9_isrForCrosstalkSources_996963_37.stderr parsl.dataflow.dflow DEBUG: Task 2628 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2628 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2628 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2628 has memoization hash a4cbe33823c24579e24aac254f9b18d5 parsl.dataflow.memoization INFO: Task 2628 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2628 try 0 launched on executor multi with executor id 2629 parsl.dataflow.dflow INFO: Standard output for task 2628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/977acd62-33c4-4826-a154-0a1cdf7e013d_isrForCrosstalkSources_996974_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/977acd62-33c4-4826-a154-0a1cdf7e013d_isrForCrosstalkSources_996974_4.stderr parsl.dataflow.dflow DEBUG: Task 2629 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2629 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2629 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2629 has memoization hash 288ca0b0e384902f59ad6e177c3852e7 parsl.dataflow.memoization INFO: Task 2629 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2629 try 0 launched on executor multi with executor id 2630 parsl.dataflow.dflow INFO: Standard output for task 2629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/1e65e63e-92ea-4f2b-b9b8-d6c388f24934_isrForCrosstalkSources_997007_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/1e65e63e-92ea-4f2b-b9b8-d6c388f24934_isrForCrosstalkSources_997007_23.stderr parsl.dataflow.dflow DEBUG: Task 2630 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2630 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2630 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2630 has memoization hash a490f01c6503feecaf9738f2945d044c parsl.dataflow.memoization INFO: Task 2630 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2630 try 0 launched on executor multi with executor id 2631 parsl.dataflow.dflow INFO: Standard output for task 2630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/64d44d0e-f5b1-4d7a-b10f-d30c639456ac_isrForCrosstalkSources_996984_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/64d44d0e-f5b1-4d7a-b10f-d30c639456ac_isrForCrosstalkSources_996984_28.stderr parsl.dataflow.dflow DEBUG: Task 2631 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2631 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2631 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2631 has memoization hash 53b7512a2595fcea9d0a92990268f2ee parsl.dataflow.memoization INFO: Task 2631 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2631 try 0 launched on executor multi with executor id 2632 parsl.dataflow.dflow INFO: Standard output for task 2631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/53027b6f-3063-4644-b8a5-7d0ffb5d906f_isrForCrosstalkSources_996986_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/53027b6f-3063-4644-b8a5-7d0ffb5d906f_isrForCrosstalkSources_996986_7.stderr parsl.dataflow.dflow DEBUG: Task 2632 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2632 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2632 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2632 has memoization hash 7f92495a8f05fbfc45c17a9a09234484 parsl.dataflow.memoization INFO: Task 2632 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e37e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2632 try 0 launched on executor multi with executor id 2633 parsl.dataflow.dflow INFO: Standard output for task 2632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5d3046f5-85a0-4a27-9d01-e237ff057162_isrForCrosstalkSources_996988_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5d3046f5-85a0-4a27-9d01-e237ff057162_isrForCrosstalkSources_996988_40.stderr parsl.dataflow.dflow DEBUG: Task 2633 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2633 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2633 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2633 has memoization hash 4a5d0464672e0eb2afbbf50c6b73145b parsl.dataflow.memoization INFO: Task 2633 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2633 try 0 launched on executor multi with executor id 2634 parsl.dataflow.dflow INFO: Standard output for task 2633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/538b2ae7-af13-41ce-949f-a07052f9342d_isrForCrosstalkSources_997027_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/538b2ae7-af13-41ce-949f-a07052f9342d_isrForCrosstalkSources_997027_1.stderr parsl.dataflow.dflow DEBUG: Task 2634 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2634 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2634 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2634 has memoization hash be34a0bb70836bb387408caad8d47c28 parsl.dataflow.memoization INFO: Task 2634 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e36a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2634 try 0 launched on executor multi with executor id 2635 parsl.dataflow.dflow INFO: Standard output for task 2634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/e7d864c1-8140-44c3-8cfc-9c4855421c6f_isrForCrosstalkSources_996969_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/e7d864c1-8140-44c3-8cfc-9c4855421c6f_isrForCrosstalkSources_996969_61.stderr parsl.dataflow.dflow DEBUG: Task 2635 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2635 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2635 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2635 has memoization hash 550b5d5be8acd87588b32970fad29155 parsl.dataflow.memoization INFO: Task 2635 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2635 try 0 launched on executor multi with executor id 2636 parsl.dataflow.dflow INFO: Standard output for task 2635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/a38df11e-d576-4f8a-85df-ffebcb59c64d_isrForCrosstalkSources_997017_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/a38df11e-d576-4f8a-85df-ffebcb59c64d_isrForCrosstalkSources_997017_28.stderr parsl.dataflow.dflow DEBUG: Task 2636 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2636 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2636 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2636 has memoization hash 6d2ee2aabe32bd57a04532375dec2354 parsl.dataflow.memoization INFO: Task 2636 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2636 try 0 launched on executor multi with executor id 2637 parsl.dataflow.dflow INFO: Standard output for task 2636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/f6a2f39d-5ee2-461c-bf99-8c20527891b5_isrForCrosstalkSources_997006_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/f6a2f39d-5ee2-461c-bf99-8c20527891b5_isrForCrosstalkSources_997006_21.stderr parsl.dataflow.dflow DEBUG: Task 2637 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2637 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2637 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2637 has memoization hash 79b3e3069253be8f4be1fcf68b59d703 parsl.dataflow.memoization INFO: Task 2637 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e34c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2637 try 0 launched on executor multi with executor id 2638 parsl.dataflow.dflow INFO: Standard output for task 2637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/770052b8-b730-46c5-850b-95466bc50c9b_isrForCrosstalkSources_997031_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/770052b8-b730-46c5-850b-95466bc50c9b_isrForCrosstalkSources_997031_41.stderr parsl.dataflow.dflow DEBUG: Task 2638 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2638 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2638 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2638 has memoization hash 5df2fd4f95c17a9cee2062b27b49bd14 parsl.dataflow.memoization INFO: Task 2638 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2638 try 0 launched on executor multi with executor id 2639 parsl.dataflow.dflow INFO: Standard output for task 2638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/db90e1b1-a7fb-4bbb-8cf6-1c4f45d4e4d2_isrForCrosstalkSources_997014_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/db90e1b1-a7fb-4bbb-8cf6-1c4f45d4e4d2_isrForCrosstalkSources_997014_19.stderr parsl.dataflow.dflow DEBUG: Task 2639 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2639 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2639 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2639 has memoization hash 25ca26ad255e75ef283eb4a230b2371e parsl.dataflow.memoization INFO: Task 2639 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2639 try 0 launched on executor multi with executor id 2640 parsl.dataflow.dflow INFO: Standard output for task 2639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d62cabdd-6cb6-4bf5-9fc5-c70d7167e314_isrForCrosstalkSources_997025_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d62cabdd-6cb6-4bf5-9fc5-c70d7167e314_isrForCrosstalkSources_997025_6.stderr parsl.dataflow.dflow DEBUG: Task 2640 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2640 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2640 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2640 has memoization hash 08ea03e09fcfb932a9ebf0f54ead0332 parsl.dataflow.memoization INFO: Task 2640 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2640 try 0 launched on executor multi with executor id 2641 parsl.dataflow.dflow INFO: Standard output for task 2640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/528a6a40-f5db-4743-a091-701bd41a88c6_isrForCrosstalkSources_997028_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/528a6a40-f5db-4743-a091-701bd41a88c6_isrForCrosstalkSources_997028_14.stderr parsl.dataflow.dflow DEBUG: Task 2641 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2641 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2641 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2641 has memoization hash e5067b321eab9cd2019b3cb612aefcb5 parsl.dataflow.memoization INFO: Task 2641 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e32e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2641 try 0 launched on executor multi with executor id 2642 parsl.dataflow.dflow INFO: Standard output for task 2641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/fe37300a-59a9-4758-bd41-a83469830801_isrForCrosstalkSources_997000_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/fe37300a-59a9-4758-bd41-a83469830801_isrForCrosstalkSources_997000_20.stderr parsl.dataflow.dflow DEBUG: Task 2642 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2642 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2642 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2642 has memoization hash 6d2481618f47b996c864c9d3a5eca92a parsl.dataflow.memoization INFO: Task 2642 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e31a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2642 try 0 launched on executor multi with executor id 2643 parsl.dataflow.dflow INFO: Standard output for task 2642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/df67196a-0f1c-4d6f-81d6-6f96a0f3d754_isrForCrosstalkSources_996986_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/df67196a-0f1c-4d6f-81d6-6f96a0f3d754_isrForCrosstalkSources_996986_37.stderr parsl.dataflow.dflow DEBUG: Task 2643 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2643 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2643 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2643 has memoization hash 82a280d4c4c8721ff4ff67abc73b9b04 parsl.dataflow.memoization INFO: Task 2643 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2643 try 0 launched on executor multi with executor id 2644 parsl.dataflow.dflow INFO: Standard output for task 2643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/5bd36215-6e00-4c3a-add0-b8ca8e61611f_isrForCrosstalkSources_997027_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/5bd36215-6e00-4c3a-add0-b8ca8e61611f_isrForCrosstalkSources_997027_9.stderr parsl.dataflow.dflow DEBUG: Task 2644 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2644 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2644 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2644 has memoization hash 208b025887b508dfa38e8df4ba574ddf parsl.dataflow.memoization INFO: Task 2644 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2644 try 0 launched on executor multi with executor id 2645 parsl.dataflow.dflow INFO: Standard output for task 2644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/853073c9-70de-4979-8775-ac96af0ff720_isrForCrosstalkSources_997024_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/853073c9-70de-4979-8775-ac96af0ff720_isrForCrosstalkSources_997024_15.stderr parsl.dataflow.dflow DEBUG: Task 2645 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2645 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2645 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2645 has memoization hash 9e68d10da1ac735a7d211031ab78205d parsl.dataflow.memoization INFO: Task 2645 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e2f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2645 try 0 launched on executor multi with executor id 2646 parsl.dataflow.dflow INFO: Standard output for task 2645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e99e4df5-bdf1-43cd-ad6c-1e41b26a2466_isrForCrosstalkSources_997031_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e99e4df5-bdf1-43cd-ad6c-1e41b26a2466_isrForCrosstalkSources_997031_18.stderr parsl.dataflow.dflow DEBUG: Task 2646 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2646 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2646 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2646 has memoization hash d5be44c45e9d753e2d9791eeb88ed625 parsl.dataflow.memoization INFO: Task 2646 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2646 try 0 launched on executor multi with executor id 2647 parsl.dataflow.dflow INFO: Standard output for task 2646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2ec96e00-1d73-41b7-be7d-2de9955a3547_isrForCrosstalkSources_996963_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2ec96e00-1d73-41b7-be7d-2de9955a3547_isrForCrosstalkSources_996963_3.stderr parsl.dataflow.dflow DEBUG: Task 2647 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2647 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2647 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2647 has memoization hash f3319a5afd276eefc8226d0ebc35d032 parsl.dataflow.memoization INFO: Task 2647 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2647 try 0 launched on executor multi with executor id 2648 parsl.dataflow.dflow INFO: Standard output for task 2647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/62a43de6-ff9a-4d40-8562-7e1d554158f9_isrForCrosstalkSources_996970_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/62a43de6-ff9a-4d40-8562-7e1d554158f9_isrForCrosstalkSources_996970_50.stderr parsl.dataflow.dflow DEBUG: Task 2648 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2648 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2648 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2648 has memoization hash 69eb14ef7fd749578c1f0397990c9cc5 parsl.dataflow.memoization INFO: Task 2648 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2648 try 0 launched on executor multi with executor id 2649 parsl.dataflow.dflow INFO: Standard output for task 2648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/166d257d-3a87-4515-8e09-5b03c5ce1a64_isrForCrosstalkSources_996974_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/166d257d-3a87-4515-8e09-5b03c5ce1a64_isrForCrosstalkSources_996974_19.stderr parsl.dataflow.dflow DEBUG: Task 2649 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2649 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2649 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2649 has memoization hash 42de1ab06c5cf001d81ca3a024013a82 parsl.dataflow.memoization INFO: Task 2649 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2649 try 0 launched on executor multi with executor id 2650 parsl.dataflow.dflow INFO: Standard output for task 2649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/cac9fac6-1829-45e9-b799-381975e69395_isrForCrosstalkSources_997010_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/cac9fac6-1829-45e9-b799-381975e69395_isrForCrosstalkSources_997010_58.stderr parsl.dataflow.dflow DEBUG: Task 2650 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2650 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2650 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2650 has memoization hash c57ceb62eea32c6450f21ede94220a0f parsl.dataflow.memoization INFO: Task 2650 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2650 try 0 launched on executor multi with executor id 2651 parsl.dataflow.dflow INFO: Standard output for task 2650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/67a6ea98-d648-4be9-b9fd-5277fb6463af_isrForCrosstalkSources_997031_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/67a6ea98-d648-4be9-b9fd-5277fb6463af_isrForCrosstalkSources_997031_38.stderr parsl.dataflow.dflow DEBUG: Task 2651 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2651 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2651 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2651 has memoization hash 047af9b4a11a8bcd377937ec610c64e1 parsl.dataflow.memoization INFO: Task 2651 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2651 try 0 launched on executor multi with executor id 2652 parsl.dataflow.dflow INFO: Standard output for task 2651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f76d4f31-0a2b-4f29-99c8-6d5128dd4990_isrForCrosstalkSources_997032_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f76d4f31-0a2b-4f29-99c8-6d5128dd4990_isrForCrosstalkSources_997032_56.stderr parsl.dataflow.dflow DEBUG: Task 2652 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2652 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2652 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2652 has memoization hash d56fd0b17a09f58ad3d18b9612dc7020 parsl.dataflow.memoization INFO: Task 2652 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2652 try 0 launched on executor multi with executor id 2653 parsl.dataflow.dflow INFO: Standard output for task 2652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/52c7c4c7-03f5-4a8b-ad82-838f9741eeaa_isrForCrosstalkSources_996961_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/52c7c4c7-03f5-4a8b-ad82-838f9741eeaa_isrForCrosstalkSources_996961_21.stderr parsl.dataflow.dflow DEBUG: Task 2653 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2653 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2653 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2653 has memoization hash 1932a311ebe0e57f1b60c5f2cb81ceff parsl.dataflow.memoization INFO: Task 2653 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2653 try 0 launched on executor multi with executor id 2654 parsl.dataflow.dflow INFO: Standard output for task 2653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/40265502-ed2e-407d-9e24-d8867da1bf25_isrForCrosstalkSources_996963_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/40265502-ed2e-407d-9e24-d8867da1bf25_isrForCrosstalkSources_996963_36.stderr parsl.dataflow.dflow DEBUG: Task 2654 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2654 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2654 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2654 has memoization hash 425dedc3e42dd69d21ee88c6b2ef171e parsl.dataflow.memoization INFO: Task 2654 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2654 try 0 launched on executor multi with executor id 2655 parsl.dataflow.dflow INFO: Standard output for task 2654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b176d987-c76e-4bad-bcc7-20cc6a226588_isrForCrosstalkSources_997004_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b176d987-c76e-4bad-bcc7-20cc6a226588_isrForCrosstalkSources_997004_16.stderr parsl.dataflow.dflow DEBUG: Task 2655 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2655 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2655 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2655 has memoization hash 074c40c51c92e6848d0931816ce7af97 parsl.dataflow.memoization INFO: Task 2655 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2655 try 0 launched on executor multi with executor id 2656 parsl.dataflow.dflow INFO: Standard output for task 2655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/60811b70-0a58-46f6-a277-d3c37f1f8b56_isrForCrosstalkSources_996992_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/60811b70-0a58-46f6-a277-d3c37f1f8b56_isrForCrosstalkSources_996992_54.stderr parsl.dataflow.dflow DEBUG: Task 2656 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2656 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2656 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2656 has memoization hash 5cb3f1e41343298515f50856d37e8aed parsl.dataflow.memoization INFO: Task 2656 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2656 try 0 launched on executor multi with executor id 2657 parsl.dataflow.dflow INFO: Standard output for task 2656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3d99429d-249b-40ce-aae5-094faa1cbca8_isrForCrosstalkSources_997014_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3d99429d-249b-40ce-aae5-094faa1cbca8_isrForCrosstalkSources_997014_6.stderr parsl.dataflow.dflow DEBUG: Task 2657 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2657 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2657 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2657 has memoization hash d4fa7cb47bf6bffda8a517eaa41be0af parsl.dataflow.memoization INFO: Task 2657 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2657 try 0 launched on executor multi with executor id 2658 parsl.dataflow.dflow INFO: Standard output for task 2657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f4e241e2-6e12-4283-bf75-56d2c133aed3_isrForCrosstalkSources_996987_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f4e241e2-6e12-4283-bf75-56d2c133aed3_isrForCrosstalkSources_996987_17.stderr parsl.dataflow.dflow DEBUG: Task 2658 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2658 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2658 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2658 has memoization hash 984a43e380a6fec91669db88b3762259 parsl.dataflow.memoization INFO: Task 2658 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2658 try 0 launched on executor multi with executor id 2659 parsl.dataflow.dflow INFO: Standard output for task 2658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/cffa26f2-d8a2-48f2-9149-4f898a057b6f_isrForCrosstalkSources_996990_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/cffa26f2-d8a2-48f2-9149-4f898a057b6f_isrForCrosstalkSources_996990_59.stderr parsl.dataflow.dflow DEBUG: Task 2659 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2659 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2659 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2659 has memoization hash 23844cc22d7ee518babe15e9175f9bcd parsl.dataflow.memoization INFO: Task 2659 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2659 try 0 launched on executor multi with executor id 2660 parsl.dataflow.dflow INFO: Standard output for task 2659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e207fd2a-2b03-43b8-8b33-df22a72a253c_isrForCrosstalkSources_996989_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e207fd2a-2b03-43b8-8b33-df22a72a253c_isrForCrosstalkSources_996989_13.stderr parsl.dataflow.dflow DEBUG: Task 2660 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2660 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2660 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2660 has memoization hash ceec27d6e2cfc75f128594a420fdd0cf parsl.dataflow.memoization INFO: Task 2660 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2660 try 0 launched on executor multi with executor id 2661 parsl.dataflow.dflow INFO: Standard output for task 2660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/eaad31b2-94a5-4b1c-bc30-164763efea1d_isrForCrosstalkSources_996953_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/eaad31b2-94a5-4b1c-bc30-164763efea1d_isrForCrosstalkSources_996953_15.stderr parsl.dataflow.dflow DEBUG: Task 2661 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2661 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2661 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2661 has memoization hash 95a1bdfc4cf0a29aec368409c77902c9 parsl.dataflow.memoization INFO: Task 2661 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2661 try 0 launched on executor multi with executor id 2662 parsl.dataflow.dflow INFO: Standard output for task 2661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4b429a0d-d051-4873-a8e6-d530c490ff94_isrForCrosstalkSources_997030_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4b429a0d-d051-4873-a8e6-d530c490ff94_isrForCrosstalkSources_997030_7.stderr parsl.dataflow.dflow DEBUG: Task 2662 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2662 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2662 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2662 has memoization hash f845264e025b13a9e01f42966f8ca8a3 parsl.dataflow.memoization INFO: Task 2662 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2662 try 0 launched on executor multi with executor id 2663 parsl.dataflow.dflow INFO: Standard output for task 2662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4f376097-7f40-44a7-b544-122def519df7_isrForCrosstalkSources_996997_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4f376097-7f40-44a7-b544-122def519df7_isrForCrosstalkSources_996997_15.stderr parsl.dataflow.dflow DEBUG: Task 2663 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2663 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2663 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2663 has memoization hash 07c7597da331da7077535cb1e2279cd0 parsl.dataflow.memoization INFO: Task 2663 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2663 try 0 launched on executor multi with executor id 2664 parsl.dataflow.dflow INFO: Standard output for task 2663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f6389cce-625e-4fd7-b9a0-af2a81dfc453_isrForCrosstalkSources_996988_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f6389cce-625e-4fd7-b9a0-af2a81dfc453_isrForCrosstalkSources_996988_54.stderr parsl.dataflow.dflow DEBUG: Task 2664 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2664 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2664 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2664 has memoization hash 49dded4e1c007a156fa24bc51d06f154 parsl.dataflow.memoization INFO: Task 2664 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2664 try 0 launched on executor multi with executor id 2665 parsl.dataflow.dflow INFO: Standard output for task 2664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e97bb3e8-30b3-4b27-ae33-272ce2b069dd_isrForCrosstalkSources_996968_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e97bb3e8-30b3-4b27-ae33-272ce2b069dd_isrForCrosstalkSources_996968_2.stderr parsl.dataflow.dflow DEBUG: Task 2665 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2665 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2665 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2665 has memoization hash 0d70425291b43ed89fa72250fe4b6114 parsl.dataflow.memoization INFO: Task 2665 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2665 try 0 launched on executor multi with executor id 2666 parsl.dataflow.dflow INFO: Standard output for task 2665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/27c67197-9865-497c-898f-98d4fe0e7769_isrForCrosstalkSources_996995_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/27c67197-9865-497c-898f-98d4fe0e7769_isrForCrosstalkSources_996995_24.stderr parsl.dataflow.dflow DEBUG: Task 2666 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2666 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2666 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2666 has memoization hash 5e1d690038956b55bafafc05cb589f50 parsl.dataflow.memoization INFO: Task 2666 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2666 try 0 launched on executor multi with executor id 2667 parsl.dataflow.dflow INFO: Standard output for task 2666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c22a55c8-2f8b-4632-a6b6-51d84d0a119c_isrForCrosstalkSources_996997_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c22a55c8-2f8b-4632-a6b6-51d84d0a119c_isrForCrosstalkSources_996997_34.stderr parsl.dataflow.dflow DEBUG: Task 2667 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2667 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2667 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2667 has memoization hash 82f8692f1a0a3dc26266e708f5b476e7 parsl.dataflow.memoization INFO: Task 2667 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2667 try 0 launched on executor multi with executor id 2668 parsl.dataflow.dflow INFO: Standard output for task 2667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d976cf3d-7e89-4a11-aa0f-4e325ad2f29e_isrForCrosstalkSources_997009_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d976cf3d-7e89-4a11-aa0f-4e325ad2f29e_isrForCrosstalkSources_997009_22.stderr parsl.dataflow.dflow DEBUG: Task 2668 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2668 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2668 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2668 has memoization hash 438ade7c3c56ce3b7e6dc544fefbced3 parsl.dataflow.memoization INFO: Task 2668 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2668 try 0 launched on executor multi with executor id 2669 parsl.dataflow.dflow INFO: Standard output for task 2668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/acace142-a7d0-4279-af54-ee6ff6f02f6f_isrForCrosstalkSources_996964_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/acace142-a7d0-4279-af54-ee6ff6f02f6f_isrForCrosstalkSources_996964_21.stderr parsl.dataflow.dflow DEBUG: Task 2669 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2669 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2669 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2669 has memoization hash bd8b1dfcb42598cbbbb3b7e2ec92d400 parsl.dataflow.memoization INFO: Task 2669 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2669 try 0 launched on executor multi with executor id 2670 parsl.dataflow.dflow INFO: Standard output for task 2669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e2430c38-b27e-4d11-b7f8-b0fc6443daa1_isrForCrosstalkSources_997005_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e2430c38-b27e-4d11-b7f8-b0fc6443daa1_isrForCrosstalkSources_997005_54.stderr parsl.dataflow.dflow DEBUG: Task 2670 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2670 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2670 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2670 has memoization hash adbed8e0701bd8d001ed939f0fb44bb0 parsl.dataflow.memoization INFO: Task 2670 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2670 try 0 launched on executor multi with executor id 2671 parsl.dataflow.dflow INFO: Standard output for task 2670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a09c2209-d2ea-452d-886f-5dcfbeff99d7_isrForCrosstalkSources_997011_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a09c2209-d2ea-452d-886f-5dcfbeff99d7_isrForCrosstalkSources_997011_25.stderr parsl.dataflow.dflow DEBUG: Task 2671 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2671 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2671 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2671 has memoization hash 50e632bc7f53f03628911f79390928b5 parsl.dataflow.memoization INFO: Task 2671 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2671 try 0 launched on executor multi with executor id 2672 parsl.dataflow.dflow INFO: Standard output for task 2671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/3055cf75-937d-44fa-b566-ac2931ad1b8e_isrForCrosstalkSources_997012_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/3055cf75-937d-44fa-b566-ac2931ad1b8e_isrForCrosstalkSources_997012_53.stderr parsl.dataflow.dflow DEBUG: Task 2672 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2672 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2672 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2672 has memoization hash 34e2f3943abcc7b5f0c40ea3abfd1179 parsl.dataflow.memoization INFO: Task 2672 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2672 try 0 launched on executor multi with executor id 2673 parsl.dataflow.dflow INFO: Standard output for task 2672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a9840771-17fb-45ae-a7cd-315d105515da_isrForCrosstalkSources_996954_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a9840771-17fb-45ae-a7cd-315d105515da_isrForCrosstalkSources_996954_36.stderr parsl.dataflow.dflow DEBUG: Task 2673 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2673 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2673 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2673 has memoization hash 958507f3782a07d7b6a55a2c3fad2695 parsl.dataflow.memoization INFO: Task 2673 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2673 try 0 launched on executor multi with executor id 2674 parsl.dataflow.dflow INFO: Standard output for task 2673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c272a04c-1427-4d59-9e39-033aea46d87e_isrForCrosstalkSources_997013_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c272a04c-1427-4d59-9e39-033aea46d87e_isrForCrosstalkSources_997013_58.stderr parsl.dataflow.dflow DEBUG: Task 2674 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2674 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2674 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2674 has memoization hash 7a55484edb1efae9e731857862b514d8 parsl.dataflow.memoization INFO: Task 2674 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2674 try 0 launched on executor multi with executor id 2675 parsl.dataflow.dflow INFO: Standard output for task 2674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4fe22c91-50d2-47e8-8867-a0eab64c042a_isrForCrosstalkSources_997017_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4fe22c91-50d2-47e8-8867-a0eab64c042a_isrForCrosstalkSources_997017_29.stderr parsl.dataflow.dflow DEBUG: Task 2675 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2675 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2675 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2675 has memoization hash 47e0b3fa35855540d854af9638d22960 parsl.dataflow.memoization INFO: Task 2675 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2675 try 0 launched on executor multi with executor id 2676 parsl.dataflow.dflow INFO: Standard output for task 2675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a366daa5-ca30-4b73-9df5-17e639fee7d0_isrForCrosstalkSources_996994_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a366daa5-ca30-4b73-9df5-17e639fee7d0_isrForCrosstalkSources_996994_50.stderr parsl.dataflow.dflow DEBUG: Task 2676 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2676 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2676 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2676 has memoization hash 3de2cdd7174ce40a42f6e3170c3b6949 parsl.dataflow.memoization INFO: Task 2676 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2676 try 0 launched on executor multi with executor id 2677 parsl.dataflow.dflow INFO: Standard output for task 2676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e4f67bde-9165-4430-9f4f-ff0589daa1e3_isrForCrosstalkSources_996986_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e4f67bde-9165-4430-9f4f-ff0589daa1e3_isrForCrosstalkSources_996986_62.stderr parsl.dataflow.dflow DEBUG: Task 2677 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2677 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2677 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2677 has memoization hash 498615d2535cc9a29829d25679efa8c8 parsl.dataflow.memoization INFO: Task 2677 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2677 try 0 launched on executor multi with executor id 2678 parsl.dataflow.dflow INFO: Standard output for task 2677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/c8f748ed-82c5-4fba-b1a9-defbb568459d_isrForCrosstalkSources_997032_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/c8f748ed-82c5-4fba-b1a9-defbb568459d_isrForCrosstalkSources_997032_35.stderr parsl.dataflow.dflow DEBUG: Task 2678 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2678 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2678 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2678 has memoization hash 25a7b572384841dcc82f9963e13623e2 parsl.dataflow.memoization INFO: Task 2678 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2678 try 0 launched on executor multi with executor id 2679 parsl.dataflow.dflow INFO: Standard output for task 2678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e1fa7cfe-df69-4854-b19f-5f96b08080ca_isrForCrosstalkSources_996985_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e1fa7cfe-df69-4854-b19f-5f96b08080ca_isrForCrosstalkSources_996985_44.stderr parsl.dataflow.dflow DEBUG: Task 2679 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2679 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2679 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2679 has memoization hash 9b811e8a83b9abb04a4027b777593a11 parsl.dataflow.memoization INFO: Task 2679 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2679 try 0 launched on executor multi with executor id 2680 parsl.dataflow.dflow INFO: Standard output for task 2679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/831b72d9-39dc-4672-850e-42e0df5a0ff6_isrForCrosstalkSources_997023_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/831b72d9-39dc-4672-850e-42e0df5a0ff6_isrForCrosstalkSources_997023_24.stderr parsl.dataflow.dflow DEBUG: Task 2680 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2680 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2680 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2680 has memoization hash 756c68b8d882a7988455ad5c4875ab18 parsl.dataflow.memoization INFO: Task 2680 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2680 try 0 launched on executor multi with executor id 2681 parsl.dataflow.dflow INFO: Standard output for task 2680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/aa049fb6-a45b-45df-9143-9b5ba7eb948a_isrForCrosstalkSources_996961_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/aa049fb6-a45b-45df-9143-9b5ba7eb948a_isrForCrosstalkSources_996961_32.stderr parsl.dataflow.dflow DEBUG: Task 2681 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2681 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2681 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2681 has memoization hash 79d58a1bdc87b1bd08ee29a816b8f38f parsl.dataflow.memoization INFO: Task 2681 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2681 try 0 launched on executor multi with executor id 2682 parsl.dataflow.dflow INFO: Standard output for task 2681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/17b41f17-df5b-4e7d-93ef-916832d2cfc0_isrForCrosstalkSources_996953_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/17b41f17-df5b-4e7d-93ef-916832d2cfc0_isrForCrosstalkSources_996953_52.stderr parsl.dataflow.dflow DEBUG: Task 2682 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2682 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2682 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2682 has memoization hash e468e4aeda841c7739501bb922952ee8 parsl.dataflow.memoization INFO: Task 2682 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2682 try 0 launched on executor multi with executor id 2683 parsl.dataflow.dflow INFO: Standard output for task 2682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d8c2c21f-94db-4e31-bb36-8d261c50d1f8_isrForCrosstalkSources_996988_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d8c2c21f-94db-4e31-bb36-8d261c50d1f8_isrForCrosstalkSources_996988_1.stderr parsl.dataflow.dflow DEBUG: Task 2683 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2683 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2683 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2683 has memoization hash ea3b3e608e504efaeede09498599ef6e parsl.dataflow.memoization INFO: Task 2683 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2683 try 0 launched on executor multi with executor id 2684 parsl.dataflow.dflow INFO: Standard output for task 2683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/16eeb570-bcbf-4ddb-8c51-7268f017d4dc_isrForCrosstalkSources_997003_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/16eeb570-bcbf-4ddb-8c51-7268f017d4dc_isrForCrosstalkSources_997003_38.stderr parsl.dataflow.dflow DEBUG: Task 2684 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2684 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2684 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2684 has memoization hash 611133bc1f74d4d8d6584f9b812f6fd3 parsl.dataflow.memoization INFO: Task 2684 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2684 try 0 launched on executor multi with executor id 2685 parsl.dataflow.dflow INFO: Standard output for task 2684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/feec94d9-faa8-4a19-b3a6-522d92eef2cb_isrForCrosstalkSources_996996_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/feec94d9-faa8-4a19-b3a6-522d92eef2cb_isrForCrosstalkSources_996996_23.stderr parsl.dataflow.dflow DEBUG: Task 2685 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2685 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2685 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2685 has memoization hash 0786c0e16de055abb01302c078a5ace4 parsl.dataflow.memoization INFO: Task 2685 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2685 try 0 launched on executor multi with executor id 2686 parsl.dataflow.dflow INFO: Standard output for task 2685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3a2003ed-0231-43a8-b6b4-2a1335e9268d_isrForCrosstalkSources_997031_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3a2003ed-0231-43a8-b6b4-2a1335e9268d_isrForCrosstalkSources_997031_52.stderr parsl.dataflow.dflow DEBUG: Task 2686 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2686 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2686 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2686 has memoization hash 11a9def7ada4db0580f98520d494c43b parsl.dataflow.memoization INFO: Task 2686 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2686 try 0 launched on executor multi with executor id 2687 parsl.dataflow.dflow INFO: Standard output for task 2686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/80cb774e-f88d-4cb9-9116-f3806cbfddf9_isrForCrosstalkSources_996993_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/80cb774e-f88d-4cb9-9116-f3806cbfddf9_isrForCrosstalkSources_996993_61.stderr parsl.dataflow.dflow DEBUG: Task 2687 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2687 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2687 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2687 has memoization hash a683723fb646284b64285bf2331a50ac parsl.dataflow.memoization INFO: Task 2687 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2687 try 0 launched on executor multi with executor id 2688 parsl.dataflow.dflow INFO: Standard output for task 2687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/0a876077-3a81-48cf-9bd2-3258a5927d72_isrForCrosstalkSources_997029_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/0a876077-3a81-48cf-9bd2-3258a5927d72_isrForCrosstalkSources_997029_62.stderr parsl.dataflow.dflow DEBUG: Task 2688 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2688 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2688 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2688 has memoization hash c7b400f22ffc471fc3c6c8f97a2c41ef parsl.dataflow.memoization INFO: Task 2688 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2688 try 0 launched on executor multi with executor id 2689 parsl.dataflow.dflow INFO: Standard output for task 2688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/42e12d3a-8818-4096-85f1-1b4a0691bf2f_isrForCrosstalkSources_996970_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/42e12d3a-8818-4096-85f1-1b4a0691bf2f_isrForCrosstalkSources_996970_9.stderr parsl.dataflow.dflow DEBUG: Task 2689 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2689 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2689 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2689 has memoization hash 1ca4c857a71580e67159995843be2180 parsl.dataflow.memoization INFO: Task 2689 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2689 try 0 launched on executor multi with executor id 2690 parsl.dataflow.dflow INFO: Standard output for task 2689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/387b9a35-7c24-4f83-bdb4-edaff4ad47ce_isrForCrosstalkSources_996965_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/387b9a35-7c24-4f83-bdb4-edaff4ad47ce_isrForCrosstalkSources_996965_8.stderr parsl.dataflow.dflow DEBUG: Task 2690 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2690 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2690 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2690 has memoization hash a5e39a44360fbece22a9c1edfc86006c parsl.dataflow.memoization INFO: Task 2690 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2690 try 0 launched on executor multi with executor id 2691 parsl.dataflow.dflow INFO: Standard output for task 2690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/966d198f-37d5-437e-a9ff-2d081e3727e3_isrForCrosstalkSources_997008_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/966d198f-37d5-437e-a9ff-2d081e3727e3_isrForCrosstalkSources_997008_20.stderr parsl.dataflow.dflow DEBUG: Task 2691 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2691 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2691 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2691 has memoization hash 415be03d99ddafc602b260d0bfc0ea0a parsl.dataflow.memoization INFO: Task 2691 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2691 try 0 launched on executor multi with executor id 2692 parsl.dataflow.dflow INFO: Standard output for task 2691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ed6f8914-70c6-477b-b285-d322a18207e6_isrForCrosstalkSources_996960_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ed6f8914-70c6-477b-b285-d322a18207e6_isrForCrosstalkSources_996960_42.stderr parsl.dataflow.dflow DEBUG: Task 2692 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2692 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2692 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2692 has memoization hash 39f8c72cc2dc9ce56438b2cb9bb4b0f4 parsl.dataflow.memoization INFO: Task 2692 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2692 try 0 launched on executor multi with executor id 2693 parsl.dataflow.dflow INFO: Standard output for task 2692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ede4c99b-ea13-4c63-9ae5-835be2cf60a8_isrForCrosstalkSources_997000_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ede4c99b-ea13-4c63-9ae5-835be2cf60a8_isrForCrosstalkSources_997000_10.stderr parsl.dataflow.dflow DEBUG: Task 2693 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2693 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2693 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2693 has memoization hash 8ec964d51f7e6f94bd2cef6f617375c0 parsl.dataflow.memoization INFO: Task 2693 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2693 try 0 launched on executor multi with executor id 2694 parsl.dataflow.dflow INFO: Standard output for task 2693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/b2ae47c9-0b8c-4ce2-b9a8-49e7accc5cbf_isrForCrosstalkSources_997020_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/b2ae47c9-0b8c-4ce2-b9a8-49e7accc5cbf_isrForCrosstalkSources_997020_20.stderr parsl.dataflow.dflow DEBUG: Task 2694 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2694 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2694 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2694 has memoization hash 5129166a6e9229c353952230a3cc1e7a parsl.dataflow.memoization INFO: Task 2694 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2694 try 0 launched on executor multi with executor id 2695 parsl.dataflow.dflow INFO: Standard output for task 2694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d9629746-9a1c-4a5c-9a95-435d85e970ed_isrForCrosstalkSources_996965_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d9629746-9a1c-4a5c-9a95-435d85e970ed_isrForCrosstalkSources_996965_61.stderr parsl.dataflow.dflow DEBUG: Task 2695 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2695 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2695 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2695 has memoization hash dd11a6405b9923356d7e33b554dbf375 parsl.dataflow.memoization INFO: Task 2695 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2695 try 0 launched on executor multi with executor id 2696 parsl.dataflow.dflow INFO: Standard output for task 2695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/6a3b757a-9e15-4dcc-b4e2-acbc0b79d592_isrForCrosstalkSources_996967_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/6a3b757a-9e15-4dcc-b4e2-acbc0b79d592_isrForCrosstalkSources_996967_47.stderr parsl.dataflow.dflow DEBUG: Task 2696 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2696 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2696 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2696 has memoization hash 7a3642b07eb47c12c093de2c08436e5e parsl.dataflow.memoization INFO: Task 2696 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2696 try 0 launched on executor multi with executor id 2697 parsl.dataflow.dflow INFO: Standard output for task 2696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/d66b7ecd-5ba6-44e8-9f01-9621a12a9c9b_isrForCrosstalkSources_996986_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/d66b7ecd-5ba6-44e8-9f01-9621a12a9c9b_isrForCrosstalkSources_996986_3.stderr parsl.dataflow.dflow DEBUG: Task 2697 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2697 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2697 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2697 has memoization hash 218cba28f0dd7efec9de39560f088b31 parsl.dataflow.memoization INFO: Task 2697 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2697 try 0 launched on executor multi with executor id 2698 parsl.dataflow.dflow INFO: Standard output for task 2697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/99448460-e315-45c5-8c2e-1aba5dea15c3_isrForCrosstalkSources_997025_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/99448460-e315-45c5-8c2e-1aba5dea15c3_isrForCrosstalkSources_997025_10.stderr parsl.dataflow.dflow DEBUG: Task 2698 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2698 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2698 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2698 has memoization hash 127310d7a3006da4a22f2c9cf5768d7f parsl.dataflow.memoization INFO: Task 2698 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2698 try 0 launched on executor multi with executor id 2699 parsl.dataflow.dflow INFO: Standard output for task 2698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/77d78653-487c-4c7c-9b9c-85eff45fe016_isrForCrosstalkSources_997004_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/77d78653-487c-4c7c-9b9c-85eff45fe016_isrForCrosstalkSources_997004_55.stderr parsl.dataflow.dflow DEBUG: Task 2699 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2699 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2699 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2699 has memoization hash 47ea78b67b8086e8269ffcb7f47d423b parsl.dataflow.memoization INFO: Task 2699 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2699 try 0 launched on executor multi with executor id 2700 parsl.dataflow.dflow INFO: Standard output for task 2699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc94f3e6-b2fd-4dbf-828e-453413ded5f0_isrForCrosstalkSources_997025_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc94f3e6-b2fd-4dbf-828e-453413ded5f0_isrForCrosstalkSources_997025_32.stderr parsl.dataflow.dflow DEBUG: Task 2700 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2700 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2700 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2700 has memoization hash 82a59cf9c558e62f142e9905aecdb15e parsl.dataflow.memoization INFO: Task 2700 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2700 try 0 launched on executor multi with executor id 2701 parsl.dataflow.dflow INFO: Standard output for task 2700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/aecbf6da-b367-42e5-8397-a8a06ddb2cc4_isrForCrosstalkSources_997006_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/aecbf6da-b367-42e5-8397-a8a06ddb2cc4_isrForCrosstalkSources_997006_10.stderr parsl.dataflow.dflow DEBUG: Task 2701 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2701 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2701 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2701 has memoization hash 9abb7fd30f3bcb64cffc624472fc07b3 parsl.dataflow.memoization INFO: Task 2701 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2701 try 0 launched on executor multi with executor id 2702 parsl.dataflow.dflow INFO: Standard output for task 2701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/29d77549-47aa-41b5-93e5-f2ffb86bb66e_isrForCrosstalkSources_996987_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/29d77549-47aa-41b5-93e5-f2ffb86bb66e_isrForCrosstalkSources_996987_57.stderr parsl.dataflow.dflow DEBUG: Task 2702 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2702 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2702 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2702 has memoization hash 976b1b7001c8e8918a89204f7405f601 parsl.dataflow.memoization INFO: Task 2702 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2702 try 0 launched on executor multi with executor id 2703 parsl.dataflow.dflow INFO: Standard output for task 2702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/12b25834-cefd-4d3e-b4ea-38528dcf5a2c_isrForCrosstalkSources_996962_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/12b25834-cefd-4d3e-b4ea-38528dcf5a2c_isrForCrosstalkSources_996962_27.stderr parsl.dataflow.dflow DEBUG: Task 2703 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2703 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2703 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2703 has memoization hash bd6da7382bcd0963d9befdc0b86759e3 parsl.dataflow.memoization INFO: Task 2703 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2703 try 0 launched on executor multi with executor id 2704 parsl.dataflow.dflow INFO: Standard output for task 2703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/e3c8bf69-2e31-4b03-a1fb-f80c2348ebd4_isrForCrosstalkSources_996990_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/e3c8bf69-2e31-4b03-a1fb-f80c2348ebd4_isrForCrosstalkSources_996990_2.stderr parsl.dataflow.dflow DEBUG: Task 2704 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2704 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2704 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2704 has memoization hash 304727c01eff49cfe9be59f80a4fdaff parsl.dataflow.memoization INFO: Task 2704 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:51 proc_lsst.multi:170 [INFO] [multi] job_id 17074877 proc_lsst.multi INFO: [multi] job_id 17074877 2024-03-15 13:49:51 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 29 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 29 2024-03-15 13:49:51 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:51 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074877 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074877 parsl.executors.status_handling DEBUG: Launched block 29 on executor multi with job ID 17074877 parsl.executors.status_handling INFO: Allocated block ID 30 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-30 2024-03-15 13:49:51 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=30 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-30 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=30 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-30 2024-03-15 13:49:51 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=30 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=30 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:51 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=30 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=30 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 29 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 29 2024-03-15 13:49:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:51 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=30 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=30 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to astro parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 2704 try 0 launched on executor multi with executor id 2705 parsl.dataflow.dflow INFO: Standard output for task 2704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/28771f35-aff3-4d0a-83e1-9eb8fdb8dbcc_isrForCrosstalkSources_997018_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/28771f35-aff3-4d0a-83e1-9eb8fdb8dbcc_isrForCrosstalkSources_997018_47.stderr parsl.dataflow.dflow DEBUG: Task 2705 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2705 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2705 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2705 has memoization hash bdbcdbbbd4a90eb8402fc394ab9f6223 parsl.dataflow.memoization INFO: Task 2705 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2705 try 0 launched on executor multi with executor id 2706 parsl.dataflow.dflow INFO: Standard output for task 2705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/48d62580-f7b0-4b3f-8742-578e4da76a1f_isrForCrosstalkSources_996977_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/48d62580-f7b0-4b3f-8742-578e4da76a1f_isrForCrosstalkSources_996977_2.stderr parsl.dataflow.dflow DEBUG: Task 2706 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2706 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2706 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2706 has memoization hash 7a07a868dee5781e4fda619a2f61e835 parsl.dataflow.memoization INFO: Task 2706 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1b380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2706 try 0 launched on executor multi with executor id 2707 parsl.dataflow.dflow INFO: Standard output for task 2706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a0795cdf-bc70-47f1-a47f-e3944f4a0479_isrForCrosstalkSources_996957_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a0795cdf-bc70-47f1-a47f-e3944f4a0479_isrForCrosstalkSources_996957_3.stderr parsl.dataflow.dflow DEBUG: Task 2707 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2707 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2707 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2707 has memoization hash 8a718a2fe2e687ab9eba0651ee21d8f3 parsl.dataflow.memoization INFO: Task 2707 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cb1bec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2707 try 0 launched on executor multi with executor id 2708 parsl.dataflow.dflow INFO: Standard output for task 2707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/47b8723c-d6f0-4baf-98bc-532cd9b3e6a5_isrForCrosstalkSources_997017_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/47b8723c-d6f0-4baf-98bc-532cd9b3e6a5_isrForCrosstalkSources_997017_30.stderr parsl.dataflow.dflow DEBUG: Task 2708 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2708 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2708 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2708 has memoization hash 6f8f6117bbffb1c52b2340a1cbabaed5 parsl.dataflow.memoization INFO: Task 2708 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2708 try 0 launched on executor multi with executor id 2709 parsl.dataflow.dflow INFO: Standard output for task 2708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/11753d3f-9ddd-46fc-ab83-7e6bd98f0c4f_isrForCrosstalkSources_996983_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/11753d3f-9ddd-46fc-ab83-7e6bd98f0c4f_isrForCrosstalkSources_996983_51.stderr parsl.dataflow.dflow DEBUG: Task 2709 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2709 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2709 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2709 has memoization hash aec4a4aca5060929e4b131310234b3b6 parsl.dataflow.memoization INFO: Task 2709 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 2709 try 0 launched on executor multi with executor id 2710 parsl.dataflow.dflow INFO: Standard output for task 2709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/53a906f5-b673-4264-befa-456de1883116_isrForCrosstalkSources_997022_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/53a906f5-b673-4264-befa-456de1883116_isrForCrosstalkSources_997022_29.stderr parsl.dataflow.dflow DEBUG: Task 2710 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2710 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2710 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2710 has memoization hash fda82ec971134675ca36a31af54e604a parsl.dataflow.memoization INFO: Task 2710 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2710 try 0 launched on executor multi with executor id 2711 parsl.dataflow.dflow INFO: Standard output for task 2710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e6b75f4c-7c01-4876-ae61-f80ba9e1806a_isrForCrosstalkSources_996997_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e6b75f4c-7c01-4876-ae61-f80ba9e1806a_isrForCrosstalkSources_996997_53.stderr parsl.dataflow.dflow DEBUG: Task 2711 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2711 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2711 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2711 has memoization hash c825fbaf9e212c87295ed3dcb5a0d815 parsl.dataflow.memoization INFO: Task 2711 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2711 try 0 launched on executor multi with executor id 2712 parsl.dataflow.dflow INFO: Standard output for task 2711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/6f12dd47-3f07-4116-9cb5-5b503c0fbb89_isrForCrosstalkSources_997029_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/6f12dd47-3f07-4116-9cb5-5b503c0fbb89_isrForCrosstalkSources_997029_51.stderr parsl.dataflow.dflow DEBUG: Task 2712 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2712 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2712 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2712 has memoization hash 9a37a86411f004c4db249c7c2e0d8e2c parsl.dataflow.memoization INFO: Task 2712 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2712 try 0 launched on executor multi with executor id 2713 parsl.dataflow.dflow INFO: Standard output for task 2712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5c742c96-72f0-4dff-8b4d-0f87ec2d544f_isrForCrosstalkSources_996985_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5c742c96-72f0-4dff-8b4d-0f87ec2d544f_isrForCrosstalkSources_996985_18.stderr parsl.dataflow.dflow DEBUG: Task 2713 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2713 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2713 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2713 has memoization hash 50c41b05f9a5dec073c84503a9c9baa6 parsl.dataflow.memoization INFO: Task 2713 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9e3060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2713 try 0 launched on executor multi with executor id 2714 parsl.dataflow.dflow INFO: Standard output for task 2713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/977142c1-c18f-43ca-b8be-03be4019a5e8_isrForCrosstalkSources_996987_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/977142c1-c18f-43ca-b8be-03be4019a5e8_isrForCrosstalkSources_996987_12.stderr parsl.dataflow.dflow DEBUG: Task 2714 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2714 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2714 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2714 has memoization hash 63c5e855c2ee494ba56e7bfb5d1e3266 parsl.dataflow.memoization INFO: Task 2714 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2714 try 0 launched on executor multi with executor id 2715 parsl.dataflow.dflow INFO: Standard output for task 2714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/35f76edf-2ea6-4a09-8e5f-2bc3142cf40b_isrForCrosstalkSources_996978_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/35f76edf-2ea6-4a09-8e5f-2bc3142cf40b_isrForCrosstalkSources_996978_40.stderr parsl.dataflow.dflow DEBUG: Task 2715 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2715 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2715 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2715 has memoization hash 9354cec10e85e567334dc3256e0c9b0f parsl.dataflow.memoization INFO: Task 2715 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2715 try 0 launched on executor multi with executor id 2716 parsl.dataflow.dflow INFO: Standard output for task 2715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/097764a2-5d18-4982-a1ed-a5eef672f84f_isrForCrosstalkSources_996982_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/097764a2-5d18-4982-a1ed-a5eef672f84f_isrForCrosstalkSources_996982_14.stderr parsl.dataflow.dflow DEBUG: Task 2716 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2716 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2716 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2716 has memoization hash d838c603f22412e9c74c5fb5b81b1686 parsl.dataflow.memoization INFO: Task 2716 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2716 try 0 launched on executor multi with executor id 2717 parsl.dataflow.dflow INFO: Standard output for task 2716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/785c7b59-701c-4ff0-8134-393e3160c4c5_isrForCrosstalkSources_997011_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/785c7b59-701c-4ff0-8134-393e3160c4c5_isrForCrosstalkSources_997011_26.stderr parsl.dataflow.dflow DEBUG: Task 2717 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2717 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2717 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2717 has memoization hash 2b3cc96dda2810d6d7d73ef7c4c7e6d2 parsl.dataflow.memoization INFO: Task 2717 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2717 try 0 launched on executor multi with executor id 2718 parsl.dataflow.dflow INFO: Standard output for task 2717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7b76f480-9a32-4023-88ff-2b1ac8474336_isrForCrosstalkSources_996991_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7b76f480-9a32-4023-88ff-2b1ac8474336_isrForCrosstalkSources_996991_19.stderr parsl.dataflow.dflow DEBUG: Task 2718 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2718 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2718 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2718 has memoization hash d02bc6b59a16db575215ad268554a7d2 parsl.dataflow.memoization INFO: Task 2718 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2718 try 0 launched on executor multi with executor id 2719 parsl.dataflow.dflow INFO: Standard output for task 2718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e51fb5d3-2398-4460-9d5c-2f7ccee6a138_isrForCrosstalkSources_997022_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e51fb5d3-2398-4460-9d5c-2f7ccee6a138_isrForCrosstalkSources_997022_54.stderr parsl.dataflow.dflow DEBUG: Task 2719 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2719 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2719 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2719 has memoization hash 599ca1ee3a7117346f003ecfde8b4242 parsl.dataflow.memoization INFO: Task 2719 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2719 try 0 launched on executor multi with executor id 2720 parsl.dataflow.dflow INFO: Standard output for task 2719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c4e738fe-f4eb-45d8-8ff9-801c20f71595_isrForCrosstalkSources_997015_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c4e738fe-f4eb-45d8-8ff9-801c20f71595_isrForCrosstalkSources_997015_46.stderr parsl.dataflow.dflow DEBUG: Task 2720 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2720 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2720 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2720 has memoization hash 20f49d6bff9fb7a5593991cb694a0668 parsl.dataflow.memoization INFO: Task 2720 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2720 try 0 launched on executor multi with executor id 2721 parsl.dataflow.dflow INFO: Standard output for task 2720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/e40e8074-143c-419d-9ca8-3918cf3eb76f_isrForCrosstalkSources_997025_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/e40e8074-143c-419d-9ca8-3918cf3eb76f_isrForCrosstalkSources_997025_47.stderr parsl.dataflow.dflow DEBUG: Task 2721 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2721 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2721 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2721 has memoization hash 5bf0d034f61f67b78412d3cac2d1acaf parsl.dataflow.memoization INFO: Task 2721 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2721 try 0 launched on executor multi with executor id 2722 parsl.dataflow.dflow INFO: Standard output for task 2721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f549354d-565c-4b4a-a947-bbb3528f6a03_isrForCrosstalkSources_997005_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f549354d-565c-4b4a-a947-bbb3528f6a03_isrForCrosstalkSources_997005_41.stderr parsl.dataflow.dflow DEBUG: Task 2722 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2722 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2722 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2722 has memoization hash ab2a9f77e60ee09e3a4811044c17aaae parsl.dataflow.memoization INFO: Task 2722 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2722 try 0 launched on executor multi with executor id 2723 parsl.dataflow.dflow INFO: Standard output for task 2722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4205fb2a-1e1d-496d-90cf-6c51c579f917_isrForCrosstalkSources_996976_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4205fb2a-1e1d-496d-90cf-6c51c579f917_isrForCrosstalkSources_996976_21.stderr parsl.dataflow.dflow DEBUG: Task 2723 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2723 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2723 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2723 has memoization hash 00bc7120c3ff7e95259a76935198eb4c parsl.dataflow.memoization INFO: Task 2723 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2723 try 0 launched on executor multi with executor id 2724 parsl.dataflow.dflow INFO: Standard output for task 2723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/1eb69ac1-fe69-4321-9372-ce07dfe9abaa_isrForCrosstalkSources_997015_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/1eb69ac1-fe69-4321-9372-ce07dfe9abaa_isrForCrosstalkSources_997015_13.stderr parsl.dataflow.dflow DEBUG: Task 2724 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2724 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2724 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2724 has memoization hash 2913fff1f2331f21715a9e3b2cd6f600 parsl.dataflow.memoization INFO: Task 2724 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2724 try 0 launched on executor multi with executor id 2725 parsl.dataflow.dflow INFO: Standard output for task 2724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e0592f54-7cc6-4649-9ef8-06eec036123c_isrForCrosstalkSources_997030_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e0592f54-7cc6-4649-9ef8-06eec036123c_isrForCrosstalkSources_997030_2.stderr parsl.dataflow.dflow DEBUG: Task 2725 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2725 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2725 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2725 has memoization hash 13b67740236c542e72b8d75e9ebf010e parsl.dataflow.memoization INFO: Task 2725 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2725 try 0 launched on executor multi with executor id 2726 parsl.dataflow.dflow INFO: Standard output for task 2725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1a4eff7c-6f00-43cb-b2b6-2ace3dd82665_isrForCrosstalkSources_996953_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1a4eff7c-6f00-43cb-b2b6-2ace3dd82665_isrForCrosstalkSources_996953_3.stderr parsl.dataflow.dflow DEBUG: Task 2726 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2726 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2726 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2726 has memoization hash 8340b287bb8480f3bd5df217faed6a96 parsl.dataflow.memoization INFO: Task 2726 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2726 try 0 launched on executor multi with executor id 2727 parsl.dataflow.dflow INFO: Standard output for task 2726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/4ff17dc3-7166-4196-8b8f-961caad43808_isrForCrosstalkSources_997024_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/4ff17dc3-7166-4196-8b8f-961caad43808_isrForCrosstalkSources_997024_11.stderr parsl.dataflow.dflow DEBUG: Task 2727 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2727 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2727 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2727 has memoization hash da55ca339f77f2b0b7bc19d80aa0ea56 parsl.dataflow.memoization INFO: Task 2727 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2727 try 0 launched on executor multi with executor id 2728 parsl.dataflow.dflow INFO: Standard output for task 2727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/2716d60b-7e3c-497a-b4d8-dff4d9b9f8f9_isrForCrosstalkSources_997021_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/2716d60b-7e3c-497a-b4d8-dff4d9b9f8f9_isrForCrosstalkSources_997021_3.stderr parsl.dataflow.dflow DEBUG: Task 2728 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2728 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2728 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2728 has memoization hash 9948f75b8cec83cbca49d53bc05034af parsl.dataflow.memoization INFO: Task 2728 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2728 try 0 launched on executor multi with executor id 2729 parsl.dataflow.dflow INFO: Standard output for task 2728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/aee5958f-6301-4f8f-a929-76b6c30ac830_isrForCrosstalkSources_996996_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/aee5958f-6301-4f8f-a929-76b6c30ac830_isrForCrosstalkSources_996996_43.stderr parsl.dataflow.dflow DEBUG: Task 2729 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2729 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2729 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2729 has memoization hash 1e887e715bf460caa11f0cfcd90b4c08 parsl.dataflow.memoization INFO: Task 2729 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2729 try 0 launched on executor multi with executor id 2730 parsl.dataflow.dflow INFO: Standard output for task 2729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/263af9d7-a423-4438-af50-fee0740ba310_isrForCrosstalkSources_997012_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/263af9d7-a423-4438-af50-fee0740ba310_isrForCrosstalkSources_997012_15.stderr parsl.dataflow.dflow DEBUG: Task 2730 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2730 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2730 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2730 has memoization hash fbb196a9d02714a79436e71011d0349d parsl.dataflow.memoization INFO: Task 2730 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2730 try 0 launched on executor multi with executor id 2731 parsl.dataflow.dflow INFO: Standard output for task 2730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/3c2b15f3-ea07-4a8f-9f66-3f7d80b8f056_isrForCrosstalkSources_997017_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/3c2b15f3-ea07-4a8f-9f66-3f7d80b8f056_isrForCrosstalkSources_997017_8.stderr parsl.dataflow.dflow DEBUG: Task 2731 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2731 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2731 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2731 has memoization hash 592217cd7ec871c4ea4c88676cc324f7 parsl.dataflow.memoization INFO: Task 2731 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2731 try 0 launched on executor multi with executor id 2732 parsl.dataflow.dflow INFO: Standard output for task 2731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d7aee4b9-8ef8-444f-9b9e-d591d4c02231_isrForCrosstalkSources_997026_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d7aee4b9-8ef8-444f-9b9e-d591d4c02231_isrForCrosstalkSources_997026_22.stderr parsl.dataflow.dflow DEBUG: Task 2732 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2732 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2732 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2732 has memoization hash 1577a26ba61ecfa6eb40577e1ac19803 parsl.dataflow.memoization INFO: Task 2732 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2732 try 0 launched on executor multi with executor id 2733 parsl.dataflow.dflow INFO: Standard output for task 2732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8955b14b-70f4-43ed-9875-623e90292554_isrForCrosstalkSources_996973_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8955b14b-70f4-43ed-9875-623e90292554_isrForCrosstalkSources_996973_7.stderr parsl.dataflow.dflow DEBUG: Task 2733 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2733 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2733 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2733 has memoization hash 88851a6ee14ea3849b09bb490a3d47df parsl.dataflow.memoization INFO: Task 2733 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2733 try 0 launched on executor multi with executor id 2734 parsl.dataflow.dflow INFO: Standard output for task 2733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e478823b-a506-4fa8-ada8-01d1a074c769_isrForCrosstalkSources_997033_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e478823b-a506-4fa8-ada8-01d1a074c769_isrForCrosstalkSources_997033_38.stderr parsl.dataflow.dflow DEBUG: Task 2734 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2734 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2734 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2734 has memoization hash 0b34a692d82a00577bb0e3a4a4fc5cf5 parsl.dataflow.memoization INFO: Task 2734 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2734 try 0 launched on executor multi with executor id 2735 parsl.dataflow.dflow INFO: Standard output for task 2734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ef252fea-f8e9-44b6-9563-155141a89ef9_isrForCrosstalkSources_997006_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ef252fea-f8e9-44b6-9563-155141a89ef9_isrForCrosstalkSources_997006_24.stderr parsl.dataflow.dflow DEBUG: Task 2735 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2735 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2735 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2735 has memoization hash 3b159a75f9c4dc61cd267aea649c82a5 parsl.dataflow.memoization INFO: Task 2735 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2735 try 0 launched on executor multi with executor id 2736 parsl.dataflow.dflow INFO: Standard output for task 2735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/6671c8aa-e5ae-4b16-8cee-21658ece6c1c_isrForCrosstalkSources_997019_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/6671c8aa-e5ae-4b16-8cee-21658ece6c1c_isrForCrosstalkSources_997019_50.stderr parsl.dataflow.dflow DEBUG: Task 2736 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2736 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2736 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2736 has memoization hash c2e46325d27b577b2fdde2a1111bce84 parsl.dataflow.memoization INFO: Task 2736 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2736 try 0 launched on executor multi with executor id 2737 parsl.dataflow.dflow INFO: Standard output for task 2736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/169cda47-e1a0-4dcd-bf0a-f4323d085694_isrForCrosstalkSources_997031_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/169cda47-e1a0-4dcd-bf0a-f4323d085694_isrForCrosstalkSources_997031_15.stderr parsl.dataflow.dflow DEBUG: Task 2737 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2737 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2737 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2737 has memoization hash 3817793ace9dd9f817a2b1f193c5ae00 parsl.dataflow.memoization INFO: Task 2737 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2737 try 0 launched on executor multi with executor id 2738 parsl.dataflow.dflow INFO: Standard output for task 2737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/99c061e2-a1fb-40c6-a6cb-8f0dc2a1beac_isrForCrosstalkSources_996968_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/99c061e2-a1fb-40c6-a6cb-8f0dc2a1beac_isrForCrosstalkSources_996968_25.stderr parsl.dataflow.dflow DEBUG: Task 2738 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2738 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2738 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2738 has memoization hash 1c68c5a002fcc144985a861975ce1dc0 parsl.dataflow.memoization INFO: Task 2738 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2738 try 0 launched on executor multi with executor id 2739 parsl.dataflow.dflow INFO: Standard output for task 2738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/355d8f3b-3de2-4fb1-8785-4cb238b5c776_isrForCrosstalkSources_997004_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/355d8f3b-3de2-4fb1-8785-4cb238b5c776_isrForCrosstalkSources_997004_10.stderr parsl.dataflow.dflow DEBUG: Task 2739 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2739 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2739 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2739 has memoization hash ee00339ff892e40a3dc9c4ba39ec5585 parsl.dataflow.memoization INFO: Task 2739 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2739 try 0 launched on executor multi with executor id 2740 parsl.dataflow.dflow INFO: Standard output for task 2739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/22d0bce4-7fee-4509-9afa-67be4bef9cbc_isrForCrosstalkSources_996954_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/22d0bce4-7fee-4509-9afa-67be4bef9cbc_isrForCrosstalkSources_996954_48.stderr parsl.dataflow.dflow DEBUG: Task 2740 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2740 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2740 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2740 has memoization hash 076681874598ab02a62943fd64884aab parsl.dataflow.memoization INFO: Task 2740 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2740 try 0 launched on executor multi with executor id 2741 parsl.dataflow.dflow INFO: Standard output for task 2740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a1b79875-844b-48e0-aab8-099f6490fdea_isrForCrosstalkSources_997011_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a1b79875-844b-48e0-aab8-099f6490fdea_isrForCrosstalkSources_997011_31.stderr parsl.dataflow.dflow DEBUG: Task 2741 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2741 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2741 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2741 has memoization hash de832a6f3087f128736c77183a3d3e7e parsl.dataflow.memoization INFO: Task 2741 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2741 try 0 launched on executor multi with executor id 2742 parsl.dataflow.dflow INFO: Standard output for task 2741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/35da15c8-08e3-498e-bc15-b07837a17895_isrForCrosstalkSources_997018_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/35da15c8-08e3-498e-bc15-b07837a17895_isrForCrosstalkSources_997018_18.stderr parsl.dataflow.dflow DEBUG: Task 2742 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2742 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2742 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2742 has memoization hash 1db63237ed69d1926cafe605feeb3eb8 parsl.dataflow.memoization INFO: Task 2742 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2742 try 0 launched on executor multi with executor id 2743 parsl.dataflow.dflow INFO: Standard output for task 2742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/73d10808-8af1-4a5e-833b-4e510c4cf206_isrForCrosstalkSources_996999_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/73d10808-8af1-4a5e-833b-4e510c4cf206_isrForCrosstalkSources_996999_16.stderr parsl.dataflow.dflow DEBUG: Task 2743 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2743 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2743 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2743 has memoization hash 1e118ddb0373183f0d6c294c7e49e812 parsl.dataflow.memoization INFO: Task 2743 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2743 try 0 launched on executor multi with executor id 2744 parsl.dataflow.dflow INFO: Standard output for task 2743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/09948192-1f6b-4c9b-a6d2-2835e80ce03b_isrForCrosstalkSources_997005_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/09948192-1f6b-4c9b-a6d2-2835e80ce03b_isrForCrosstalkSources_997005_21.stderr parsl.dataflow.dflow DEBUG: Task 2744 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2744 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2744 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2744 has memoization hash 3b377526c75a13cd4977bb1d14d690fe parsl.dataflow.memoization INFO: Task 2744 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2744 try 0 launched on executor multi with executor id 2745 parsl.dataflow.dflow INFO: Standard output for task 2744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d8517c4d-a561-47a4-b3f2-ebd6dafe87bd_isrForCrosstalkSources_996971_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d8517c4d-a561-47a4-b3f2-ebd6dafe87bd_isrForCrosstalkSources_996971_24.stderr parsl.dataflow.dflow DEBUG: Task 2745 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2745 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2745 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2745 has memoization hash 4130e5392ff65e6bcd73ae355f9e8e8f parsl.dataflow.memoization INFO: Task 2745 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2745 try 0 launched on executor multi with executor id 2746 parsl.dataflow.dflow INFO: Standard output for task 2745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/7e43c469-0b13-4b47-a039-c2217ef5eac0_isrForCrosstalkSources_997000_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/7e43c469-0b13-4b47-a039-c2217ef5eac0_isrForCrosstalkSources_997000_2.stderr parsl.dataflow.dflow DEBUG: Task 2746 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2746 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2746 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2746 has memoization hash 4b596a80cb782ed1003e5a7f74807d90 parsl.dataflow.memoization INFO: Task 2746 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2746 try 0 launched on executor multi with executor id 2747 parsl.dataflow.dflow INFO: Standard output for task 2746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/c3b23665-8dc2-40f5-9f27-a3f2fbd63d43_isrForCrosstalkSources_996964_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/c3b23665-8dc2-40f5-9f27-a3f2fbd63d43_isrForCrosstalkSources_996964_44.stderr parsl.dataflow.dflow DEBUG: Task 2747 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2747 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2747 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2747 has memoization hash fe4577e91a48a3a5bb6fbef87ff6b7c6 parsl.dataflow.memoization INFO: Task 2747 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2747 try 0 launched on executor multi with executor id 2748 parsl.dataflow.dflow INFO: Standard output for task 2747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e7102f36-e8b8-4f1f-b687-8577ab974bea_isrForCrosstalkSources_997033_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e7102f36-e8b8-4f1f-b687-8577ab974bea_isrForCrosstalkSources_997033_20.stderr parsl.dataflow.dflow DEBUG: Task 2748 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2748 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2748 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2748 has memoization hash 8b02c891cab416ecb2702ddaa47f2967 parsl.dataflow.memoization INFO: Task 2748 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2748 try 0 launched on executor multi with executor id 2749 parsl.dataflow.dflow INFO: Standard output for task 2748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/62198719-c0d6-4cb6-8a79-ae06b1a2df66_isrForCrosstalkSources_997003_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/62198719-c0d6-4cb6-8a79-ae06b1a2df66_isrForCrosstalkSources_997003_43.stderr parsl.dataflow.dflow DEBUG: Task 2749 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2749 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2749 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2749 has memoization hash f1cd897e365e5da0ef013a3e6245dc56 parsl.dataflow.memoization INFO: Task 2749 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2749 try 0 launched on executor multi with executor id 2750 parsl.dataflow.dflow INFO: Standard output for task 2749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6d4fd76e-1368-4851-a147-1b29a3e6028a_isrForCrosstalkSources_996957_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6d4fd76e-1368-4851-a147-1b29a3e6028a_isrForCrosstalkSources_996957_51.stderr parsl.dataflow.dflow DEBUG: Task 2750 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2750 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2750 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2750 has memoization hash b26977ea553afb01daf97d4095f77dc3 parsl.dataflow.memoization INFO: Task 2750 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2750 try 0 launched on executor multi with executor id 2751 parsl.dataflow.dflow INFO: Standard output for task 2750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5f5d6688-d8fb-476c-aeef-1abfeef189bf_isrForCrosstalkSources_996955_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5f5d6688-d8fb-476c-aeef-1abfeef189bf_isrForCrosstalkSources_996955_43.stderr parsl.dataflow.dflow DEBUG: Task 2751 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2751 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2751 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2751 has memoization hash 9f7f6befd9b881b86c9e11a374f4d7fc parsl.dataflow.memoization INFO: Task 2751 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2751 try 0 launched on executor multi with executor id 2752 parsl.dataflow.dflow INFO: Standard output for task 2751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/180fe417-d6ab-4562-bc09-0a4083589a24_isrForCrosstalkSources_997026_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/180fe417-d6ab-4562-bc09-0a4083589a24_isrForCrosstalkSources_997026_46.stderr parsl.dataflow.dflow DEBUG: Task 2752 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2752 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2752 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2752 has memoization hash 3af71f789396e53bbe2804451214ef92 parsl.dataflow.memoization INFO: Task 2752 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2752 try 0 launched on executor multi with executor id 2753 parsl.dataflow.dflow INFO: Standard output for task 2752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0c228d1d-0875-4cfa-9616-fd1a21fcdd6f_isrForCrosstalkSources_996980_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0c228d1d-0875-4cfa-9616-fd1a21fcdd6f_isrForCrosstalkSources_996980_4.stderr parsl.dataflow.dflow DEBUG: Task 2753 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2753 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2753 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2753 has memoization hash efdfcdd58c5f834fec5a49fc720eac42 parsl.dataflow.memoization INFO: Task 2753 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2753 try 0 launched on executor multi with executor id 2754 parsl.dataflow.dflow INFO: Standard output for task 2753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a9d773bf-5477-4483-a07b-7468093ba10a_isrForCrosstalkSources_996970_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a9d773bf-5477-4483-a07b-7468093ba10a_isrForCrosstalkSources_996970_20.stderr parsl.dataflow.dflow DEBUG: Task 2754 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2754 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2754 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2754 has memoization hash 1323a18e9e7047c54add2c00743985cf parsl.dataflow.memoization INFO: Task 2754 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca882c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2754 try 0 launched on executor multi with executor id 2755 parsl.dataflow.dflow INFO: Standard output for task 2754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ca0065b3-20fd-4db9-a76d-ae78612dd75f_isrForCrosstalkSources_996969_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ca0065b3-20fd-4db9-a76d-ae78612dd75f_isrForCrosstalkSources_996969_26.stderr parsl.dataflow.dflow DEBUG: Task 2755 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2755 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2755 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2755 has memoization hash dbe173eb9b6b4300df5e433a417867ea parsl.dataflow.memoization INFO: Task 2755 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca880e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2755 try 0 launched on executor multi with executor id 2756 parsl.dataflow.dflow INFO: Standard output for task 2755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0e6380ec-c176-4cc2-8bc0-444541cf5f49_isrForCrosstalkSources_997013_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0e6380ec-c176-4cc2-8bc0-444541cf5f49_isrForCrosstalkSources_997013_56.stderr parsl.dataflow.dflow DEBUG: Task 2756 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2756 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2756 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2756 has memoization hash 001c1b41f7a297a20679e8d94fcde4a0 parsl.dataflow.memoization INFO: Task 2756 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2756 try 0 launched on executor multi with executor id 2757 parsl.dataflow.dflow INFO: Standard output for task 2756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8dc0fba1-16c5-4773-937a-36af601e09a1_isrForCrosstalkSources_997032_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8dc0fba1-16c5-4773-937a-36af601e09a1_isrForCrosstalkSources_997032_32.stderr parsl.dataflow.dflow DEBUG: Task 2757 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2757 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2757 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2757 has memoization hash 4e0413f801a28b6a3cdeb4fe6c942d3a parsl.dataflow.memoization INFO: Task 2757 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2757 try 0 launched on executor multi with executor id 2758 parsl.dataflow.dflow INFO: Standard output for task 2757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fc99ac1e-85a6-4e0f-841b-2353732dd4ac_isrForCrosstalkSources_997008_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fc99ac1e-85a6-4e0f-841b-2353732dd4ac_isrForCrosstalkSources_997008_44.stderr parsl.dataflow.dflow DEBUG: Task 2758 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2758 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2758 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2758 has memoization hash 627412657227bceb5bdae67939cfdcfe parsl.dataflow.memoization INFO: Task 2758 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca884a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2758 try 0 launched on executor multi with executor id 2759 parsl.dataflow.dflow INFO: Standard output for task 2758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/01b47f1e-4529-4aaa-9594-58f5475b701c_isrForCrosstalkSources_997017_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/01b47f1e-4529-4aaa-9594-58f5475b701c_isrForCrosstalkSources_997017_7.stderr parsl.dataflow.dflow DEBUG: Task 2759 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2759 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2759 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2759 has memoization hash 1c892a66ab1bb17526267da55ae5a702 parsl.dataflow.memoization INFO: Task 2759 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca885e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2759 try 0 launched on executor multi with executor id 2760 parsl.dataflow.dflow INFO: Standard output for task 2759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ddd9ca66-29fe-47ae-b4eb-c29e4cc207de_isrForCrosstalkSources_996988_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ddd9ca66-29fe-47ae-b4eb-c29e4cc207de_isrForCrosstalkSources_996988_24.stderr parsl.dataflow.dflow DEBUG: Task 2760 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2760 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2760 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2760 has memoization hash 54e4c1db8592c713464b0a97bd471ac6 parsl.dataflow.memoization INFO: Task 2760 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2760 try 0 launched on executor multi with executor id 2761 parsl.dataflow.dflow INFO: Standard output for task 2760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e1f69863-7fbd-4e54-84c6-7048c8c1d2b9_isrForCrosstalkSources_997001_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e1f69863-7fbd-4e54-84c6-7048c8c1d2b9_isrForCrosstalkSources_997001_53.stderr parsl.dataflow.dflow DEBUG: Task 2761 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2761 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2761 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2761 has memoization hash 344319d79aa25bb4a946f4672421780a parsl.dataflow.memoization INFO: Task 2761 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2761 try 0 launched on executor multi with executor id 2762 parsl.dataflow.dflow INFO: Standard output for task 2761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6298f33-5797-4f3a-add9-c8747b0acba1_isrForCrosstalkSources_996976_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6298f33-5797-4f3a-add9-c8747b0acba1_isrForCrosstalkSources_996976_14.stderr parsl.dataflow.dflow DEBUG: Task 2762 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2762 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2762 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2762 has memoization hash a9525fc261cf499b1260417285406e5c parsl.dataflow.memoization INFO: Task 2762 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2762 try 0 launched on executor multi with executor id 2763 parsl.dataflow.dflow INFO: Standard output for task 2762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/bada876c-d775-4b4b-a2f2-f2953a786b18_isrForCrosstalkSources_996997_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/bada876c-d775-4b4b-a2f2-f2953a786b18_isrForCrosstalkSources_996997_12.stderr parsl.dataflow.dflow DEBUG: Task 2763 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2763 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2763 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2763 has memoization hash c687bcb6e75937ff6dd37d820792fc3e parsl.dataflow.memoization INFO: Task 2763 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca887c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2763 try 0 launched on executor multi with executor id 2764 parsl.dataflow.dflow INFO: Standard output for task 2763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fcf07496-8789-4b05-a97e-d54f7687ed38_isrForCrosstalkSources_997012_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fcf07496-8789-4b05-a97e-d54f7687ed38_isrForCrosstalkSources_997012_5.stderr parsl.dataflow.dflow DEBUG: Task 2764 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2764 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2764 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2764 has memoization hash 3061bb43590e51afc39f111566bfec91 parsl.dataflow.memoization INFO: Task 2764 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2764 try 0 launched on executor multi with executor id 2765 parsl.dataflow.dflow INFO: Standard output for task 2764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/74aaabf7-4766-4600-b4dc-a85b4d021585_isrForCrosstalkSources_997026_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/74aaabf7-4766-4600-b4dc-a85b4d021585_isrForCrosstalkSources_997026_48.stderr parsl.dataflow.dflow DEBUG: Task 2765 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2765 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2765 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2765 has memoization hash 45d2e36f1297d5a3e85a091e604ba168 parsl.dataflow.memoization INFO: Task 2765 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2765 try 0 launched on executor multi with executor id 2766 parsl.dataflow.dflow INFO: Standard output for task 2765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fb6cd9e1-2ec5-474a-b8e4-792a00120b1b_isrForCrosstalkSources_997008_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fb6cd9e1-2ec5-474a-b8e4-792a00120b1b_isrForCrosstalkSources_997008_27.stderr parsl.dataflow.dflow DEBUG: Task 2766 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2766 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2766 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2766 has memoization hash 69283de9c9f979259a819601893636a3 parsl.dataflow.memoization INFO: Task 2766 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2766 try 0 launched on executor multi with executor id 2767 parsl.dataflow.dflow INFO: Standard output for task 2766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3dfc5091-2d50-43f7-acda-5b3d79e7a5e5_isrForCrosstalkSources_997001_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3dfc5091-2d50-43f7-acda-5b3d79e7a5e5_isrForCrosstalkSources_997001_2.stderr parsl.dataflow.dflow DEBUG: Task 2767 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2767 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2767 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2767 has memoization hash fce09c6a6efde1c970de365148e59877 parsl.dataflow.memoization INFO: Task 2767 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca889a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2767 try 0 launched on executor multi with executor id 2768 parsl.dataflow.dflow INFO: Standard output for task 2767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f4ad5a95-c08e-44f8-a439-161f879b9dbc_isrForCrosstalkSources_996966_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f4ad5a95-c08e-44f8-a439-161f879b9dbc_isrForCrosstalkSources_996966_58.stderr parsl.dataflow.dflow DEBUG: Task 2768 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2768 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2768 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2768 has memoization hash 5209171d0d98fc7b548e9ff7a07e83fe parsl.dataflow.memoization INFO: Task 2768 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2768 try 0 launched on executor multi with executor id 2769 parsl.dataflow.dflow INFO: Standard output for task 2768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9a1dd9f6-14b0-4ba7-8519-74878b8d685c_isrForCrosstalkSources_996979_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9a1dd9f6-14b0-4ba7-8519-74878b8d685c_isrForCrosstalkSources_996979_3.stderr parsl.dataflow.dflow DEBUG: Task 2769 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2769 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2769 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2769 has memoization hash f0fc418e1900e0e685712650d94d1a24 parsl.dataflow.memoization INFO: Task 2769 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2769 try 0 launched on executor multi with executor id 2770 parsl.dataflow.dflow INFO: Standard output for task 2769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/7290379d-f020-4986-b3ec-d6efe8157972_isrForCrosstalkSources_996957_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/7290379d-f020-4986-b3ec-d6efe8157972_isrForCrosstalkSources_996957_12.stderr parsl.dataflow.dflow DEBUG: Task 2770 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2770 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2770 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2770 has memoization hash 706d7b58f17ff32dfe3191df0b90226f parsl.dataflow.memoization INFO: Task 2770 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2770 try 0 launched on executor multi with executor id 2771 parsl.dataflow.dflow INFO: Standard output for task 2770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fde6959b-c59f-4217-bb08-82cf397234c7_isrForCrosstalkSources_997002_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fde6959b-c59f-4217-bb08-82cf397234c7_isrForCrosstalkSources_997002_5.stderr parsl.dataflow.dflow DEBUG: Task 2771 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2771 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2771 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2771 has memoization hash 2ad6938befdca9d3a753bb8d8bf62af6 parsl.dataflow.memoization INFO: Task 2771 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2771 try 0 launched on executor multi with executor id 2772 parsl.dataflow.dflow INFO: Standard output for task 2771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/93dd7b97-ecd4-4d66-afc1-a284443c5a70_isrForCrosstalkSources_997008_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/93dd7b97-ecd4-4d66-afc1-a284443c5a70_isrForCrosstalkSources_997008_13.stderr parsl.dataflow.dflow DEBUG: Task 2772 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2772 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2772 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2772 has memoization hash 3af1e7f148e2daef9ff293aeea27b93d parsl.dataflow.memoization INFO: Task 2772 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca899e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2772 try 0 launched on executor multi with executor id 2773 parsl.dataflow.dflow INFO: Standard output for task 2772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/06302638-8e75-4ee2-b6f7-d3160659ceb6_isrForCrosstalkSources_996984_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/06302638-8e75-4ee2-b6f7-d3160659ceb6_isrForCrosstalkSources_996984_51.stderr parsl.dataflow.dflow DEBUG: Task 2773 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2773 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2773 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2773 has memoization hash e6af409b08a0b6b6a23ce5c6b0c5a5ae parsl.dataflow.memoization INFO: Task 2773 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2773 try 0 launched on executor multi with executor id 2774 parsl.dataflow.dflow INFO: Standard output for task 2773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/89b10738-4811-43ad-883a-066c26af2843_isrForCrosstalkSources_996997_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/89b10738-4811-43ad-883a-066c26af2843_isrForCrosstalkSources_996997_40.stderr parsl.dataflow.dflow DEBUG: Task 2774 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2774 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2774 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2774 has memoization hash d69c9255f70d729d8d5d0330468b6d1b parsl.dataflow.memoization INFO: Task 2774 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2774 try 0 launched on executor multi with executor id 2775 parsl.dataflow.dflow INFO: Standard output for task 2774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/2dd7a00a-1fdb-452b-8e11-32e888e49c6d_isrForCrosstalkSources_996975_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/2dd7a00a-1fdb-452b-8e11-32e888e49c6d_isrForCrosstalkSources_996975_48.stderr parsl.dataflow.dflow DEBUG: Task 2775 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2775 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2775 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2775 has memoization hash 1f88a95c1badda884c5a3ebf54c2671c parsl.dataflow.memoization INFO: Task 2775 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca898a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2775 try 0 launched on executor multi with executor id 2776 parsl.dataflow.dflow INFO: Standard output for task 2775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7e84250d-b7e5-4f44-8221-56171d3ccb92_isrForCrosstalkSources_996971_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7e84250d-b7e5-4f44-8221-56171d3ccb92_isrForCrosstalkSources_996971_16.stderr parsl.dataflow.dflow DEBUG: Task 2776 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2776 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2776 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2776 has memoization hash cd357c5e803b41e30d8389d602773f66 parsl.dataflow.memoization INFO: Task 2776 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2776 try 0 launched on executor multi with executor id 2777 parsl.dataflow.dflow INFO: Standard output for task 2776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ae8a9225-4ddc-4cd2-89aa-d4d8ca2edf20_isrForCrosstalkSources_996976_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ae8a9225-4ddc-4cd2-89aa-d4d8ca2edf20_isrForCrosstalkSources_996976_22.stderr parsl.dataflow.dflow DEBUG: Task 2777 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2777 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2777 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2777 has memoization hash d54930bf54e9f9de8fc0e627f7e29c5b parsl.dataflow.memoization INFO: Task 2777 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca896c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2777 try 0 launched on executor multi with executor id 2778 parsl.dataflow.dflow INFO: Standard output for task 2777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/eeb9fdb9-ee04-445a-aa81-2173dc06a2d4_isrForCrosstalkSources_997022_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/eeb9fdb9-ee04-445a-aa81-2173dc06a2d4_isrForCrosstalkSources_997022_53.stderr parsl.dataflow.dflow DEBUG: Task 2778 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2778 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2778 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2778 has memoization hash ecc68fd8b68ffe8cb54132cd2e40d886 parsl.dataflow.memoization INFO: Task 2778 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2778 try 0 launched on executor multi with executor id 2779 parsl.dataflow.dflow INFO: Standard output for task 2778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/3747aa24-0450-4ba1-9e20-1f0881a868cf_isrForCrosstalkSources_996976_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/3747aa24-0450-4ba1-9e20-1f0881a868cf_isrForCrosstalkSources_996976_2.stderr parsl.dataflow.dflow DEBUG: Task 2779 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2779 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2779 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2779 has memoization hash faf69495ac9800d675f31c8689319e37 parsl.dataflow.memoization INFO: Task 2779 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca894e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2779 try 0 launched on executor multi with executor id 2780 parsl.dataflow.dflow INFO: Standard output for task 2779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/84673b74-329e-4474-b660-e5311fc0a238_isrForCrosstalkSources_997029_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/84673b74-329e-4474-b660-e5311fc0a238_isrForCrosstalkSources_997029_50.stderr parsl.dataflow.dflow DEBUG: Task 2780 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2780 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2780 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2780 has memoization hash 578aae7e809141c60516591c08c89fcf parsl.dataflow.memoization INFO: Task 2780 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2780 try 0 launched on executor multi with executor id 2781 parsl.dataflow.dflow INFO: Standard output for task 2780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0589061b-d9de-471b-a2be-1e19f8d50d25_isrForCrosstalkSources_996958_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0589061b-d9de-471b-a2be-1e19f8d50d25_isrForCrosstalkSources_996958_61.stderr parsl.dataflow.dflow DEBUG: Task 2781 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2781 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2781 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2781 has memoization hash d16992d30d8e5254cda8b92889c64062 parsl.dataflow.memoization INFO: Task 2781 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2781 try 0 launched on executor multi with executor id 2782 parsl.dataflow.dflow INFO: Standard output for task 2781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/a385a6e7-cc7d-4cb6-8598-d69deefe602a_isrForCrosstalkSources_997016_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/a385a6e7-cc7d-4cb6-8598-d69deefe602a_isrForCrosstalkSources_997016_14.stderr parsl.dataflow.dflow DEBUG: Task 2782 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2782 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2782 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2782 has memoization hash 4f40b0631f74746c3168c3cadce29b33 parsl.dataflow.memoization INFO: Task 2782 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca893a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 10 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7034bfd87ee59cbb0819e7a07c3d51bd with result from task 10 parsl.dataflow.dflow DEBUG: Task 10 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 10 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5bcf6bcc-0b38-41c1-abb1-fb4a31357258_isrForCrosstalkSources_997017_12.stdout parsl.dataflow.dflow INFO: Standard error for task 10 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5bcf6bcc-0b38-41c1-abb1-fb4a31357258_isrForCrosstalkSources_997017_12.stderr parsl.dataflow.dflow INFO: Parsl task 2782 try 0 launched on executor multi with executor id 2783 parsl.dataflow.dflow INFO: Standard output for task 2782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/76fd61ba-5bc4-4b76-91fc-629fe212ea43_isrForCrosstalkSources_997018_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/76fd61ba-5bc4-4b76-91fc-629fe212ea43_isrForCrosstalkSources_997018_50.stderr parsl.dataflow.dflow DEBUG: Task 2783 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2783 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2783 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2783 has memoization hash 6051f94995306f625138d803c0e12c9d parsl.dataflow.memoization INFO: Task 2783 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2783 try 0 launched on executor multi with executor id 2784 parsl.dataflow.dflow INFO: Standard output for task 2783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6a9bc977-d840-4912-a8e1-4cef99537d6b_isrForCrosstalkSources_997006_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6a9bc977-d840-4912-a8e1-4cef99537d6b_isrForCrosstalkSources_997006_52.stderr parsl.dataflow.dflow DEBUG: Task 2784 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2784 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2784 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2784 has memoization hash 7b530ed491f3bcac52c55953646510ba parsl.dataflow.memoization INFO: Task 2784 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2784 try 0 launched on executor multi with executor id 2785 parsl.dataflow.dflow INFO: Standard output for task 2784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/51fb3834-996f-445b-a482-417ea9222335_isrForCrosstalkSources_997030_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/51fb3834-996f-445b-a482-417ea9222335_isrForCrosstalkSources_997030_19.stderr parsl.dataflow.dflow DEBUG: Task 2785 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2785 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2785 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2785 has memoization hash 0fb769181c926dd73771729a82f3c29e parsl.dataflow.memoization INFO: Task 2785 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2785 try 0 launched on executor multi with executor id 2786 parsl.dataflow.dflow INFO: Standard output for task 2785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/9f8df356-5868-4678-9a63-db8f7d32123e_isrForCrosstalkSources_997033_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/9f8df356-5868-4678-9a63-db8f7d32123e_isrForCrosstalkSources_997033_59.stderr parsl.dataflow.dflow DEBUG: Task 2786 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2786 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2786 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2786 has memoization hash 9b9747c878e3dac62f5be78a8b3bccfc parsl.dataflow.memoization INFO: Task 2786 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca891c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2786 try 0 launched on executor multi with executor id 2787 parsl.dataflow.dflow INFO: Standard output for task 2786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/795d9a79-d9b4-4ca0-aca6-2b1ae706e10b_isrForCrosstalkSources_996992_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/795d9a79-d9b4-4ca0-aca6-2b1ae706e10b_isrForCrosstalkSources_996992_28.stderr parsl.dataflow.dflow DEBUG: Task 2787 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2787 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2787 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2787 has memoization hash ba601f4f121ed54535e0a5c123195d38 parsl.dataflow.memoization INFO: Task 2787 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2787 try 0 launched on executor multi with executor id 2788 parsl.dataflow.dflow INFO: Standard output for task 2787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d0b65c7a-8a85-4497-8fbf-650dc87e2143_isrForCrosstalkSources_997000_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d0b65c7a-8a85-4497-8fbf-650dc87e2143_isrForCrosstalkSources_997000_28.stderr parsl.dataflow.dflow DEBUG: Task 2788 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2788 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2788 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2788 has memoization hash aab4f92cfc8a4758cedab5b2638aa0ba parsl.dataflow.memoization INFO: Task 2788 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2788 try 0 launched on executor multi with executor id 2789 parsl.dataflow.dflow INFO: Standard output for task 2788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ad9cea19-ce2b-44c4-8d89-6e4cdcfa1fea_isrForCrosstalkSources_996994_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ad9cea19-ce2b-44c4-8d89-6e4cdcfa1fea_isrForCrosstalkSources_996994_51.stderr parsl.dataflow.dflow DEBUG: Task 2789 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2789 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2789 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2789 has memoization hash d0647137c55b097fd830c3dd15e43ec3 parsl.dataflow.memoization INFO: Task 2789 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2789 try 0 launched on executor multi with executor id 2790 parsl.dataflow.dflow INFO: Standard output for task 2789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/ea7008d9-e15c-46c9-b9bd-cf0f94410fb7_isrForCrosstalkSources_997017_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/ea7008d9-e15c-46c9-b9bd-cf0f94410fb7_isrForCrosstalkSources_997017_46.stderr parsl.dataflow.dflow DEBUG: Task 2790 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2790 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2790 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2790 has memoization hash 70d943684594990f5c2ea4c0bf60a5b2 parsl.dataflow.memoization INFO: Task 2790 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2790 try 0 launched on executor multi with executor id 2791 parsl.dataflow.dflow INFO: Standard output for task 2790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/00f23fae-8122-4a87-b78b-48f88067f203_isrForCrosstalkSources_996996_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/00f23fae-8122-4a87-b78b-48f88067f203_isrForCrosstalkSources_996996_59.stderr parsl.dataflow.dflow DEBUG: Task 2791 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2791 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2791 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2791 has memoization hash f385f59a978e8e63f815ae2ae82abe55 parsl.dataflow.memoization INFO: Task 2791 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2791 try 0 launched on executor multi with executor id 2792 parsl.dataflow.dflow INFO: Standard output for task 2791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c6d13f08-d725-4dcd-aa39-537672c557a2_isrForCrosstalkSources_997013_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c6d13f08-d725-4dcd-aa39-537672c557a2_isrForCrosstalkSources_997013_47.stderr parsl.dataflow.dflow DEBUG: Task 2792 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2792 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2792 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2792 has memoization hash ce3b87983da0e53e5e32db025d23d7c8 parsl.dataflow.memoization INFO: Task 2792 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2792 try 0 launched on executor multi with executor id 2793 parsl.dataflow.dflow INFO: Standard output for task 2792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/2eaba9d2-77a6-4945-b2c3-9b37cef27c20_isrForCrosstalkSources_997001_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/2eaba9d2-77a6-4945-b2c3-9b37cef27c20_isrForCrosstalkSources_997001_17.stderr parsl.dataflow.dflow DEBUG: Task 2793 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2793 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2793 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2793 has memoization hash cd04ef7e30955a8974eb14f070aa0a28 parsl.dataflow.memoization INFO: Task 2793 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2793 try 0 launched on executor multi with executor id 2794 parsl.dataflow.dflow INFO: Standard output for task 2793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9dc3bd22-ccd1-4a74-a1ce-bcf5a292f29e_isrForCrosstalkSources_996984_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9dc3bd22-ccd1-4a74-a1ce-bcf5a292f29e_isrForCrosstalkSources_996984_21.stderr parsl.dataflow.dflow DEBUG: Task 2794 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2794 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2794 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2794 has memoization hash fac549a60a6990685f7d681efedaa825 parsl.dataflow.memoization INFO: Task 2794 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2794 try 0 launched on executor multi with executor id 2795 parsl.dataflow.dflow INFO: Standard output for task 2794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/344e9d0f-b558-481b-8093-4476711a0584_isrForCrosstalkSources_996991_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/344e9d0f-b558-481b-8093-4476711a0584_isrForCrosstalkSources_996991_46.stderr parsl.dataflow.dflow DEBUG: Task 2795 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2795 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2795 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2795 has memoization hash 77a0f827e5205c5fc4982277590df78c parsl.dataflow.memoization INFO: Task 2795 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2795 try 0 launched on executor multi with executor id 2796 parsl.dataflow.dflow INFO: Standard output for task 2795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/b964848f-0a85-430c-905e-7d6a3acfaddf_isrForCrosstalkSources_996970_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/b964848f-0a85-430c-905e-7d6a3acfaddf_isrForCrosstalkSources_996970_58.stderr parsl.dataflow.dflow DEBUG: Task 2796 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2796 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2796 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2796 has memoization hash 2bb97e7eef3b94e477bd0f35dc59b181 parsl.dataflow.memoization INFO: Task 2796 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca88a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2796 try 0 launched on executor multi with executor id 2797 parsl.dataflow.dflow INFO: Standard output for task 2796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/cde69ff8-7560-4eb6-9dbb-1d470c8bd091_isrForCrosstalkSources_996964_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/cde69ff8-7560-4eb6-9dbb-1d470c8bd091_isrForCrosstalkSources_996964_47.stderr parsl.dataflow.dflow DEBUG: Task 2797 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2797 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2797 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2797 has memoization hash 3d0230ce4cc1c4db536faded9b1d5d0c parsl.dataflow.memoization INFO: Task 2797 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2797 try 0 launched on executor multi with executor id 2798 parsl.dataflow.dflow INFO: Standard output for task 2797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/73bac0a0-24d6-4897-a51e-b5eda58281b4_isrForCrosstalkSources_996965_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/73bac0a0-24d6-4897-a51e-b5eda58281b4_isrForCrosstalkSources_996965_19.stderr parsl.dataflow.dflow DEBUG: Task 2798 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2798 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2798 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2798 has memoization hash 8a9e65c77dee021b02395898043745e8 parsl.dataflow.memoization INFO: Task 2798 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2798 try 0 launched on executor multi with executor id 2799 parsl.dataflow.dflow INFO: Standard output for task 2798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a246a335-1cdd-46cb-aa35-ad7cb32d9011_isrForCrosstalkSources_997008_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a246a335-1cdd-46cb-aa35-ad7cb32d9011_isrForCrosstalkSources_997008_59.stderr parsl.dataflow.dflow DEBUG: Task 2799 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2799 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2799 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2799 has memoization hash 436901567ae10a704f262a02e13baf16 parsl.dataflow.memoization INFO: Task 2799 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2799 try 0 launched on executor multi with executor id 2800 parsl.dataflow.dflow INFO: Standard output for task 2799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/49bc34a5-e473-465c-9cac-8ec26f8413ab_isrForCrosstalkSources_997013_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/49bc34a5-e473-465c-9cac-8ec26f8413ab_isrForCrosstalkSources_997013_3.stderr parsl.dataflow.dflow DEBUG: Task 2800 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2800 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2800 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2800 has memoization hash 6944ae5a20bbaeaabfec5c4d3a17ca28 parsl.dataflow.memoization INFO: Task 2800 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2800 try 0 launched on executor multi with executor id 2801 parsl.dataflow.dflow INFO: Standard output for task 2800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7afabad8-4875-4e13-8488-e1e4d23e55ed_isrForCrosstalkSources_997028_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7afabad8-4875-4e13-8488-e1e4d23e55ed_isrForCrosstalkSources_997028_13.stderr parsl.dataflow.dflow DEBUG: Task 2801 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2801 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2801 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2801 has memoization hash 58f190d3d1dd88081e7a7f35e84677c3 parsl.dataflow.memoization INFO: Task 2801 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2801 try 0 launched on executor multi with executor id 2802 parsl.dataflow.dflow INFO: Standard output for task 2801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/b9fca640-6e7b-4665-a8e5-19121e799350_isrForCrosstalkSources_996976_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/b9fca640-6e7b-4665-a8e5-19121e799350_isrForCrosstalkSources_996976_29.stderr parsl.dataflow.dflow DEBUG: Task 2802 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2802 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2802 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2802 has memoization hash b37e99199f07566516f23051d6047ef4 parsl.dataflow.memoization INFO: Task 2802 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2802 try 0 launched on executor multi with executor id 2803 parsl.dataflow.dflow INFO: Standard output for task 2802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ea36fbee-7a36-4a8f-8782-e1430997f1f9_isrForCrosstalkSources_996972_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ea36fbee-7a36-4a8f-8782-e1430997f1f9_isrForCrosstalkSources_996972_57.stderr parsl.dataflow.dflow DEBUG: Task 2803 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2803 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2803 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2803 has memoization hash b4a26fd134ffbfe2902380d6256fdf7b parsl.dataflow.memoization INFO: Task 2803 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2803 try 0 launched on executor multi with executor id 2804 parsl.dataflow.dflow INFO: Standard output for task 2803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cd60d8d3-9998-4128-a61b-ce1e30023d48_isrForCrosstalkSources_996957_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cd60d8d3-9998-4128-a61b-ce1e30023d48_isrForCrosstalkSources_996957_47.stderr parsl.dataflow.dflow DEBUG: Task 2804 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2804 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2804 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2804 has memoization hash 32dd83cd69e301ad98e70ed9502ea51f parsl.dataflow.memoization INFO: Task 2804 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2804 try 0 launched on executor multi with executor id 2805 parsl.dataflow.dflow INFO: Standard output for task 2804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0146f99c-03df-40ca-849b-bbe9ef43c06d_isrForCrosstalkSources_996966_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0146f99c-03df-40ca-849b-bbe9ef43c06d_isrForCrosstalkSources_996966_8.stderr parsl.dataflow.dflow DEBUG: Task 2805 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2805 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2805 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2805 has memoization hash 78c42576504f79173bf77707adfda7b1 parsl.dataflow.memoization INFO: Task 2805 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2805 try 0 launched on executor multi with executor id 2806 parsl.dataflow.dflow INFO: Standard output for task 2805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ff958221-a5c2-4896-8d23-4dc68fec81d2_isrForCrosstalkSources_996969_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ff958221-a5c2-4896-8d23-4dc68fec81d2_isrForCrosstalkSources_996969_8.stderr parsl.dataflow.dflow DEBUG: Task 2806 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2806 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2806 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2806 has memoization hash 834da497916f624c02566e86c523ece5 parsl.dataflow.memoization INFO: Task 2806 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2806 try 0 launched on executor multi with executor id 2807 parsl.dataflow.dflow INFO: Standard output for task 2806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7f7fde2b-b86d-46f5-b45e-c8446a7d0efe_isrForCrosstalkSources_997008_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7f7fde2b-b86d-46f5-b45e-c8446a7d0efe_isrForCrosstalkSources_997008_14.stderr parsl.dataflow.dflow DEBUG: Task 2807 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2807 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2807 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2807 has memoization hash f402b4a49583a03911f1ab300c4e51c9 parsl.dataflow.memoization INFO: Task 2807 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2807 try 0 launched on executor multi with executor id 2808 parsl.dataflow.dflow INFO: Standard output for task 2807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9052b222-f575-43e2-aab1-828f3d992794_isrForCrosstalkSources_997000_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9052b222-f575-43e2-aab1-828f3d992794_isrForCrosstalkSources_997000_51.stderr parsl.dataflow.dflow DEBUG: Task 2808 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2808 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2808 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2808 has memoization hash 7cb399f9229fc12aa0ee2e435d9898b6 parsl.dataflow.memoization INFO: Task 2808 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2808 try 0 launched on executor multi with executor id 2809 parsl.dataflow.dflow INFO: Standard output for task 2808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f1e2caa8-7242-4fa0-bf49-63c64503c99a_isrForCrosstalkSources_996956_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f1e2caa8-7242-4fa0-bf49-63c64503c99a_isrForCrosstalkSources_996956_26.stderr parsl.dataflow.dflow DEBUG: Task 2809 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2809 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2809 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2809 has memoization hash 485bb949d0da834c4c497483c7a7206c parsl.dataflow.memoization INFO: Task 2809 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2809 try 0 launched on executor multi with executor id 2810 parsl.dataflow.dflow INFO: Standard output for task 2809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/594fbdb2-bf1e-440d-a042-31f63ec5265a_isrForCrosstalkSources_996961_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/594fbdb2-bf1e-440d-a042-31f63ec5265a_isrForCrosstalkSources_996961_22.stderr parsl.dataflow.dflow DEBUG: Task 2810 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2810 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2810 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2810 has memoization hash a552668bf1daf17aa17887f86f0bb60b parsl.dataflow.memoization INFO: Task 2810 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2810 try 0 launched on executor multi with executor id 2811 parsl.dataflow.dflow INFO: Standard output for task 2810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b4166ea4-f80d-4cde-9267-16e6470b5fbe_isrForCrosstalkSources_997004_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b4166ea4-f80d-4cde-9267-16e6470b5fbe_isrForCrosstalkSources_997004_56.stderr parsl.dataflow.dflow DEBUG: Task 2811 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2811 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2811 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2811 has memoization hash 18ea5f3f2341c7b7a98f090434dd8e0b parsl.dataflow.memoization INFO: Task 2811 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2811 try 0 launched on executor multi with executor id 2812 parsl.dataflow.dflow INFO: Standard output for task 2811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ee30ae68-7c5e-4f53-b4d6-a9a6c80bcc07_isrForCrosstalkSources_996982_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ee30ae68-7c5e-4f53-b4d6-a9a6c80bcc07_isrForCrosstalkSources_996982_15.stderr parsl.dataflow.dflow DEBUG: Task 2812 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2812 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2812 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2812 has memoization hash ef1071854e0e33bfb9188c6532ecca9b parsl.dataflow.memoization INFO: Task 2812 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2812 try 0 launched on executor multi with executor id 2813 parsl.dataflow.dflow INFO: Standard output for task 2812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/3104352d-e608-42dd-8345-33e886562fec_isrForCrosstalkSources_996954_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/3104352d-e608-42dd-8345-33e886562fec_isrForCrosstalkSources_996954_41.stderr parsl.dataflow.dflow DEBUG: Task 2813 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2813 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2813 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2813 has memoization hash 0729006b52ef876228258564a005945f parsl.dataflow.memoization INFO: Task 2813 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8a980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2813 try 0 launched on executor multi with executor id 2814 parsl.dataflow.dflow INFO: Standard output for task 2813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2c9728c1-7f0a-44dc-be89-e08615f02005_isrForCrosstalkSources_996977_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2c9728c1-7f0a-44dc-be89-e08615f02005_isrForCrosstalkSources_996977_18.stderr parsl.dataflow.dflow DEBUG: Task 2814 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2814 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2814 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2814 has memoization hash 5c9b127c6e18e363569b4950483ed3ef parsl.dataflow.memoization INFO: Task 2814 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8aac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2814 try 0 launched on executor multi with executor id 2815 parsl.dataflow.dflow INFO: Standard output for task 2814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/be469688-d600-4e38-92dc-84d3f1ce5508_isrForCrosstalkSources_997017_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/be469688-d600-4e38-92dc-84d3f1ce5508_isrForCrosstalkSources_997017_24.stderr parsl.dataflow.dflow DEBUG: Task 2815 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2815 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2815 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2815 has memoization hash ab1aef9caebe71bdc37927e0c4503963 parsl.dataflow.memoization INFO: Task 2815 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8aa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2815 try 0 launched on executor multi with executor id 2816 parsl.dataflow.dflow INFO: Standard output for task 2815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e8c0a489-9fda-4e76-8194-5c72d39d5316_isrForCrosstalkSources_997010_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e8c0a489-9fda-4e76-8194-5c72d39d5316_isrForCrosstalkSources_997010_46.stderr parsl.dataflow.dflow DEBUG: Task 2816 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2816 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2816 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2816 has memoization hash 6e1d9a8a155fef1ac50b92bc90e31a4f parsl.dataflow.memoization INFO: Task 2816 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8ab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2816 try 0 launched on executor multi with executor id 2817 parsl.dataflow.dflow INFO: Standard output for task 2816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/cf650ffd-45d6-4704-a963-60a7b5ce0eb4_isrForCrosstalkSources_996969_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/cf650ffd-45d6-4704-a963-60a7b5ce0eb4_isrForCrosstalkSources_996969_53.stderr parsl.dataflow.dflow DEBUG: Task 2817 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2817 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2817 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2817 has memoization hash 6c01cb8595b0e42e6bf01ff8bcf9ba84 parsl.dataflow.memoization INFO: Task 2817 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8ac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2817 try 0 launched on executor multi with executor id 2818 parsl.dataflow.dflow INFO: Standard output for task 2817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/144029fa-a480-4c4c-aa18-0eed805a64a4_isrForCrosstalkSources_996992_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/144029fa-a480-4c4c-aa18-0eed805a64a4_isrForCrosstalkSources_996992_5.stderr parsl.dataflow.dflow DEBUG: Task 2818 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2818 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2818 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2818 has memoization hash 4d364dc9780a4fba85f1b5243337135a parsl.dataflow.memoization INFO: Task 2818 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8aca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2818 try 0 launched on executor multi with executor id 2819 parsl.dataflow.dflow INFO: Standard output for task 2818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f8f8091c-a2aa-4554-8fd0-fa5e43b49c2e_isrForCrosstalkSources_997016_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f8f8091c-a2aa-4554-8fd0-fa5e43b49c2e_isrForCrosstalkSources_997016_15.stderr parsl.dataflow.dflow DEBUG: Task 2819 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2819 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2819 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2819 has memoization hash 25fbd4e21f4a5fa118f42167c39d2fb3 parsl.dataflow.memoization INFO: Task 2819 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8ade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2819 try 0 launched on executor multi with executor id 2820 parsl.dataflow.dflow INFO: Standard output for task 2819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3eefe7dc-2ae2-4b05-bbbc-d1eb83666ed4_isrForCrosstalkSources_996969_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3eefe7dc-2ae2-4b05-bbbc-d1eb83666ed4_isrForCrosstalkSources_996969_12.stderr parsl.dataflow.dflow DEBUG: Task 2820 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2820 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2820 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2820 has memoization hash 0261985c9c1afd0419ff7b3024327aa6 parsl.dataflow.memoization INFO: Task 2820 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8ad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2820 try 0 launched on executor multi with executor id 2821 parsl.dataflow.dflow INFO: Standard output for task 2820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2c0ec059-31c6-4908-9bfd-f467b1a97ec4_isrForCrosstalkSources_996961_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2c0ec059-31c6-4908-9bfd-f467b1a97ec4_isrForCrosstalkSources_996961_55.stderr parsl.dataflow.dflow DEBUG: Task 2821 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2821 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2821 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2821 has memoization hash b269bfd1776323dcc1422a155bf6e4de parsl.dataflow.memoization INFO: Task 2821 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8ae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2821 try 0 launched on executor multi with executor id 2822 parsl.dataflow.dflow INFO: Standard output for task 2821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7791c427-db51-4a84-bbd5-596ab4e172d3_isrForCrosstalkSources_997015_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7791c427-db51-4a84-bbd5-596ab4e172d3_isrForCrosstalkSources_997015_33.stderr parsl.dataflow.dflow DEBUG: Task 2822 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2822 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2822 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2822 has memoization hash 3cf4fe82890250692de92793b3846b15 parsl.dataflow.memoization INFO: Task 2822 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8af20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2822 try 0 launched on executor multi with executor id 2823 parsl.dataflow.dflow INFO: Standard output for task 2822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d5b2bb89-aeb0-4486-a182-6f3ef66ea2b2_isrForCrosstalkSources_996993_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d5b2bb89-aeb0-4486-a182-6f3ef66ea2b2_isrForCrosstalkSources_996993_33.stderr parsl.dataflow.dflow DEBUG: Task 2823 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2823 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2823 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2823 has memoization hash 9989e35d7ff6c01176c10417f6631a26 parsl.dataflow.memoization INFO: Task 2823 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8afc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2823 try 0 launched on executor multi with executor id 2824 parsl.dataflow.dflow INFO: Standard output for task 2823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/c2179676-2c98-412f-92f8-6337b28e3d6f_isrForCrosstalkSources_996983_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/c2179676-2c98-412f-92f8-6337b28e3d6f_isrForCrosstalkSources_996983_1.stderr parsl.dataflow.dflow DEBUG: Task 2824 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2824 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2824 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2824 has memoization hash 6758d5dd18ea7842cc4c7681440b66fe parsl.dataflow.memoization INFO: Task 2824 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2824 try 0 launched on executor multi with executor id 2825 parsl.dataflow.dflow INFO: Standard output for task 2824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/34ce65a2-210f-4646-a7b2-78357a9114dc_isrForCrosstalkSources_996975_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/34ce65a2-210f-4646-a7b2-78357a9114dc_isrForCrosstalkSources_996975_40.stderr parsl.dataflow.dflow DEBUG: Task 2825 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2825 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2825 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2825 has memoization hash 6f5c158f2b26c8db7d00892aefa63722 parsl.dataflow.memoization INFO: Task 2825 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2825 try 0 launched on executor multi with executor id 2826 parsl.dataflow.dflow INFO: Standard output for task 2825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9a4710c0-c6ef-4b0c-a4fd-75e38645ee93_isrForCrosstalkSources_996968_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9a4710c0-c6ef-4b0c-a4fd-75e38645ee93_isrForCrosstalkSources_996968_35.stderr parsl.dataflow.dflow DEBUG: Task 2826 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2826 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2826 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2826 has memoization hash 552f6dfae671d0c138d68533a4ac382e parsl.dataflow.memoization INFO: Task 2826 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2826 try 0 launched on executor multi with executor id 2827 parsl.dataflow.dflow INFO: Standard output for task 2826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/6208dbf6-ba78-4896-8345-4b43a658825a_isrForCrosstalkSources_996992_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/6208dbf6-ba78-4896-8345-4b43a658825a_isrForCrosstalkSources_996992_6.stderr parsl.dataflow.dflow DEBUG: Task 2827 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2827 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2827 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2827 has memoization hash 6a129f392900a6aadb1989701d60fc74 parsl.dataflow.memoization INFO: Task 2827 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2827 try 0 launched on executor multi with executor id 2828 parsl.dataflow.dflow INFO: Standard output for task 2827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/000d4ea1-3e3c-4ef8-b2da-eac9ae1ccf63_isrForCrosstalkSources_996992_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/000d4ea1-3e3c-4ef8-b2da-eac9ae1ccf63_isrForCrosstalkSources_996992_45.stderr parsl.dataflow.dflow DEBUG: Task 2828 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2828 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2828 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2828 has memoization hash e50d364edf51f8b0d5d8546e649ee09b parsl.dataflow.memoization INFO: Task 2828 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2828 try 0 launched on executor multi with executor id 2829 parsl.dataflow.dflow INFO: Standard output for task 2828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/007a4596-b88b-4c02-8895-55beaf05e71b_isrForCrosstalkSources_997019_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/007a4596-b88b-4c02-8895-55beaf05e71b_isrForCrosstalkSources_997019_39.stderr parsl.dataflow.dflow DEBUG: Task 2829 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2829 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2829 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2829 has memoization hash 3fc955bd9b8452964e5bb73c6e1243a2 parsl.dataflow.memoization INFO: Task 2829 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2829 try 0 launched on executor multi with executor id 2830 parsl.dataflow.dflow INFO: Standard output for task 2829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/2e6a3bb8-d391-417c-bab9-61795d2ebb3f_isrForCrosstalkSources_996959_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/2e6a3bb8-d391-417c-bab9-61795d2ebb3f_isrForCrosstalkSources_996959_16.stderr parsl.dataflow.dflow DEBUG: Task 2830 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2830 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2830 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2830 has memoization hash 970a871a7115ad35f72619f297629c54 parsl.dataflow.memoization INFO: Task 2830 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2830 try 0 launched on executor multi with executor id 2831 parsl.dataflow.dflow INFO: Standard output for task 2830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6d36a060-b1e6-49fc-9ffa-fe1762def735_isrForCrosstalkSources_997006_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6d36a060-b1e6-49fc-9ffa-fe1762def735_isrForCrosstalkSources_997006_54.stderr parsl.dataflow.dflow DEBUG: Task 2831 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2831 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2831 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2831 has memoization hash 466676ee5f95ebac7369a2330bd5aa61 parsl.dataflow.memoization INFO: Task 2831 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2831 try 0 launched on executor multi with executor id 2832 parsl.dataflow.dflow INFO: Standard output for task 2831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b310dacb-a51b-415a-a393-1354369d3c36_isrForCrosstalkSources_997033_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b310dacb-a51b-415a-a393-1354369d3c36_isrForCrosstalkSources_997033_58.stderr parsl.dataflow.dflow DEBUG: Task 2832 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2832 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2832 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2832 has memoization hash 4109a575bfc014d41ae2bb6d458944a5 parsl.dataflow.memoization INFO: Task 2832 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2832 try 0 launched on executor multi with executor id 2833 parsl.dataflow.dflow INFO: Standard output for task 2832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e3cc12da-4671-4157-8dc3-49107cc1bc43_isrForCrosstalkSources_997028_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e3cc12da-4671-4157-8dc3-49107cc1bc43_isrForCrosstalkSources_997028_10.stderr parsl.dataflow.dflow DEBUG: Task 2833 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2833 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2833 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2833 has memoization hash 0222c8fb075f3ad0ca02bc810151b5ce parsl.dataflow.memoization INFO: Task 2833 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2833 try 0 launched on executor multi with executor id 2834 parsl.dataflow.dflow INFO: Standard output for task 2833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/3bf4ba14-50a3-4f2b-806c-e2ddbdddcdce_isrForCrosstalkSources_997022_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/3bf4ba14-50a3-4f2b-806c-e2ddbdddcdce_isrForCrosstalkSources_997022_6.stderr parsl.dataflow.dflow DEBUG: Task 2834 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2834 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2834 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2834 has memoization hash edbd735bf22df590be923516834c731f parsl.dataflow.memoization INFO: Task 2834 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2834 try 0 launched on executor multi with executor id 2835 parsl.dataflow.dflow INFO: Standard output for task 2834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/fbd6fd52-9e0d-4b2a-9586-b7f637666daf_isrForCrosstalkSources_997007_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/fbd6fd52-9e0d-4b2a-9586-b7f637666daf_isrForCrosstalkSources_997007_45.stderr parsl.dataflow.dflow DEBUG: Task 2835 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2835 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2835 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2835 has memoization hash ae80533c71cb9b72f4b0876866eaedbd parsl.dataflow.memoization INFO: Task 2835 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2835 try 0 launched on executor multi with executor id 2836 parsl.dataflow.dflow INFO: Standard output for task 2835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/546d477c-5271-480f-a584-934ad65006b2_isrForCrosstalkSources_996953_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/546d477c-5271-480f-a584-934ad65006b2_isrForCrosstalkSources_996953_59.stderr parsl.dataflow.dflow DEBUG: Task 2836 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2836 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2836 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2836 has memoization hash 7f3a4f9a45b376ebe710d8ee9115428d parsl.dataflow.memoization INFO: Task 2836 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2836 try 0 launched on executor multi with executor id 2837 parsl.dataflow.dflow INFO: Standard output for task 2836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8676e33b-6910-4987-86eb-88948aac7ae7_isrForCrosstalkSources_996952_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8676e33b-6910-4987-86eb-88948aac7ae7_isrForCrosstalkSources_996952_57.stderr parsl.dataflow.dflow DEBUG: Task 2837 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2837 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2837 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2837 has memoization hash a72d3cb13f2d40b4a005cad0dfdcc6a1 parsl.dataflow.memoization INFO: Task 2837 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2837 try 0 launched on executor multi with executor id 2838 parsl.dataflow.dflow INFO: Standard output for task 2837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/95f1616e-e8c3-4f8d-9f9e-e6af59b3cab4_isrForCrosstalkSources_996998_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/95f1616e-e8c3-4f8d-9f9e-e6af59b3cab4_isrForCrosstalkSources_996998_9.stderr parsl.dataflow.dflow DEBUG: Task 2838 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2838 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2838 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2838 has memoization hash e6da502a84ee129d51031aab28c11a2b parsl.dataflow.memoization INFO: Task 2838 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2838 try 0 launched on executor multi with executor id 2839 parsl.dataflow.dflow INFO: Standard output for task 2838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4d22b2cf-8f1b-4b2e-a998-9f61b5e46dbd_isrForCrosstalkSources_996986_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4d22b2cf-8f1b-4b2e-a998-9f61b5e46dbd_isrForCrosstalkSources_996986_31.stderr parsl.dataflow.dflow DEBUG: Task 2839 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2839 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2839 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2839 has memoization hash adacd6c06f53f905c9d04ae6716970e2 parsl.dataflow.memoization INFO: Task 2839 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8ba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2839 try 0 launched on executor multi with executor id 2840 parsl.dataflow.dflow INFO: Standard output for task 2839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/294da685-f945-4704-90a4-7274aeec8f21_isrForCrosstalkSources_996953_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/294da685-f945-4704-90a4-7274aeec8f21_isrForCrosstalkSources_996953_35.stderr parsl.dataflow.dflow DEBUG: Task 2840 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2840 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2840 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2840 has memoization hash cbb309b446a21e28d2d2155e0a7b4ed7 parsl.dataflow.memoization INFO: Task 2840 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8b9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2840 try 0 launched on executor multi with executor id 2841 parsl.dataflow.dflow INFO: Standard output for task 2840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/b108b748-696e-4f2f-935d-8d29a0184da7_isrForCrosstalkSources_996965_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/b108b748-696e-4f2f-935d-8d29a0184da7_isrForCrosstalkSources_996965_14.stderr parsl.dataflow.dflow DEBUG: Task 2841 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2841 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2841 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2841 has memoization hash 54e371ecba867a6988618232f8541447 parsl.dataflow.memoization INFO: Task 2841 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8bb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2841 try 0 launched on executor multi with executor id 2842 parsl.dataflow.dflow INFO: Standard output for task 2841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b9c6432c-2b5b-4a42-bd02-cb0e759ff511_isrForCrosstalkSources_997014_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b9c6432c-2b5b-4a42-bd02-cb0e759ff511_isrForCrosstalkSources_997014_4.stderr parsl.dataflow.dflow DEBUG: Task 2842 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2842 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2842 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2842 has memoization hash 2c23ddbabb0c6f6f8dc82e084de5a605 parsl.dataflow.memoization INFO: Task 2842 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8bba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2842 try 0 launched on executor multi with executor id 2843 parsl.dataflow.dflow INFO: Standard output for task 2842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/4735e082-169a-4b6c-ac70-c366dccf6be2_isrForCrosstalkSources_996983_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/4735e082-169a-4b6c-ac70-c366dccf6be2_isrForCrosstalkSources_996983_7.stderr parsl.dataflow.dflow DEBUG: Task 2843 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2843 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2843 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2843 has memoization hash f941a711b5f4a3eab69624c9de222ca7 parsl.dataflow.memoization INFO: Task 2843 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8bc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2843 try 0 launched on executor multi with executor id 2844 parsl.dataflow.dflow INFO: Standard output for task 2843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/54aa51a1-3c2e-4381-8eac-16065b18ab3a_isrForCrosstalkSources_996995_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/54aa51a1-3c2e-4381-8eac-16065b18ab3a_isrForCrosstalkSources_996995_12.stderr parsl.dataflow.dflow DEBUG: Task 2844 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2844 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2844 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2844 has memoization hash c1af92b33e50c7fc5d537476fd9d5cdc parsl.dataflow.memoization INFO: Task 2844 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8bd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2844 try 0 launched on executor multi with executor id 2845 parsl.dataflow.dflow INFO: Standard output for task 2844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/353923cc-d8e6-4f2b-b8fa-d799bc8bcdf5_isrForCrosstalkSources_997003_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/353923cc-d8e6-4f2b-b8fa-d799bc8bcdf5_isrForCrosstalkSources_997003_47.stderr parsl.dataflow.dflow DEBUG: Task 2845 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2845 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2845 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2845 has memoization hash ef0a12038773dd84eddbbeae278873b3 parsl.dataflow.memoization INFO: Task 2845 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8bce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2845 try 0 launched on executor multi with executor id 2846 parsl.dataflow.dflow INFO: Standard output for task 2845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/15293833-442d-4733-a076-8764f9e48dbf_isrForCrosstalkSources_996994_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/15293833-442d-4733-a076-8764f9e48dbf_isrForCrosstalkSources_996994_36.stderr parsl.dataflow.dflow DEBUG: Task 2846 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2846 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2846 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2846 has memoization hash 090d0a896288336de03ee08fd4e550e8 parsl.dataflow.memoization INFO: Task 2846 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8be20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2846 try 0 launched on executor multi with executor id 2847 parsl.dataflow.dflow INFO: Standard output for task 2846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/81bd8950-a8a6-42f8-9f34-ec8b7054af85_isrForCrosstalkSources_996957_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/81bd8950-a8a6-42f8-9f34-ec8b7054af85_isrForCrosstalkSources_996957_30.stderr parsl.dataflow.dflow DEBUG: Task 2847 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2847 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2847 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2847 has memoization hash c325e06a517290bb493e62e30220b447 parsl.dataflow.memoization INFO: Task 2847 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8bec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2847 try 0 launched on executor multi with executor id 2848 parsl.dataflow.dflow INFO: Standard output for task 2847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/196cba82-9e1a-47ba-abd9-eab40d470994_isrForCrosstalkSources_996980_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/196cba82-9e1a-47ba-abd9-eab40d470994_isrForCrosstalkSources_996980_34.stderr parsl.dataflow.dflow DEBUG: Task 2848 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2848 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2848 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2848 has memoization hash 53f2d3cfdec586152fdda366b672c624 parsl.dataflow.memoization INFO: Task 2848 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca8bf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2848 try 0 launched on executor multi with executor id 2849 parsl.dataflow.dflow INFO: Standard output for task 2848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/68fc4a28-1d21-4ac1-8c9d-ee02d906f14b_isrForCrosstalkSources_997021_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/68fc4a28-1d21-4ac1-8c9d-ee02d906f14b_isrForCrosstalkSources_997021_35.stderr parsl.dataflow.dflow DEBUG: Task 2849 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2849 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2849 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2849 has memoization hash 1edb84b678369c179022cfc860ed8320 parsl.dataflow.memoization INFO: Task 2849 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca89120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2849 try 0 launched on executor multi with executor id 2850 parsl.dataflow.dflow INFO: Standard output for task 2849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/761bfac8-fd31-486f-b742-8962ad5183e1_isrForCrosstalkSources_996958_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/761bfac8-fd31-486f-b742-8962ad5183e1_isrForCrosstalkSources_996958_35.stderr parsl.dataflow.dflow DEBUG: Task 2850 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2850 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2850 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2850 has memoization hash 2ff4e469ef57e6b4feee0ecda7225ee3 parsl.dataflow.memoization INFO: Task 2850 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2850 try 0 launched on executor multi with executor id 2851 parsl.dataflow.dflow INFO: Standard output for task 2850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/472ef291-56c8-4fa4-9332-31aa2d9c4a46_isrForCrosstalkSources_997026_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/472ef291-56c8-4fa4-9332-31aa2d9c4a46_isrForCrosstalkSources_997026_55.stderr parsl.dataflow.dflow DEBUG: Task 2851 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2851 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2851 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2851 has memoization hash a3d215860ff1afc24b1a1fbbcbb9f449 parsl.dataflow.memoization INFO: Task 2851 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2851 try 0 launched on executor multi with executor id 2852 parsl.dataflow.dflow INFO: Standard output for task 2851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/70c11f32-7018-42ff-afe1-0dc18882af6f_isrForCrosstalkSources_997019_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/70c11f32-7018-42ff-afe1-0dc18882af6f_isrForCrosstalkSources_997019_22.stderr parsl.dataflow.dflow DEBUG: Task 2852 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2852 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2852 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2852 has memoization hash 0f18c9faac81ad8561074024d0825b3f parsl.dataflow.memoization INFO: Task 2852 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2852 try 0 launched on executor multi with executor id 2853 parsl.dataflow.dflow INFO: Standard output for task 2852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/51b35427-f3e8-43a8-8b89-c3103bc22a30_isrForCrosstalkSources_997007_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/51b35427-f3e8-43a8-8b89-c3103bc22a30_isrForCrosstalkSources_997007_36.stderr parsl.dataflow.dflow DEBUG: Task 2853 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2853 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2853 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2853 has memoization hash 03987f5b94ae22e24600f42e6022ea06 parsl.dataflow.memoization INFO: Task 2853 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9382c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2853 try 0 launched on executor multi with executor id 2854 parsl.dataflow.dflow INFO: Standard output for task 2853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/5a6be999-5741-4a0c-b25f-162301207710_isrForCrosstalkSources_996983_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/5a6be999-5741-4a0c-b25f-162301207710_isrForCrosstalkSources_996983_12.stderr parsl.dataflow.dflow DEBUG: Task 2854 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2854 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2854 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2854 has memoization hash 11c2d68890bd4934c69c4278aded8d34 parsl.dataflow.memoization INFO: Task 2854 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2854 try 0 launched on executor multi with executor id 2855 parsl.dataflow.dflow INFO: Standard output for task 2854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/a26a73ae-e1d9-4fc0-b5a4-7895bdc7c964_isrForCrosstalkSources_996997_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/a26a73ae-e1d9-4fc0-b5a4-7895bdc7c964_isrForCrosstalkSources_996997_41.stderr parsl.dataflow.dflow DEBUG: Task 2855 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2855 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2855 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2855 has memoization hash 1d66a3c0d51d576b970a2af1d688be1c parsl.dataflow.memoization INFO: Task 2855 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2855 try 0 launched on executor multi with executor id 2856 parsl.dataflow.dflow INFO: Standard output for task 2855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/38caacbd-6102-45cd-8c8d-e2233bcf191b_isrForCrosstalkSources_996989_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/38caacbd-6102-45cd-8c8d-e2233bcf191b_isrForCrosstalkSources_996989_23.stderr parsl.dataflow.dflow DEBUG: Task 2856 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2856 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2856 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2856 has memoization hash 1ce982d27d4ceacc9028857bece750f0 parsl.dataflow.memoization INFO: Task 2856 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9384a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2856 try 0 launched on executor multi with executor id 2857 parsl.dataflow.dflow INFO: Standard output for task 2856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/bd6ca355-1875-425d-958f-64a37c2b347a_isrForCrosstalkSources_996990_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/bd6ca355-1875-425d-958f-64a37c2b347a_isrForCrosstalkSources_996990_24.stderr parsl.dataflow.dflow DEBUG: Task 2857 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2857 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2857 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2857 has memoization hash 6fda3ea0743dd71bed7990be1e41b0a5 parsl.dataflow.memoization INFO: Task 2857 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:52 proc_lsst.multi:170 [INFO] [multi] job_id 17074879 proc_lsst.multi INFO: [multi] job_id 17074879 2024-03-15 13:49:52 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:49:52 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:52 proc_lsst.multi:178 [INFO] [multi] provider astro accepted submit and returned 17074879 proc_lsst.multi INFO: [multi] provider astro accepted submit and returned 17074879 parsl.executors.status_handling DEBUG: Launched block 30 on executor multi with job ID 17074879 parsl.executors.status_handling INFO: Allocated block ID 31 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-31 2024-03-15 13:49:52 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-31 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-31 2024-03-15 13:49:52 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:52 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:49:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:52 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 0 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 0 2024-03-15 13:49:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:52 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=31 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 2857 try 0 launched on executor multi with executor id 2858 parsl.dataflow.dflow INFO: Standard output for task 2857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/58d22a18-9e2f-4a29-ae36-4a9e356e7635_isrForCrosstalkSources_997029_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/58d22a18-9e2f-4a29-ae36-4a9e356e7635_isrForCrosstalkSources_997029_60.stderr parsl.dataflow.dflow DEBUG: Task 2858 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2858 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2858 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2858 has memoization hash f5c60c2e978804978255b1fb1a10798a parsl.dataflow.memoization INFO: Task 2858 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2858 try 0 launched on executor multi with executor id 2859 parsl.dataflow.dflow INFO: Standard output for task 2858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/bcb85892-ef4c-4ffe-a8fc-8f2cbe694779_isrForCrosstalkSources_996993_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/bcb85892-ef4c-4ffe-a8fc-8f2cbe694779_isrForCrosstalkSources_996993_35.stderr parsl.dataflow.dflow DEBUG: Task 2859 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2859 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2859 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2859 has memoization hash d2d734b69f7d74724875112dcadf7633 parsl.dataflow.memoization INFO: Task 2859 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2859 try 0 launched on executor multi with executor id 2860 parsl.dataflow.dflow INFO: Standard output for task 2859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/64dc6861-944c-4986-a801-a1b21df3fb50_isrForCrosstalkSources_997023_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/64dc6861-944c-4986-a801-a1b21df3fb50_isrForCrosstalkSources_997023_17.stderr parsl.dataflow.dflow DEBUG: Task 2860 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2860 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2860 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2860 has memoization hash bf9a36531b8a7261e336c8a53833a4a3 parsl.dataflow.memoization INFO: Task 2860 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 2860 try 0 launched on executor multi with executor id 2861 parsl.dataflow.dflow INFO: Standard output for task 2860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3cd9f0e1-6d68-4e1e-adff-919e9fe5569c_isrForCrosstalkSources_997031_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3cd9f0e1-6d68-4e1e-adff-919e9fe5569c_isrForCrosstalkSources_997031_51.stderr parsl.dataflow.dflow DEBUG: Task 2861 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2861 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2861 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2861 has memoization hash f7c1a744ceb3ae93d37e5ba494e77100 parsl.dataflow.memoization INFO: Task 2861 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9385e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2861 try 0 launched on executor multi with executor id 2862 parsl.dataflow.dflow INFO: Standard output for task 2861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/84975943-1ff6-49d2-b5f6-0abcb6cc78fb_isrForCrosstalkSources_996955_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/84975943-1ff6-49d2-b5f6-0abcb6cc78fb_isrForCrosstalkSources_996955_53.stderr parsl.dataflow.dflow DEBUG: Task 2862 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2862 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2862 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2862 has memoization hash 339af58f17a8c911ec2677e1817a20f3 parsl.dataflow.memoization INFO: Task 2862 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2862 try 0 launched on executor multi with executor id 2863 parsl.dataflow.dflow INFO: Standard output for task 2862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/df15e4f0-8547-4820-918f-18c7864b0685_isrForCrosstalkSources_996986_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/df15e4f0-8547-4820-918f-18c7864b0685_isrForCrosstalkSources_996986_61.stderr parsl.dataflow.dflow DEBUG: Task 2863 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2863 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2863 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2863 has memoization hash 7c22d5313e658d40dbd5905a4f083bed parsl.dataflow.memoization INFO: Task 2863 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2863 try 0 launched on executor multi with executor id 2864 parsl.dataflow.dflow INFO: Standard output for task 2863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/21ffdaef-4a89-4225-9282-1464bdd9b68d_isrForCrosstalkSources_996984_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/21ffdaef-4a89-4225-9282-1464bdd9b68d_isrForCrosstalkSources_996984_55.stderr parsl.dataflow.dflow DEBUG: Task 2864 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2864 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2864 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2864 has memoization hash 3d5c1ee1394056c1ce711790fc6b6fba parsl.dataflow.memoization INFO: Task 2864 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2864 try 0 launched on executor multi with executor id 2865 parsl.dataflow.dflow INFO: Standard output for task 2864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a8e1e54a-706e-457d-989a-8bd3b5511bb2_isrForCrosstalkSources_997020_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a8e1e54a-706e-457d-989a-8bd3b5511bb2_isrForCrosstalkSources_997020_19.stderr parsl.dataflow.dflow DEBUG: Task 2865 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2865 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2865 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2865 has memoization hash 8d233af6b6ba67e159a9c28cd6adca48 parsl.dataflow.memoization INFO: Task 2865 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93bce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2865 try 0 launched on executor multi with executor id 2866 parsl.dataflow.dflow INFO: Standard output for task 2865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/6c5c5afc-c619-49a5-ad89-d73121b6fa83_isrForCrosstalkSources_997011_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/6c5c5afc-c619-49a5-ad89-d73121b6fa83_isrForCrosstalkSources_997011_32.stderr parsl.dataflow.dflow DEBUG: Task 2866 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2866 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2866 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2866 has memoization hash 54411aedecbe0155a52efd0c5fd2752e parsl.dataflow.memoization INFO: Task 2866 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2866 try 0 launched on executor multi with executor id 2867 parsl.dataflow.dflow INFO: Standard output for task 2866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/32fb4123-a4d8-4e5e-b0ac-0cf0ccb354cb_isrForCrosstalkSources_996953_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/32fb4123-a4d8-4e5e-b0ac-0cf0ccb354cb_isrForCrosstalkSources_996953_62.stderr parsl.dataflow.dflow DEBUG: Task 2867 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2867 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2867 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2867 has memoization hash 792aec35622ffe07bf764dc17ebc5f4b parsl.dataflow.memoization INFO: Task 2867 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93bd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2867 try 0 launched on executor multi with executor id 2868 parsl.dataflow.dflow INFO: Standard output for task 2867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/98547152-fdad-4d6a-85f6-490fe247dc54_isrForCrosstalkSources_996967_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/98547152-fdad-4d6a-85f6-490fe247dc54_isrForCrosstalkSources_996967_32.stderr parsl.dataflow.dflow DEBUG: Task 2868 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2868 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2868 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2868 has memoization hash 773c9f937e782ae4a77dd13c68adbbe4 parsl.dataflow.memoization INFO: Task 2868 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93bc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2868 try 0 launched on executor multi with executor id 2869 parsl.dataflow.dflow INFO: Standard output for task 2868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/b9ed2a27-fed9-4dca-aaff-5fc5b7355954_isrForCrosstalkSources_996965_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/b9ed2a27-fed9-4dca-aaff-5fc5b7355954_isrForCrosstalkSources_996965_27.stderr parsl.dataflow.dflow DEBUG: Task 2869 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2869 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2869 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2869 has memoization hash d73444ad55fee393a0eb32cb9b27774e parsl.dataflow.memoization INFO: Task 2869 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93bb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2869 try 0 launched on executor multi with executor id 2870 parsl.dataflow.dflow INFO: Standard output for task 2869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/9e69b7f6-d131-49a6-8a92-9e0575977a83_isrForCrosstalkSources_996996_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/9e69b7f6-d131-49a6-8a92-9e0575977a83_isrForCrosstalkSources_996996_11.stderr parsl.dataflow.dflow DEBUG: Task 2870 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2870 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2870 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2870 has memoization hash e354b12a4c6c439ec7af0dbd460cbfb6 parsl.dataflow.memoization INFO: Task 2870 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9389a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2870 try 0 launched on executor multi with executor id 2871 parsl.dataflow.dflow INFO: Standard output for task 2870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/45e48bc8-85fd-4239-b740-4bfafaa83600_isrForCrosstalkSources_996969_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/45e48bc8-85fd-4239-b740-4bfafaa83600_isrForCrosstalkSources_996969_48.stderr parsl.dataflow.dflow DEBUG: Task 2871 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2871 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2871 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2871 has memoization hash 41e89d9a58e73b7bae171812a3bc6adf parsl.dataflow.memoization INFO: Task 2871 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2871 try 0 launched on executor multi with executor id 2872 parsl.dataflow.dflow INFO: Standard output for task 2871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/1c6957ca-0e25-4b13-8bf7-6e07ad27ef3d_isrForCrosstalkSources_996980_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/1c6957ca-0e25-4b13-8bf7-6e07ad27ef3d_isrForCrosstalkSources_996980_25.stderr parsl.dataflow.dflow DEBUG: Task 2872 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2872 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2872 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2872 has memoization hash 4f40a01e52ec90794bc22e8a21bdd19d parsl.dataflow.memoization INFO: Task 2872 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93ba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2872 try 0 launched on executor multi with executor id 2873 parsl.dataflow.dflow INFO: Standard output for task 2872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/c0e870b7-0e31-4bac-8fb1-c07a565d6a4d_isrForCrosstalkSources_996988_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/c0e870b7-0e31-4bac-8fb1-c07a565d6a4d_isrForCrosstalkSources_996988_16.stderr parsl.dataflow.dflow DEBUG: Task 2873 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2873 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2873 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2873 has memoization hash baa102b5f7c579a6957a005d6c16d753 parsl.dataflow.memoization INFO: Task 2873 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2873 try 0 launched on executor multi with executor id 2874 parsl.dataflow.dflow INFO: Standard output for task 2873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3528f0cf-6506-4fc1-81dd-d952514849ce_isrForCrosstalkSources_996955_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3528f0cf-6506-4fc1-81dd-d952514849ce_isrForCrosstalkSources_996955_60.stderr parsl.dataflow.dflow DEBUG: Task 2874 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2874 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2874 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2874 has memoization hash c2401e277dcae74c819d9d5cf1e48db6 parsl.dataflow.memoization INFO: Task 2874 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2874 try 0 launched on executor multi with executor id 2875 parsl.dataflow.dflow INFO: Standard output for task 2874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/460b18e5-b238-444a-b5e5-a558f72c2588_isrForCrosstalkSources_996969_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/460b18e5-b238-444a-b5e5-a558f72c2588_isrForCrosstalkSources_996969_16.stderr parsl.dataflow.dflow DEBUG: Task 2875 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2875 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2875 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2875 has memoization hash 62a87642ca1c8313fd2aed199e9e53ef parsl.dataflow.memoization INFO: Task 2875 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2875 try 0 launched on executor multi with executor id 2876 parsl.dataflow.dflow INFO: Standard output for task 2875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/aa983a72-5840-4fbd-a5d8-45a2dd3dc9e4_isrForCrosstalkSources_996960_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/aa983a72-5840-4fbd-a5d8-45a2dd3dc9e4_isrForCrosstalkSources_996960_1.stderr parsl.dataflow.dflow DEBUG: Task 2876 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2876 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2876 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2876 has memoization hash 030f3b98eb340f5237e0a834ae79b47b parsl.dataflow.memoization INFO: Task 2876 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2876 try 0 launched on executor multi with executor id 2877 parsl.dataflow.dflow INFO: Standard output for task 2876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/2715cc49-889b-4337-938b-3111a5efae49_isrForCrosstalkSources_997025_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/2715cc49-889b-4337-938b-3111a5efae49_isrForCrosstalkSources_997025_38.stderr parsl.dataflow.dflow DEBUG: Task 2877 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2877 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2877 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2877 has memoization hash ff5467530ed8534f9f3814a60f7247a5 parsl.dataflow.memoization INFO: Task 2877 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2877 try 0 launched on executor multi with executor id 2878 parsl.dataflow.dflow INFO: Standard output for task 2877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/016824c8-e6cc-49bf-a866-ac3c2b21fa20_isrForCrosstalkSources_996969_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/016824c8-e6cc-49bf-a866-ac3c2b21fa20_isrForCrosstalkSources_996969_5.stderr parsl.dataflow.dflow DEBUG: Task 2878 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2878 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2878 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2878 has memoization hash ec4e9960c51f23bd211fbdded437d6b2 parsl.dataflow.memoization INFO: Task 2878 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2878 try 0 launched on executor multi with executor id 2879 parsl.dataflow.dflow INFO: Standard output for task 2878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/ad322615-2246-4e8a-ae11-2144da57d3b6_isrForCrosstalkSources_997032_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/ad322615-2246-4e8a-ae11-2144da57d3b6_isrForCrosstalkSources_997032_28.stderr parsl.dataflow.dflow DEBUG: Task 2879 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2879 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2879 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2879 has memoization hash 6aa853d564a4970f8015e46f48f0463c parsl.dataflow.memoization INFO: Task 2879 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2879 try 0 launched on executor multi with executor id 2880 parsl.dataflow.dflow INFO: Standard output for task 2879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a898564e-bd29-490a-a84f-a732751b7fbc_isrForCrosstalkSources_996993_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a898564e-bd29-490a-a84f-a732751b7fbc_isrForCrosstalkSources_996993_31.stderr parsl.dataflow.dflow DEBUG: Task 2880 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2880 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2880 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2880 has memoization hash 7bebe503391a43539352b7c6e92850d0 parsl.dataflow.memoization INFO: Task 2880 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9391c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2880 try 0 launched on executor multi with executor id 2881 parsl.dataflow.dflow INFO: Standard output for task 2880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/404ee456-4526-49c4-a05b-e5684a267a06_isrForCrosstalkSources_996976_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/404ee456-4526-49c4-a05b-e5684a267a06_isrForCrosstalkSources_996976_49.stderr parsl.dataflow.dflow DEBUG: Task 2881 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2881 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2881 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2881 has memoization hash 8cf15f6f6cc6d53ebb9b149049bc9d21 parsl.dataflow.memoization INFO: Task 2881 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2881 try 0 launched on executor multi with executor id 2882 parsl.dataflow.dflow INFO: Standard output for task 2881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/a1b09f11-b1d4-41a1-b8b5-0c4f62a5bb08_isrForCrosstalkSources_996985_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/a1b09f11-b1d4-41a1-b8b5-0c4f62a5bb08_isrForCrosstalkSources_996985_42.stderr parsl.dataflow.dflow DEBUG: Task 2882 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2882 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2882 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2882 has memoization hash 53d6dd31779878c04bff5fa174590bf2 parsl.dataflow.memoization INFO: Task 2882 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2882 try 0 launched on executor multi with executor id 2883 parsl.dataflow.dflow INFO: Standard output for task 2882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/67df462c-89b2-4a04-8f19-db8cbe9b780d_isrForCrosstalkSources_997023_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/67df462c-89b2-4a04-8f19-db8cbe9b780d_isrForCrosstalkSources_997023_26.stderr parsl.dataflow.dflow DEBUG: Task 2883 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2883 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2883 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2883 has memoization hash 659f3b6632323c25982b7e7cf90265b2 parsl.dataflow.memoization INFO: Task 2883 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2883 try 0 launched on executor multi with executor id 2884 parsl.dataflow.dflow INFO: Standard output for task 2883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/54f7eb22-59e6-4be4-9afc-7775ce6fe912_isrForCrosstalkSources_996959_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/54f7eb22-59e6-4be4-9afc-7775ce6fe912_isrForCrosstalkSources_996959_47.stderr parsl.dataflow.dflow DEBUG: Task 2884 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2884 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2884 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2884 has memoization hash 51f8510f8225e3b31782a1360ea25f97 parsl.dataflow.memoization INFO: Task 2884 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2884 try 0 launched on executor multi with executor id 2885 parsl.dataflow.dflow INFO: Standard output for task 2884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/398c8453-16d2-4550-9416-fa89a8f69dba_isrForCrosstalkSources_996981_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/398c8453-16d2-4550-9416-fa89a8f69dba_isrForCrosstalkSources_996981_24.stderr parsl.dataflow.dflow DEBUG: Task 2885 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2885 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2885 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2885 has memoization hash 8008012574cebbf5c8fa29e9fc3a1c77 parsl.dataflow.memoization INFO: Task 2885 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9394e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2885 try 0 launched on executor multi with executor id 2886 parsl.dataflow.dflow INFO: Standard output for task 2885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5e50a1b4-61e9-4e3f-871d-a90389ccd63c_isrForCrosstalkSources_997031_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5e50a1b4-61e9-4e3f-871d-a90389ccd63c_isrForCrosstalkSources_997031_13.stderr parsl.dataflow.dflow DEBUG: Task 2886 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2886 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2886 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2886 has memoization hash 5247f995f2a607c73597c74694b322a7 parsl.dataflow.memoization INFO: Task 2886 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9393a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2886 try 0 launched on executor multi with executor id 2887 parsl.dataflow.dflow INFO: Standard output for task 2886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/69d59daf-f7b0-4c3d-a2a1-b30bfef6664c_isrForCrosstalkSources_996956_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/69d59daf-f7b0-4c3d-a2a1-b30bfef6664c_isrForCrosstalkSources_996956_37.stderr parsl.dataflow.dflow DEBUG: Task 2887 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2887 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2887 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2887 has memoization hash aa6c72d24c1edc3ab26c900703130a59 parsl.dataflow.memoization INFO: Task 2887 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2887 try 0 launched on executor multi with executor id 2888 parsl.dataflow.dflow INFO: Standard output for task 2887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ff015dcb-4cce-4aaa-80e2-8524000672c7_isrForCrosstalkSources_996982_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ff015dcb-4cce-4aaa-80e2-8524000672c7_isrForCrosstalkSources_996982_57.stderr parsl.dataflow.dflow DEBUG: Task 2888 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2888 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2888 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2888 has memoization hash 7b8f965610a4609d3f0ec81e26131962 parsl.dataflow.memoization INFO: Task 2888 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2888 try 0 launched on executor multi with executor id 2889 parsl.dataflow.dflow INFO: Standard output for task 2888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ddfd41e1-42b7-4702-9548-7b7feb6a9018_isrForCrosstalkSources_996986_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ddfd41e1-42b7-4702-9548-7b7feb6a9018_isrForCrosstalkSources_996986_34.stderr parsl.dataflow.dflow DEBUG: Task 2889 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2889 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2889 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2889 has memoization hash a00849f477a790891b5fedb86d1572a4 parsl.dataflow.memoization INFO: Task 2889 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2889 try 0 launched on executor multi with executor id 2890 parsl.dataflow.dflow INFO: Standard output for task 2889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/f7ed49d4-1980-4834-8012-2619aa020684_isrForCrosstalkSources_997004_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/f7ed49d4-1980-4834-8012-2619aa020684_isrForCrosstalkSources_997004_57.stderr parsl.dataflow.dflow DEBUG: Task 2890 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2890 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2890 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2890 has memoization hash 27e73c797ba4dc2b35562344783853b8 parsl.dataflow.memoization INFO: Task 2890 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2890 try 0 launched on executor multi with executor id 2891 parsl.dataflow.dflow INFO: Standard output for task 2890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/adbc9df0-7930-4b00-a2d8-99a3a373f036_isrForCrosstalkSources_996994_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/adbc9df0-7930-4b00-a2d8-99a3a373f036_isrForCrosstalkSources_996994_62.stderr parsl.dataflow.dflow DEBUG: Task 2891 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2891 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2891 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2891 has memoization hash ae27760d3d2c4c28363f401be1797f41 parsl.dataflow.memoization INFO: Task 2891 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9396c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2891 try 0 launched on executor multi with executor id 2892 parsl.dataflow.dflow INFO: Standard output for task 2891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/b144cdf3-8180-49b2-9196-cbd1c308faae_isrForCrosstalkSources_997020_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/b144cdf3-8180-49b2-9196-cbd1c308faae_isrForCrosstalkSources_997020_28.stderr parsl.dataflow.dflow DEBUG: Task 2892 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2892 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2892 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2892 has memoization hash 8a49ac471029611ae44d179b088e425c parsl.dataflow.memoization INFO: Task 2892 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9398a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2892 try 0 launched on executor multi with executor id 2893 parsl.dataflow.dflow INFO: Standard output for task 2892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/07c78432-d744-4dd9-9af5-d367fe07d9ad_isrForCrosstalkSources_997032_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/07c78432-d744-4dd9-9af5-d367fe07d9ad_isrForCrosstalkSources_997032_3.stderr parsl.dataflow.dflow DEBUG: Task 2893 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2893 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2893 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2893 has memoization hash 37e5df953fd6d0497fbe639b8b8a9d22 parsl.dataflow.memoization INFO: Task 2893 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2893 try 0 launched on executor multi with executor id 2894 parsl.dataflow.dflow INFO: Standard output for task 2893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e296968c-2108-44c5-8972-ac351c63a21a_isrForCrosstalkSources_996955_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e296968c-2108-44c5-8972-ac351c63a21a_isrForCrosstalkSources_996955_22.stderr parsl.dataflow.dflow DEBUG: Task 2894 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2894 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2894 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2894 has memoization hash 6588ca6b0fb7b40eb66c897f8ec5f243 parsl.dataflow.memoization INFO: Task 2894 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2894 try 0 launched on executor multi with executor id 2895 parsl.dataflow.dflow INFO: Standard output for task 2894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/430af234-623f-43f6-800f-d18f71cda8c3_isrForCrosstalkSources_997014_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/430af234-623f-43f6-800f-d18f71cda8c3_isrForCrosstalkSources_997014_18.stderr parsl.dataflow.dflow DEBUG: Task 2895 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2895 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2895 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2895 has memoization hash c06e9fe4bed6c36bbcced307a8084069 parsl.dataflow.memoization INFO: Task 2895 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9399e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2895 try 0 launched on executor multi with executor id 2896 parsl.dataflow.dflow INFO: Standard output for task 2895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bc4cbcd9-898d-4093-9e32-30ec1735c3ca_isrForCrosstalkSources_996960_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bc4cbcd9-898d-4093-9e32-30ec1735c3ca_isrForCrosstalkSources_996960_12.stderr parsl.dataflow.dflow DEBUG: Task 2896 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2896 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2896 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2896 has memoization hash de47e409225523efad5cadc758bb0874 parsl.dataflow.memoization INFO: Task 2896 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2896 try 0 launched on executor multi with executor id 2897 parsl.dataflow.dflow INFO: Standard output for task 2896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8159cd99-79a8-437c-8416-de7ec5a6ac98_isrForCrosstalkSources_996987_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8159cd99-79a8-437c-8416-de7ec5a6ac98_isrForCrosstalkSources_996987_4.stderr parsl.dataflow.dflow DEBUG: Task 2897 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2897 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2897 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2897 has memoization hash f5da18f3b401cf188f618e55f0ebe20d parsl.dataflow.memoization INFO: Task 2897 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2897 try 0 launched on executor multi with executor id 2898 parsl.dataflow.dflow INFO: Standard output for task 2897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/177edfb3-95aa-478b-ba6e-eb8df23f0c3f_isrForCrosstalkSources_997010_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/177edfb3-95aa-478b-ba6e-eb8df23f0c3f_isrForCrosstalkSources_997010_17.stderr parsl.dataflow.dflow DEBUG: Task 2898 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2898 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2898 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2898 has memoization hash 291b4f5d391ca6205ee0049117d8b4d5 parsl.dataflow.memoization INFO: Task 2898 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2898 try 0 launched on executor multi with executor id 2899 parsl.dataflow.dflow INFO: Standard output for task 2898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/a93a4ca0-0e11-4315-ada0-9a469480afd0_isrForCrosstalkSources_996962_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/a93a4ca0-0e11-4315-ada0-9a469480afd0_isrForCrosstalkSources_996962_16.stderr parsl.dataflow.dflow DEBUG: Task 2899 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2899 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2899 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2899 has memoization hash df51685c14efbe62f6322c87bf1b4773 parsl.dataflow.memoization INFO: Task 2899 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2899 try 0 launched on executor multi with executor id 2900 parsl.dataflow.dflow INFO: Standard output for task 2899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/74cb9618-e6c2-4682-8556-42b8d406947b_isrForCrosstalkSources_996979_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/74cb9618-e6c2-4682-8556-42b8d406947b_isrForCrosstalkSources_996979_33.stderr parsl.dataflow.dflow DEBUG: Task 2900 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2900 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2900 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2900 has memoization hash d4ea9e344c17c9cc0cfd559eac1ce120 parsl.dataflow.memoization INFO: Task 2900 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2900 try 0 launched on executor multi with executor id 2901 parsl.dataflow.dflow INFO: Standard output for task 2900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/f992f867-009c-44be-a67f-5e77316f227a_isrForCrosstalkSources_996998_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/f992f867-009c-44be-a67f-5e77316f227a_isrForCrosstalkSources_996998_34.stderr parsl.dataflow.dflow DEBUG: Task 2901 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2901 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2901 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2901 has memoization hash d388d3568050b0114c5f77d46bb2a0a7 parsl.dataflow.memoization INFO: Task 2901 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2901 try 0 launched on executor multi with executor id 2902 parsl.dataflow.dflow INFO: Standard output for task 2901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/77b3c704-cf5b-46ae-aa08-d481a89a931d_isrForCrosstalkSources_996999_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/77b3c704-cf5b-46ae-aa08-d481a89a931d_isrForCrosstalkSources_996999_38.stderr parsl.dataflow.dflow DEBUG: Task 2902 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2902 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2902 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2902 has memoization hash d54607ec0bf51b544b79d623125ffba7 parsl.dataflow.memoization INFO: Task 2902 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2902 try 0 launched on executor multi with executor id 2903 parsl.dataflow.dflow INFO: Standard output for task 2902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/46a05853-9e64-437b-970a-9f24d3262cd4_isrForCrosstalkSources_997011_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/46a05853-9e64-437b-970a-9f24d3262cd4_isrForCrosstalkSources_997011_44.stderr parsl.dataflow.dflow DEBUG: Task 2903 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2903 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2903 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2903 has memoization hash 2944e3b294ecfb0da9970edde83a9720 parsl.dataflow.memoization INFO: Task 2903 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2903 try 0 launched on executor multi with executor id 2904 parsl.dataflow.dflow INFO: Standard output for task 2903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ada51573-b39a-43a9-b66e-66c8dc8e2b4f_isrForCrosstalkSources_996978_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ada51573-b39a-43a9-b66e-66c8dc8e2b4f_isrForCrosstalkSources_996978_13.stderr parsl.dataflow.dflow DEBUG: Task 2904 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2904 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2904 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2904 has memoization hash 1338e1484eb33b4203d83ff87c718813 parsl.dataflow.memoization INFO: Task 2904 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c939ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2904 try 0 launched on executor multi with executor id 2905 parsl.dataflow.dflow INFO: Standard output for task 2904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/18599bbd-7a14-488b-bceb-e2749a2d046e_isrForCrosstalkSources_997006_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/18599bbd-7a14-488b-bceb-e2749a2d046e_isrForCrosstalkSources_997006_42.stderr parsl.dataflow.dflow DEBUG: Task 2905 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2905 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2905 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2905 has memoization hash c8b1bc111c073d78c375098445a7d315 parsl.dataflow.memoization INFO: Task 2905 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2905 try 0 launched on executor multi with executor id 2906 parsl.dataflow.dflow INFO: Standard output for task 2905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/baa2ee2d-40b1-4731-b328-ab721fcf8f21_isrForCrosstalkSources_997004_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/baa2ee2d-40b1-4731-b328-ab721fcf8f21_isrForCrosstalkSources_997004_27.stderr parsl.dataflow.dflow DEBUG: Task 2906 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2906 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2906 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2906 has memoization hash 2edba1b97d4a941c528c6dd88f86ba29 parsl.dataflow.memoization INFO: Task 2906 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2906 try 0 launched on executor multi with executor id 2907 parsl.dataflow.dflow INFO: Standard output for task 2906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/933f54bc-0ee3-4629-8b3f-ca3ff201976c_isrForCrosstalkSources_997024_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/933f54bc-0ee3-4629-8b3f-ca3ff201976c_isrForCrosstalkSources_997024_31.stderr parsl.dataflow.dflow DEBUG: Task 2907 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2907 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2907 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2907 has memoization hash 2951dba98378d765e1e7206473e749b7 parsl.dataflow.memoization INFO: Task 2907 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2907 try 0 launched on executor multi with executor id 2908 parsl.dataflow.dflow INFO: Standard output for task 2907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ff4a8135-d2c5-40ce-a5be-46c4807fc22b_isrForCrosstalkSources_996989_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ff4a8135-d2c5-40ce-a5be-46c4807fc22b_isrForCrosstalkSources_996989_2.stderr parsl.dataflow.dflow DEBUG: Task 2908 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2908 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2908 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2908 has memoization hash b1e86007b26668dd55d5a60e71f6b106 parsl.dataflow.memoization INFO: Task 2908 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2908 try 0 launched on executor multi with executor id 2909 parsl.dataflow.dflow INFO: Standard output for task 2908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/83e0a3cb-8b30-4078-9d15-e769011f44ff_isrForCrosstalkSources_996954_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/83e0a3cb-8b30-4078-9d15-e769011f44ff_isrForCrosstalkSources_996954_26.stderr parsl.dataflow.dflow DEBUG: Task 2909 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2909 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2909 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2909 has memoization hash 621e2b42134ab12a06e6f7d134684e46 parsl.dataflow.memoization INFO: Task 2909 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2909 try 0 launched on executor multi with executor id 2910 parsl.dataflow.dflow INFO: Standard output for task 2909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e2c08f50-5d6b-4ca1-b6ee-0df6bccee3a0_isrForCrosstalkSources_997019_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e2c08f50-5d6b-4ca1-b6ee-0df6bccee3a0_isrForCrosstalkSources_997019_2.stderr parsl.dataflow.dflow DEBUG: Task 2910 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2910 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2910 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2910 has memoization hash 5761af31c7e4f127c2663e80aa2e3ca5 parsl.dataflow.memoization INFO: Task 2910 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2910 try 0 launched on executor multi with executor id 2911 parsl.dataflow.dflow INFO: Standard output for task 2910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ec33553f-3450-4aa8-b976-d243a6577371_isrForCrosstalkSources_997015_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ec33553f-3450-4aa8-b976-d243a6577371_isrForCrosstalkSources_997015_7.stderr parsl.dataflow.dflow DEBUG: Task 2911 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2911 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2911 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2911 has memoization hash 21a3fdf5a250178738cd3a5b5da6722d parsl.dataflow.memoization INFO: Task 2911 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2911 try 0 launched on executor multi with executor id 2912 parsl.dataflow.dflow INFO: Standard output for task 2911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a4987a81-1dcd-4b92-9804-91ba6df6ec36_isrForCrosstalkSources_996956_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a4987a81-1dcd-4b92-9804-91ba6df6ec36_isrForCrosstalkSources_996956_36.stderr parsl.dataflow.dflow DEBUG: Task 2912 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2912 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2912 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2912 has memoization hash f9488b8add8aa78f66f597d9d831195d parsl.dataflow.memoization INFO: Task 2912 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2912 try 0 launched on executor multi with executor id 2913 parsl.dataflow.dflow INFO: Standard output for task 2912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/00644a80-af0e-4674-983f-78c18e1f1552_isrForCrosstalkSources_996962_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/00644a80-af0e-4674-983f-78c18e1f1552_isrForCrosstalkSources_996962_42.stderr parsl.dataflow.dflow DEBUG: Task 2913 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2913 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2913 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2913 has memoization hash 0d653fd1723abd1e7e9d2c7f8d24a338 parsl.dataflow.memoization INFO: Task 2913 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2913 try 0 launched on executor multi with executor id 2914 parsl.dataflow.dflow INFO: Standard output for task 2913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d2276757-709c-4f0c-929d-f0c7f13fd065_isrForCrosstalkSources_996965_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d2276757-709c-4f0c-929d-f0c7f13fd065_isrForCrosstalkSources_996965_33.stderr parsl.dataflow.dflow DEBUG: Task 2914 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2914 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2914 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2914 has memoization hash fceca0e18408a0a20620f53a6cc24f37 parsl.dataflow.memoization INFO: Task 2914 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2914 try 0 launched on executor multi with executor id 2915 parsl.dataflow.dflow INFO: Standard output for task 2914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/edc974c1-fc9c-44c8-954f-9f7e3de2a281_isrForCrosstalkSources_996958_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/edc974c1-fc9c-44c8-954f-9f7e3de2a281_isrForCrosstalkSources_996958_36.stderr parsl.dataflow.dflow DEBUG: Task 2915 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2915 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2915 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2915 has memoization hash 5a2c081e8ca5dd2603ad96e42e0757fc parsl.dataflow.memoization INFO: Task 2915 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2915 try 0 launched on executor multi with executor id 2916 parsl.dataflow.dflow INFO: Standard output for task 2915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/199053dc-5b7a-478c-9875-8a6fda4a8c9d_isrForCrosstalkSources_996968_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/199053dc-5b7a-478c-9875-8a6fda4a8c9d_isrForCrosstalkSources_996968_24.stderr parsl.dataflow.dflow DEBUG: Task 2916 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2916 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2916 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2916 has memoization hash ac8387780ee1c71ff1331359f7e4e484 parsl.dataflow.memoization INFO: Task 2916 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2916 try 0 launched on executor multi with executor id 2917 parsl.dataflow.dflow INFO: Standard output for task 2916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/505e72f3-8f0a-46fe-b614-c08c1f469103_isrForCrosstalkSources_996987_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/505e72f3-8f0a-46fe-b614-c08c1f469103_isrForCrosstalkSources_996987_38.stderr parsl.dataflow.dflow DEBUG: Task 2917 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2917 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2917 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2917 has memoization hash ea4971aa6e284a68b709ebb1847eafff parsl.dataflow.memoization INFO: Task 2917 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2917 try 0 launched on executor multi with executor id 2918 parsl.dataflow.dflow INFO: Standard output for task 2917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a934996c-d7cd-4178-a803-a1d2cc28bd1f_isrForCrosstalkSources_996956_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a934996c-d7cd-4178-a803-a1d2cc28bd1f_isrForCrosstalkSources_996956_43.stderr parsl.dataflow.dflow DEBUG: Task 2918 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2918 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2918 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2918 has memoization hash 8981d19f6eb0908f0f6077fe3e96c1d7 parsl.dataflow.memoization INFO: Task 2918 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2918 try 0 launched on executor multi with executor id 2919 parsl.dataflow.dflow INFO: Standard output for task 2918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/cd6e702e-5598-4311-a31f-659f52d1b054_isrForCrosstalkSources_997007_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/cd6e702e-5598-4311-a31f-659f52d1b054_isrForCrosstalkSources_997007_50.stderr parsl.dataflow.dflow DEBUG: Task 2919 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2919 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2919 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2919 has memoization hash 3a0ace76b9cfb9d27893ac716ea2402e parsl.dataflow.memoization INFO: Task 2919 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2919 try 0 launched on executor multi with executor id 2920 parsl.dataflow.dflow INFO: Standard output for task 2919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/abb611a2-7362-4a40-82f9-f5fd643cf6f6_isrForCrosstalkSources_997005_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/abb611a2-7362-4a40-82f9-f5fd643cf6f6_isrForCrosstalkSources_997005_30.stderr parsl.dataflow.dflow DEBUG: Task 2920 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2920 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2920 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2920 has memoization hash 9196ab1154569ac9268e6b2832d55848 parsl.dataflow.memoization INFO: Task 2920 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93be20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2920 try 0 launched on executor multi with executor id 2921 parsl.dataflow.dflow INFO: Standard output for task 2920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/49f24e6e-724b-4f18-9c03-02279f27d04f_isrForCrosstalkSources_996979_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/49f24e6e-724b-4f18-9c03-02279f27d04f_isrForCrosstalkSources_996979_1.stderr parsl.dataflow.dflow DEBUG: Task 2921 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2921 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2921 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2921 has memoization hash 63acbd83dd52349387386f0f37c76491 parsl.dataflow.memoization INFO: Task 2921 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93aa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2921 try 0 launched on executor multi with executor id 2922 parsl.dataflow.dflow INFO: Standard output for task 2921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/cbd947ee-9ed1-40fd-a64c-391c72c7c10b_isrForCrosstalkSources_997029_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/cbd947ee-9ed1-40fd-a64c-391c72c7c10b_isrForCrosstalkSources_997029_37.stderr parsl.dataflow.dflow DEBUG: Task 2922 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2922 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2922 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2922 has memoization hash 021e3a3fec33810f843289e51e7f9236 parsl.dataflow.memoization INFO: Task 2922 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93bba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2922 try 0 launched on executor multi with executor id 2923 parsl.dataflow.dflow INFO: Standard output for task 2922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/46b75174-985b-471d-a143-9f8c3b731039_isrForCrosstalkSources_996984_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/46b75174-985b-471d-a143-9f8c3b731039_isrForCrosstalkSources_996984_36.stderr parsl.dataflow.dflow DEBUG: Task 2923 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2923 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2923 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2923 has memoization hash 3f66ab91aaded5a002dc8e9601e10ef0 parsl.dataflow.memoization INFO: Task 2923 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2923 try 0 launched on executor multi with executor id 2924 parsl.dataflow.dflow INFO: Standard output for task 2923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/25621a10-68e8-4f6e-8ff3-686ad8cf4dbc_isrForCrosstalkSources_997006_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/25621a10-68e8-4f6e-8ff3-686ad8cf4dbc_isrForCrosstalkSources_997006_34.stderr parsl.dataflow.dflow DEBUG: Task 2924 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2924 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2924 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2924 has memoization hash 0917ec1cf83f8639e2d6aabeb0788576 parsl.dataflow.memoization INFO: Task 2924 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2924 try 0 launched on executor multi with executor id 2925 parsl.dataflow.dflow INFO: Standard output for task 2924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9e691935-3518-4313-9d09-371b0f5d307c_isrForCrosstalkSources_996968_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9e691935-3518-4313-9d09-371b0f5d307c_isrForCrosstalkSources_996968_57.stderr parsl.dataflow.dflow DEBUG: Task 2925 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2925 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2925 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2925 has memoization hash 08949a5cf1b329a62d55556f7180ca49 parsl.dataflow.memoization INFO: Task 2925 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2925 try 0 launched on executor multi with executor id 2926 parsl.dataflow.dflow INFO: Standard output for task 2925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/098becc5-a9cf-4d93-bef9-e8c2839b5cb9_isrForCrosstalkSources_997020_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/098becc5-a9cf-4d93-bef9-e8c2839b5cb9_isrForCrosstalkSources_997020_52.stderr parsl.dataflow.dflow DEBUG: Task 2926 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2926 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2926 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2926 has memoization hash d211c9d107a4d8083908e7bedb3078c8 parsl.dataflow.memoization INFO: Task 2926 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2926 try 0 launched on executor multi with executor id 2927 parsl.dataflow.dflow INFO: Standard output for task 2926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/d751e76a-c671-4f1c-8d1b-4baa46865040_isrForCrosstalkSources_997031_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/d751e76a-c671-4f1c-8d1b-4baa46865040_isrForCrosstalkSources_997031_37.stderr parsl.dataflow.dflow DEBUG: Task 2927 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2927 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2927 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2927 has memoization hash 5e2db0eca09259024757577ba2978923 parsl.dataflow.memoization INFO: Task 2927 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2927 try 0 launched on executor multi with executor id 2928 parsl.dataflow.dflow INFO: Standard output for task 2927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/2c26d667-a37b-452d-bec0-5600ef579fe4_isrForCrosstalkSources_996994_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/2c26d667-a37b-452d-bec0-5600ef579fe4_isrForCrosstalkSources_996994_45.stderr parsl.dataflow.dflow DEBUG: Task 2928 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2928 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2928 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2928 has memoization hash a080b2721f332bf3f6617a47a851dc7b parsl.dataflow.memoization INFO: Task 2928 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2928 try 0 launched on executor multi with executor id 2929 parsl.dataflow.dflow INFO: Standard output for task 2928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/8bbbe872-e6a2-4d64-9e55-ffb5c184de39_isrForCrosstalkSources_996962_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/8bbbe872-e6a2-4d64-9e55-ffb5c184de39_isrForCrosstalkSources_996962_28.stderr parsl.dataflow.dflow DEBUG: Task 2929 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2929 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2929 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2929 has memoization hash 0b47b2a5a710f24eb9ca661295590b53 parsl.dataflow.memoization INFO: Task 2929 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2929 try 0 launched on executor multi with executor id 2930 parsl.dataflow.dflow INFO: Standard output for task 2929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/5f79c0fa-f93c-4bae-a8b2-7775cd60ebff_isrForCrosstalkSources_997014_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/5f79c0fa-f93c-4bae-a8b2-7775cd60ebff_isrForCrosstalkSources_997014_55.stderr parsl.dataflow.dflow DEBUG: Task 2930 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2930 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2930 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2930 has memoization hash 043da3a70c15f79dbccc3bfb4a3e4cf5 parsl.dataflow.memoization INFO: Task 2930 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2930 try 0 launched on executor multi with executor id 2931 parsl.dataflow.dflow INFO: Standard output for task 2930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/07a28806-fcea-4117-904f-dd6e6a8c40d9_isrForCrosstalkSources_997007_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/07a28806-fcea-4117-904f-dd6e6a8c40d9_isrForCrosstalkSources_997007_31.stderr parsl.dataflow.dflow DEBUG: Task 2931 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2931 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2931 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2931 has memoization hash a158b7acfcc8072f2a2ee95d20e08cb1 parsl.dataflow.memoization INFO: Task 2931 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2931 try 0 launched on executor multi with executor id 2932 parsl.dataflow.dflow INFO: Standard output for task 2931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ea803b00-3faa-48f6-8b3b-ad11dc6481aa_isrForCrosstalkSources_996969_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ea803b00-3faa-48f6-8b3b-ad11dc6481aa_isrForCrosstalkSources_996969_6.stderr parsl.dataflow.dflow DEBUG: Task 2932 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2932 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2932 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2932 has memoization hash 4a091f30ab106a544e513f2107e88274 parsl.dataflow.memoization INFO: Task 2932 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2932 try 0 launched on executor multi with executor id 2933 parsl.dataflow.dflow INFO: Standard output for task 2932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/010eea8c-b489-4982-bab7-ffcbef1d7d8e_isrForCrosstalkSources_996984_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/010eea8c-b489-4982-bab7-ffcbef1d7d8e_isrForCrosstalkSources_996984_45.stderr parsl.dataflow.dflow DEBUG: Task 2933 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2933 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2933 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2933 has memoization hash 2793d2ab3274f1e5e055f529e3463a7b parsl.dataflow.memoization INFO: Task 2933 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2933 try 0 launched on executor multi with executor id 2934 parsl.dataflow.dflow INFO: Standard output for task 2933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f3e31e8b-b134-4fde-8c6f-6778e251ca74_isrForCrosstalkSources_996960_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f3e31e8b-b134-4fde-8c6f-6778e251ca74_isrForCrosstalkSources_996960_6.stderr parsl.dataflow.dflow DEBUG: Task 2934 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2934 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2934 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2934 has memoization hash f19e470cf7ef474ed1657bdf6fe2f326 parsl.dataflow.memoization INFO: Task 2934 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2934 try 0 launched on executor multi with executor id 2935 parsl.dataflow.dflow INFO: Standard output for task 2934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/7b952721-e0cd-49bd-b200-48c67fb6fb9e_isrForCrosstalkSources_996954_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/7b952721-e0cd-49bd-b200-48c67fb6fb9e_isrForCrosstalkSources_996954_2.stderr parsl.dataflow.dflow DEBUG: Task 2935 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2935 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2935 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2935 has memoization hash d556533580a3f38d10e465b92c84b10c parsl.dataflow.memoization INFO: Task 2935 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2935 try 0 launched on executor multi with executor id 2936 parsl.dataflow.dflow INFO: Standard output for task 2935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/884b6e24-1b7e-4219-a3d4-2dc2c26d38bf_isrForCrosstalkSources_996957_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/884b6e24-1b7e-4219-a3d4-2dc2c26d38bf_isrForCrosstalkSources_996957_49.stderr parsl.dataflow.dflow DEBUG: Task 2936 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2936 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2936 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2936 has memoization hash f823219e002b46015f431d863d3cdaae parsl.dataflow.memoization INFO: Task 2936 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2936 try 0 launched on executor multi with executor id 2937 parsl.dataflow.dflow INFO: Standard output for task 2936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/494120fc-a4fc-4bd5-a764-67e54947eaf1_isrForCrosstalkSources_997022_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/494120fc-a4fc-4bd5-a764-67e54947eaf1_isrForCrosstalkSources_997022_33.stderr parsl.dataflow.dflow DEBUG: Task 2937 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2937 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2937 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2937 has memoization hash ecb8ca175f3abd05d4fe0848e406813b parsl.dataflow.memoization INFO: Task 2937 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2937 try 0 launched on executor multi with executor id 2938 parsl.dataflow.dflow INFO: Standard output for task 2937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1a217e41-6131-41cb-bf3f-a5386f353b5a_isrForCrosstalkSources_996968_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1a217e41-6131-41cb-bf3f-a5386f353b5a_isrForCrosstalkSources_996968_27.stderr parsl.dataflow.dflow DEBUG: Task 2938 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2938 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2938 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2938 has memoization hash bb5ced68b2cdc4ddac5e00b3a2e6faf9 parsl.dataflow.memoization INFO: Task 2938 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93b060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2938 try 0 launched on executor multi with executor id 2939 parsl.dataflow.dflow INFO: Standard output for task 2938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/04a00608-a6e8-4b15-942e-df1b8489f115_isrForCrosstalkSources_997006_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/04a00608-a6e8-4b15-942e-df1b8489f115_isrForCrosstalkSources_997006_55.stderr parsl.dataflow.dflow DEBUG: Task 2939 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2939 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2939 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2939 has memoization hash 2cbbddf2a4ddd1925e76f1233a93d355 parsl.dataflow.memoization INFO: Task 2939 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93afc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2939 try 0 launched on executor multi with executor id 2940 parsl.dataflow.dflow INFO: Standard output for task 2939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/416cc84b-2de0-46fd-b393-b03e999491c6_isrForCrosstalkSources_997017_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/416cc84b-2de0-46fd-b393-b03e999491c6_isrForCrosstalkSources_997017_53.stderr parsl.dataflow.dflow DEBUG: Task 2940 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2940 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2940 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2940 has memoization hash 82d2fa97d4eb0c391ba9395a30be071f parsl.dataflow.memoization INFO: Task 2940 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93ae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2940 try 0 launched on executor multi with executor id 2941 parsl.dataflow.dflow INFO: Standard output for task 2940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/e6a8eefa-feee-4def-a899-542b79e05a0f_isrForCrosstalkSources_996962_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/e6a8eefa-feee-4def-a899-542b79e05a0f_isrForCrosstalkSources_996962_7.stderr parsl.dataflow.dflow DEBUG: Task 2941 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2941 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2941 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2941 has memoization hash 80791cfe21f8648010695ad6fd5d9290 parsl.dataflow.memoization INFO: Task 2941 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93af20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2941 try 0 launched on executor multi with executor id 2942 parsl.dataflow.dflow INFO: Standard output for task 2941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a6669c99-16fa-4376-a7ca-7811bab32df5_isrForCrosstalkSources_996981_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a6669c99-16fa-4376-a7ca-7811bab32df5_isrForCrosstalkSources_996981_57.stderr parsl.dataflow.dflow DEBUG: Task 2942 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2942 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2942 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2942 has memoization hash 913870cf0b593c02acf55505e35a762d parsl.dataflow.memoization INFO: Task 2942 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93ade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2942 try 0 launched on executor multi with executor id 2943 parsl.dataflow.dflow INFO: Standard output for task 2942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/ab79b855-35ef-4502-b426-023352ff219f_isrForCrosstalkSources_997022_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/ab79b855-35ef-4502-b426-023352ff219f_isrForCrosstalkSources_997022_3.stderr parsl.dataflow.dflow DEBUG: Task 2943 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2943 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2943 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2943 has memoization hash b4684439a4745998711d8dc3e0aa3a9c parsl.dataflow.memoization INFO: Task 2943 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93ad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2943 try 0 launched on executor multi with executor id 2944 parsl.dataflow.dflow INFO: Standard output for task 2943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/165a4f75-14af-4ac0-98ff-da55136073d5_isrForCrosstalkSources_997008_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/165a4f75-14af-4ac0-98ff-da55136073d5_isrForCrosstalkSources_997008_56.stderr parsl.dataflow.dflow DEBUG: Task 2944 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2944 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2944 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2944 has memoization hash ddf7fbe4dc7e956ec90f58c3b8c774c8 parsl.dataflow.memoization INFO: Task 2944 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93aca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2944 try 0 launched on executor multi with executor id 2945 parsl.dataflow.dflow INFO: Standard output for task 2944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9e2a194-c615-49cb-8682-d2693655fb5b_isrForCrosstalkSources_997009_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9e2a194-c615-49cb-8682-d2693655fb5b_isrForCrosstalkSources_997009_30.stderr parsl.dataflow.dflow DEBUG: Task 2945 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2945 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2945 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2945 has memoization hash 11297064c1d9f9bd122d4a8eb7ab0b3d parsl.dataflow.memoization INFO: Task 2945 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93ab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2945 try 0 launched on executor multi with executor id 2946 parsl.dataflow.dflow INFO: Standard output for task 2945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4064624e-0cd6-4147-b713-09fa64f0bfa8_isrForCrosstalkSources_996987_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4064624e-0cd6-4147-b713-09fa64f0bfa8_isrForCrosstalkSources_996987_61.stderr parsl.dataflow.dflow DEBUG: Task 2946 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2946 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2946 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2946 has memoization hash 128ca4e13c5952cfae74bacf0f2611b8 parsl.dataflow.memoization INFO: Task 2946 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93ac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2946 try 0 launched on executor multi with executor id 2947 parsl.dataflow.dflow INFO: Standard output for task 2946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/9a179c26-5938-47ad-9be7-4af07ecf8c65_isrForCrosstalkSources_997032_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/9a179c26-5938-47ad-9be7-4af07ecf8c65_isrForCrosstalkSources_997032_55.stderr parsl.dataflow.dflow DEBUG: Task 2947 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2947 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2947 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2947 has memoization hash ecff021fda3e5944f7aa343daa65183f parsl.dataflow.memoization INFO: Task 2947 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93aac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2947 try 0 launched on executor multi with executor id 2948 parsl.dataflow.dflow INFO: Standard output for task 2947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f44f50d9-b109-413e-9596-411dc23cbfce_isrForCrosstalkSources_996966_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f44f50d9-b109-413e-9596-411dc23cbfce_isrForCrosstalkSources_996966_40.stderr parsl.dataflow.dflow DEBUG: Task 2948 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2948 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2948 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2948 has memoization hash 3eb111c4ab84a14178cab0e11ed75db9 parsl.dataflow.memoization INFO: Task 2948 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93a980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2948 try 0 launched on executor multi with executor id 2949 parsl.dataflow.dflow INFO: Standard output for task 2948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/800f343f-4cfa-4646-b994-048922fe17b1_isrForCrosstalkSources_996985_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/800f343f-4cfa-4646-b994-048922fe17b1_isrForCrosstalkSources_996985_21.stderr parsl.dataflow.dflow DEBUG: Task 2949 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2949 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2949 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2949 has memoization hash 984b3aed66eb65a77460451e538b6bd3 parsl.dataflow.memoization INFO: Task 2949 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9380e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2949 try 0 launched on executor multi with executor id 2950 parsl.dataflow.dflow INFO: Standard output for task 2949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b21cf7f6-1ce3-4a53-bbd8-4f1c55267741_isrForCrosstalkSources_996986_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b21cf7f6-1ce3-4a53-bbd8-4f1c55267741_isrForCrosstalkSources_996986_47.stderr parsl.dataflow.dflow DEBUG: Task 2950 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2950 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2950 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2950 has memoization hash b87cbf5fce3ff75a01e1f89c2a3ba7f2 parsl.dataflow.memoization INFO: Task 2950 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93bf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2950 try 0 launched on executor multi with executor id 2951 parsl.dataflow.dflow INFO: Standard output for task 2950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ae6010cb-5cd3-4d1f-86f1-6b0e18c360ec_isrForCrosstalkSources_997030_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ae6010cb-5cd3-4d1f-86f1-6b0e18c360ec_isrForCrosstalkSources_997030_23.stderr parsl.dataflow.dflow DEBUG: Task 2951 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2951 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2951 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2951 has memoization hash d24c3b18bc6c19030780fb56e62639a2 parsl.dataflow.memoization INFO: Task 2951 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2951 try 0 launched on executor multi with executor id 2952 parsl.dataflow.dflow INFO: Standard output for task 2951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/89f12b47-6ac2-4c31-abb2-6d837180e865_isrForCrosstalkSources_996985_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/89f12b47-6ac2-4c31-abb2-6d837180e865_isrForCrosstalkSources_996985_52.stderr parsl.dataflow.dflow DEBUG: Task 2952 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2952 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2952 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2952 has memoization hash 8650f08b4ec7a1d34a536009edb93b2d parsl.dataflow.memoization INFO: Task 2952 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d00e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2952 try 0 launched on executor multi with executor id 2953 parsl.dataflow.dflow INFO: Standard output for task 2952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/58522cbc-b35d-4f3c-b97f-e0c4230a502c_isrForCrosstalkSources_996987_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/58522cbc-b35d-4f3c-b97f-e0c4230a502c_isrForCrosstalkSources_996987_9.stderr parsl.dataflow.dflow DEBUG: Task 2953 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2953 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2953 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2953 has memoization hash 47f619a6c3e612692f4e0c8ed5480539 parsl.dataflow.memoization INFO: Task 2953 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2953 try 0 launched on executor multi with executor id 2954 parsl.dataflow.dflow INFO: Standard output for task 2953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/678a5929-8c7e-4044-8801-ea74bc5847e9_isrForCrosstalkSources_996964_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/678a5929-8c7e-4044-8801-ea74bc5847e9_isrForCrosstalkSources_996964_7.stderr parsl.dataflow.dflow DEBUG: Task 2954 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2954 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2954 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2954 has memoization hash 2f05c463baebbd03560b10915784c623 parsl.dataflow.memoization INFO: Task 2954 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2954 try 0 launched on executor multi with executor id 2955 parsl.dataflow.dflow INFO: Standard output for task 2954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d2de8f0d-6bcb-462a-a24b-7924907818a0_isrForCrosstalkSources_997023_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d2de8f0d-6bcb-462a-a24b-7924907818a0_isrForCrosstalkSources_997023_39.stderr parsl.dataflow.dflow DEBUG: Task 2955 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2955 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2955 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2955 has memoization hash 4460ed457e9708b582bd2a6b8a8b000c parsl.dataflow.memoization INFO: Task 2955 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2955 try 0 launched on executor multi with executor id 2956 parsl.dataflow.dflow INFO: Standard output for task 2955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/e6a2bcae-1c1a-4f9c-96f3-f2e83fc74d11_isrForCrosstalkSources_996959_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/e6a2bcae-1c1a-4f9c-96f3-f2e83fc74d11_isrForCrosstalkSources_996959_18.stderr parsl.dataflow.dflow DEBUG: Task 2956 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2956 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2956 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2956 has memoization hash 736f77632354c5ca17447455c85ea475 parsl.dataflow.memoization INFO: Task 2956 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2956 try 0 launched on executor multi with executor id 2957 parsl.dataflow.dflow INFO: Standard output for task 2956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c66fdfde-2efb-4bdd-9f71-c91c5b516b11_isrForCrosstalkSources_996959_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c66fdfde-2efb-4bdd-9f71-c91c5b516b11_isrForCrosstalkSources_996959_12.stderr parsl.dataflow.dflow DEBUG: Task 2957 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2957 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2957 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2957 has memoization hash dceba03f4845ce9a23f44ae7fa0db8ec parsl.dataflow.memoization INFO: Task 2957 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2957 try 0 launched on executor multi with executor id 2958 parsl.dataflow.dflow INFO: Standard output for task 2957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/2c27e4eb-8e44-478c-8e3d-214e8c410dc6_isrForCrosstalkSources_997032_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/2c27e4eb-8e44-478c-8e3d-214e8c410dc6_isrForCrosstalkSources_997032_6.stderr parsl.dataflow.dflow DEBUG: Task 2958 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2958 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2958 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2958 has memoization hash 0251facc2da8d2eb8a264dfab10238d8 parsl.dataflow.memoization INFO: Task 2958 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d04a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2958 try 0 launched on executor multi with executor id 2959 parsl.dataflow.dflow INFO: Standard output for task 2958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/02dcf7c9-6669-4d58-ac90-ad7962d0df8b_isrForCrosstalkSources_997022_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/02dcf7c9-6669-4d58-ac90-ad7962d0df8b_isrForCrosstalkSources_997022_8.stderr parsl.dataflow.dflow DEBUG: Task 2959 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2959 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2959 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2959 has memoization hash 8f0e09d85d921b508a2c078b5da3691e parsl.dataflow.memoization INFO: Task 2959 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d05e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2959 try 0 launched on executor multi with executor id 2960 parsl.dataflow.dflow INFO: Standard output for task 2959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d5b0b451-8f69-4543-a26e-7549bc59a00c_isrForCrosstalkSources_996988_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d5b0b451-8f69-4543-a26e-7549bc59a00c_isrForCrosstalkSources_996988_25.stderr parsl.dataflow.dflow DEBUG: Task 2960 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2960 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2960 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2960 has memoization hash 819538a2a4fc75d4f1724f6cf4c1c3aa parsl.dataflow.memoization INFO: Task 2960 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2960 try 0 launched on executor multi with executor id 2961 parsl.dataflow.dflow INFO: Standard output for task 2960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1bea7713-906c-4477-a981-162e3488a6a5_isrForCrosstalkSources_996961_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1bea7713-906c-4477-a981-162e3488a6a5_isrForCrosstalkSources_996961_58.stderr parsl.dataflow.dflow DEBUG: Task 2961 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2961 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2961 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2961 has memoization hash aeb0f535be2e3fc975cd181843b6657b parsl.dataflow.memoization INFO: Task 2961 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:53 proc_lsst.multi:170 [INFO] [multi] job_id 17074880 proc_lsst.multi INFO: [multi] job_id 17074880 2024-03-15 13:49:53 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:53 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:53 proc_lsst.multi:178 [INFO] [multi] provider escience_rtx6k accepted submit and returned 17074880 proc_lsst.multi INFO: [multi] provider escience_rtx6k accepted submit and returned 17074880 parsl.executors.status_handling DEBUG: Launched block 31 on executor multi with job ID 17074880 parsl.executors.status_handling INFO: Allocated block ID 32 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-32 2024-03-15 13:49:53 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-32 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-32 2024-03-15 13:49:53 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:53 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:49:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:53 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:53 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=32 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 2961 try 0 launched on executor multi with executor id 2962 parsl.dataflow.dflow INFO: Standard output for task 2961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b65c622e-559e-4aa7-827f-e738b7467506_isrForCrosstalkSources_996983_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b65c622e-559e-4aa7-827f-e738b7467506_isrForCrosstalkSources_996983_55.stderr parsl.dataflow.dflow DEBUG: Task 2962 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2962 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2962 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2962 has memoization hash f526bbf48a3c2c912c0e4f8ce423a8b0 parsl.dataflow.memoization INFO: Task 2962 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c93bec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2962 try 0 launched on executor multi with executor id 2963 parsl.dataflow.dflow INFO: Standard output for task 2962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/0c6b8235-683a-4d3b-8712-d57ea5f06093_isrForCrosstalkSources_996990_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/0c6b8235-683a-4d3b-8712-d57ea5f06093_isrForCrosstalkSources_996990_62.stderr parsl.dataflow.dflow DEBUG: Task 2963 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2963 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2963 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2963 has memoization hash 3611dbaea1e41483e3b28d1e99788134 parsl.dataflow.memoization INFO: Task 2963 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c9387c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2963 try 0 launched on executor multi with executor id 2964 parsl.dataflow.dflow INFO: Standard output for task 2963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/6954c032-28f4-4794-a486-10bd429c8dc2_isrForCrosstalkSources_996968_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/6954c032-28f4-4794-a486-10bd429c8dc2_isrForCrosstalkSources_996968_50.stderr parsl.dataflow.dflow DEBUG: Task 2964 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2964 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2964 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2964 has memoization hash 7e7000a403e903d53f0c4cc95a76eb35 parsl.dataflow.memoization INFO: Task 2964 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c938a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2964 try 0 launched on executor multi with executor id 2965 parsl.dataflow.dflow INFO: Standard output for task 2964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d57ff839-0a93-4591-bf46-f6f519989dc1_isrForCrosstalkSources_996980_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d57ff839-0a93-4591-bf46-f6f519989dc1_isrForCrosstalkSources_996980_48.stderr parsl.dataflow.dflow DEBUG: Task 2965 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2965 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2965 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2965 has memoization hash 7302b38f051f3b466aad9abe526672f2 parsl.dataflow.memoization INFO: Task 2965 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2965 try 0 launched on executor multi with executor id 2966 parsl.dataflow.dflow INFO: Standard output for task 2965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/b67c6557-6aa0-430a-9463-2b072b6db208_isrForCrosstalkSources_997000_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/b67c6557-6aa0-430a-9463-2b072b6db208_isrForCrosstalkSources_997000_44.stderr parsl.dataflow.dflow DEBUG: Task 2966 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2966 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2966 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2966 has memoization hash 93d6e7faf56712308e476563de2e7409 parsl.dataflow.memoization INFO: Task 2966 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 9 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b48068ecbca8b669488fafb11c857245 with result from task 9 parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Task 9 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 9 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/917da520-a20f-49da-b8ad-bf5d15f9315e_isrForCrosstalkSources_996982_59.stdout parsl.dataflow.dflow INFO: Standard error for task 9 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/917da520-a20f-49da-b8ad-bf5d15f9315e_isrForCrosstalkSources_996982_59.stderr parsl.dataflow.dflow INFO: Parsl task 2966 try 0 launched on executor multi with executor id 2967 parsl.dataflow.dflow INFO: Standard output for task 2966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/66f338f4-14da-4010-bc29-b1d9b1c8c9ef_isrForCrosstalkSources_996953_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/66f338f4-14da-4010-bc29-b1d9b1c8c9ef_isrForCrosstalkSources_996953_1.stderr parsl.dataflow.dflow DEBUG: Task 2967 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2967 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2967 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2967 has memoization hash 451b943be4f43716757040dc88f5fb0f parsl.dataflow.memoization INFO: Task 2967 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2967 try 0 launched on executor multi with executor id 2968 parsl.dataflow.dflow INFO: Standard output for task 2967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ab031285-508e-4019-b453-edadec8b4998_isrForCrosstalkSources_997002_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ab031285-508e-4019-b453-edadec8b4998_isrForCrosstalkSources_997002_49.stderr parsl.dataflow.dflow DEBUG: Task 2968 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2968 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2968 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2968 has memoization hash a4018652108056f0b2222a43d28cfafd parsl.dataflow.memoization INFO: Task 2968 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2968 try 0 launched on executor multi with executor id 2969 parsl.dataflow.dflow INFO: Standard output for task 2968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c1c8518a-fd2f-4416-b825-3af0ce9729af_isrForCrosstalkSources_996972_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c1c8518a-fd2f-4416-b825-3af0ce9729af_isrForCrosstalkSources_996972_17.stderr parsl.dataflow.dflow DEBUG: Task 2969 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2969 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2969 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2969 has memoization hash 1e2da2e237653e9f8bc63f417587fcd8 parsl.dataflow.memoization INFO: Task 2969 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2969 try 0 launched on executor multi with executor id 2970 parsl.dataflow.dflow INFO: Standard output for task 2969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/84bc61bd-f05b-47fc-80e5-efc034398aa3_isrForCrosstalkSources_996957_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/84bc61bd-f05b-47fc-80e5-efc034398aa3_isrForCrosstalkSources_996957_44.stderr parsl.dataflow.dflow DEBUG: Task 2970 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2970 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2970 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2970 has memoization hash 48692d6964073c993e370c49dc8a28f9 parsl.dataflow.memoization INFO: Task 2970 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2970 try 0 launched on executor multi with executor id 2971 parsl.dataflow.dflow INFO: Standard output for task 2970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/44659357-96ab-435f-9e97-94638dba2fa0_isrForCrosstalkSources_996953_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/44659357-96ab-435f-9e97-94638dba2fa0_isrForCrosstalkSources_996953_6.stderr parsl.dataflow.dflow DEBUG: Task 2971 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2971 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2971 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2971 has memoization hash 444f005941b11c45dc57b6a7ca011ab7 parsl.dataflow.memoization INFO: Task 2971 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2971 try 0 launched on executor multi with executor id 2972 parsl.dataflow.dflow INFO: Standard output for task 2971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fbd49976-f9cf-4ec8-b6a5-e9cf59b4542f_isrForCrosstalkSources_996976_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fbd49976-f9cf-4ec8-b6a5-e9cf59b4542f_isrForCrosstalkSources_996976_39.stderr parsl.dataflow.dflow DEBUG: Task 2972 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2972 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2972 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2972 has memoization hash 586e80af006b6bd62b1e5eb5c578269f parsl.dataflow.memoization INFO: Task 2972 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2972 try 0 launched on executor multi with executor id 2973 parsl.dataflow.dflow INFO: Standard output for task 2972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/edaa37de-22da-4dd2-b00f-ae171167dbe3_isrForCrosstalkSources_997016_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/edaa37de-22da-4dd2-b00f-ae171167dbe3_isrForCrosstalkSources_997016_20.stderr parsl.dataflow.dflow DEBUG: Task 2973 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2973 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2973 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2973 has memoization hash 28c7214b31c333dbb104e92071d6bbe5 parsl.dataflow.memoization INFO: Task 2973 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2973 try 0 launched on executor multi with executor id 2974 parsl.dataflow.dflow INFO: Standard output for task 2973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f693525c-1bd5-4d6c-aa81-f9ead8265f15_isrForCrosstalkSources_996999_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f693525c-1bd5-4d6c-aa81-f9ead8265f15_isrForCrosstalkSources_996999_25.stderr parsl.dataflow.dflow DEBUG: Task 2974 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2974 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2974 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2974 has memoization hash 470e0acb829f638b146693c282bcb5d0 parsl.dataflow.memoization INFO: Task 2974 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2974 try 0 launched on executor multi with executor id 2975 parsl.dataflow.dflow INFO: Standard output for task 2974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/f9706ef5-79a6-400c-be60-50023384ca1d_isrForCrosstalkSources_996984_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/f9706ef5-79a6-400c-be60-50023384ca1d_isrForCrosstalkSources_996984_58.stderr parsl.dataflow.dflow DEBUG: Task 2975 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2975 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2975 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2975 has memoization hash 742b203f0b83f64d90b8c6da2e79d7b1 parsl.dataflow.memoization INFO: Task 2975 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2975 try 0 launched on executor multi with executor id 2976 parsl.dataflow.dflow INFO: Standard output for task 2975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c72a0b5e-73f4-486c-8251-8585b8a30750_isrForCrosstalkSources_997019_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c72a0b5e-73f4-486c-8251-8585b8a30750_isrForCrosstalkSources_997019_3.stderr parsl.dataflow.dflow DEBUG: Task 2976 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2976 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2976 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2976 has memoization hash 389ef78db7ca48d7ef87459585db25f0 parsl.dataflow.memoization INFO: Task 2976 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2976 try 0 launched on executor multi with executor id 2977 parsl.dataflow.dflow INFO: Standard output for task 2976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/37fd0a1b-f2b1-4132-bbc7-5986f3c5c14f_isrForCrosstalkSources_996978_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/37fd0a1b-f2b1-4132-bbc7-5986f3c5c14f_isrForCrosstalkSources_996978_11.stderr parsl.dataflow.dflow DEBUG: Task 2977 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2977 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2977 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2977 has memoization hash 9756fbe22918c39da570762529784fd6 parsl.dataflow.memoization INFO: Task 2977 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2977 try 0 launched on executor multi with executor id 2978 parsl.dataflow.dflow INFO: Standard output for task 2977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/30bc09cc-feba-46a6-8812-47917bf415b3_isrForCrosstalkSources_997016_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/30bc09cc-feba-46a6-8812-47917bf415b3_isrForCrosstalkSources_997016_45.stderr parsl.dataflow.dflow DEBUG: Task 2978 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2978 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2978 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2978 has memoization hash 6ad796ae475d1f20c68de92484afa067 parsl.dataflow.memoization INFO: Task 2978 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2978 try 0 launched on executor multi with executor id 2979 parsl.dataflow.dflow INFO: Standard output for task 2978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/2f9cacad-7d33-4b2c-a525-70f97971f187_isrForCrosstalkSources_997029_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/2f9cacad-7d33-4b2c-a525-70f97971f187_isrForCrosstalkSources_997029_16.stderr parsl.dataflow.dflow DEBUG: Task 2979 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2979 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2979 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2979 has memoization hash 79bd4bea7fb2d518caa8cd2621017efd parsl.dataflow.memoization INFO: Task 2979 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2979 try 0 launched on executor multi with executor id 2980 parsl.dataflow.dflow INFO: Standard output for task 2979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f83c5495-d647-42fe-b58b-33d7f8e932c7_isrForCrosstalkSources_997018_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f83c5495-d647-42fe-b58b-33d7f8e932c7_isrForCrosstalkSources_997018_2.stderr parsl.dataflow.dflow DEBUG: Task 2980 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2980 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2980 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2980 has memoization hash 55474ddf23c05c44e3adef81962f1bbb parsl.dataflow.memoization INFO: Task 2980 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2980 try 0 launched on executor multi with executor id 2981 parsl.dataflow.dflow INFO: Standard output for task 2980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/93a5aae3-5cab-409c-a9ed-a3afffc1c5fe_isrForCrosstalkSources_996973_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/93a5aae3-5cab-409c-a9ed-a3afffc1c5fe_isrForCrosstalkSources_996973_17.stderr parsl.dataflow.dflow DEBUG: Task 2981 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2981 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2981 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2981 has memoization hash 8fe9928b92c6d1661fc295946b2fc5b0 parsl.dataflow.memoization INFO: Task 2981 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2981 try 0 launched on executor multi with executor id 2982 parsl.dataflow.dflow INFO: Standard output for task 2981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/51c6b59f-6de9-4400-a231-a050bc85aafa_isrForCrosstalkSources_997008_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/51c6b59f-6de9-4400-a231-a050bc85aafa_isrForCrosstalkSources_997008_50.stderr parsl.dataflow.dflow DEBUG: Task 2982 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2982 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2982 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2982 has memoization hash a5f94a86ce3e6f49ea6bf799c2532b07 parsl.dataflow.memoization INFO: Task 2982 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d11c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2982 try 0 launched on executor multi with executor id 2983 parsl.dataflow.dflow INFO: Standard output for task 2982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2c7d4765-92d3-487a-a15a-eee9c7940738_isrForCrosstalkSources_996990_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2c7d4765-92d3-487a-a15a-eee9c7940738_isrForCrosstalkSources_996990_14.stderr parsl.dataflow.dflow DEBUG: Task 2983 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2983 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2983 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2983 has memoization hash 1b9ecc97cca9fdd5a9d0deeda77e9ea4 parsl.dataflow.memoization INFO: Task 2983 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d13a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2983 try 0 launched on executor multi with executor id 2984 parsl.dataflow.dflow INFO: Standard output for task 2983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/90e67a6a-0c3a-49a1-9cdc-b6780fadfde7_isrForCrosstalkSources_996961_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/90e67a6a-0c3a-49a1-9cdc-b6780fadfde7_isrForCrosstalkSources_996961_28.stderr parsl.dataflow.dflow DEBUG: Task 2984 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2984 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2984 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2984 has memoization hash 78ded03b5e3ca54a1cbb8d3ce1c8b285 parsl.dataflow.memoization INFO: Task 2984 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d14e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2984 try 0 launched on executor multi with executor id 2985 parsl.dataflow.dflow INFO: Standard output for task 2984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/0e3d9767-4b54-414e-add0-004083a540a3_isrForCrosstalkSources_996989_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/0e3d9767-4b54-414e-add0-004083a540a3_isrForCrosstalkSources_996989_43.stderr parsl.dataflow.dflow DEBUG: Task 2985 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2985 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2985 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2985 has memoization hash cc761d8ada7074e444eb15b60feb273e parsl.dataflow.memoization INFO: Task 2985 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2985 try 0 launched on executor multi with executor id 2986 parsl.dataflow.dflow INFO: Standard output for task 2985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/8b94f7eb-bb0d-4350-bb3d-2ff2b683b086_isrForCrosstalkSources_997014_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/8b94f7eb-bb0d-4350-bb3d-2ff2b683b086_isrForCrosstalkSources_997014_41.stderr parsl.dataflow.dflow DEBUG: Task 2986 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2986 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2986 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2986 has memoization hash f3c29899fe0a2aa08426899f1210eaa9 parsl.dataflow.memoization INFO: Task 2986 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2986 try 0 launched on executor multi with executor id 2987 parsl.dataflow.dflow INFO: Standard output for task 2986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/64e68dd7-36f9-4bf4-9166-1da476bf6b83_isrForCrosstalkSources_996984_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/64e68dd7-36f9-4bf4-9166-1da476bf6b83_isrForCrosstalkSources_996984_22.stderr parsl.dataflow.dflow DEBUG: Task 2987 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2987 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2987 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2987 has memoization hash dc2f397698ae3080173d563d24dc8105 parsl.dataflow.memoization INFO: Task 2987 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d16c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2987 try 0 launched on executor multi with executor id 2988 parsl.dataflow.dflow INFO: Standard output for task 2987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/dd173c1f-ec9c-4524-a3e2-3b5b716521e6_isrForCrosstalkSources_996990_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/dd173c1f-ec9c-4524-a3e2-3b5b716521e6_isrForCrosstalkSources_996990_46.stderr parsl.dataflow.dflow DEBUG: Task 2988 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2988 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2988 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2988 has memoization hash f42b49810b95117895148eae901d8de6 parsl.dataflow.memoization INFO: Task 2988 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2988 try 0 launched on executor multi with executor id 2989 parsl.dataflow.dflow INFO: Standard output for task 2988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b90339c2-f14f-4f66-b751-01a753fbcfdf_isrForCrosstalkSources_996980_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b90339c2-f14f-4f66-b751-01a753fbcfdf_isrForCrosstalkSources_996980_57.stderr parsl.dataflow.dflow DEBUG: Task 2989 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2989 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2989 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2989 has memoization hash c9e5b44027ec0e13c31852afc1bec811 parsl.dataflow.memoization INFO: Task 2989 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2989 try 0 launched on executor multi with executor id 2990 parsl.dataflow.dflow INFO: Standard output for task 2989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1ca61b3b-c605-4273-95ec-1e54c9a8b7a5_isrForCrosstalkSources_996960_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1ca61b3b-c605-4273-95ec-1e54c9a8b7a5_isrForCrosstalkSources_996960_23.stderr parsl.dataflow.dflow DEBUG: Task 2990 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2990 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2990 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2990 has memoization hash cf8f2471ba00367ff1195eef8ba67a8c parsl.dataflow.memoization INFO: Task 2990 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2990 try 0 launched on executor multi with executor id 2991 parsl.dataflow.dflow INFO: Standard output for task 2990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c523b3ce-37fc-46e8-b926-bbda289a93d2_isrForCrosstalkSources_997006_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c523b3ce-37fc-46e8-b926-bbda289a93d2_isrForCrosstalkSources_997006_33.stderr parsl.dataflow.dflow DEBUG: Task 2991 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2991 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2991 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2991 has memoization hash 0eb294141394b2d7d3a4a64216e79585 parsl.dataflow.memoization INFO: Task 2991 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2991 try 0 launched on executor multi with executor id 2992 parsl.dataflow.dflow INFO: Standard output for task 2991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7e67da05-bd8e-4d67-b18d-ae6bd3b35fef_isrForCrosstalkSources_997028_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7e67da05-bd8e-4d67-b18d-ae6bd3b35fef_isrForCrosstalkSources_997028_2.stderr parsl.dataflow.dflow DEBUG: Task 2992 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2992 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2992 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2992 has memoization hash 6c31036224c30115b3421e29b1081da1 parsl.dataflow.memoization INFO: Task 2992 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d19e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2992 try 0 launched on executor multi with executor id 2993 parsl.dataflow.dflow INFO: Standard output for task 2992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e2783aa9-560a-4986-8d30-22097e037ee0_isrForCrosstalkSources_997019_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e2783aa9-560a-4986-8d30-22097e037ee0_isrForCrosstalkSources_997019_27.stderr parsl.dataflow.dflow DEBUG: Task 2993 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2993 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2993 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2993 has memoization hash a0dd778bd18c39c787bb2d70170d7f93 parsl.dataflow.memoization INFO: Task 2993 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2993 try 0 launched on executor multi with executor id 2994 parsl.dataflow.dflow INFO: Standard output for task 2993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/1ac4efb5-a844-4938-b608-b89fb1358188_isrForCrosstalkSources_997012_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/1ac4efb5-a844-4938-b608-b89fb1358188_isrForCrosstalkSources_997012_18.stderr parsl.dataflow.dflow DEBUG: Task 2994 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2994 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2994 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2994 has memoization hash c4e5a30978ee6573cddd3d633e9544e4 parsl.dataflow.memoization INFO: Task 2994 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2994 try 0 launched on executor multi with executor id 2995 parsl.dataflow.dflow INFO: Standard output for task 2994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/2ad7c6ab-eedd-4eaa-b8d5-8c22eef7d5f7_isrForCrosstalkSources_996967_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/2ad7c6ab-eedd-4eaa-b8d5-8c22eef7d5f7_isrForCrosstalkSources_996967_56.stderr parsl.dataflow.dflow DEBUG: Task 2995 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2995 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2995 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2995 has memoization hash 4026ef945ccf57504440976ca8702490 parsl.dataflow.memoization INFO: Task 2995 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d18a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2995 try 0 launched on executor multi with executor id 2996 parsl.dataflow.dflow INFO: Standard output for task 2995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b59dddc9-c889-4d5f-9719-03e7fb23e039_isrForCrosstalkSources_997014_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b59dddc9-c889-4d5f-9719-03e7fb23e039_isrForCrosstalkSources_997014_20.stderr parsl.dataflow.dflow DEBUG: Task 2996 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2996 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2996 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2996 has memoization hash 1647d5406a71b994da2196f58d05e74e parsl.dataflow.memoization INFO: Task 2996 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2996 try 0 launched on executor multi with executor id 2997 parsl.dataflow.dflow INFO: Standard output for task 2996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/e8310b60-1d43-4921-872e-3021bece0224_isrForCrosstalkSources_997024_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/e8310b60-1d43-4921-872e-3021bece0224_isrForCrosstalkSources_997024_55.stderr parsl.dataflow.dflow DEBUG: Task 2997 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2997 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2997 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2997 has memoization hash 95f9d825d79ac23c4db16b101c755a8e parsl.dataflow.memoization INFO: Task 2997 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2997 try 0 launched on executor multi with executor id 2998 parsl.dataflow.dflow INFO: Standard output for task 2997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/3a7c7af3-d949-4eb1-b94b-20a61af155d9_isrForCrosstalkSources_997007_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/3a7c7af3-d949-4eb1-b94b-20a61af155d9_isrForCrosstalkSources_997007_7.stderr parsl.dataflow.dflow DEBUG: Task 2998 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2998 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2998 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2998 has memoization hash 62bfbc81827d3ea9c83687d53de6b620 parsl.dataflow.memoization INFO: Task 2998 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2998 try 0 launched on executor multi with executor id 2999 parsl.dataflow.dflow INFO: Standard output for task 2998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/1a0096a0-174e-419f-a4b4-941877ebbe74_isrForCrosstalkSources_996970_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/1a0096a0-174e-419f-a4b4-941877ebbe74_isrForCrosstalkSources_996970_37.stderr parsl.dataflow.dflow DEBUG: Task 2999 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 2999 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 2999 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 2999 has memoization hash 2198311c132f0259f50b82979ae27a8b parsl.dataflow.memoization INFO: Task 2999 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 2999 try 0 launched on executor multi with executor id 3000 parsl.dataflow.dflow INFO: Standard output for task 2999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1f29cc4f-a5cf-4062-9011-b23e01fcc860_isrForCrosstalkSources_996960_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1f29cc4f-a5cf-4062-9011-b23e01fcc860_isrForCrosstalkSources_996960_32.stderr parsl.dataflow.dflow DEBUG: Task 3000 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3000 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3000 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3000 has memoization hash d642e1834ee0224f0fc51da3d12af5fb parsl.dataflow.memoization INFO: Task 3000 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3000 try 0 launched on executor multi with executor id 3001 parsl.dataflow.dflow INFO: Standard output for task 3000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/d8e9740d-24df-4853-a004-2eea1fcd88ae_isrForCrosstalkSources_996986_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/d8e9740d-24df-4853-a004-2eea1fcd88ae_isrForCrosstalkSources_996986_60.stderr parsl.dataflow.dflow DEBUG: Task 3001 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3001 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3001 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3001 has memoization hash 3349f2cfc28ebb6346995609957d30a0 parsl.dataflow.memoization INFO: Task 3001 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3001 try 0 launched on executor multi with executor id 3002 parsl.dataflow.dflow INFO: Standard output for task 3001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/b9566bbd-120e-4169-8edd-c1f90b064009_isrForCrosstalkSources_997024_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/b9566bbd-120e-4169-8edd-c1f90b064009_isrForCrosstalkSources_997024_45.stderr parsl.dataflow.dflow DEBUG: Task 3002 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3002 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3002 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3002 has memoization hash a81825ac63cad0e50465addab5c72904 parsl.dataflow.memoization INFO: Task 3002 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d1f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3002 try 0 launched on executor multi with executor id 3003 parsl.dataflow.dflow INFO: Standard output for task 3002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/508c5c8b-5da9-4478-bde0-afa4c793a7ae_isrForCrosstalkSources_996962_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/508c5c8b-5da9-4478-bde0-afa4c793a7ae_isrForCrosstalkSources_996962_23.stderr parsl.dataflow.dflow DEBUG: Task 3003 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3003 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3003 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3003 has memoization hash 223853d25f6d285cabc53ec68f199eaa parsl.dataflow.memoization INFO: Task 3003 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d20c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3003 try 0 launched on executor multi with executor id 3004 parsl.dataflow.dflow INFO: Standard output for task 3003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/c8f9caf0-eac8-4719-a7da-252635471289_isrForCrosstalkSources_996964_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/c8f9caf0-eac8-4719-a7da-252635471289_isrForCrosstalkSources_996964_32.stderr parsl.dataflow.dflow DEBUG: Task 3004 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3004 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3004 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3004 has memoization hash ba3f491bea3f5e158e0fc54437ff48ad parsl.dataflow.memoization INFO: Task 3004 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3004 try 0 launched on executor multi with executor id 3005 parsl.dataflow.dflow INFO: Standard output for task 3004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0ff322b0-33f5-40ea-b0e2-275bd9d6c434_isrForCrosstalkSources_996991_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0ff322b0-33f5-40ea-b0e2-275bd9d6c434_isrForCrosstalkSources_996991_33.stderr parsl.dataflow.dflow DEBUG: Task 3005 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3005 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3005 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3005 has memoization hash bd78a79b5960b37063e4b926c86c474a parsl.dataflow.memoization INFO: Task 3005 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3005 try 0 launched on executor multi with executor id 3006 parsl.dataflow.dflow INFO: Standard output for task 3005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/e51df0eb-a9e5-4ddc-90ba-54c995125c78_isrForCrosstalkSources_996969_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/e51df0eb-a9e5-4ddc-90ba-54c995125c78_isrForCrosstalkSources_996969_62.stderr parsl.dataflow.dflow DEBUG: Task 3006 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3006 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3006 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3006 has memoization hash 9b3f54311e3fa2c19d49c509e5efdccb parsl.dataflow.memoization INFO: Task 3006 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3006 try 0 launched on executor multi with executor id 3007 parsl.dataflow.dflow INFO: Standard output for task 3006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/7960fcf0-3d7f-413d-accf-09a1d981c18e_isrForCrosstalkSources_996955_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/7960fcf0-3d7f-413d-accf-09a1d981c18e_isrForCrosstalkSources_996955_36.stderr parsl.dataflow.dflow DEBUG: Task 3007 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3007 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3007 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3007 has memoization hash 4986fd3210b61596c7a19b23262ee766 parsl.dataflow.memoization INFO: Task 3007 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3007 try 0 launched on executor multi with executor id 3008 parsl.dataflow.dflow INFO: Standard output for task 3007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6171fc96-2df4-4c67-8438-48dd01f0e84d_isrForCrosstalkSources_996963_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6171fc96-2df4-4c67-8438-48dd01f0e84d_isrForCrosstalkSources_996963_46.stderr parsl.dataflow.dflow DEBUG: Task 3008 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3008 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3008 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3008 has memoization hash 11d7a8667530c2498ba5de2d2e8dd5ea parsl.dataflow.memoization INFO: Task 3008 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d23e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3008 try 0 launched on executor multi with executor id 3009 parsl.dataflow.dflow INFO: Standard output for task 3008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/ce723bf2-010a-42bf-9843-a4e565f9b571_isrForCrosstalkSources_996967_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/ce723bf2-010a-42bf-9843-a4e565f9b571_isrForCrosstalkSources_996967_3.stderr parsl.dataflow.dflow DEBUG: Task 3009 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3009 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3009 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3009 has memoization hash f2c1bd0d7e858a30d7b6ee5b68089946 parsl.dataflow.memoization INFO: Task 3009 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3009 try 0 launched on executor multi with executor id 3010 parsl.dataflow.dflow INFO: Standard output for task 3009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/75a8615d-e098-470a-8ba2-ceb4b6f71336_isrForCrosstalkSources_997009_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/75a8615d-e098-470a-8ba2-ceb4b6f71336_isrForCrosstalkSources_997009_12.stderr parsl.dataflow.dflow DEBUG: Task 3010 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3010 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3010 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3010 has memoization hash 6b2773fb5320f1cbfaea41b099cfe884 parsl.dataflow.memoization INFO: Task 3010 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d22a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3010 try 0 launched on executor multi with executor id 3011 parsl.dataflow.dflow INFO: Standard output for task 3010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/49fb95d8-08b3-4bf2-a71d-418ed13d6c07_isrForCrosstalkSources_996984_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/49fb95d8-08b3-4bf2-a71d-418ed13d6c07_isrForCrosstalkSources_996984_23.stderr parsl.dataflow.dflow DEBUG: Task 3011 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3011 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3011 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3011 has memoization hash 1ba5407d313de982727a6d343004dacc parsl.dataflow.memoization INFO: Task 3011 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d25c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3011 try 0 launched on executor multi with executor id 3012 parsl.dataflow.dflow INFO: Standard output for task 3011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/096802de-d9fb-4ce7-b096-93deecc06472_isrForCrosstalkSources_996979_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/096802de-d9fb-4ce7-b096-93deecc06472_isrForCrosstalkSources_996979_52.stderr parsl.dataflow.dflow DEBUG: Task 3012 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3012 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3012 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3012 has memoization hash 4517a23d92db807add388ea6ff141d9e parsl.dataflow.memoization INFO: Task 3012 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3012 try 0 launched on executor multi with executor id 3013 parsl.dataflow.dflow INFO: Standard output for task 3012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b2e85978-c066-4be5-aff5-dd6e33156c59_isrForCrosstalkSources_997007_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b2e85978-c066-4be5-aff5-dd6e33156c59_isrForCrosstalkSources_997007_8.stderr parsl.dataflow.dflow DEBUG: Task 3013 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3013 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3013 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3013 has memoization hash 3fd7e17a7ea53101b18b1e0071f2195b parsl.dataflow.memoization INFO: Task 3013 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3013 try 0 launched on executor multi with executor id 3014 parsl.dataflow.dflow INFO: Standard output for task 3013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e4ad14f8-0715-4151-8270-6a8b903740ee_isrForCrosstalkSources_996961_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e4ad14f8-0715-4151-8270-6a8b903740ee_isrForCrosstalkSources_996961_10.stderr parsl.dataflow.dflow DEBUG: Task 3014 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3014 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3014 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3014 has memoization hash 9540fb540ad382d123de1cac84a5e488 parsl.dataflow.memoization INFO: Task 3014 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3014 try 0 launched on executor multi with executor id 3015 parsl.dataflow.dflow INFO: Standard output for task 3014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3c71d742-8ab3-4bfa-a118-bb48e06c6bd3_isrForCrosstalkSources_996967_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3c71d742-8ab3-4bfa-a118-bb48e06c6bd3_isrForCrosstalkSources_996967_18.stderr parsl.dataflow.dflow DEBUG: Task 3015 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3015 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3015 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3015 has memoization hash 24e90229e560960a68be2e96433464ea parsl.dataflow.memoization INFO: Task 3015 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d27a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3015 try 0 launched on executor multi with executor id 3016 parsl.dataflow.dflow INFO: Standard output for task 3015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ad1dc883-53d4-4c07-802f-9a2b7cd76b52_isrForCrosstalkSources_996988_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ad1dc883-53d4-4c07-802f-9a2b7cd76b52_isrForCrosstalkSources_996988_22.stderr parsl.dataflow.dflow DEBUG: Task 3016 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3016 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3016 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3016 has memoization hash df8f7e420b748cae4e52452e086c895c parsl.dataflow.memoization INFO: Task 3016 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d28e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3016 try 0 launched on executor multi with executor id 3017 parsl.dataflow.dflow INFO: Standard output for task 3016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d3aba8e9-9d9a-4c87-9c0a-0a87098b674c_isrForCrosstalkSources_996991_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d3aba8e9-9d9a-4c87-9c0a-0a87098b674c_isrForCrosstalkSources_996991_24.stderr parsl.dataflow.dflow DEBUG: Task 3017 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3017 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3017 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3017 has memoization hash 73c1d4bb33f2f5420eeb6df79a34017c parsl.dataflow.memoization INFO: Task 3017 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3017 try 0 launched on executor multi with executor id 3018 parsl.dataflow.dflow INFO: Standard output for task 3017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/11090e71-c76d-4b99-bdd6-787452ca6ef2_isrForCrosstalkSources_996989_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/11090e71-c76d-4b99-bdd6-787452ca6ef2_isrForCrosstalkSources_996989_44.stderr parsl.dataflow.dflow DEBUG: Task 3018 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3018 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3018 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3018 has memoization hash 986b7a6a25701b3f659a41bf273d9388 parsl.dataflow.memoization INFO: Task 3018 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3018 try 0 launched on executor multi with executor id 3019 parsl.dataflow.dflow INFO: Standard output for task 3018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/163b6b2d-d170-49d3-a54a-411b5a357d28_isrForCrosstalkSources_997025_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/163b6b2d-d170-49d3-a54a-411b5a357d28_isrForCrosstalkSources_997025_40.stderr parsl.dataflow.dflow DEBUG: Task 3019 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3019 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3019 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3019 has memoization hash 55b843784df6eb37bf0c7d2b1682ddc8 parsl.dataflow.memoization INFO: Task 3019 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3019 try 0 launched on executor multi with executor id 3020 parsl.dataflow.dflow INFO: Standard output for task 3019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c40c26ae-2d87-4e6a-9c52-685cdcd45848_isrForCrosstalkSources_996970_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c40c26ae-2d87-4e6a-9c52-685cdcd45848_isrForCrosstalkSources_996970_49.stderr parsl.dataflow.dflow DEBUG: Task 3020 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3020 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3020 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3020 has memoization hash 679cf53c243e3cb94539bf9c066a2eb8 parsl.dataflow.memoization INFO: Task 3020 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3020 try 0 launched on executor multi with executor id 3021 parsl.dataflow.dflow INFO: Standard output for task 3020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/bcf42695-00e0-4eb6-986c-f0230c73d5f2_isrForCrosstalkSources_996979_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/bcf42695-00e0-4eb6-986c-f0230c73d5f2_isrForCrosstalkSources_996979_44.stderr parsl.dataflow.dflow DEBUG: Task 3021 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3021 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3021 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3021 has memoization hash 4da64c92e5ac73b939465f8833b38850 parsl.dataflow.memoization INFO: Task 3021 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3021 try 0 launched on executor multi with executor id 3022 parsl.dataflow.dflow INFO: Standard output for task 3021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8681f580-71db-4641-a5e6-d610a12319af_isrForCrosstalkSources_996967_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8681f580-71db-4641-a5e6-d610a12319af_isrForCrosstalkSources_996967_59.stderr parsl.dataflow.dflow DEBUG: Task 3022 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3022 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3022 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3022 has memoization hash 0c02509749be012c06954e298074da6f parsl.dataflow.memoization INFO: Task 3022 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3022 try 0 launched on executor multi with executor id 3023 parsl.dataflow.dflow INFO: Standard output for task 3022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b20d992f-affc-4174-a002-f3f74316c023_isrForCrosstalkSources_997001_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b20d992f-affc-4174-a002-f3f74316c023_isrForCrosstalkSources_997001_58.stderr parsl.dataflow.dflow DEBUG: Task 3023 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3023 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3023 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3023 has memoization hash 9d071794ebeb1c9eebf89fe919280b15 parsl.dataflow.memoization INFO: Task 3023 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3023 try 0 launched on executor multi with executor id 3024 parsl.dataflow.dflow INFO: Standard output for task 3023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/62bc8d37-203f-4fc1-82c9-9a571b534dd1_isrForCrosstalkSources_996952_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/62bc8d37-203f-4fc1-82c9-9a571b534dd1_isrForCrosstalkSources_996952_54.stderr parsl.dataflow.dflow DEBUG: Task 3024 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3024 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3024 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3024 has memoization hash 56e34e04fbd21279873be22974384d06 parsl.dataflow.memoization INFO: Task 3024 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3024 try 0 launched on executor multi with executor id 3025 parsl.dataflow.dflow INFO: Standard output for task 3024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/7962e892-a9f6-429e-90ff-b0c71b42b6e7_isrForCrosstalkSources_996969_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/7962e892-a9f6-429e-90ff-b0c71b42b6e7_isrForCrosstalkSources_996969_56.stderr parsl.dataflow.dflow DEBUG: Task 3025 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3025 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3025 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3025 has memoization hash 7f7839af827a32677337352bf06fdf2c parsl.dataflow.memoization INFO: Task 3025 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3025 try 0 launched on executor multi with executor id 3026 parsl.dataflow.dflow INFO: Standard output for task 3025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/c04f563b-66f6-41b6-9166-0df8a2f43e19_isrForCrosstalkSources_996986_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/c04f563b-66f6-41b6-9166-0df8a2f43e19_isrForCrosstalkSources_996986_54.stderr parsl.dataflow.dflow DEBUG: Task 3026 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3026 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3026 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3026 has memoization hash e4421269593a4e3a122c53922eaaf1a1 parsl.dataflow.memoization INFO: Task 3026 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3026 try 0 launched on executor multi with executor id 3027 parsl.dataflow.dflow INFO: Standard output for task 3026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a9624c59-12f7-4a0d-aaf7-8d8b6f927b67_isrForCrosstalkSources_996970_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a9624c59-12f7-4a0d-aaf7-8d8b6f927b67_isrForCrosstalkSources_996970_24.stderr parsl.dataflow.dflow DEBUG: Task 3027 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3027 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3027 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3027 has memoization hash 9158eb78cd6caea7271d5f25506fbe65 parsl.dataflow.memoization INFO: Task 3027 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d39c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3027 try 0 launched on executor multi with executor id 3028 parsl.dataflow.dflow INFO: Standard output for task 3027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/4254ade1-db05-418f-8ec8-8cceab8dbf76_isrForCrosstalkSources_996967_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/4254ade1-db05-418f-8ec8-8cceab8dbf76_isrForCrosstalkSources_996967_39.stderr parsl.dataflow.dflow DEBUG: Task 3028 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3028 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3028 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3028 has memoization hash 65d043739a3f761b427a13790ba45a8f parsl.dataflow.memoization INFO: Task 3028 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3028 try 0 launched on executor multi with executor id 3029 parsl.dataflow.dflow INFO: Standard output for task 3028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/074a8b4c-4388-44e5-b25a-b1f8246f24d5_isrForCrosstalkSources_996980_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/074a8b4c-4388-44e5-b25a-b1f8246f24d5_isrForCrosstalkSources_996980_42.stderr parsl.dataflow.dflow DEBUG: Task 3029 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3029 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3029 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3029 has memoization hash 575db023553de509e300963ed25c8930 parsl.dataflow.memoization INFO: Task 3029 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d37e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3029 try 0 launched on executor multi with executor id 3030 parsl.dataflow.dflow INFO: Standard output for task 3029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/742f9bfc-45da-41ce-bc77-7737838d3d73_isrForCrosstalkSources_996970_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/742f9bfc-45da-41ce-bc77-7737838d3d73_isrForCrosstalkSources_996970_36.stderr parsl.dataflow.dflow DEBUG: Task 3030 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3030 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3030 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3030 has memoization hash 5f4106babb5979ff50cf79ba6e175f65 parsl.dataflow.memoization INFO: Task 3030 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3030 try 0 launched on executor multi with executor id 3031 parsl.dataflow.dflow INFO: Standard output for task 3030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6ed15fa8-107b-4b98-92a5-31c9a81e6f39_isrForCrosstalkSources_996954_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6ed15fa8-107b-4b98-92a5-31c9a81e6f39_isrForCrosstalkSources_996954_52.stderr parsl.dataflow.dflow DEBUG: Task 3031 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3031 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3031 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3031 has memoization hash 008f843f6b8087b4e4a1f48b1369cef8 parsl.dataflow.memoization INFO: Task 3031 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3031 try 0 launched on executor multi with executor id 3032 parsl.dataflow.dflow INFO: Standard output for task 3031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2b47ded9-08ee-40e8-b47e-a7ca75de44ca_isrForCrosstalkSources_997015_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2b47ded9-08ee-40e8-b47e-a7ca75de44ca_isrForCrosstalkSources_997015_35.stderr parsl.dataflow.dflow DEBUG: Task 3032 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3032 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3032 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3032 has memoization hash 08ae6bba737f296405fc834cad6ccf2d parsl.dataflow.memoization INFO: Task 3032 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d36a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3032 try 0 launched on executor multi with executor id 3033 parsl.dataflow.dflow INFO: Standard output for task 3032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7b4889b1-ee25-4134-92b1-014523f4c265_isrForCrosstalkSources_997028_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7b4889b1-ee25-4134-92b1-014523f4c265_isrForCrosstalkSources_997028_29.stderr parsl.dataflow.dflow DEBUG: Task 3033 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3033 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3033 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3033 has memoization hash 627e464f9e5bd9d5e175c4989ba3076a parsl.dataflow.memoization INFO: Task 3033 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3033 try 0 launched on executor multi with executor id 3034 parsl.dataflow.dflow INFO: Standard output for task 3033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/78696bd3-faa9-441a-8330-ae78d8cc2b36_isrForCrosstalkSources_997007_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/78696bd3-faa9-441a-8330-ae78d8cc2b36_isrForCrosstalkSources_997007_9.stderr parsl.dataflow.dflow DEBUG: Task 3034 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3034 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3034 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3034 has memoization hash c6b07c7126c9593520e6b5b26d2b7831 parsl.dataflow.memoization INFO: Task 3034 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d34c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3034 try 0 launched on executor multi with executor id 3035 parsl.dataflow.dflow INFO: Standard output for task 3034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/87e92dad-93e0-4dfe-85be-989408a65c17_isrForCrosstalkSources_996954_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/87e92dad-93e0-4dfe-85be-989408a65c17_isrForCrosstalkSources_996954_59.stderr parsl.dataflow.dflow DEBUG: Task 3035 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3035 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3035 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3035 has memoization hash 70a4bb55a596a9cc89cba13d6d162da5 parsl.dataflow.memoization INFO: Task 3035 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3035 try 0 launched on executor multi with executor id 3036 parsl.dataflow.dflow INFO: Standard output for task 3035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/ae3f09d9-7711-4199-aa1a-9927207e3f3c_isrForCrosstalkSources_996977_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/ae3f09d9-7711-4199-aa1a-9927207e3f3c_isrForCrosstalkSources_996977_42.stderr parsl.dataflow.dflow DEBUG: Task 3036 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3036 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3036 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3036 has memoization hash 9f7f9f3962a4d2e82f560f2c1f597a3e parsl.dataflow.memoization INFO: Task 3036 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3036 try 0 launched on executor multi with executor id 3037 parsl.dataflow.dflow INFO: Standard output for task 3036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/edc50e1d-c603-458c-befc-f63731bc9f9a_isrForCrosstalkSources_997002_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/edc50e1d-c603-458c-befc-f63731bc9f9a_isrForCrosstalkSources_997002_25.stderr parsl.dataflow.dflow DEBUG: Task 3037 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3037 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3037 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3037 has memoization hash 53e0156b65b5cc008ae22b3001c7bf51 parsl.dataflow.memoization INFO: Task 3037 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3037 try 0 launched on executor multi with executor id 3038 parsl.dataflow.dflow INFO: Standard output for task 3037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e67031e3-f9f9-4313-8889-8c6f7d73b36d_isrForCrosstalkSources_996968_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e67031e3-f9f9-4313-8889-8c6f7d73b36d_isrForCrosstalkSources_996968_18.stderr parsl.dataflow.dflow DEBUG: Task 3038 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3038 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3038 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3038 has memoization hash eb85e83d5b96bcfe4be2ead17b57c028 parsl.dataflow.memoization INFO: Task 3038 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d32e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3038 try 0 launched on executor multi with executor id 3039 parsl.dataflow.dflow INFO: Standard output for task 3038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/bd47b642-33f5-4921-9ed2-c2f07e6f030f_isrForCrosstalkSources_996979_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/bd47b642-33f5-4921-9ed2-c2f07e6f030f_isrForCrosstalkSources_996979_2.stderr parsl.dataflow.dflow DEBUG: Task 3039 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3039 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3039 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3039 has memoization hash 5d6b7f3673fa94e6a8f05af7d87c9555 parsl.dataflow.memoization INFO: Task 3039 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d31a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3039 try 0 launched on executor multi with executor id 3040 parsl.dataflow.dflow INFO: Standard output for task 3039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/aba108d4-09e3-47de-b6bd-31eb36118d2d_isrForCrosstalkSources_997013_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/aba108d4-09e3-47de-b6bd-31eb36118d2d_isrForCrosstalkSources_997013_16.stderr parsl.dataflow.dflow DEBUG: Task 3040 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3040 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3040 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3040 has memoization hash e0d7abd151a88a4967d90ec33a4c4791 parsl.dataflow.memoization INFO: Task 3040 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3040 try 0 launched on executor multi with executor id 3041 parsl.dataflow.dflow INFO: Standard output for task 3040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4f192782-6a2f-4ba9-b257-ba906efb6eee_isrForCrosstalkSources_996957_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4f192782-6a2f-4ba9-b257-ba906efb6eee_isrForCrosstalkSources_996957_7.stderr parsl.dataflow.dflow DEBUG: Task 3041 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3041 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3041 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3041 has memoization hash 560958136f08fa185114e5e5295a91e7 parsl.dataflow.memoization INFO: Task 3041 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3041 try 0 launched on executor multi with executor id 3042 parsl.dataflow.dflow INFO: Standard output for task 3041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ce4f8ba5-ee26-4628-9073-ea03ce3b9ad9_isrForCrosstalkSources_997019_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ce4f8ba5-ee26-4628-9073-ea03ce3b9ad9_isrForCrosstalkSources_997019_34.stderr parsl.dataflow.dflow DEBUG: Task 3042 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3042 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3042 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3042 has memoization hash 67b3a3f6bd0e138872a776709561f725 parsl.dataflow.memoization INFO: Task 3042 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3042 try 0 launched on executor multi with executor id 3043 parsl.dataflow.dflow INFO: Standard output for task 3042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/cb55b33a-6658-4147-9939-6a6eb093c1e3_isrForCrosstalkSources_997000_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/cb55b33a-6658-4147-9939-6a6eb093c1e3_isrForCrosstalkSources_997000_49.stderr parsl.dataflow.dflow DEBUG: Task 3043 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3043 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3043 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3043 has memoization hash 934fc07552a38e8c32648a4761d81fed parsl.dataflow.memoization INFO: Task 3043 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3043 try 0 launched on executor multi with executor id 3044 parsl.dataflow.dflow INFO: Standard output for task 3043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/73ff5190-3455-4e3f-90ea-922e4541da3a_isrForCrosstalkSources_997020_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/73ff5190-3455-4e3f-90ea-922e4541da3a_isrForCrosstalkSources_997020_29.stderr parsl.dataflow.dflow DEBUG: Task 3044 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3044 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3044 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3044 has memoization hash a828a4fb09995d7e2fd427d82ec1e3da parsl.dataflow.memoization INFO: Task 3044 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3044 try 0 launched on executor multi with executor id 3045 parsl.dataflow.dflow INFO: Standard output for task 3044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/de9eab24-f14e-4e6e-9997-78e47e03d8b0_isrForCrosstalkSources_996968_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/de9eab24-f14e-4e6e-9997-78e47e03d8b0_isrForCrosstalkSources_996968_32.stderr parsl.dataflow.dflow DEBUG: Task 3045 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3045 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3045 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3045 has memoization hash cd83cadaf3bb0fe4ca8bf6355c9b171e parsl.dataflow.memoization INFO: Task 3045 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3045 try 0 launched on executor multi with executor id 3046 parsl.dataflow.dflow INFO: Standard output for task 3045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8b2458fe-5931-4bbd-a726-2d7ac86aee50_isrForCrosstalkSources_996973_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8b2458fe-5931-4bbd-a726-2d7ac86aee50_isrForCrosstalkSources_996973_43.stderr parsl.dataflow.dflow DEBUG: Task 3046 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3046 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3046 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3046 has memoization hash d82e86f6206c18045809b624d6d0b677 parsl.dataflow.memoization INFO: Task 3046 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3046 try 0 launched on executor multi with executor id 3047 parsl.dataflow.dflow INFO: Standard output for task 3046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/13561822-c4e7-4aac-9028-1ece5f90f6f8_isrForCrosstalkSources_996992_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/13561822-c4e7-4aac-9028-1ece5f90f6f8_isrForCrosstalkSources_996992_42.stderr parsl.dataflow.dflow DEBUG: Task 3047 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3047 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3047 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3047 has memoization hash a37e742db8518be3b31c3267b4ee3ba0 parsl.dataflow.memoization INFO: Task 3047 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3047 try 0 launched on executor multi with executor id 3048 parsl.dataflow.dflow INFO: Standard output for task 3047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/e4606221-2e5b-4561-86c4-9c47e3842139_isrForCrosstalkSources_996990_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/e4606221-2e5b-4561-86c4-9c47e3842139_isrForCrosstalkSources_996990_25.stderr parsl.dataflow.dflow DEBUG: Task 3048 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3048 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3048 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3048 has memoization hash b5de7e7a72462c1694826d0acebb88c8 parsl.dataflow.memoization INFO: Task 3048 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3048 try 0 launched on executor multi with executor id 3049 parsl.dataflow.dflow INFO: Standard output for task 3048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e30533f4-7a58-4bd9-ae38-8f0f2f4e6d79_isrForCrosstalkSources_996955_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e30533f4-7a58-4bd9-ae38-8f0f2f4e6d79_isrForCrosstalkSources_996955_59.stderr parsl.dataflow.dflow DEBUG: Task 3049 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3049 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3049 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3049 has memoization hash d7f03cdee32c0bdf43925622a477bc18 parsl.dataflow.memoization INFO: Task 3049 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3049 try 0 launched on executor multi with executor id 3050 parsl.dataflow.dflow INFO: Standard output for task 3049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/130878b3-1575-41e6-90b3-fa68938a3d49_isrForCrosstalkSources_996958_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/130878b3-1575-41e6-90b3-fa68938a3d49_isrForCrosstalkSources_996958_30.stderr parsl.dataflow.dflow DEBUG: Task 3050 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3050 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3050 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3050 has memoization hash ed684e77275beeb6c4b93eb717def826 parsl.dataflow.memoization INFO: Task 3050 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d2c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3050 try 0 launched on executor multi with executor id 3051 parsl.dataflow.dflow INFO: Standard output for task 3050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/619385ea-1176-42ab-b4b6-5d8632b2b716_isrForCrosstalkSources_996995_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/619385ea-1176-42ab-b4b6-5d8632b2b716_isrForCrosstalkSources_996995_27.stderr parsl.dataflow.dflow DEBUG: Task 3051 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3051 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3051 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3051 has memoization hash 83f8d39ef60f6e9d0afb96861625f527 parsl.dataflow.memoization INFO: Task 3051 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d07c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3051 try 0 launched on executor multi with executor id 3052 parsl.dataflow.dflow INFO: Standard output for task 3051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/4b464d88-c991-41f9-ab32-349746c1bcbe_isrForCrosstalkSources_997004_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/4b464d88-c991-41f9-ab32-349746c1bcbe_isrForCrosstalkSources_997004_43.stderr parsl.dataflow.dflow DEBUG: Task 3052 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3052 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3052 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3052 has memoization hash c0cb8b186c2505105d368712b7909b8b parsl.dataflow.memoization INFO: Task 3052 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d09a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3052 try 0 launched on executor multi with executor id 3053 parsl.dataflow.dflow INFO: Standard output for task 3052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/81732adc-1400-47a3-9ce9-abd941898f64_isrForCrosstalkSources_997024_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/81732adc-1400-47a3-9ce9-abd941898f64_isrForCrosstalkSources_997024_46.stderr parsl.dataflow.dflow DEBUG: Task 3053 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3053 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3053 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3053 has memoization hash ba1e28c2a3b67efa6d2d8e1f4fa2b997 parsl.dataflow.memoization INFO: Task 3053 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d3f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3053 try 0 launched on executor multi with executor id 3054 parsl.dataflow.dflow INFO: Standard output for task 3053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ea9bcd71-cbee-4a99-8614-7e63a244f79e_isrForCrosstalkSources_996962_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ea9bcd71-cbee-4a99-8614-7e63a244f79e_isrForCrosstalkSources_996962_58.stderr parsl.dataflow.dflow DEBUG: Task 3054 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3054 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3054 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3054 has memoization hash 2d88d58c5a6a02df367eb41e434dcd7c parsl.dataflow.memoization INFO: Task 3054 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3054 try 0 launched on executor multi with executor id 3055 parsl.dataflow.dflow INFO: Standard output for task 3054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/4474889b-c5cf-42a8-8efb-830e6c40b76d_isrForCrosstalkSources_996993_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/4474889b-c5cf-42a8-8efb-830e6c40b76d_isrForCrosstalkSources_996993_22.stderr parsl.dataflow.dflow DEBUG: Task 3055 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3055 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3055 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3055 has memoization hash 6ffca3351927062b82cdff3a214eebbd parsl.dataflow.memoization INFO: Task 3055 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8640e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3055 try 0 launched on executor multi with executor id 3056 parsl.dataflow.dflow INFO: Standard output for task 3055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/41f4562c-ebe4-460c-bf78-bed56f269c54_isrForCrosstalkSources_996993_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/41f4562c-ebe4-460c-bf78-bed56f269c54_isrForCrosstalkSources_996993_52.stderr parsl.dataflow.dflow DEBUG: Task 3056 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3056 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3056 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3056 has memoization hash 1b890dc6706d094f64b0f3b4414046d3 parsl.dataflow.memoization INFO: Task 3056 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3056 try 0 launched on executor multi with executor id 3057 parsl.dataflow.dflow INFO: Standard output for task 3056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5933159c-d0cf-483a-a641-6176e8c78ae2_isrForCrosstalkSources_997012_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5933159c-d0cf-483a-a641-6176e8c78ae2_isrForCrosstalkSources_997012_35.stderr parsl.dataflow.dflow DEBUG: Task 3057 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3057 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3057 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3057 has memoization hash af6893baadc9d2430bfc1704fb6c27dd parsl.dataflow.memoization INFO: Task 3057 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8642c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3057 try 0 launched on executor multi with executor id 3058 parsl.dataflow.dflow INFO: Standard output for task 3057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/191efaa9-c41e-4d52-a1c0-2538f57a6238_isrForCrosstalkSources_996956_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/191efaa9-c41e-4d52-a1c0-2538f57a6238_isrForCrosstalkSources_996956_8.stderr parsl.dataflow.dflow DEBUG: Task 3058 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3058 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3058 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3058 has memoization hash 94953f46788d3c84ee60ca6c4e59c98c parsl.dataflow.memoization INFO: Task 3058 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3058 try 0 launched on executor multi with executor id 3059 parsl.dataflow.dflow INFO: Standard output for task 3058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e523d38a-79d3-4c59-b931-be2703aa69f8_isrForCrosstalkSources_997022_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e523d38a-79d3-4c59-b931-be2703aa69f8_isrForCrosstalkSources_997022_40.stderr parsl.dataflow.dflow DEBUG: Task 3059 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3059 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3059 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3059 has memoization hash b37c6b0491915c353181b377dc90684e parsl.dataflow.memoization INFO: Task 3059 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8644a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3059 try 0 launched on executor multi with executor id 3060 parsl.dataflow.dflow INFO: Standard output for task 3059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1f0d9875-0028-485b-9f96-2e1afeb00a1c_isrForCrosstalkSources_996983_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1f0d9875-0028-485b-9f96-2e1afeb00a1c_isrForCrosstalkSources_996983_26.stderr parsl.dataflow.dflow DEBUG: Task 3060 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3060 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3060 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3060 has memoization hash 97be1d7c4067d57ad90cf8e1a0acf3a4 parsl.dataflow.memoization INFO: Task 3060 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3060 try 0 launched on executor multi with executor id 3061 parsl.dataflow.dflow INFO: Standard output for task 3060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/5f8e2515-8a34-493e-b3b4-4427fa6c02f7_isrForCrosstalkSources_997016_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/5f8e2515-8a34-493e-b3b4-4427fa6c02f7_isrForCrosstalkSources_997016_30.stderr parsl.dataflow.dflow DEBUG: Task 3061 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3061 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3061 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3061 has memoization hash 33424ab6dd3192f7f45394d487327882 parsl.dataflow.memoization INFO: Task 3061 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3061 try 0 launched on executor multi with executor id 3062 parsl.dataflow.dflow INFO: Standard output for task 3061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/7871d022-24c9-4a5f-8f85-1afd5d1579ed_isrForCrosstalkSources_996952_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/7871d022-24c9-4a5f-8f85-1afd5d1579ed_isrForCrosstalkSources_996952_53.stderr parsl.dataflow.dflow DEBUG: Task 3062 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3062 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3062 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3062 has memoization hash 44ff9380a95aa21ccf4523c8e05f59f4 parsl.dataflow.memoization INFO: Task 3062 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8645e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3062 try 0 launched on executor multi with executor id 3063 parsl.dataflow.dflow INFO: Standard output for task 3062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/29741fc6-0174-44e5-875b-10bc9cb125d5_isrForCrosstalkSources_996985_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/29741fc6-0174-44e5-875b-10bc9cb125d5_isrForCrosstalkSources_996985_39.stderr parsl.dataflow.dflow DEBUG: Task 3063 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3063 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3063 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3063 has memoization hash 8deb75307181dab2025cbf53cef9c4db parsl.dataflow.memoization INFO: Task 3063 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3063 try 0 launched on executor multi with executor id 3064 parsl.dataflow.dflow INFO: Standard output for task 3063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e7eb1d2d-6c90-4cfa-b969-af0313cfe4fe_isrForCrosstalkSources_997018_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e7eb1d2d-6c90-4cfa-b969-af0313cfe4fe_isrForCrosstalkSources_997018_19.stderr parsl.dataflow.dflow DEBUG: Task 3064 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3064 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3064 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3064 has memoization hash 1798184e89d7b636eb28a1a878ccf3b5 parsl.dataflow.memoization INFO: Task 3064 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8647c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3064 try 0 launched on executor multi with executor id 3065 parsl.dataflow.dflow INFO: Standard output for task 3064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/420478ac-58c6-4aaf-a22c-070409b37787_isrForCrosstalkSources_996962_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/420478ac-58c6-4aaf-a22c-070409b37787_isrForCrosstalkSources_996962_17.stderr parsl.dataflow.dflow DEBUG: Task 3065 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3065 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3065 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3065 has memoization hash 87ce4923a46d31d8328e663bf24ed76f parsl.dataflow.memoization INFO: Task 3065 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3065 try 0 launched on executor multi with executor id 3066 parsl.dataflow.dflow INFO: Standard output for task 3065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/cb3e7e4f-dfa8-462d-8b5d-9bd9df0c9b77_isrForCrosstalkSources_997025_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/cb3e7e4f-dfa8-462d-8b5d-9bd9df0c9b77_isrForCrosstalkSources_997025_45.stderr parsl.dataflow.dflow DEBUG: Task 3066 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3066 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3066 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3066 has memoization hash 785c447f3b96c406c96139de630f0c37 parsl.dataflow.memoization INFO: Task 3066 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3066 try 0 launched on executor multi with executor id 3067 parsl.dataflow.dflow INFO: Standard output for task 3066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/4188305a-b57c-4e4e-85ae-edacab1dda95_isrForCrosstalkSources_996965_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/4188305a-b57c-4e4e-85ae-edacab1dda95_isrForCrosstalkSources_996965_22.stderr parsl.dataflow.dflow DEBUG: Task 3067 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3067 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3067 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3067 has memoization hash 003497dc6ecb95758657ef41cf23c216 parsl.dataflow.memoization INFO: Task 3067 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3067 try 0 launched on executor multi with executor id 3068 parsl.dataflow.dflow INFO: Standard output for task 3067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e3c2a0bd-c124-40e8-b6c6-536607cd459f_isrForCrosstalkSources_997016_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e3c2a0bd-c124-40e8-b6c6-536607cd459f_isrForCrosstalkSources_997016_17.stderr parsl.dataflow.dflow DEBUG: Task 3068 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3068 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3068 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3068 has memoization hash 04cdbe6adb31abe15ffa383d5a64ede3 parsl.dataflow.memoization INFO: Task 3068 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8649a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3068 try 0 launched on executor multi with executor id 3069 parsl.dataflow.dflow INFO: Standard output for task 3068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c38d23e7-f619-4a33-9782-28f1a0097b54_isrForCrosstalkSources_996959_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c38d23e7-f619-4a33-9782-28f1a0097b54_isrForCrosstalkSources_996959_44.stderr parsl.dataflow.dflow DEBUG: Task 3069 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3069 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3069 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3069 has memoization hash a0164a1690d34b94bdcd3e463fda6477 parsl.dataflow.memoization INFO: Task 3069 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3069 try 0 launched on executor multi with executor id 3070 parsl.dataflow.dflow INFO: Standard output for task 3069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/521780b1-6310-4515-9363-f24b014d246e_isrForCrosstalkSources_996992_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/521780b1-6310-4515-9363-f24b014d246e_isrForCrosstalkSources_996992_37.stderr parsl.dataflow.dflow DEBUG: Task 3070 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3070 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3070 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3070 has memoization hash aea16c65f58fed389af265bbf1f20a76 parsl.dataflow.memoization INFO: Task 3070 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3070 try 0 launched on executor multi with executor id 3071 parsl.dataflow.dflow INFO: Standard output for task 3070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/84a660c8-461b-4b00-a6fb-ec4d8d31750c_isrForCrosstalkSources_997031_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/84a660c8-461b-4b00-a6fb-ec4d8d31750c_isrForCrosstalkSources_997031_8.stderr parsl.dataflow.dflow DEBUG: Task 3071 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3071 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3071 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3071 has memoization hash 3fb121b54c197e273f317eff481c3cfc parsl.dataflow.memoization INFO: Task 3071 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3071 try 0 launched on executor multi with executor id 3072 parsl.dataflow.dflow INFO: Standard output for task 3071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f9efee13-a9ba-43a2-8531-8814b297afe1_isrForCrosstalkSources_996977_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f9efee13-a9ba-43a2-8531-8814b297afe1_isrForCrosstalkSources_996977_41.stderr parsl.dataflow.dflow DEBUG: Task 3072 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3072 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3072 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3072 has memoization hash 8e2cb257591334989bfeb53f7eb7c4b0 parsl.dataflow.memoization INFO: Task 3072 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3072 try 0 launched on executor multi with executor id 3073 parsl.dataflow.dflow INFO: Standard output for task 3072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f523bb7f-d585-44e8-a8af-751adfa125be_isrForCrosstalkSources_996995_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f523bb7f-d585-44e8-a8af-751adfa125be_isrForCrosstalkSources_996995_2.stderr parsl.dataflow.dflow DEBUG: Task 3073 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3073 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3073 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3073 has memoization hash 1ced6d00428a59b25242b303e46c13e1 parsl.dataflow.memoization INFO: Task 3073 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3073 try 0 launched on executor multi with executor id 3074 parsl.dataflow.dflow INFO: Standard output for task 3073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7c2b48fc-d2d8-4dd4-9d8b-eee41d38de81_isrForCrosstalkSources_997024_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7c2b48fc-d2d8-4dd4-9d8b-eee41d38de81_isrForCrosstalkSources_997024_33.stderr parsl.dataflow.dflow DEBUG: Task 3074 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3074 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3074 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3074 has memoization hash f92d1652de701c3c86b1540f3cf66141 parsl.dataflow.memoization INFO: Task 3074 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3074 try 0 launched on executor multi with executor id 3075 parsl.dataflow.dflow INFO: Standard output for task 3074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/26816f57-3aa2-405b-bd68-e5f1dad1ca62_isrForCrosstalkSources_997024_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/26816f57-3aa2-405b-bd68-e5f1dad1ca62_isrForCrosstalkSources_997024_14.stderr parsl.dataflow.dflow DEBUG: Task 3075 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3075 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3075 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3075 has memoization hash dc70eab1f75cc6ff20fea2ab78b9b2ad parsl.dataflow.memoization INFO: Task 3075 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3075 try 0 launched on executor multi with executor id 3076 parsl.dataflow.dflow INFO: Standard output for task 3075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5ced96d3-2e74-4217-8972-c490cd2c3c7a_isrForCrosstalkSources_997031_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5ced96d3-2e74-4217-8972-c490cd2c3c7a_isrForCrosstalkSources_997031_9.stderr parsl.dataflow.dflow DEBUG: Task 3076 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3076 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3076 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3076 has memoization hash 3560b63f2a955ae3b3ae29d48e37cd4c parsl.dataflow.memoization INFO: Task 3076 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3076 try 0 launched on executor multi with executor id 3077 parsl.dataflow.dflow INFO: Standard output for task 3076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2ecd8fb3-30a9-4ea5-95b4-8459a8cfd81a_isrForCrosstalkSources_997002_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2ecd8fb3-30a9-4ea5-95b4-8459a8cfd81a_isrForCrosstalkSources_997002_58.stderr parsl.dataflow.dflow DEBUG: Task 3077 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3077 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3077 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3077 has memoization hash 0a600fc3e8075ad4a2a93f16a7d06b06 parsl.dataflow.memoization INFO: Task 3077 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3077 try 0 launched on executor multi with executor id 3078 parsl.dataflow.dflow INFO: Standard output for task 3077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/b9d0a722-8984-411e-8b68-bdde85514447_isrForCrosstalkSources_996952_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/b9d0a722-8984-411e-8b68-bdde85514447_isrForCrosstalkSources_996952_26.stderr parsl.dataflow.dflow DEBUG: Task 3078 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3078 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3078 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3078 has memoization hash 27f4b76b7b83a423912066d7f94e2430 parsl.dataflow.memoization INFO: Task 3078 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3078 try 0 launched on executor multi with executor id 3079 parsl.dataflow.dflow INFO: Standard output for task 3078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/35716663-aec6-479c-9084-2f66155c907a_isrForCrosstalkSources_997024_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/35716663-aec6-479c-9084-2f66155c907a_isrForCrosstalkSources_997024_18.stderr parsl.dataflow.dflow DEBUG: Task 3079 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3079 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3079 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3079 has memoization hash 4af89d52232ba630520afc79fe3495df parsl.dataflow.memoization INFO: Task 3079 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 11 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da51ccefdcedfecca7598ea1960be85a with result from task 11 parsl.dataflow.dflow DEBUG: Task 11 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 11 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/bb67a28d-d110-4cb7-bf72-78298de41a92_isrForCrosstalkSources_996969_34.stdout parsl.dataflow.dflow INFO: Standard error for task 11 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/bb67a28d-d110-4cb7-bf72-78298de41a92_isrForCrosstalkSources_996969_34.stderr parsl.dataflow.dflow INFO: Parsl task 3079 try 0 launched on executor multi with executor id 3080 parsl.dataflow.dflow INFO: Standard output for task 3079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b7abc7c7-c418-4f58-874f-4a66dc54d27a_isrForCrosstalkSources_996999_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b7abc7c7-c418-4f58-874f-4a66dc54d27a_isrForCrosstalkSources_996999_26.stderr parsl.dataflow.dflow DEBUG: Task 3080 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3080 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3080 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3080 has memoization hash 55e89831026d40a28ad0ea88df80421f parsl.dataflow.memoization INFO: Task 3080 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3080 try 0 launched on executor multi with executor id 3081 parsl.dataflow.dflow INFO: Standard output for task 3080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/79fa66d1-7c04-4c34-aac0-d5afc5866baa_isrForCrosstalkSources_996964_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/79fa66d1-7c04-4c34-aac0-d5afc5866baa_isrForCrosstalkSources_996964_55.stderr parsl.dataflow.dflow DEBUG: Task 3081 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3081 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3081 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3081 has memoization hash 82f5b6ae7a3a8ba39da706872c2a04f2 parsl.dataflow.memoization INFO: Task 3081 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8651c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3081 try 0 launched on executor multi with executor id 3082 parsl.dataflow.dflow INFO: Standard output for task 3081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/182393d4-414f-41f8-8559-13e1f91e3da9_isrForCrosstalkSources_996976_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/182393d4-414f-41f8-8559-13e1f91e3da9_isrForCrosstalkSources_996976_54.stderr parsl.dataflow.dflow DEBUG: Task 3082 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3082 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3082 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3082 has memoization hash b3f5efe427bf53ebb2c60a4a02ff87fa parsl.dataflow.memoization INFO: Task 3082 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3082 try 0 launched on executor multi with executor id 3083 parsl.dataflow.dflow INFO: Standard output for task 3082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/60c2bfe6-bdf4-4fe1-b679-a87204a74e51_isrForCrosstalkSources_996959_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/60c2bfe6-bdf4-4fe1-b679-a87204a74e51_isrForCrosstalkSources_996959_36.stderr parsl.dataflow.dflow DEBUG: Task 3083 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3083 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3083 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3083 has memoization hash 262d0d412fc91639dc2df7f10d5f2fc3 parsl.dataflow.memoization INFO: Task 3083 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3083 try 0 launched on executor multi with executor id 3084 parsl.dataflow.dflow INFO: Standard output for task 3083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ca264f78-0ef3-465e-816a-9325e1941a5f_isrForCrosstalkSources_997030_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ca264f78-0ef3-465e-816a-9325e1941a5f_isrForCrosstalkSources_997030_17.stderr parsl.dataflow.dflow DEBUG: Task 3084 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3084 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3084 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3084 has memoization hash 2082acaa1a69f63151b40c1e75f247ad parsl.dataflow.memoization INFO: Task 3084 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3084 try 0 launched on executor multi with executor id 3085 parsl.dataflow.dflow INFO: Standard output for task 3084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/5bd2a0f1-f73b-40a5-ace0-41896dde4848_isrForCrosstalkSources_996952_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/5bd2a0f1-f73b-40a5-ace0-41896dde4848_isrForCrosstalkSources_996952_3.stderr parsl.dataflow.dflow DEBUG: Task 3085 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3085 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3085 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3085 has memoization hash a897478c1caa9cfd659ae6b5f2ec1584 parsl.dataflow.memoization INFO: Task 3085 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8653a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3085 try 0 launched on executor multi with executor id 3086 parsl.dataflow.dflow INFO: Standard output for task 3085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/0d1b2c91-5bf1-43a2-8fbb-3a4e47a33076_isrForCrosstalkSources_996965_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/0d1b2c91-5bf1-43a2-8fbb-3a4e47a33076_isrForCrosstalkSources_996965_54.stderr parsl.dataflow.dflow DEBUG: Task 3086 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3086 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3086 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3086 has memoization hash c97f0a192d9b17c8a88e093f17309551 parsl.dataflow.memoization INFO: Task 3086 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8654e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3086 try 0 launched on executor multi with executor id 3087 parsl.dataflow.dflow INFO: Standard output for task 3086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/f921eb32-bafd-4bf8-9e1a-408f7294ed8c_isrForCrosstalkSources_997014_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/f921eb32-bafd-4bf8-9e1a-408f7294ed8c_isrForCrosstalkSources_997014_49.stderr parsl.dataflow.dflow DEBUG: Task 3087 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3087 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3087 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3087 has memoization hash b147eccca8403b687e3e5c44c281d0c1 parsl.dataflow.memoization INFO: Task 3087 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3087 try 0 launched on executor multi with executor id 3088 parsl.dataflow.dflow INFO: Standard output for task 3087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68aa2644-aebf-4fe4-b2e7-b56d61cece5b_isrForCrosstalkSources_996985_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68aa2644-aebf-4fe4-b2e7-b56d61cece5b_isrForCrosstalkSources_996985_22.stderr parsl.dataflow.dflow DEBUG: Task 3088 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3088 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3088 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3088 has memoization hash 886241d1ef415e60ccbc87dd85eb4b65 parsl.dataflow.memoization INFO: Task 3088 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3088 try 0 launched on executor multi with executor id 3089 parsl.dataflow.dflow INFO: Standard output for task 3088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/39ff7f8d-2a5a-4d06-8488-64b9e6a5655a_isrForCrosstalkSources_996996_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/39ff7f8d-2a5a-4d06-8488-64b9e6a5655a_isrForCrosstalkSources_996996_41.stderr parsl.dataflow.dflow DEBUG: Task 3089 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3089 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3089 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3089 has memoization hash cba951673e6891a41c795e0bc77f3de5 parsl.dataflow.memoization INFO: Task 3089 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3089 try 0 launched on executor multi with executor id 3090 parsl.dataflow.dflow INFO: Standard output for task 3089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/66e037d2-fd7b-4001-bc19-5bd3b90bcd75_isrForCrosstalkSources_996975_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/66e037d2-fd7b-4001-bc19-5bd3b90bcd75_isrForCrosstalkSources_996975_16.stderr parsl.dataflow.dflow DEBUG: Task 3090 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3090 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3090 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3090 has memoization hash c3aa58d1cbda89600b2b790e5aa43f2d parsl.dataflow.memoization INFO: Task 3090 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8656c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3090 try 0 launched on executor multi with executor id 3091 parsl.dataflow.dflow INFO: Standard output for task 3090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/389e64d5-77a8-4369-932d-7c8d66f9b244_isrForCrosstalkSources_996984_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/389e64d5-77a8-4369-932d-7c8d66f9b244_isrForCrosstalkSources_996984_26.stderr parsl.dataflow.dflow DEBUG: Task 3091 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3091 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3091 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3091 has memoization hash 30c7176750d3955a681957a622d4e6a0 parsl.dataflow.memoization INFO: Task 3091 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3091 try 0 launched on executor multi with executor id 3092 parsl.dataflow.dflow INFO: Standard output for task 3091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/48df183e-0a3b-4eec-a849-947c545c5628_isrForCrosstalkSources_996956_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/48df183e-0a3b-4eec-a849-947c545c5628_isrForCrosstalkSources_996956_12.stderr parsl.dataflow.dflow DEBUG: Task 3092 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3092 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3092 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3092 has memoization hash cb1a2069d81441632688400a9851e131 parsl.dataflow.memoization INFO: Task 3092 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8658a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3092 try 0 launched on executor multi with executor id 3093 parsl.dataflow.dflow INFO: Standard output for task 3092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1ccfd1b0-f028-482b-b5f1-2e24563e8c51_isrForCrosstalkSources_996979_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1ccfd1b0-f028-482b-b5f1-2e24563e8c51_isrForCrosstalkSources_996979_48.stderr parsl.dataflow.dflow DEBUG: Task 3093 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3093 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3093 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3093 has memoization hash bfae83f0cd735d78786510dc30ef8546 parsl.dataflow.memoization INFO: Task 3093 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:54 proc_lsst.multi:170 [INFO] [multi] job_id 17074881 proc_lsst.multi INFO: [multi] job_id 17074881 2024-03-15 13:49:54 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 2 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 2 2024-03-15 13:49:54 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:54 proc_lsst.multi:178 [INFO] [multi] provider escience_rtx6k accepted submit and returned 17074881 proc_lsst.multi INFO: [multi] provider escience_rtx6k accepted submit and returned 17074881 parsl.executors.status_handling DEBUG: Launched block 32 on executor multi with job ID 17074881 parsl.executors.status_handling INFO: Allocated block ID 33 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-33 2024-03-15 13:49:54 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-33 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-33 2024-03-15 13:49:54 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:54 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:49:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:54 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 2 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 2 2024-03-15 13:49:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:54 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=33 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 3093 try 0 launched on executor multi with executor id 3094 parsl.dataflow.dflow INFO: Standard output for task 3093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f2d9a9d9-4686-4257-8d10-8ac7df53babd_isrForCrosstalkSources_996953_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f2d9a9d9-4686-4257-8d10-8ac7df53babd_isrForCrosstalkSources_996953_44.stderr parsl.dataflow.dflow DEBUG: Task 3094 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3094 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3094 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3094 has memoization hash 1fa8ff7f3a7600943b4041b7e3034b50 parsl.dataflow.memoization INFO: Task 3094 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3094 try 0 launched on executor multi with executor id 3095 parsl.dataflow.dflow INFO: Standard output for task 3094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5b03ed22-6a15-4441-aaca-4554315f2b98_isrForCrosstalkSources_996954_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5b03ed22-6a15-4441-aaca-4554315f2b98_isrForCrosstalkSources_996954_7.stderr parsl.dataflow.dflow DEBUG: Task 3095 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3095 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3095 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3095 has memoization hash 37fdd7bc349da7e5549d9d4591b6404b parsl.dataflow.memoization INFO: Task 3095 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 3095 try 0 launched on executor multi with executor id 3096 parsl.dataflow.dflow INFO: Standard output for task 3095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/285b4a39-a344-4ae3-a649-6b93608cf698_isrForCrosstalkSources_996974_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/285b4a39-a344-4ae3-a649-6b93608cf698_isrForCrosstalkSources_996974_41.stderr parsl.dataflow.dflow DEBUG: Task 3096 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3096 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3096 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3096 has memoization hash 497208e1a19bcab4a018e85c630eb77f parsl.dataflow.memoization INFO: Task 3096 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3096 try 0 launched on executor multi with executor id 3097 parsl.dataflow.dflow INFO: Standard output for task 3096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/368b6bb3-c27f-48c4-a00d-025c4b678794_isrForCrosstalkSources_996963_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/368b6bb3-c27f-48c4-a00d-025c4b678794_isrForCrosstalkSources_996963_18.stderr parsl.dataflow.dflow DEBUG: Task 3097 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3097 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3097 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3097 has memoization hash 2c55708fcf9b972fbeeb9be3b49051fc parsl.dataflow.memoization INFO: Task 3097 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3097 try 0 launched on executor multi with executor id 3098 parsl.dataflow.dflow INFO: Standard output for task 3097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/279e6f80-5e82-4457-834c-72ef418c2b02_isrForCrosstalkSources_997020_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/279e6f80-5e82-4457-834c-72ef418c2b02_isrForCrosstalkSources_997020_9.stderr parsl.dataflow.dflow DEBUG: Task 3098 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3098 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3098 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3098 has memoization hash 180600c45864f6d578cfb31ae7488e43 parsl.dataflow.memoization INFO: Task 3098 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3098 try 0 launched on executor multi with executor id 3099 parsl.dataflow.dflow INFO: Standard output for task 3098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/e790e64c-cb47-4986-9eb4-fa42d4eb815f_isrForCrosstalkSources_996994_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/e790e64c-cb47-4986-9eb4-fa42d4eb815f_isrForCrosstalkSources_996994_31.stderr parsl.dataflow.dflow DEBUG: Task 3099 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3099 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3099 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3099 has memoization hash 19e0f9d712357ae1078f2ff5ba1cdbfc parsl.dataflow.memoization INFO: Task 3099 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3099 try 0 launched on executor multi with executor id 3100 parsl.dataflow.dflow INFO: Standard output for task 3099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7730d3ff-6a12-46dc-a079-841b49c896cc_isrForCrosstalkSources_996975_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7730d3ff-6a12-46dc-a079-841b49c896cc_isrForCrosstalkSources_996975_27.stderr parsl.dataflow.dflow DEBUG: Task 3100 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3100 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3100 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3100 has memoization hash aa7207ec20147d8fe4904e4de6940449 parsl.dataflow.memoization INFO: Task 3100 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8659e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3100 try 0 launched on executor multi with executor id 3101 parsl.dataflow.dflow INFO: Standard output for task 3100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a0ffffbf-321b-4899-a4c4-2eba50ec08e0_isrForCrosstalkSources_997026_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a0ffffbf-321b-4899-a4c4-2eba50ec08e0_isrForCrosstalkSources_997026_2.stderr parsl.dataflow.dflow DEBUG: Task 3101 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3101 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3101 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3101 has memoization hash 3f33e9de2894de45d3c4d6fca6f96cd9 parsl.dataflow.memoization INFO: Task 3101 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3101 try 0 launched on executor multi with executor id 3102 parsl.dataflow.dflow INFO: Standard output for task 3101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/3530f1e3-e453-4de0-a99d-0446f3855ec4_isrForCrosstalkSources_996965_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/3530f1e3-e453-4de0-a99d-0446f3855ec4_isrForCrosstalkSources_996965_51.stderr parsl.dataflow.dflow DEBUG: Task 3102 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3102 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3102 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3102 has memoization hash f9dafe2a5b69924eff9062ccf5816c36 parsl.dataflow.memoization INFO: Task 3102 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3102 try 0 launched on executor multi with executor id 3103 parsl.dataflow.dflow INFO: Standard output for task 3102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/46e468d9-9990-45f7-a69a-4b5205ac1e08_isrForCrosstalkSources_996966_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/46e468d9-9990-45f7-a69a-4b5205ac1e08_isrForCrosstalkSources_996966_32.stderr parsl.dataflow.dflow DEBUG: Task 3103 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3103 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3103 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3103 has memoization hash e67ecbad58793f8b7121ae3d69f4479c parsl.dataflow.memoization INFO: Task 3103 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3103 try 0 launched on executor multi with executor id 3104 parsl.dataflow.dflow INFO: Standard output for task 3103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9cc8d5f3-5e08-48f0-9a15-1f8817f10633_isrForCrosstalkSources_997004_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9cc8d5f3-5e08-48f0-9a15-1f8817f10633_isrForCrosstalkSources_997004_9.stderr parsl.dataflow.dflow DEBUG: Task 3104 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3104 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3104 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3104 has memoization hash bedb35816a4e19650beaf25180f21f5a parsl.dataflow.memoization INFO: Task 3104 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3104 try 0 launched on executor multi with executor id 3105 parsl.dataflow.dflow INFO: Standard output for task 3104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7b1c9eba-c856-4402-b4c8-95d39c66adb0_isrForCrosstalkSources_996990_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7b1c9eba-c856-4402-b4c8-95d39c66adb0_isrForCrosstalkSources_996990_47.stderr parsl.dataflow.dflow DEBUG: Task 3105 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3105 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3105 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3105 has memoization hash 6cfcf14356d35aa284df62dd8c74a70c parsl.dataflow.memoization INFO: Task 3105 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3105 try 0 launched on executor multi with executor id 3106 parsl.dataflow.dflow INFO: Standard output for task 3105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/4c0751d9-8b30-4ee5-998b-12cda46dd6ad_isrForCrosstalkSources_997022_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/4c0751d9-8b30-4ee5-998b-12cda46dd6ad_isrForCrosstalkSources_997022_55.stderr parsl.dataflow.dflow DEBUG: Task 3106 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3106 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3106 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3106 has memoization hash b9bc1287b384d9fd9f9c584b287b1e2b parsl.dataflow.memoization INFO: Task 3106 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3106 try 0 launched on executor multi with executor id 3107 parsl.dataflow.dflow INFO: Standard output for task 3106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/9f122589-9761-4d5b-ac08-40955bafe2ec_isrForCrosstalkSources_996973_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/9f122589-9761-4d5b-ac08-40955bafe2ec_isrForCrosstalkSources_996973_26.stderr parsl.dataflow.dflow DEBUG: Task 3107 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3107 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3107 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3107 has memoization hash 26bc1987cb97ce25ee77af592f31e363 parsl.dataflow.memoization INFO: Task 3107 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3107 try 0 launched on executor multi with executor id 3108 parsl.dataflow.dflow INFO: Standard output for task 3107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6ad526e-b6dd-450d-9bcc-d81d9214255b_isrForCrosstalkSources_997026_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6ad526e-b6dd-450d-9bcc-d81d9214255b_isrForCrosstalkSources_997026_31.stderr parsl.dataflow.dflow DEBUG: Task 3108 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3108 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3108 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3108 has memoization hash 15ef9771c71d493721d4b68e52fc311d parsl.dataflow.memoization INFO: Task 3108 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8660c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3108 try 0 launched on executor multi with executor id 3109 parsl.dataflow.dflow INFO: Standard output for task 3108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/b395012e-69da-4242-b61b-94c42509bbcd_isrForCrosstalkSources_997002_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/b395012e-69da-4242-b61b-94c42509bbcd_isrForCrosstalkSources_997002_44.stderr parsl.dataflow.dflow DEBUG: Task 3109 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3109 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3109 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3109 has memoization hash 6e20ca28c72d72794e24fdcc20c8fe12 parsl.dataflow.memoization INFO: Task 3109 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8662a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3109 try 0 launched on executor multi with executor id 3110 parsl.dataflow.dflow INFO: Standard output for task 3109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6de43bfc-5256-4de3-8f63-fc196048deeb_isrForCrosstalkSources_997010_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6de43bfc-5256-4de3-8f63-fc196048deeb_isrForCrosstalkSources_997010_22.stderr parsl.dataflow.dflow DEBUG: Task 3110 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3110 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3110 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3110 has memoization hash 26d3f4027769e0088069529f1c84d5ea parsl.dataflow.memoization INFO: Task 3110 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3110 try 0 launched on executor multi with executor id 3111 parsl.dataflow.dflow INFO: Standard output for task 3110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3a4adde7-5487-4cd9-8b26-2a4ff67a6749_isrForCrosstalkSources_996999_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3a4adde7-5487-4cd9-8b26-2a4ff67a6749_isrForCrosstalkSources_996999_17.stderr parsl.dataflow.dflow DEBUG: Task 3111 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3111 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3111 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3111 has memoization hash 764dee5722845ef8ec8c71de72431c85 parsl.dataflow.memoization INFO: Task 3111 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3111 try 0 launched on executor multi with executor id 3112 parsl.dataflow.dflow INFO: Standard output for task 3111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/4bd4510a-3d84-4a85-bd77-19425eb979f8_isrForCrosstalkSources_996979_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/4bd4510a-3d84-4a85-bd77-19425eb979f8_isrForCrosstalkSources_996979_37.stderr parsl.dataflow.dflow DEBUG: Task 3112 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3112 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3112 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3112 has memoization hash 951a0793ac0518620bc6ba6f10485116 parsl.dataflow.memoization INFO: Task 3112 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3112 try 0 launched on executor multi with executor id 3113 parsl.dataflow.dflow INFO: Standard output for task 3112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5dddb490-a647-468d-a66b-62a2421419b7_isrForCrosstalkSources_996999_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5dddb490-a647-468d-a66b-62a2421419b7_isrForCrosstalkSources_996999_47.stderr parsl.dataflow.dflow DEBUG: Task 3113 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3113 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3113 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3113 has memoization hash 394eea11f85f9a5d4964f09fab96666a parsl.dataflow.memoization INFO: Task 3113 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8663e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3113 try 0 launched on executor multi with executor id 3114 parsl.dataflow.dflow INFO: Standard output for task 3113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1da4932f-8f04-413d-9548-c2a75bb14451_isrForCrosstalkSources_996983_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1da4932f-8f04-413d-9548-c2a75bb14451_isrForCrosstalkSources_996983_61.stderr parsl.dataflow.dflow DEBUG: Task 3114 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3114 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3114 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3114 has memoization hash 3444236807e198f35ec878a4fe817655 parsl.dataflow.memoization INFO: Task 3114 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3114 try 0 launched on executor multi with executor id 3115 parsl.dataflow.dflow INFO: Standard output for task 3114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c80befd4-e15b-4634-b961-a8a18cc51c7f_isrForCrosstalkSources_997025_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c80befd4-e15b-4634-b961-a8a18cc51c7f_isrForCrosstalkSources_997025_30.stderr parsl.dataflow.dflow DEBUG: Task 3115 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3115 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3115 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3115 has memoization hash 6b21373053c85a1ab697bc40d2569c47 parsl.dataflow.memoization INFO: Task 3115 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3115 try 0 launched on executor multi with executor id 3116 parsl.dataflow.dflow INFO: Standard output for task 3115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7c716cb8-4fb8-4fa0-b214-1a5a4c22d133_isrForCrosstalkSources_997015_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7c716cb8-4fb8-4fa0-b214-1a5a4c22d133_isrForCrosstalkSources_997015_29.stderr parsl.dataflow.dflow DEBUG: Task 3116 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3116 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3116 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3116 has memoization hash c75f4072f02c12b265d76f27f6368cd1 parsl.dataflow.memoization INFO: Task 3116 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3116 try 0 launched on executor multi with executor id 3117 parsl.dataflow.dflow INFO: Standard output for task 3116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/f48792d7-f5fc-4a03-9a8e-c75d8f20fe0b_isrForCrosstalkSources_996982_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/f48792d7-f5fc-4a03-9a8e-c75d8f20fe0b_isrForCrosstalkSources_996982_47.stderr parsl.dataflow.dflow DEBUG: Task 3117 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3117 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3117 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3117 has memoization hash 389083b5900eb03d84672acb0b180bde parsl.dataflow.memoization INFO: Task 3117 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3117 try 0 launched on executor multi with executor id 3118 parsl.dataflow.dflow INFO: Standard output for task 3117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4f5a8ca7-6a17-43fe-a1ea-f76beb932e5f_isrForCrosstalkSources_997017_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4f5a8ca7-6a17-43fe-a1ea-f76beb932e5f_isrForCrosstalkSources_997017_26.stderr parsl.dataflow.dflow DEBUG: Task 3118 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3118 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3118 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3118 has memoization hash bf2556ccdd73aa5fbdee97679c2fa7d7 parsl.dataflow.memoization INFO: Task 3118 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8667a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3118 try 0 launched on executor multi with executor id 3119 parsl.dataflow.dflow INFO: Standard output for task 3118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c3a7ede0-813a-4a97-882e-ae92257cf998_isrForCrosstalkSources_996997_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c3a7ede0-813a-4a97-882e-ae92257cf998_isrForCrosstalkSources_996997_50.stderr parsl.dataflow.dflow DEBUG: Task 3119 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3119 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3119 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3119 has memoization hash e569f5ff155b716177261845369ff2c3 parsl.dataflow.memoization INFO: Task 3119 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3119 try 0 launched on executor multi with executor id 3120 parsl.dataflow.dflow INFO: Standard output for task 3119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/2a755af7-6a2b-48f9-a2f1-66211cd606b2_isrForCrosstalkSources_996999_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/2a755af7-6a2b-48f9-a2f1-66211cd606b2_isrForCrosstalkSources_996999_10.stderr parsl.dataflow.dflow DEBUG: Task 3120 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3120 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3120 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3120 has memoization hash 691d8d110f5d4316f526b3098b40e0d4 parsl.dataflow.memoization INFO: Task 3120 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8665c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3120 try 0 launched on executor multi with executor id 3121 parsl.dataflow.dflow INFO: Standard output for task 3120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bb5bc034-5bc5-4141-9c31-524ccc5f4461_isrForCrosstalkSources_997027_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bb5bc034-5bc5-4141-9c31-524ccc5f4461_isrForCrosstalkSources_997027_19.stderr parsl.dataflow.dflow DEBUG: Task 3121 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3121 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3121 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3121 has memoization hash 44c687d791b32b7b7a328db637480e22 parsl.dataflow.memoization INFO: Task 3121 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3121 try 0 launched on executor multi with executor id 3122 parsl.dataflow.dflow INFO: Standard output for task 3121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/04e5ceeb-74e4-48a5-84a1-2e2065138976_isrForCrosstalkSources_996982_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/04e5ceeb-74e4-48a5-84a1-2e2065138976_isrForCrosstalkSources_996982_22.stderr parsl.dataflow.dflow DEBUG: Task 3122 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3122 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3122 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3122 has memoization hash 0a850727c791f6fde01a2d3ecf20d00f parsl.dataflow.memoization INFO: Task 3122 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3122 try 0 launched on executor multi with executor id 3123 parsl.dataflow.dflow INFO: Standard output for task 3122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/339d5cc7-23d1-4ed7-83cc-cab8c579d7db_isrForCrosstalkSources_997000_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/339d5cc7-23d1-4ed7-83cc-cab8c579d7db_isrForCrosstalkSources_997000_17.stderr parsl.dataflow.dflow DEBUG: Task 3123 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3123 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3123 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3123 has memoization hash a30ba5c20f150490d27ba325db850c22 parsl.dataflow.memoization INFO: Task 3123 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3123 try 0 launched on executor multi with executor id 3124 parsl.dataflow.dflow INFO: Standard output for task 3123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/73c2d3ff-bd0e-450e-a6fc-744d61452613_isrForCrosstalkSources_997007_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/73c2d3ff-bd0e-450e-a6fc-744d61452613_isrForCrosstalkSources_997007_2.stderr parsl.dataflow.dflow DEBUG: Task 3124 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3124 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3124 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3124 has memoization hash e3764dc1b25175570f789edb0ddadeee parsl.dataflow.memoization INFO: Task 3124 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3124 try 0 launched on executor multi with executor id 3125 parsl.dataflow.dflow INFO: Standard output for task 3124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8bebce0f-7570-49d6-96fe-017ee46effeb_isrForCrosstalkSources_996977_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8bebce0f-7570-49d6-96fe-017ee46effeb_isrForCrosstalkSources_996977_50.stderr parsl.dataflow.dflow DEBUG: Task 3125 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3125 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3125 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3125 has memoization hash 749eb5031b872ac119547e9ef85aaa73 parsl.dataflow.memoization INFO: Task 3125 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8668e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3125 try 0 launched on executor multi with executor id 3126 parsl.dataflow.dflow INFO: Standard output for task 3125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/c015e3e3-02d3-4077-879b-90c9ab42cadd_isrForCrosstalkSources_996996_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/c015e3e3-02d3-4077-879b-90c9ab42cadd_isrForCrosstalkSources_996996_57.stderr parsl.dataflow.dflow DEBUG: Task 3126 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3126 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3126 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3126 has memoization hash 1c18a51a645470399e98f273e63b8a26 parsl.dataflow.memoization INFO: Task 3126 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3126 try 0 launched on executor multi with executor id 3127 parsl.dataflow.dflow INFO: Standard output for task 3126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b38e3d1f-ebca-4505-84ca-616208312679_isrForCrosstalkSources_996953_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b38e3d1f-ebca-4505-84ca-616208312679_isrForCrosstalkSources_996953_2.stderr parsl.dataflow.dflow DEBUG: Task 3127 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3127 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3127 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3127 has memoization hash 7be184d86b7100c1f2a0f971957001b2 parsl.dataflow.memoization INFO: Task 3127 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3127 try 0 launched on executor multi with executor id 3128 parsl.dataflow.dflow INFO: Standard output for task 3127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/66b0731f-c75b-4a9d-bf41-cb06cbdd79d1_isrForCrosstalkSources_996975_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/66b0731f-c75b-4a9d-bf41-cb06cbdd79d1_isrForCrosstalkSources_996975_34.stderr parsl.dataflow.dflow DEBUG: Task 3128 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3128 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3128 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3128 has memoization hash 4030b158143fdb918a92e880184485f1 parsl.dataflow.memoization INFO: Task 3128 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3128 try 0 launched on executor multi with executor id 3129 parsl.dataflow.dflow INFO: Standard output for task 3128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/ede89e6e-fd7b-42d1-b261-93e11390b812_isrForCrosstalkSources_996973_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/ede89e6e-fd7b-42d1-b261-93e11390b812_isrForCrosstalkSources_996973_52.stderr parsl.dataflow.dflow DEBUG: Task 3129 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3129 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3129 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3129 has memoization hash d7f935eeb786560f7b795f9bc719a003 parsl.dataflow.memoization INFO: Task 3129 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3129 try 0 launched on executor multi with executor id 3130 parsl.dataflow.dflow INFO: Standard output for task 3129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/8172a616-97d8-4e43-9d82-d388d709dca4_isrForCrosstalkSources_997015_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/8172a616-97d8-4e43-9d82-d388d709dca4_isrForCrosstalkSources_997015_16.stderr parsl.dataflow.dflow DEBUG: Task 3130 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3130 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3130 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3130 has memoization hash 86152e23a80610e31dd2760e5af9d805 parsl.dataflow.memoization INFO: Task 3130 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3130 try 0 launched on executor multi with executor id 3131 parsl.dataflow.dflow INFO: Standard output for task 3130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9425560b-0a13-4543-bd34-b3c392d524a5_isrForCrosstalkSources_996995_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9425560b-0a13-4543-bd34-b3c392d524a5_isrForCrosstalkSources_996995_55.stderr parsl.dataflow.dflow DEBUG: Task 3131 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3131 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3131 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3131 has memoization hash 3df1d4915aca9144deb17244d2d15314 parsl.dataflow.memoization INFO: Task 3131 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3131 try 0 launched on executor multi with executor id 3132 parsl.dataflow.dflow INFO: Standard output for task 3131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/1c409f64-86ba-49ae-807f-9e9c87630170_isrForCrosstalkSources_996972_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/1c409f64-86ba-49ae-807f-9e9c87630170_isrForCrosstalkSources_996972_6.stderr parsl.dataflow.dflow DEBUG: Task 3132 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3132 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3132 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3132 has memoization hash b7eae3fb0c4478e7489ffb4b740adfd5 parsl.dataflow.memoization INFO: Task 3132 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c866fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3132 try 0 launched on executor multi with executor id 3133 parsl.dataflow.dflow INFO: Standard output for task 3132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/59b23405-670e-45bd-9b4a-bec2074b20f0_isrForCrosstalkSources_996979_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/59b23405-670e-45bd-9b4a-bec2074b20f0_isrForCrosstalkSources_996979_6.stderr parsl.dataflow.dflow DEBUG: Task 3133 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3133 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3133 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3133 has memoization hash b44e000a29ad2ec4f665a0fdca1fb72e parsl.dataflow.memoization INFO: Task 3133 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3133 try 0 launched on executor multi with executor id 3134 parsl.dataflow.dflow INFO: Standard output for task 3133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/be429ecb-6fae-46c9-aff1-8eb7905f187c_isrForCrosstalkSources_996984_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/be429ecb-6fae-46c9-aff1-8eb7905f187c_isrForCrosstalkSources_996984_10.stderr parsl.dataflow.dflow DEBUG: Task 3134 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3134 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3134 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3134 has memoization hash 522f81db866844d47dcf49ae32ef2fad parsl.dataflow.memoization INFO: Task 3134 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3134 try 0 launched on executor multi with executor id 3135 parsl.dataflow.dflow INFO: Standard output for task 3134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0f2a29cc-7016-4ad6-807f-8983c658f848_isrForCrosstalkSources_997025_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0f2a29cc-7016-4ad6-807f-8983c658f848_isrForCrosstalkSources_997025_29.stderr parsl.dataflow.dflow DEBUG: Task 3135 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3135 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3135 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3135 has memoization hash 3c9f704135b70c3f7d9f478912200262 parsl.dataflow.memoization INFO: Task 3135 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3135 try 0 launched on executor multi with executor id 3136 parsl.dataflow.dflow INFO: Standard output for task 3135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/895104c2-d698-495e-87c7-ef5e67a883ec_isrForCrosstalkSources_996983_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/895104c2-d698-495e-87c7-ef5e67a883ec_isrForCrosstalkSources_996983_29.stderr parsl.dataflow.dflow DEBUG: Task 3136 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3136 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3136 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3136 has memoization hash 61448a1b2b8ddce78d98cf0bfef431cd parsl.dataflow.memoization INFO: Task 3136 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8671a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3136 try 0 launched on executor multi with executor id 3137 parsl.dataflow.dflow INFO: Standard output for task 3136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/93b9336d-dc5f-418f-a833-fd9ef095c264_isrForCrosstalkSources_996965_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/93b9336d-dc5f-418f-a833-fd9ef095c264_isrForCrosstalkSources_996965_20.stderr parsl.dataflow.dflow DEBUG: Task 3137 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3137 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3137 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3137 has memoization hash c0ed36c1b391fa89d532d99d9ce8d15a parsl.dataflow.memoization INFO: Task 3137 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3137 try 0 launched on executor multi with executor id 3138 parsl.dataflow.dflow INFO: Standard output for task 3137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d4353d73-e580-4be7-9b3e-ddd33147ef95_isrForCrosstalkSources_996995_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d4353d73-e580-4be7-9b3e-ddd33147ef95_isrForCrosstalkSources_996995_25.stderr parsl.dataflow.dflow DEBUG: Task 3138 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3138 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3138 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3138 has memoization hash 8a2127cd4c3e3d17660fc35523f81962 parsl.dataflow.memoization INFO: Task 3138 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3138 try 0 launched on executor multi with executor id 3139 parsl.dataflow.dflow INFO: Standard output for task 3138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6208a248-de2f-4c5b-b420-9fa95195ec11_isrForCrosstalkSources_996973_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6208a248-de2f-4c5b-b420-9fa95195ec11_isrForCrosstalkSources_996973_54.stderr parsl.dataflow.dflow DEBUG: Task 3139 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3139 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3139 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3139 has memoization hash 53c26fe370bfad2c01ce4eafa99f9473 parsl.dataflow.memoization INFO: Task 3139 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3139 try 0 launched on executor multi with executor id 3140 parsl.dataflow.dflow INFO: Standard output for task 3139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cf36eba1-0543-4f97-bfb8-e85df080d21d_isrForCrosstalkSources_997015_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cf36eba1-0543-4f97-bfb8-e85df080d21d_isrForCrosstalkSources_997015_59.stderr parsl.dataflow.dflow DEBUG: Task 3140 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3140 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3140 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3140 has memoization hash d56d4714f8f8bc18bf5f5d6a66328838 parsl.dataflow.memoization INFO: Task 3140 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8672e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3140 try 0 launched on executor multi with executor id 3141 parsl.dataflow.dflow INFO: Standard output for task 3140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/99c3227f-de5c-4527-b325-734793fea5b2_isrForCrosstalkSources_997003_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/99c3227f-de5c-4527-b325-734793fea5b2_isrForCrosstalkSources_997003_55.stderr parsl.dataflow.dflow DEBUG: Task 3141 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3141 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3141 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3141 has memoization hash 4f16131a29fc0b5a3503f8a358377318 parsl.dataflow.memoization INFO: Task 3141 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3141 try 0 launched on executor multi with executor id 3142 parsl.dataflow.dflow INFO: Standard output for task 3141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/34a10a89-8452-4b47-ba7c-2e9ef67574d3_isrForCrosstalkSources_996993_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/34a10a89-8452-4b47-ba7c-2e9ef67574d3_isrForCrosstalkSources_996993_58.stderr parsl.dataflow.dflow DEBUG: Task 3142 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3142 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3142 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3142 has memoization hash d54b65c734a1f6319665379620f06d10 parsl.dataflow.memoization INFO: Task 3142 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8676a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3142 try 0 launched on executor multi with executor id 3143 parsl.dataflow.dflow INFO: Standard output for task 3142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/98afd34f-4e19-4c3e-bc3f-9258668b6d09_isrForCrosstalkSources_997016_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/98afd34f-4e19-4c3e-bc3f-9258668b6d09_isrForCrosstalkSources_997016_22.stderr parsl.dataflow.dflow DEBUG: Task 3143 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3143 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3143 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3143 has memoization hash c6e889051b4526851d450b0f63f2030e parsl.dataflow.memoization INFO: Task 3143 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8674c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3143 try 0 launched on executor multi with executor id 3144 parsl.dataflow.dflow INFO: Standard output for task 3143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a9b3f19f-531b-4ac7-9004-61c5fe35f8c1_isrForCrosstalkSources_997001_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a9b3f19f-531b-4ac7-9004-61c5fe35f8c1_isrForCrosstalkSources_997001_40.stderr parsl.dataflow.dflow DEBUG: Task 3144 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3144 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3144 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3144 has memoization hash d58189355bb3be76db2198b863f643e7 parsl.dataflow.memoization INFO: Task 3144 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3144 try 0 launched on executor multi with executor id 3145 parsl.dataflow.dflow INFO: Standard output for task 3144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/df32e980-cdd0-44c8-a8ee-4c6485f8a8b8_isrForCrosstalkSources_997014_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/df32e980-cdd0-44c8-a8ee-4c6485f8a8b8_isrForCrosstalkSources_997014_47.stderr parsl.dataflow.dflow DEBUG: Task 3145 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3145 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3145 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3145 has memoization hash 8751070e658da5b5b40a452e9a09cb14 parsl.dataflow.memoization INFO: Task 3145 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8677e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3145 try 0 launched on executor multi with executor id 3146 parsl.dataflow.dflow INFO: Standard output for task 3145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f016a26b-f0c5-47cb-837d-31d7075c10a3_isrForCrosstalkSources_996995_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f016a26b-f0c5-47cb-837d-31d7075c10a3_isrForCrosstalkSources_996995_32.stderr parsl.dataflow.dflow DEBUG: Task 3146 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3146 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3146 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3146 has memoization hash e828027cb0c8bba16f9f88488c7b4ea9 parsl.dataflow.memoization INFO: Task 3146 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3146 try 0 launched on executor multi with executor id 3147 parsl.dataflow.dflow INFO: Standard output for task 3146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/7339dc9d-f267-40b8-899b-696ed2eb495e_isrForCrosstalkSources_996962_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/7339dc9d-f267-40b8-899b-696ed2eb495e_isrForCrosstalkSources_996962_47.stderr parsl.dataflow.dflow DEBUG: Task 3147 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3147 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3147 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3147 has memoization hash 438442738c0386bf3aa575dc1eacaf36 parsl.dataflow.memoization INFO: Task 3147 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3147 try 0 launched on executor multi with executor id 3148 parsl.dataflow.dflow INFO: Standard output for task 3147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/0002c6d8-4816-4ab0-b94a-4d22376af9bc_isrForCrosstalkSources_997004_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/0002c6d8-4816-4ab0-b94a-4d22376af9bc_isrForCrosstalkSources_997004_28.stderr parsl.dataflow.dflow DEBUG: Task 3148 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3148 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3148 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3148 has memoization hash 02273af2a1732ce70cc43b730ee0999f parsl.dataflow.memoization INFO: Task 3148 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8679c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3148 try 0 launched on executor multi with executor id 3149 parsl.dataflow.dflow INFO: Standard output for task 3148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9fddd4e6-59d1-49cf-9a67-e96517e280a8_isrForCrosstalkSources_996979_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9fddd4e6-59d1-49cf-9a67-e96517e280a8_isrForCrosstalkSources_996979_29.stderr parsl.dataflow.dflow DEBUG: Task 3149 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3149 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3149 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3149 has memoization hash d2e2a91c6dc53df599ea6d0667f1907f parsl.dataflow.memoization INFO: Task 3149 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3149 try 0 launched on executor multi with executor id 3150 parsl.dataflow.dflow INFO: Standard output for task 3149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b2f368cf-7e45-42ec-bfc0-c335c44337be_isrForCrosstalkSources_996980_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b2f368cf-7e45-42ec-bfc0-c335c44337be_isrForCrosstalkSources_996980_33.stderr parsl.dataflow.dflow DEBUG: Task 3150 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3150 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3150 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3150 has memoization hash 9404d7809289e208029a76d31fadb8aa parsl.dataflow.memoization INFO: Task 3150 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3150 try 0 launched on executor multi with executor id 3151 parsl.dataflow.dflow INFO: Standard output for task 3150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/aa111026-4922-4292-b946-8e5048ec3a96_isrForCrosstalkSources_996970_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/aa111026-4922-4292-b946-8e5048ec3a96_isrForCrosstalkSources_996970_62.stderr parsl.dataflow.dflow DEBUG: Task 3151 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3151 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3151 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3151 has memoization hash 4a385c570f209d04c2dbb6b07326601a parsl.dataflow.memoization INFO: Task 3151 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3151 try 0 launched on executor multi with executor id 3152 parsl.dataflow.dflow INFO: Standard output for task 3151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/19a784a4-4f66-4da4-8d2b-253bc9c9e230_isrForCrosstalkSources_996979_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/19a784a4-4f66-4da4-8d2b-253bc9c9e230_isrForCrosstalkSources_996979_17.stderr parsl.dataflow.dflow DEBUG: Task 3152 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3152 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3152 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3152 has memoization hash 33943e32b47c0d7beb06ca1fe2ede106 parsl.dataflow.memoization INFO: Task 3152 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3152 try 0 launched on executor multi with executor id 3153 parsl.dataflow.dflow INFO: Standard output for task 3152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f04a59e9-646a-40eb-aaf8-2e8c05a747cf_isrForCrosstalkSources_996967_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f04a59e9-646a-40eb-aaf8-2e8c05a747cf_isrForCrosstalkSources_996967_53.stderr parsl.dataflow.dflow DEBUG: Task 3153 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3153 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3153 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3153 has memoization hash 32772c53f8c086769e3d37a892c8f45f parsl.dataflow.memoization INFO: Task 3153 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3153 try 0 launched on executor multi with executor id 3154 parsl.dataflow.dflow INFO: Standard output for task 3153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/d7bb80ab-9dd9-409f-8678-05819181ca3b_isrForCrosstalkSources_997007_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/d7bb80ab-9dd9-409f-8678-05819181ca3b_isrForCrosstalkSources_997007_62.stderr parsl.dataflow.dflow DEBUG: Task 3154 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3154 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3154 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3154 has memoization hash 1323c9c5b405432cf9ea3a6144a6f749 parsl.dataflow.memoization INFO: Task 3154 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3154 try 0 launched on executor multi with executor id 3155 parsl.dataflow.dflow INFO: Standard output for task 3154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e7811ed1-e5d6-48aa-9adb-45f5486b89ef_isrForCrosstalkSources_996965_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e7811ed1-e5d6-48aa-9adb-45f5486b89ef_isrForCrosstalkSources_996965_36.stderr parsl.dataflow.dflow DEBUG: Task 3155 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3155 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3155 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3155 has memoization hash aab8cdb9902c3a57eefe4b4ab30eab4c parsl.dataflow.memoization INFO: Task 3155 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c867b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3155 try 0 launched on executor multi with executor id 3156 parsl.dataflow.dflow INFO: Standard output for task 3155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/c0e852b8-8146-4e5f-b967-8b8c45fd5c53_isrForCrosstalkSources_996979_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/c0e852b8-8146-4e5f-b967-8b8c45fd5c53_isrForCrosstalkSources_996979_16.stderr parsl.dataflow.dflow DEBUG: Task 3156 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3156 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3156 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3156 has memoization hash 8c918c8bc0f2959762072811f8264819 parsl.dataflow.memoization INFO: Task 3156 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7000e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3156 try 0 launched on executor multi with executor id 3157 parsl.dataflow.dflow INFO: Standard output for task 3156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dc1bfb09-a884-47f8-a8b1-e7606fe00f8a_isrForCrosstalkSources_997001_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dc1bfb09-a884-47f8-a8b1-e7606fe00f8a_isrForCrosstalkSources_997001_25.stderr parsl.dataflow.dflow DEBUG: Task 3157 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3157 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3157 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3157 has memoization hash 2b46bccf2fc18ebe463c5f9a203eca4f parsl.dataflow.memoization INFO: Task 3157 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3157 try 0 launched on executor multi with executor id 3158 parsl.dataflow.dflow INFO: Standard output for task 3157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/60c8e9ce-83b6-43e6-84bb-20f333a3b041_isrForCrosstalkSources_996987_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/60c8e9ce-83b6-43e6-84bb-20f333a3b041_isrForCrosstalkSources_996987_3.stderr parsl.dataflow.dflow DEBUG: Task 3158 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3158 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3158 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3158 has memoization hash f00399d665afe62d758db29eae6dda12 parsl.dataflow.memoization INFO: Task 3158 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3158 try 0 launched on executor multi with executor id 3159 parsl.dataflow.dflow INFO: Standard output for task 3158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6116f23a-3507-45bf-96f9-7a5efcce218a_isrForCrosstalkSources_996952_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6116f23a-3507-45bf-96f9-7a5efcce218a_isrForCrosstalkSources_996952_42.stderr parsl.dataflow.dflow DEBUG: Task 3159 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3159 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3159 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3159 has memoization hash eaf53363b23609f8475312c8ddcbe149 parsl.dataflow.memoization INFO: Task 3159 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3159 try 0 launched on executor multi with executor id 3160 parsl.dataflow.dflow INFO: Standard output for task 3159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/85034533-19fb-49e3-b22d-2037d5f40064_isrForCrosstalkSources_996952_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/85034533-19fb-49e3-b22d-2037d5f40064_isrForCrosstalkSources_996952_50.stderr parsl.dataflow.dflow DEBUG: Task 3160 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3160 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3160 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3160 has memoization hash 4831bdc06c1432376fb89b2ef54df015 parsl.dataflow.memoization INFO: Task 3160 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7002c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3160 try 0 launched on executor multi with executor id 3161 parsl.dataflow.dflow INFO: Standard output for task 3160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a90bb34-aab8-4624-8a8e-a7721ec89e84_isrForCrosstalkSources_997004_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a90bb34-aab8-4624-8a8e-a7721ec89e84_isrForCrosstalkSources_997004_18.stderr parsl.dataflow.dflow DEBUG: Task 3161 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3161 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3161 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3161 has memoization hash 43e4ad3d641b8d0431f6e3aeeb56389b parsl.dataflow.memoization INFO: Task 3161 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3161 try 0 launched on executor multi with executor id 3162 parsl.dataflow.dflow INFO: Standard output for task 3161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/addf0e15-c7f6-4ef1-bcf0-fd979ae157fa_isrForCrosstalkSources_997032_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/addf0e15-c7f6-4ef1-bcf0-fd979ae157fa_isrForCrosstalkSources_997032_39.stderr parsl.dataflow.dflow DEBUG: Task 3162 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3162 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3162 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3162 has memoization hash 6480b2e32f6900dded4c54ebe7a44ade parsl.dataflow.memoization INFO: Task 3162 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7004a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3162 try 0 launched on executor multi with executor id 3163 parsl.dataflow.dflow INFO: Standard output for task 3162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/08bd9fab-9e60-44a8-9553-277644ac2a6d_isrForCrosstalkSources_996975_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/08bd9fab-9e60-44a8-9553-277644ac2a6d_isrForCrosstalkSources_996975_2.stderr parsl.dataflow.dflow DEBUG: Task 3163 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3163 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3163 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3163 has memoization hash 148535e55fccb5a7c0c3ba732623c8a4 parsl.dataflow.memoization INFO: Task 3163 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3163 try 0 launched on executor multi with executor id 3164 parsl.dataflow.dflow INFO: Standard output for task 3163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/54cf8afc-3bbe-4801-ac45-d71d794e0f9f_isrForCrosstalkSources_996969_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/54cf8afc-3bbe-4801-ac45-d71d794e0f9f_isrForCrosstalkSources_996969_46.stderr parsl.dataflow.dflow DEBUG: Task 3164 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3164 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3164 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3164 has memoization hash 4011d81fa105c3b64fa90dde2e36bcc9 parsl.dataflow.memoization INFO: Task 3164 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3164 try 0 launched on executor multi with executor id 3165 parsl.dataflow.dflow INFO: Standard output for task 3164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c9a449e0-666a-4457-ade1-3e522b8face0_isrForCrosstalkSources_996959_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c9a449e0-666a-4457-ade1-3e522b8face0_isrForCrosstalkSources_996959_50.stderr parsl.dataflow.dflow DEBUG: Task 3165 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3165 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3165 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3165 has memoization hash 53b4786b06e3203ce310e818d8fc9c64 parsl.dataflow.memoization INFO: Task 3165 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7005e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3165 try 0 launched on executor multi with executor id 3166 parsl.dataflow.dflow INFO: Standard output for task 3165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8ad98ed8-93c2-4b71-b642-a7b97369f422_isrForCrosstalkSources_996952_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8ad98ed8-93c2-4b71-b642-a7b97369f422_isrForCrosstalkSources_996952_40.stderr parsl.dataflow.dflow DEBUG: Task 3166 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3166 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3166 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3166 has memoization hash e941e3c5a870e55f88111dc9af94500b parsl.dataflow.memoization INFO: Task 3166 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3166 try 0 launched on executor multi with executor id 3167 parsl.dataflow.dflow INFO: Standard output for task 3166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/325e4e20-e9fe-43a5-a936-ee2fd509b29c_isrForCrosstalkSources_997033_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/325e4e20-e9fe-43a5-a936-ee2fd509b29c_isrForCrosstalkSources_997033_41.stderr parsl.dataflow.dflow DEBUG: Task 3167 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3167 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3167 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3167 has memoization hash 2639005c90196ace37b8d59c99c096f5 parsl.dataflow.memoization INFO: Task 3167 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7007c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3167 try 0 launched on executor multi with executor id 3168 parsl.dataflow.dflow INFO: Standard output for task 3167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/88366b76-4063-4fe2-9b3a-e6a45b5d50a3_isrForCrosstalkSources_996959_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/88366b76-4063-4fe2-9b3a-e6a45b5d50a3_isrForCrosstalkSources_996959_10.stderr parsl.dataflow.dflow DEBUG: Task 3168 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3168 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3168 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3168 has memoization hash 96707aa4d85b40e6df5cd2a06ef7d09c parsl.dataflow.memoization INFO: Task 3168 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3168 try 0 launched on executor multi with executor id 3169 parsl.dataflow.dflow INFO: Standard output for task 3168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/68baef19-176a-492a-a31e-31f08e43866a_isrForCrosstalkSources_996958_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/68baef19-176a-492a-a31e-31f08e43866a_isrForCrosstalkSources_996958_60.stderr parsl.dataflow.dflow DEBUG: Task 3169 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3169 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3169 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3169 has memoization hash ba409ad8a8836e69eea5ee61e31ea48a parsl.dataflow.memoization INFO: Task 3169 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7009a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3169 try 0 launched on executor multi with executor id 3170 parsl.dataflow.dflow INFO: Standard output for task 3169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/1fb32f38-b0ce-4ed6-9fd2-946df923d91f_isrForCrosstalkSources_997010_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/1fb32f38-b0ce-4ed6-9fd2-946df923d91f_isrForCrosstalkSources_997010_6.stderr parsl.dataflow.dflow DEBUG: Task 3170 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3170 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3170 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3170 has memoization hash 0cc616ea6768c3f482b5f295325b73f4 parsl.dataflow.memoization INFO: Task 3170 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3170 try 0 launched on executor multi with executor id 3171 parsl.dataflow.dflow INFO: Standard output for task 3170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/e7ef8337-5c35-4fd2-a32e-89dfcc0fd117_isrForCrosstalkSources_996999_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/e7ef8337-5c35-4fd2-a32e-89dfcc0fd117_isrForCrosstalkSources_996999_43.stderr parsl.dataflow.dflow DEBUG: Task 3171 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3171 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3171 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3171 has memoization hash 469a621e714336b43019338db0a0f22f parsl.dataflow.memoization INFO: Task 3171 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3171 try 0 launched on executor multi with executor id 3172 parsl.dataflow.dflow INFO: Standard output for task 3171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/437220b3-6e19-465e-a097-00b151591ce3_isrForCrosstalkSources_997008_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/437220b3-6e19-465e-a097-00b151591ce3_isrForCrosstalkSources_997008_31.stderr parsl.dataflow.dflow DEBUG: Task 3172 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3172 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3172 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3172 has memoization hash d7ae3bd09a498dc07663abcc2673d90b parsl.dataflow.memoization INFO: Task 3172 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3172 try 0 launched on executor multi with executor id 3173 parsl.dataflow.dflow INFO: Standard output for task 3172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/a6e61a35-3c57-444f-bdbd-3260b70fad49_isrForCrosstalkSources_997023_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/a6e61a35-3c57-444f-bdbd-3260b70fad49_isrForCrosstalkSources_997023_14.stderr parsl.dataflow.dflow DEBUG: Task 3173 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3173 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3173 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3173 has memoization hash c72f80f7522a34592129bc70cf1f44e9 parsl.dataflow.memoization INFO: Task 3173 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3173 try 0 launched on executor multi with executor id 3174 parsl.dataflow.dflow INFO: Standard output for task 3173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/461d7d06-4b7f-4609-978b-47f387289f97_isrForCrosstalkSources_997016_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/461d7d06-4b7f-4609-978b-47f387289f97_isrForCrosstalkSources_997016_3.stderr parsl.dataflow.dflow DEBUG: Task 3174 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3174 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3174 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3174 has memoization hash 27ca5b9f854e56d4005d7f20ab057ba8 parsl.dataflow.memoization INFO: Task 3174 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3174 try 0 launched on executor multi with executor id 3175 parsl.dataflow.dflow INFO: Standard output for task 3174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f3fefa24-3e4c-4fcc-8afc-1b35a55e5157_isrForCrosstalkSources_997017_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f3fefa24-3e4c-4fcc-8afc-1b35a55e5157_isrForCrosstalkSources_997017_11.stderr parsl.dataflow.dflow DEBUG: Task 3175 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3175 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3175 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3175 has memoization hash cd16ed989bcda8511717918efd3d796a parsl.dataflow.memoization INFO: Task 3175 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3175 try 0 launched on executor multi with executor id 3176 parsl.dataflow.dflow INFO: Standard output for task 3175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0755dc5c-1f8b-4114-ae24-aa5505bbc756_isrForCrosstalkSources_996981_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0755dc5c-1f8b-4114-ae24-aa5505bbc756_isrForCrosstalkSources_996981_59.stderr parsl.dataflow.dflow DEBUG: Task 3176 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3176 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3176 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3176 has memoization hash 70f700c6ef6f75887bbe4ad0f86c3dfc parsl.dataflow.memoization INFO: Task 3176 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3176 try 0 launched on executor multi with executor id 3177 parsl.dataflow.dflow INFO: Standard output for task 3176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/99953982-306c-4cc3-86f7-4e623d98e9be_isrForCrosstalkSources_997019_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/99953982-306c-4cc3-86f7-4e623d98e9be_isrForCrosstalkSources_997019_56.stderr parsl.dataflow.dflow DEBUG: Task 3177 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3177 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3177 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3177 has memoization hash 2642c4d5307e0c552344e583bb528232 parsl.dataflow.memoization INFO: Task 3177 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3177 try 0 launched on executor multi with executor id 3178 parsl.dataflow.dflow INFO: Standard output for task 3177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/d95366c6-c062-4937-a6e8-f0571254861a_isrForCrosstalkSources_997016_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/d95366c6-c062-4937-a6e8-f0571254861a_isrForCrosstalkSources_997016_55.stderr parsl.dataflow.dflow DEBUG: Task 3178 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3178 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3178 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3178 has memoization hash 71bafc59c464a804466aa61b1381cd0c parsl.dataflow.memoization INFO: Task 3178 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3178 try 0 launched on executor multi with executor id 3179 parsl.dataflow.dflow INFO: Standard output for task 3178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2b97f651-1d5e-4ef9-885f-30a95de95838_isrForCrosstalkSources_996974_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2b97f651-1d5e-4ef9-885f-30a95de95838_isrForCrosstalkSources_996974_26.stderr parsl.dataflow.dflow DEBUG: Task 3179 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3179 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3179 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3179 has memoization hash d9e7bb2bb2437719015da1351adbe8c9 parsl.dataflow.memoization INFO: Task 3179 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3179 try 0 launched on executor multi with executor id 3180 parsl.dataflow.dflow INFO: Standard output for task 3179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/df55b068-9a67-47b4-a055-3f65ee831236_isrForCrosstalkSources_996988_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/df55b068-9a67-47b4-a055-3f65ee831236_isrForCrosstalkSources_996988_49.stderr parsl.dataflow.dflow DEBUG: Task 3180 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3180 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3180 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3180 has memoization hash 4eedcf5ae37c75d5c78bb7e449c314b0 parsl.dataflow.memoization INFO: Task 3180 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3180 try 0 launched on executor multi with executor id 3181 parsl.dataflow.dflow INFO: Standard output for task 3180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/bf2508e1-7a2e-4228-94d8-18f2b9673ea3_isrForCrosstalkSources_997018_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/bf2508e1-7a2e-4228-94d8-18f2b9673ea3_isrForCrosstalkSources_997018_59.stderr parsl.dataflow.dflow DEBUG: Task 3181 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3181 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3181 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3181 has memoization hash 6a44c317bca64b5741934dcf31b511a9 parsl.dataflow.memoization INFO: Task 3181 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3181 try 0 launched on executor multi with executor id 3182 parsl.dataflow.dflow INFO: Standard output for task 3181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5bf6f98c-2ae7-427e-92eb-bbb3e0c1bc0e_isrForCrosstalkSources_997031_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5bf6f98c-2ae7-427e-92eb-bbb3e0c1bc0e_isrForCrosstalkSources_997031_5.stderr parsl.dataflow.dflow DEBUG: Task 3182 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3182 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3182 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3182 has memoization hash 5a2f031b987544e15b883f2cc351bc97 parsl.dataflow.memoization INFO: Task 3182 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3182 try 0 launched on executor multi with executor id 3183 parsl.dataflow.dflow INFO: Standard output for task 3182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ed7b8605-f528-48bf-ab81-4b1a999e09e5_isrForCrosstalkSources_997019_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ed7b8605-f528-48bf-ab81-4b1a999e09e5_isrForCrosstalkSources_997019_10.stderr parsl.dataflow.dflow DEBUG: Task 3183 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3183 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3183 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3183 has memoization hash 421558814e2f9b6c19102ab04820c084 parsl.dataflow.memoization INFO: Task 3183 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7011c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3183 try 0 launched on executor multi with executor id 3184 parsl.dataflow.dflow INFO: Standard output for task 3183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e7498fe2-e98d-42bf-b097-d0c095c8b9ea_isrForCrosstalkSources_997021_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e7498fe2-e98d-42bf-b097-d0c095c8b9ea_isrForCrosstalkSources_997021_27.stderr parsl.dataflow.dflow DEBUG: Task 3184 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3184 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3184 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3184 has memoization hash 8bd3742b7a752933f5eb619cdd56cec2 parsl.dataflow.memoization INFO: Task 3184 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3184 try 0 launched on executor multi with executor id 3185 parsl.dataflow.dflow INFO: Standard output for task 3184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5baefdda-e37a-4609-bcbe-d4ff07371fc6_isrForCrosstalkSources_997031_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5baefdda-e37a-4609-bcbe-d4ff07371fc6_isrForCrosstalkSources_997031_2.stderr parsl.dataflow.dflow DEBUG: Task 3185 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3185 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3185 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3185 has memoization hash 28e4d2536685e96b676b62d09567785f parsl.dataflow.memoization INFO: Task 3185 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3185 try 0 launched on executor multi with executor id 3186 parsl.dataflow.dflow INFO: Standard output for task 3185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/33dc09a7-321f-4e7f-b6fb-f042b3bfdc25_isrForCrosstalkSources_997026_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/33dc09a7-321f-4e7f-b6fb-f042b3bfdc25_isrForCrosstalkSources_997026_62.stderr parsl.dataflow.dflow DEBUG: Task 3186 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3186 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3186 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3186 has memoization hash 75a3aec052d32ae817a8a131e8d6c63c parsl.dataflow.memoization INFO: Task 3186 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7013a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3186 try 0 launched on executor multi with executor id 3187 parsl.dataflow.dflow INFO: Standard output for task 3186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ab5e1184-05bb-4108-b197-1697526cc7c5_isrForCrosstalkSources_997009_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ab5e1184-05bb-4108-b197-1697526cc7c5_isrForCrosstalkSources_997009_51.stderr parsl.dataflow.dflow DEBUG: Task 3187 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3187 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3187 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3187 has memoization hash 8277203a29961a1a2beb422f2f621232 parsl.dataflow.memoization INFO: Task 3187 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3187 try 0 launched on executor multi with executor id 3188 parsl.dataflow.dflow INFO: Standard output for task 3187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/989592d4-d748-4d6c-9972-8ccce586e4ae_isrForCrosstalkSources_997022_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/989592d4-d748-4d6c-9972-8ccce586e4ae_isrForCrosstalkSources_997022_37.stderr parsl.dataflow.dflow DEBUG: Task 3188 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3188 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3188 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3188 has memoization hash 4348d42e43a843b57f7735c7aab4e6a2 parsl.dataflow.memoization INFO: Task 3188 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7014e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3188 try 0 launched on executor multi with executor id 3189 parsl.dataflow.dflow INFO: Standard output for task 3188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d834e7d4-05f9-47fc-b2a0-6c7948c6101e_isrForCrosstalkSources_996987_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d834e7d4-05f9-47fc-b2a0-6c7948c6101e_isrForCrosstalkSources_996987_44.stderr parsl.dataflow.dflow DEBUG: Task 3189 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3189 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3189 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3189 has memoization hash fa08bdd6e501d0d8128a176f385c5e22 parsl.dataflow.memoization INFO: Task 3189 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3189 try 0 launched on executor multi with executor id 3190 parsl.dataflow.dflow INFO: Standard output for task 3189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/97aa5fed-56be-43a2-a6bf-65697ab0b301_isrForCrosstalkSources_996979_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/97aa5fed-56be-43a2-a6bf-65697ab0b301_isrForCrosstalkSources_996979_25.stderr parsl.dataflow.dflow DEBUG: Task 3190 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3190 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3190 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3190 has memoization hash 1a63b388ed538c4dfca18e5bf15450c1 parsl.dataflow.memoization INFO: Task 3190 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3190 try 0 launched on executor multi with executor id 3191 parsl.dataflow.dflow INFO: Standard output for task 3190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c026c71b-061a-4bfe-8d8b-73732be2701c_isrForCrosstalkSources_996980_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c026c71b-061a-4bfe-8d8b-73732be2701c_isrForCrosstalkSources_996980_61.stderr parsl.dataflow.dflow DEBUG: Task 3191 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3191 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3191 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3191 has memoization hash 24e2cf565d6915e11541a2c446be9210 parsl.dataflow.memoization INFO: Task 3191 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7016c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3191 try 0 launched on executor multi with executor id 3192 parsl.dataflow.dflow INFO: Standard output for task 3191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/693f49f3-8127-431d-a3c6-eb542140b73f_isrForCrosstalkSources_996952_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/693f49f3-8127-431d-a3c6-eb542140b73f_isrForCrosstalkSources_996952_8.stderr parsl.dataflow.dflow DEBUG: Task 3192 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3192 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3192 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3192 has memoization hash a57dc4910e6826a595583fe932ee6e69 parsl.dataflow.memoization INFO: Task 3192 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3192 try 0 launched on executor multi with executor id 3193 parsl.dataflow.dflow INFO: Standard output for task 3192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/fd8acc59-2c74-4276-96e9-5f89bb77bc54_isrForCrosstalkSources_997025_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/fd8acc59-2c74-4276-96e9-5f89bb77bc54_isrForCrosstalkSources_997025_23.stderr parsl.dataflow.dflow DEBUG: Task 3193 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3193 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3193 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3193 has memoization hash e275cdba30e928cc44dd043623af20fd parsl.dataflow.memoization INFO: Task 3193 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3193 try 0 launched on executor multi with executor id 3194 parsl.dataflow.dflow INFO: Standard output for task 3193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4e0424d2-6906-4ebb-b126-15afc314da59_isrForCrosstalkSources_996986_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4e0424d2-6906-4ebb-b126-15afc314da59_isrForCrosstalkSources_996986_38.stderr parsl.dataflow.dflow DEBUG: Task 3194 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3194 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3194 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3194 has memoization hash 61ce7eb9838adddd2644ea67d6d4bbb5 parsl.dataflow.memoization INFO: Task 3194 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3194 try 0 launched on executor multi with executor id 3195 parsl.dataflow.dflow INFO: Standard output for task 3194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/cfecac5c-b504-497a-b2be-57abca298442_isrForCrosstalkSources_996952_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/cfecac5c-b504-497a-b2be-57abca298442_isrForCrosstalkSources_996952_56.stderr parsl.dataflow.dflow DEBUG: Task 3195 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3195 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3195 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3195 has memoization hash 56cbd41d09cc771decf0cac8a3d3f928 parsl.dataflow.memoization INFO: Task 3195 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7018a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3195 try 0 launched on executor multi with executor id 3196 parsl.dataflow.dflow INFO: Standard output for task 3195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/16a7ad76-6e43-41f6-a8fd-4012b050c09f_isrForCrosstalkSources_996970_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/16a7ad76-6e43-41f6-a8fd-4012b050c09f_isrForCrosstalkSources_996970_38.stderr parsl.dataflow.dflow DEBUG: Task 3196 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3196 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3196 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3196 has memoization hash 984aaf6e4368cb4a579920df3705dfd0 parsl.dataflow.memoization INFO: Task 3196 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7019e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3196 try 0 launched on executor multi with executor id 3197 parsl.dataflow.dflow INFO: Standard output for task 3196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/832d4c01-2894-4029-aed2-b7c94a034eec_isrForCrosstalkSources_996955_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/832d4c01-2894-4029-aed2-b7c94a034eec_isrForCrosstalkSources_996955_3.stderr parsl.dataflow.dflow DEBUG: Task 3197 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3197 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3197 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3197 has memoization hash 6f17be8c97e434fc9c84142ee53db97e parsl.dataflow.memoization INFO: Task 3197 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3197 try 0 launched on executor multi with executor id 3198 parsl.dataflow.dflow INFO: Standard output for task 3197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/ab35c226-fd67-4578-a157-50aa03d3a46d_isrForCrosstalkSources_997033_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/ab35c226-fd67-4578-a157-50aa03d3a46d_isrForCrosstalkSources_997033_37.stderr parsl.dataflow.dflow DEBUG: Task 3198 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3198 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3198 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3198 has memoization hash ec8b69df9fa866c44575d2785b850ff1 parsl.dataflow.memoization INFO: Task 3198 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3198 try 0 launched on executor multi with executor id 3199 parsl.dataflow.dflow INFO: Standard output for task 3198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/dd134de8-0688-4481-964c-5b306078ac1d_isrForCrosstalkSources_996992_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/dd134de8-0688-4481-964c-5b306078ac1d_isrForCrosstalkSources_996992_36.stderr parsl.dataflow.dflow DEBUG: Task 3199 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3199 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3199 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3199 has memoization hash 29b17907d3d7b65af0570c3e50c47f0e parsl.dataflow.memoization INFO: Task 3199 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3199 try 0 launched on executor multi with executor id 3200 parsl.dataflow.dflow INFO: Standard output for task 3199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/b7728c5e-5135-4129-85df-10b1813746ec_isrForCrosstalkSources_997005_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/b7728c5e-5135-4129-85df-10b1813746ec_isrForCrosstalkSources_997005_34.stderr parsl.dataflow.dflow DEBUG: Task 3200 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3200 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3200 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3200 has memoization hash f7838c5007eee0ecedb11c737aa8900c parsl.dataflow.memoization INFO: Task 3200 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3200 try 0 launched on executor multi with executor id 3201 parsl.dataflow.dflow INFO: Standard output for task 3200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/136de1f6-f40b-4957-aff4-09207c2e05b6_isrForCrosstalkSources_996952_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/136de1f6-f40b-4957-aff4-09207c2e05b6_isrForCrosstalkSources_996952_32.stderr parsl.dataflow.dflow DEBUG: Task 3201 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3201 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3201 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3201 has memoization hash 3a99b146e457aade6bcd753730eb4bb4 parsl.dataflow.memoization INFO: Task 3201 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3201 try 0 launched on executor multi with executor id 3202 parsl.dataflow.dflow INFO: Standard output for task 3201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e0281d8e-1939-4c9a-be15-1699da61eb74_isrForCrosstalkSources_997022_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e0281d8e-1939-4c9a-be15-1699da61eb74_isrForCrosstalkSources_997022_56.stderr parsl.dataflow.dflow DEBUG: Task 3202 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3202 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3202 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3202 has memoization hash 7c4c99170ab8b289c676a0e0b44a2ddd parsl.dataflow.memoization INFO: Task 3202 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3202 try 0 launched on executor multi with executor id 3203 parsl.dataflow.dflow INFO: Standard output for task 3202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/63080e83-ed69-4fcd-9b9e-c4339949d1fd_isrForCrosstalkSources_996989_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/63080e83-ed69-4fcd-9b9e-c4339949d1fd_isrForCrosstalkSources_996989_18.stderr parsl.dataflow.dflow DEBUG: Task 3203 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3203 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3203 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3203 has memoization hash 5b6afc47a2854cbf4ca130b91f9a95c4 parsl.dataflow.memoization INFO: Task 3203 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3203 try 0 launched on executor multi with executor id 3204 parsl.dataflow.dflow INFO: Standard output for task 3203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a109e3e1-d97f-4b12-816a-c1910c4f77ce_isrForCrosstalkSources_997026_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a109e3e1-d97f-4b12-816a-c1910c4f77ce_isrForCrosstalkSources_997026_10.stderr parsl.dataflow.dflow DEBUG: Task 3204 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3204 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3204 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3204 has memoization hash a4e27834e1a6dc1c0ac9a19e5156a006 parsl.dataflow.memoization INFO: Task 3204 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3204 try 0 launched on executor multi with executor id 3205 parsl.dataflow.dflow INFO: Standard output for task 3204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/9c1d3748-3da0-4251-9b5e-148c1083bde2_isrForCrosstalkSources_997011_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/9c1d3748-3da0-4251-9b5e-148c1083bde2_isrForCrosstalkSources_997011_47.stderr parsl.dataflow.dflow DEBUG: Task 3205 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3205 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3205 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3205 has memoization hash 546f776a7e1413b85204393e24b74a41 parsl.dataflow.memoization INFO: Task 3205 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c701ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3205 try 0 launched on executor multi with executor id 3206 parsl.dataflow.dflow INFO: Standard output for task 3205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/595c9d4b-c952-46de-865f-47916708198f_isrForCrosstalkSources_996972_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/595c9d4b-c952-46de-865f-47916708198f_isrForCrosstalkSources_996972_22.stderr parsl.dataflow.dflow DEBUG: Task 3206 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3206 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3206 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3206 has memoization hash 9fef557c642f066c7df06bbeb628d64a parsl.dataflow.memoization INFO: Task 3206 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3206 try 0 launched on executor multi with executor id 3207 parsl.dataflow.dflow INFO: Standard output for task 3206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/6ad7d136-3349-478e-a140-7fbcf288940c_isrForCrosstalkSources_996964_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/6ad7d136-3349-478e-a140-7fbcf288940c_isrForCrosstalkSources_996964_43.stderr parsl.dataflow.dflow DEBUG: Task 3207 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3207 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3207 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3207 has memoization hash 289213aed312d6fdc007cb47cfaba8bc parsl.dataflow.memoization INFO: Task 3207 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7020c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3207 try 0 launched on executor multi with executor id 3208 parsl.dataflow.dflow INFO: Standard output for task 3207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/b17c6dab-10ab-4980-a8b6-eba66d4b846b_isrForCrosstalkSources_997024_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/b17c6dab-10ab-4980-a8b6-eba66d4b846b_isrForCrosstalkSources_997024_27.stderr parsl.dataflow.dflow DEBUG: Task 3208 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3208 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3208 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3208 has memoization hash 535956d6fd936a901eb3b0cef5e8b6cf parsl.dataflow.memoization INFO: Task 3208 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3208 try 0 launched on executor multi with executor id 3209 parsl.dataflow.dflow INFO: Standard output for task 3208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/cf294e22-f430-4ea0-9732-dce56a91db6b_isrForCrosstalkSources_997026_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/cf294e22-f430-4ea0-9732-dce56a91db6b_isrForCrosstalkSources_997026_6.stderr parsl.dataflow.dflow DEBUG: Task 3209 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3209 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3209 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3209 has memoization hash f97f88aaf4aae4363138f4957416a7a2 parsl.dataflow.memoization INFO: Task 3209 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7022a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3209 try 0 launched on executor multi with executor id 3210 parsl.dataflow.dflow INFO: Standard output for task 3209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/bbd9647a-50a9-4b3d-9f29-185799677689_isrForCrosstalkSources_997024_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/bbd9647a-50a9-4b3d-9f29-185799677689_isrForCrosstalkSources_997024_48.stderr parsl.dataflow.dflow DEBUG: Task 3210 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3210 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3210 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3210 has memoization hash e720632593d1eb1d493e62312f2afd70 parsl.dataflow.memoization INFO: Task 3210 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3210 try 0 launched on executor multi with executor id 3211 parsl.dataflow.dflow INFO: Standard output for task 3210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2c647f14-3b60-4da6-8d24-9580fe73e375_isrForCrosstalkSources_996980_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2c647f14-3b60-4da6-8d24-9580fe73e375_isrForCrosstalkSources_996980_46.stderr parsl.dataflow.dflow DEBUG: Task 3211 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3211 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3211 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3211 has memoization hash b1dc6c3ae1e468dfb2f89f9c74c7e1be parsl.dataflow.memoization INFO: Task 3211 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3211 try 0 launched on executor multi with executor id 3212 parsl.dataflow.dflow INFO: Standard output for task 3211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/bcbcc29e-e829-4bed-b604-8aadc676f129_isrForCrosstalkSources_997013_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/bcbcc29e-e829-4bed-b604-8aadc676f129_isrForCrosstalkSources_997013_32.stderr parsl.dataflow.dflow DEBUG: Task 3212 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3212 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3212 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3212 has memoization hash 32083de5205e4e945152f97293099521 parsl.dataflow.memoization INFO: Task 3212 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7023e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3212 try 0 launched on executor multi with executor id 3213 parsl.dataflow.dflow INFO: Standard output for task 3212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/11905bef-a0f5-4c3c-abd1-26584f9b6ee4_isrForCrosstalkSources_997023_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/11905bef-a0f5-4c3c-abd1-26584f9b6ee4_isrForCrosstalkSources_997023_59.stderr parsl.dataflow.dflow DEBUG: Task 3213 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3213 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3213 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3213 has memoization hash c87ae4eb968b9205f8ec3aa060e05087 parsl.dataflow.memoization INFO: Task 3213 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3213 try 0 launched on executor multi with executor id 3214 parsl.dataflow.dflow INFO: Standard output for task 3213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/ff5aaf99-f39d-46f5-87a0-12e6465798e5_isrForCrosstalkSources_996963_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/ff5aaf99-f39d-46f5-87a0-12e6465798e5_isrForCrosstalkSources_996963_34.stderr parsl.dataflow.dflow DEBUG: Task 3214 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3214 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3214 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3214 has memoization hash ac8c2a18f52a0cdee322405ca4813c5a parsl.dataflow.memoization INFO: Task 3214 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7025c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3214 try 0 launched on executor multi with executor id 3215 parsl.dataflow.dflow INFO: Standard output for task 3214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/25bf0146-fcb1-49ab-8cbf-5db33681836f_isrForCrosstalkSources_997023_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/25bf0146-fcb1-49ab-8cbf-5db33681836f_isrForCrosstalkSources_997023_50.stderr parsl.dataflow.dflow DEBUG: Task 3215 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3215 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3215 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3215 has memoization hash c4b63358512be60e00037361fa92c544 parsl.dataflow.memoization INFO: Task 3215 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3215 try 0 launched on executor multi with executor id 3216 parsl.dataflow.dflow INFO: Standard output for task 3215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a7503e82-2977-43b2-8867-24ec5485b316_isrForCrosstalkSources_997032_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a7503e82-2977-43b2-8867-24ec5485b316_isrForCrosstalkSources_997032_12.stderr parsl.dataflow.dflow DEBUG: Task 3216 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3216 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3216 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3216 has memoization hash a4553aa0464171f12abee82c6432f6c9 parsl.dataflow.memoization INFO: Task 3216 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3216 try 0 launched on executor multi with executor id 3217 parsl.dataflow.dflow INFO: Standard output for task 3216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4032d8d8-232f-433c-9f09-6e1b30e8b6e3_isrForCrosstalkSources_997020_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4032d8d8-232f-433c-9f09-6e1b30e8b6e3_isrForCrosstalkSources_997020_32.stderr parsl.dataflow.dflow DEBUG: Task 3217 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3217 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3217 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3217 has memoization hash f99a43e745a260a83f7b5a4970e10ffd parsl.dataflow.memoization INFO: Task 3217 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3217 try 0 launched on executor multi with executor id 3218 parsl.dataflow.dflow INFO: Standard output for task 3217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0c01677f-8ccf-42ac-9622-860c9dffbb78_isrForCrosstalkSources_997020_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0c01677f-8ccf-42ac-9622-860c9dffbb78_isrForCrosstalkSources_997020_13.stderr parsl.dataflow.dflow DEBUG: Task 3218 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3218 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3218 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3218 has memoization hash ad9b17e58731dd95d73c303c23881ea2 parsl.dataflow.memoization INFO: Task 3218 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7027a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3218 try 0 launched on executor multi with executor id 3219 parsl.dataflow.dflow INFO: Standard output for task 3218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4d38a619-42b3-49c8-b027-0b8c3f110290_isrForCrosstalkSources_997014_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4d38a619-42b3-49c8-b027-0b8c3f110290_isrForCrosstalkSources_997014_22.stderr parsl.dataflow.dflow DEBUG: Task 3219 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3219 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3219 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3219 has memoization hash f5b0e12f27ef5ceab52c1b3ccc4d1bb7 parsl.dataflow.memoization INFO: Task 3219 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7028e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3219 try 0 launched on executor multi with executor id 3220 parsl.dataflow.dflow INFO: Standard output for task 3219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3bdac7ba-5aae-40dc-af5e-cd299b29c9f8_isrForCrosstalkSources_996959_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3bdac7ba-5aae-40dc-af5e-cd299b29c9f8_isrForCrosstalkSources_996959_30.stderr parsl.dataflow.dflow DEBUG: Task 3220 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3220 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3220 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3220 has memoization hash 39294c674e8523ae40f3eb849e56e7f9 parsl.dataflow.memoization INFO: Task 3220 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3220 try 0 launched on executor multi with executor id 3221 parsl.dataflow.dflow INFO: Standard output for task 3220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/05260c5a-fefd-4162-9220-65ecb2a345e6_isrForCrosstalkSources_997001_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/05260c5a-fefd-4162-9220-65ecb2a345e6_isrForCrosstalkSources_997001_49.stderr parsl.dataflow.dflow DEBUG: Task 3221 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3221 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3221 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3221 has memoization hash 23ec3891b820c3b04ccb98c815594998 parsl.dataflow.memoization INFO: Task 3221 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3221 try 0 launched on executor multi with executor id 3222 parsl.dataflow.dflow INFO: Standard output for task 3221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/340afdf2-b31c-4d58-b94d-d7c682bb662e_isrForCrosstalkSources_996976_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/340afdf2-b31c-4d58-b94d-d7c682bb662e_isrForCrosstalkSources_996976_13.stderr parsl.dataflow.dflow DEBUG: Task 3222 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3222 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3222 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3222 has memoization hash 9856d82af50d4c8cb93832e5f6919ddb parsl.dataflow.memoization INFO: Task 3222 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3222 try 0 launched on executor multi with executor id 3223 parsl.dataflow.dflow INFO: Standard output for task 3222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2e6095da-83a9-4c51-a794-fc09dbc8ee6c_isrForCrosstalkSources_997003_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2e6095da-83a9-4c51-a794-fc09dbc8ee6c_isrForCrosstalkSources_997003_15.stderr parsl.dataflow.dflow DEBUG: Task 3223 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3223 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3223 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3223 has memoization hash 2d74fbc6adda3bd961a610fbb85ffc96 parsl.dataflow.memoization INFO: Task 3223 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3223 try 0 launched on executor multi with executor id 3224 parsl.dataflow.dflow INFO: Standard output for task 3223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/42ec242c-8f73-4020-bdb7-8e0a087bcfee_isrForCrosstalkSources_997013_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/42ec242c-8f73-4020-bdb7-8e0a087bcfee_isrForCrosstalkSources_997013_26.stderr parsl.dataflow.dflow DEBUG: Task 3224 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3224 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3224 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3224 has memoization hash 429cd8521d54b161ee3c3530ffe847e7 parsl.dataflow.memoization INFO: Task 3224 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3224 try 0 launched on executor multi with executor id 3225 parsl.dataflow.dflow INFO: Standard output for task 3224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4c57b535-8b03-42ec-8e3d-5fa17950b991_isrForCrosstalkSources_996958_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4c57b535-8b03-42ec-8e3d-5fa17950b991_isrForCrosstalkSources_996958_1.stderr parsl.dataflow.dflow DEBUG: Task 3225 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3225 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3225 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3225 has memoization hash 26dd0d67199157ebc836f44ee9bba511 parsl.dataflow.memoization INFO: Task 3225 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3225 try 0 launched on executor multi with executor id 3226 parsl.dataflow.dflow INFO: Standard output for task 3225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a22f5aa2-608d-4edd-ad6b-47f9a4ddd9f3_isrForCrosstalkSources_996972_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a22f5aa2-608d-4edd-ad6b-47f9a4ddd9f3_isrForCrosstalkSources_996972_38.stderr parsl.dataflow.dflow DEBUG: Task 3226 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3226 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3226 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3226 has memoization hash c294770f6bd9a8f3048014f32ba04299 parsl.dataflow.memoization INFO: Task 3226 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3226 try 0 launched on executor multi with executor id 3227 parsl.dataflow.dflow INFO: Standard output for task 3226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/13c58d9f-d80d-47e1-a3d9-1c65e3ba11ff_isrForCrosstalkSources_996972_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/13c58d9f-d80d-47e1-a3d9-1c65e3ba11ff_isrForCrosstalkSources_996972_55.stderr parsl.dataflow.dflow DEBUG: Task 3227 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3227 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3227 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3227 has memoization hash 19c77aa013f7dbcefe5feabe3ec2d2a2 parsl.dataflow.memoization INFO: Task 3227 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:55 proc_lsst.multi:170 [INFO] [multi] job_id 17074887 proc_lsst.multi INFO: [multi] job_id 17074887 2024-03-15 13:49:55 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 3 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 3 2024-03-15 13:49:55 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:55 proc_lsst.multi:178 [INFO] [multi] provider escience_rtx6k accepted submit and returned 17074887 proc_lsst.multi INFO: [multi] provider escience_rtx6k accepted submit and returned 17074887 parsl.executors.status_handling DEBUG: Launched block 33 on executor multi with job ID 17074887 parsl.executors.status_handling INFO: Allocated block ID 34 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-34 2024-03-15 13:49:55 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-34 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-34 2024-03-15 13:49:55 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:55 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:49:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:55 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 3 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 3 2024-03-15 13:49:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:55 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=34 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 3227 try 0 launched on executor multi with executor id 3228 parsl.dataflow.dflow INFO: Standard output for task 3227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/eb288399-d1b8-46a2-aa8b-08a8236b9bcb_isrForCrosstalkSources_996972_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/eb288399-d1b8-46a2-aa8b-08a8236b9bcb_isrForCrosstalkSources_996972_51.stderr parsl.dataflow.dflow DEBUG: Task 3228 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3228 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3228 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3228 has memoization hash a243ccc59b492874983637b9d84f7c1b parsl.dataflow.memoization INFO: Task 3228 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3228 try 0 launched on executor multi with executor id 3229 parsl.dataflow.dflow INFO: Standard output for task 3228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/59edb1b1-1d1f-45f6-bd87-5c13ab002150_isrForCrosstalkSources_997019_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/59edb1b1-1d1f-45f6-bd87-5c13ab002150_isrForCrosstalkSources_997019_37.stderr parsl.dataflow.dflow DEBUG: Task 3229 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3229 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3229 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3229 has memoization hash 8840aa387c799cbbc09ef6e0d43f5808 parsl.dataflow.memoization INFO: Task 3229 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c700040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3229 try 0 launched on executor multi with executor id 3230 parsl.dataflow.dflow INFO: Standard output for task 3229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e8aa1511-1b91-4821-9ac8-384037c20fd9_isrForCrosstalkSources_996957_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e8aa1511-1b91-4821-9ac8-384037c20fd9_isrForCrosstalkSources_996957_15.stderr parsl.dataflow.dflow DEBUG: Task 3230 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3230 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3230 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3230 has memoization hash 240c4556fff2a80af4b94083082b8d29 parsl.dataflow.memoization INFO: Task 3230 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3230 try 0 launched on executor multi with executor id 3231 parsl.dataflow.dflow INFO: Standard output for task 3230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/1a6db936-ace2-43ce-a445-7a4aba41b2aa_isrForCrosstalkSources_996986_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/1a6db936-ace2-43ce-a445-7a4aba41b2aa_isrForCrosstalkSources_996986_56.stderr parsl.dataflow.dflow DEBUG: Task 3231 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3231 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3231 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3231 has memoization hash 7cf52dbaac5accbb7554c297a19c0d43 parsl.dataflow.memoization INFO: Task 3231 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3231 try 0 launched on executor multi with executor id 3232 parsl.dataflow.dflow INFO: Standard output for task 3231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a013966a-19be-41b4-8c41-5045986ac35f_isrForCrosstalkSources_997013_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a013966a-19be-41b4-8c41-5045986ac35f_isrForCrosstalkSources_997013_41.stderr parsl.dataflow.dflow DEBUG: Task 3232 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3232 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3232 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3232 has memoization hash 5a589f78d4258f9f201b9ea44c7c14d4 parsl.dataflow.memoization INFO: Task 3232 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7031a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3232 try 0 launched on executor multi with executor id 3233 parsl.dataflow.dflow INFO: Standard output for task 3232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1c012e16-2d3f-47eb-8685-90768efb91d3_isrForCrosstalkSources_996968_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1c012e16-2d3f-47eb-8685-90768efb91d3_isrForCrosstalkSources_996968_15.stderr parsl.dataflow.dflow DEBUG: Task 3233 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3233 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3233 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3233 has memoization hash 9e7f2e367290bffc9dc8830015f88500 parsl.dataflow.memoization INFO: Task 3233 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3233 try 0 launched on executor multi with executor id 3234 parsl.dataflow.dflow INFO: Standard output for task 3233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/5d53d407-0c5c-4959-8802-5db729c38656_isrForCrosstalkSources_996960_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/5d53d407-0c5c-4959-8802-5db729c38656_isrForCrosstalkSources_996960_59.stderr parsl.dataflow.dflow DEBUG: Task 3234 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3234 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3234 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3234 has memoization hash 22ef5b4a538b78ce20756470c0a9c7f0 parsl.dataflow.memoization INFO: Task 3234 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3234 try 0 launched on executor multi with executor id 3235 parsl.dataflow.dflow INFO: Standard output for task 3234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0e70d68d-4749-4c2b-9bd4-3de6e2bb6fd9_isrForCrosstalkSources_996953_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0e70d68d-4749-4c2b-9bd4-3de6e2bb6fd9_isrForCrosstalkSources_996953_25.stderr parsl.dataflow.dflow DEBUG: Task 3235 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3235 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3235 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3235 has memoization hash 5a6072decff83a732a31bcb0f6805109 parsl.dataflow.memoization INFO: Task 3235 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3235 try 0 launched on executor multi with executor id 3236 parsl.dataflow.dflow INFO: Standard output for task 3235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/488a7fa9-5740-4806-8ad7-3712178010f7_isrForCrosstalkSources_997009_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/488a7fa9-5740-4806-8ad7-3712178010f7_isrForCrosstalkSources_997009_60.stderr parsl.dataflow.dflow DEBUG: Task 3236 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3236 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3236 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3236 has memoization hash 905fceff28397cb45aeab80ab95046e3 parsl.dataflow.memoization INFO: Task 3236 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7032e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3236 try 0 launched on executor multi with executor id 3237 parsl.dataflow.dflow INFO: Standard output for task 3236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/368ecd73-a0e6-4b7a-8f48-bedd1b787564_isrForCrosstalkSources_997022_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/368ecd73-a0e6-4b7a-8f48-bedd1b787564_isrForCrosstalkSources_997022_38.stderr parsl.dataflow.dflow DEBUG: Task 3237 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3237 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3237 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3237 has memoization hash c91bad7b6b00b90832cb9724a5b0c5ec parsl.dataflow.memoization INFO: Task 3237 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3237 try 0 launched on executor multi with executor id 3238 parsl.dataflow.dflow INFO: Standard output for task 3237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/dc447bb4-5b0b-48a2-9573-6b48c97e0bff_isrForCrosstalkSources_996959_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/dc447bb4-5b0b-48a2-9573-6b48c97e0bff_isrForCrosstalkSources_996959_52.stderr parsl.dataflow.dflow DEBUG: Task 3238 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3238 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3238 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3238 has memoization hash 2afa13fd64c70ed8d53453d4e13df877 parsl.dataflow.memoization INFO: Task 3238 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3238 try 0 launched on executor multi with executor id 3239 parsl.dataflow.dflow INFO: Standard output for task 3238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/37aafbf1-aa93-461f-bc9d-6cf19c01a040_isrForCrosstalkSources_996985_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/37aafbf1-aa93-461f-bc9d-6cf19c01a040_isrForCrosstalkSources_996985_47.stderr parsl.dataflow.dflow DEBUG: Task 3239 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3239 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3239 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3239 has memoization hash ee921717ec0e995762ebc4edb83e3a96 parsl.dataflow.memoization INFO: Task 3239 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3239 try 0 launched on executor multi with executor id 3240 parsl.dataflow.dflow INFO: Standard output for task 3239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/5372d3e2-06f5-4b33-a68a-f559fe177799_isrForCrosstalkSources_996982_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/5372d3e2-06f5-4b33-a68a-f559fe177799_isrForCrosstalkSources_996982_60.stderr parsl.dataflow.dflow DEBUG: Task 3240 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3240 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3240 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3240 has memoization hash 9e55480f5e37484d14598960ac1541d2 parsl.dataflow.memoization INFO: Task 3240 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3240 try 0 launched on executor multi with executor id 3241 parsl.dataflow.dflow INFO: Standard output for task 3240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/5917730d-c6f2-43bf-9f21-9d134b393339_isrForCrosstalkSources_997014_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/5917730d-c6f2-43bf-9f21-9d134b393339_isrForCrosstalkSources_997014_60.stderr parsl.dataflow.dflow DEBUG: Task 3241 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3241 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3241 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3241 has memoization hash cff97c3cde5ff7ca353c805f3d99d6a3 parsl.dataflow.memoization INFO: Task 3241 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7034c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3241 try 0 launched on executor multi with executor id 3242 parsl.dataflow.dflow INFO: Standard output for task 3241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/22b0b4ad-c9dc-4eb5-8c32-fb32e4ab5ba3_isrForCrosstalkSources_997031_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/22b0b4ad-c9dc-4eb5-8c32-fb32e4ab5ba3_isrForCrosstalkSources_997031_47.stderr parsl.dataflow.dflow DEBUG: Task 3242 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3242 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3242 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3242 has memoization hash b754a289f7912ce67d24645786e33e96 parsl.dataflow.memoization INFO: Task 3242 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7036a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3242 try 0 launched on executor multi with executor id 3243 parsl.dataflow.dflow INFO: Standard output for task 3242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/69ec19b2-61eb-40f9-a248-5ef03557c973_isrForCrosstalkSources_997029_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/69ec19b2-61eb-40f9-a248-5ef03557c973_isrForCrosstalkSources_997029_8.stderr parsl.dataflow.dflow DEBUG: Task 3243 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3243 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3243 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3243 has memoization hash 63c66e5464f916c58a99f0b0d8046a91 parsl.dataflow.memoization INFO: Task 3243 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3243 try 0 launched on executor multi with executor id 3244 parsl.dataflow.dflow INFO: Standard output for task 3243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/14e76306-cf6c-4d3e-bcc1-dfde3c4fe763_isrForCrosstalkSources_996990_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/14e76306-cf6c-4d3e-bcc1-dfde3c4fe763_isrForCrosstalkSources_996990_7.stderr parsl.dataflow.dflow DEBUG: Task 3244 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3244 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3244 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3244 has memoization hash 38a02fab8a83c4fe5500e7cb7edd4b0a parsl.dataflow.memoization INFO: Task 3244 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3244 try 0 launched on executor multi with executor id 3245 parsl.dataflow.dflow INFO: Standard output for task 3244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c79c2b08-71c5-4e0b-af45-6583d718e0db_isrForCrosstalkSources_996993_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c79c2b08-71c5-4e0b-af45-6583d718e0db_isrForCrosstalkSources_996993_34.stderr parsl.dataflow.dflow DEBUG: Task 3245 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3245 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3245 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3245 has memoization hash 63577645c79c083c98cdd9dcd02dff9f parsl.dataflow.memoization INFO: Task 3245 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7037e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3245 try 0 launched on executor multi with executor id 3246 parsl.dataflow.dflow INFO: Standard output for task 3245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b14cf562-4445-4dc7-9d26-cd5fa9a36309_isrForCrosstalkSources_996964_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b14cf562-4445-4dc7-9d26-cd5fa9a36309_isrForCrosstalkSources_996964_25.stderr parsl.dataflow.dflow DEBUG: Task 3246 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3246 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3246 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3246 has memoization hash 541831a4fe9d34430ac4ddd4f6ff7e7d parsl.dataflow.memoization INFO: Task 3246 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7039c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3246 try 0 launched on executor multi with executor id 3247 parsl.dataflow.dflow INFO: Standard output for task 3246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/aac934b1-5a9b-476a-9f0e-3876951a15fa_isrForCrosstalkSources_996977_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/aac934b1-5a9b-476a-9f0e-3876951a15fa_isrForCrosstalkSources_996977_19.stderr parsl.dataflow.dflow DEBUG: Task 3247 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3247 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3247 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3247 has memoization hash a898838f6105f1d1332d86b0ea3b75d4 parsl.dataflow.memoization INFO: Task 3247 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3247 try 0 launched on executor multi with executor id 3248 parsl.dataflow.dflow INFO: Standard output for task 3247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/62247415-14c2-493f-8bfd-6997185a0768_isrForCrosstalkSources_997024_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/62247415-14c2-493f-8bfd-6997185a0768_isrForCrosstalkSources_997024_1.stderr parsl.dataflow.dflow DEBUG: Task 3248 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3248 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3248 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3248 has memoization hash fd83eb5ca8a1e909b127f7ab67361fc1 parsl.dataflow.memoization INFO: Task 3248 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3248 try 0 launched on executor multi with executor id 3249 parsl.dataflow.dflow INFO: Standard output for task 3248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/e86c4ec8-9086-4eba-b67b-a13a4d655b86_isrForCrosstalkSources_996974_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/e86c4ec8-9086-4eba-b67b-a13a4d655b86_isrForCrosstalkSources_996974_5.stderr parsl.dataflow.dflow DEBUG: Task 3249 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3249 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3249 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3249 has memoization hash 3250826321c75e6ff868e22d6f977035 parsl.dataflow.memoization INFO: Task 3249 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3249 try 0 launched on executor multi with executor id 3250 parsl.dataflow.dflow INFO: Standard output for task 3249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/babd6167-1f35-43a8-9c44-606f251a3425_isrForCrosstalkSources_997018_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/babd6167-1f35-43a8-9c44-606f251a3425_isrForCrosstalkSources_997018_38.stderr parsl.dataflow.dflow DEBUG: Task 3250 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3250 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3250 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3250 has memoization hash f7089c735da1a579adbd73667d2a2d71 parsl.dataflow.memoization INFO: Task 3250 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3250 try 0 launched on executor multi with executor id 3251 parsl.dataflow.dflow INFO: Standard output for task 3250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/559b6395-6573-4880-8d9e-e8131d5b8d92_isrForCrosstalkSources_996960_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/559b6395-6573-4880-8d9e-e8131d5b8d92_isrForCrosstalkSources_996960_60.stderr parsl.dataflow.dflow DEBUG: Task 3251 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3251 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3251 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3251 has memoization hash 5fe1c9cfd0bd42ea05008bde6247973b parsl.dataflow.memoization INFO: Task 3251 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3251 try 0 launched on executor multi with executor id 3252 parsl.dataflow.dflow INFO: Standard output for task 3251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a7381447-183c-4ada-bf9b-4dec0aedfaaf_isrForCrosstalkSources_996996_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a7381447-183c-4ada-bf9b-4dec0aedfaaf_isrForCrosstalkSources_996996_32.stderr parsl.dataflow.dflow DEBUG: Task 3252 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3252 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3252 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3252 has memoization hash aa1cfa9a3b89e472f45c48c3e6f51b10 parsl.dataflow.memoization INFO: Task 3252 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3252 try 0 launched on executor multi with executor id 3253 parsl.dataflow.dflow INFO: Standard output for task 3252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/449ca920-a62e-4ede-a622-77395e28dab0_isrForCrosstalkSources_996995_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/449ca920-a62e-4ede-a622-77395e28dab0_isrForCrosstalkSources_996995_56.stderr parsl.dataflow.dflow DEBUG: Task 3253 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3253 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3253 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3253 has memoization hash 75dea96d32364a4577ed3517e8476675 parsl.dataflow.memoization INFO: Task 3253 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3253 try 0 launched on executor multi with executor id 3254 parsl.dataflow.dflow INFO: Standard output for task 3253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ed44ad6e-db55-4b53-b962-bd48f489f6e4_isrForCrosstalkSources_996998_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ed44ad6e-db55-4b53-b962-bd48f489f6e4_isrForCrosstalkSources_996998_19.stderr parsl.dataflow.dflow DEBUG: Task 3254 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3254 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3254 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3254 has memoization hash 858d78dbbd2ff6412092ecf65ac0c2be parsl.dataflow.memoization INFO: Task 3254 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3254 try 0 launched on executor multi with executor id 3255 parsl.dataflow.dflow INFO: Standard output for task 3254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ac4aea4e-e619-4e2d-b5be-1a55ec42e026_isrForCrosstalkSources_997016_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ac4aea4e-e619-4e2d-b5be-1a55ec42e026_isrForCrosstalkSources_997016_8.stderr parsl.dataflow.dflow DEBUG: Task 3255 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3255 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3255 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3255 has memoization hash c2cc96e7645927b04e5caea642d7e569 parsl.dataflow.memoization INFO: Task 3255 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3255 try 0 launched on executor multi with executor id 3256 parsl.dataflow.dflow INFO: Standard output for task 3255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/17406ab4-da21-4b7d-a4fa-af0f2e9bf5ce_isrForCrosstalkSources_997010_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/17406ab4-da21-4b7d-a4fa-af0f2e9bf5ce_isrForCrosstalkSources_997010_45.stderr parsl.dataflow.dflow DEBUG: Task 3256 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3256 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3256 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3256 has memoization hash 54ccbe542c4a90af27d170b557972507 parsl.dataflow.memoization INFO: Task 3256 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c703f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3256 try 0 launched on executor multi with executor id 3257 parsl.dataflow.dflow INFO: Standard output for task 3256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7c90b4ad-e86b-4d13-bee4-dd7f9d24e903_isrForCrosstalkSources_997024_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7c90b4ad-e86b-4d13-bee4-dd7f9d24e903_isrForCrosstalkSources_997024_57.stderr parsl.dataflow.dflow DEBUG: Task 3257 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3257 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3257 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3257 has memoization hash bd5bdd6dd3dc4b7efc26c009c729a7a7 parsl.dataflow.memoization INFO: Task 3257 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3257 try 0 launched on executor multi with executor id 3258 parsl.dataflow.dflow INFO: Standard output for task 3257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/abef7e08-46f3-4906-a3fb-7e35b692096d_isrForCrosstalkSources_996979_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/abef7e08-46f3-4906-a3fb-7e35b692096d_isrForCrosstalkSources_996979_4.stderr parsl.dataflow.dflow DEBUG: Task 3258 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3258 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3258 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3258 has memoization hash 657fbebe513a188bff6e8d009ecfe893 parsl.dataflow.memoization INFO: Task 3258 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5980e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3258 try 0 launched on executor multi with executor id 3259 parsl.dataflow.dflow INFO: Standard output for task 3258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c981a80c-1090-4a38-941b-42787fcdffdc_isrForCrosstalkSources_996965_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c981a80c-1090-4a38-941b-42787fcdffdc_isrForCrosstalkSources_996965_10.stderr parsl.dataflow.dflow DEBUG: Task 3259 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3259 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3259 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3259 has memoization hash 640e361a73ca5e7cc7d63d0b2dded230 parsl.dataflow.memoization INFO: Task 3259 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3259 try 0 launched on executor multi with executor id 3260 parsl.dataflow.dflow INFO: Standard output for task 3259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1ac4f8d7-b0b4-444e-a353-71809351af62_isrForCrosstalkSources_996998_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1ac4f8d7-b0b4-444e-a353-71809351af62_isrForCrosstalkSources_996998_50.stderr parsl.dataflow.dflow DEBUG: Task 3260 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3260 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3260 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3260 has memoization hash b6fc7c59a615e550c61d42216152c6f0 parsl.dataflow.memoization INFO: Task 3260 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3260 try 0 launched on executor multi with executor id 3261 parsl.dataflow.dflow INFO: Standard output for task 3260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/18c76de1-d6e0-4c43-a4fa-392a5167efbb_isrForCrosstalkSources_996987_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/18c76de1-d6e0-4c43-a4fa-392a5167efbb_isrForCrosstalkSources_996987_52.stderr parsl.dataflow.dflow DEBUG: Task 3261 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3261 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3261 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3261 has memoization hash bf5ea47092edabbb1eac68cb4ca3e6ae parsl.dataflow.memoization INFO: Task 3261 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5982c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3261 try 0 launched on executor multi with executor id 3262 parsl.dataflow.dflow INFO: Standard output for task 3261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a91af4c3-256b-4c0a-9bd6-8b7a53b97160_isrForCrosstalkSources_997000_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a91af4c3-256b-4c0a-9bd6-8b7a53b97160_isrForCrosstalkSources_997000_54.stderr parsl.dataflow.dflow DEBUG: Task 3262 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3262 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3262 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3262 has memoization hash 93157f2ce45130bd026afb24afb81da1 parsl.dataflow.memoization INFO: Task 3262 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3262 try 0 launched on executor multi with executor id 3263 parsl.dataflow.dflow INFO: Standard output for task 3262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ea18decf-2bf9-44c5-b2d4-96d535c3ab7b_isrForCrosstalkSources_997009_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ea18decf-2bf9-44c5-b2d4-96d535c3ab7b_isrForCrosstalkSources_997009_49.stderr parsl.dataflow.dflow DEBUG: Task 3263 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3263 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3263 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3263 has memoization hash 9fb601e505cbed81bb0380ee9d915236 parsl.dataflow.memoization INFO: Task 3263 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3263 try 0 launched on executor multi with executor id 3264 parsl.dataflow.dflow INFO: Standard output for task 3263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/0440f9e3-c56d-4b48-9d7d-02e38adcf7aa_isrForCrosstalkSources_997001_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/0440f9e3-c56d-4b48-9d7d-02e38adcf7aa_isrForCrosstalkSources_997001_10.stderr parsl.dataflow.dflow DEBUG: Task 3264 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3264 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3264 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3264 has memoization hash bd6d783f2f8a4fa98df3029a0ee911ab parsl.dataflow.memoization INFO: Task 3264 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3264 try 0 launched on executor multi with executor id 3265 parsl.dataflow.dflow INFO: Standard output for task 3264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/e7fa2deb-dc2b-40f0-a3ff-30cc8193ca32_isrForCrosstalkSources_996976_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/e7fa2deb-dc2b-40f0-a3ff-30cc8193ca32_isrForCrosstalkSources_996976_3.stderr parsl.dataflow.dflow DEBUG: Task 3265 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3265 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3265 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3265 has memoization hash 0768f8a4e924581ae4fa30f502d5287c parsl.dataflow.memoization INFO: Task 3265 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5984a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3265 try 0 launched on executor multi with executor id 3266 parsl.dataflow.dflow INFO: Standard output for task 3265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/913c0388-3dff-46d2-9293-02a87fbf1b39_isrForCrosstalkSources_996961_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/913c0388-3dff-46d2-9293-02a87fbf1b39_isrForCrosstalkSources_996961_43.stderr parsl.dataflow.dflow DEBUG: Task 3266 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3266 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3266 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3266 has memoization hash bfeeacc7ae0290e3821760e16674fc43 parsl.dataflow.memoization INFO: Task 3266 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5985e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3266 try 0 launched on executor multi with executor id 3267 parsl.dataflow.dflow INFO: Standard output for task 3266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f156071d-ba4c-4b79-a24c-54d7ddf5f602_isrForCrosstalkSources_997016_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f156071d-ba4c-4b79-a24c-54d7ddf5f602_isrForCrosstalkSources_997016_34.stderr parsl.dataflow.dflow DEBUG: Task 3267 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3267 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3267 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3267 has memoization hash c18137992ed4c50a6efdd38fdb7f1e34 parsl.dataflow.memoization INFO: Task 3267 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3267 try 0 launched on executor multi with executor id 3268 parsl.dataflow.dflow INFO: Standard output for task 3267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4f6280d3-742f-4fac-8500-22dcc2b4d494_isrForCrosstalkSources_997015_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4f6280d3-742f-4fac-8500-22dcc2b4d494_isrForCrosstalkSources_997015_48.stderr parsl.dataflow.dflow DEBUG: Task 3268 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3268 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3268 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3268 has memoization hash 976134703dcc78faed33550b1d47def4 parsl.dataflow.memoization INFO: Task 3268 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3268 try 0 launched on executor multi with executor id 3269 parsl.dataflow.dflow INFO: Standard output for task 3268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/7eec73bf-2f77-43ca-92ac-c720da21b397_isrForCrosstalkSources_996995_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/7eec73bf-2f77-43ca-92ac-c720da21b397_isrForCrosstalkSources_996995_44.stderr parsl.dataflow.dflow DEBUG: Task 3269 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3269 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3269 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3269 has memoization hash 9b653374c703f787ffc660e9bcdd748a parsl.dataflow.memoization INFO: Task 3269 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3269 try 0 launched on executor multi with executor id 3270 parsl.dataflow.dflow INFO: Standard output for task 3269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1fbd26ac-687a-4813-953e-190ff06b9178_isrForCrosstalkSources_996957_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1fbd26ac-687a-4813-953e-190ff06b9178_isrForCrosstalkSources_996957_43.stderr parsl.dataflow.dflow DEBUG: Task 3270 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3270 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3270 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3270 has memoization hash 9a6b66538863c308b7363c9cc1f85430 parsl.dataflow.memoization INFO: Task 3270 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5987c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3270 try 0 launched on executor multi with executor id 3271 parsl.dataflow.dflow INFO: Standard output for task 3270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/94b8b631-a128-4520-ac76-5b0a03f4a8f2_isrForCrosstalkSources_996967_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/94b8b631-a128-4520-ac76-5b0a03f4a8f2_isrForCrosstalkSources_996967_27.stderr parsl.dataflow.dflow DEBUG: Task 3271 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3271 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3271 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3271 has memoization hash 41f3efa72a6cd497838c45c77a2af504 parsl.dataflow.memoization INFO: Task 3271 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3271 try 0 launched on executor multi with executor id 3272 parsl.dataflow.dflow INFO: Standard output for task 3271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/4f7aad10-7c8b-46b6-b008-6a21e5d89c55_isrForCrosstalkSources_997031_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/4f7aad10-7c8b-46b6-b008-6a21e5d89c55_isrForCrosstalkSources_997031_22.stderr parsl.dataflow.dflow DEBUG: Task 3272 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3272 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3272 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3272 has memoization hash 8730b2182725b4df375f518847b02778 parsl.dataflow.memoization INFO: Task 3272 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5989a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3272 try 0 launched on executor multi with executor id 3273 parsl.dataflow.dflow INFO: Standard output for task 3272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f5d5213f-81bc-4776-9cbb-b7a3e5569636_isrForCrosstalkSources_997024_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f5d5213f-81bc-4776-9cbb-b7a3e5569636_isrForCrosstalkSources_997024_56.stderr parsl.dataflow.dflow DEBUG: Task 3273 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3273 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3273 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3273 has memoization hash ae31d5438f4746dbce3c85bfc895503b parsl.dataflow.memoization INFO: Task 3273 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3273 try 0 launched on executor multi with executor id 3274 parsl.dataflow.dflow INFO: Standard output for task 3273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/ff4bfee7-7e82-432d-996d-131f8f60b886_isrForCrosstalkSources_997027_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/ff4bfee7-7e82-432d-996d-131f8f60b886_isrForCrosstalkSources_997027_57.stderr parsl.dataflow.dflow DEBUG: Task 3274 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3274 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3274 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3274 has memoization hash 2210ff7eee818cc57f064684e451333a parsl.dataflow.memoization INFO: Task 3274 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3274 try 0 launched on executor multi with executor id 3275 parsl.dataflow.dflow INFO: Standard output for task 3274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ca35e9b3-b27c-4989-bf4a-379b8c7385be_isrForCrosstalkSources_996978_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ca35e9b3-b27c-4989-bf4a-379b8c7385be_isrForCrosstalkSources_996978_42.stderr parsl.dataflow.dflow DEBUG: Task 3275 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3275 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3275 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3275 has memoization hash 6430537eb2d7199fcf9700bb5da94a9a parsl.dataflow.memoization INFO: Task 3275 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3275 try 0 launched on executor multi with executor id 3276 parsl.dataflow.dflow INFO: Standard output for task 3275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/618f7062-520d-48e8-a599-878d57908ae6_isrForCrosstalkSources_997007_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/618f7062-520d-48e8-a599-878d57908ae6_isrForCrosstalkSources_997007_32.stderr parsl.dataflow.dflow DEBUG: Task 3276 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3276 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3276 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3276 has memoization hash 4e689b9362e8a02da874d6576d4203aa parsl.dataflow.memoization INFO: Task 3276 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3276 try 0 launched on executor multi with executor id 3277 parsl.dataflow.dflow INFO: Standard output for task 3276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/827e5923-9b3b-4865-aa46-355fc5f10efa_isrForCrosstalkSources_996958_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/827e5923-9b3b-4865-aa46-355fc5f10efa_isrForCrosstalkSources_996958_45.stderr parsl.dataflow.dflow DEBUG: Task 3277 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3277 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3277 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3277 has memoization hash 13b98b7389647af614e5064ded88a2ef parsl.dataflow.memoization INFO: Task 3277 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3277 try 0 launched on executor multi with executor id 3278 parsl.dataflow.dflow INFO: Standard output for task 3277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/bd321c43-faa5-4e9d-b421-9d54b2cbdd2e_isrForCrosstalkSources_996981_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/bd321c43-faa5-4e9d-b421-9d54b2cbdd2e_isrForCrosstalkSources_996981_34.stderr parsl.dataflow.dflow DEBUG: Task 3278 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3278 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3278 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3278 has memoization hash 549859a81909c448e5b00e1bc6cf3907 parsl.dataflow.memoization INFO: Task 3278 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3278 try 0 launched on executor multi with executor id 3279 parsl.dataflow.dflow INFO: Standard output for task 3278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2715a42f-9501-4677-b1ae-de85f57e9e8f_isrForCrosstalkSources_996963_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2715a42f-9501-4677-b1ae-de85f57e9e8f_isrForCrosstalkSources_996963_23.stderr parsl.dataflow.dflow DEBUG: Task 3279 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3279 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3279 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3279 has memoization hash 4dcdc60785d10c8d4307a89a1123683e parsl.dataflow.memoization INFO: Task 3279 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3279 try 0 launched on executor multi with executor id 3280 parsl.dataflow.dflow INFO: Standard output for task 3279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/906cebc8-5af4-4d8d-aa54-4501ca8ee5cd_isrForCrosstalkSources_996956_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/906cebc8-5af4-4d8d-aa54-4501ca8ee5cd_isrForCrosstalkSources_996956_16.stderr parsl.dataflow.dflow DEBUG: Task 3280 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3280 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3280 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3280 has memoization hash 2c170007e2f4f5d8e3e05622ecbe3ba0 parsl.dataflow.memoization INFO: Task 3280 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3280 try 0 launched on executor multi with executor id 3281 parsl.dataflow.dflow INFO: Standard output for task 3280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/fee0cd93-a087-4253-af6b-424797133f8a_isrForCrosstalkSources_996989_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/fee0cd93-a087-4253-af6b-424797133f8a_isrForCrosstalkSources_996989_36.stderr parsl.dataflow.dflow DEBUG: Task 3281 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3281 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3281 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3281 has memoization hash 032d525a2166102d6d2847f587fda1d9 parsl.dataflow.memoization INFO: Task 3281 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3281 try 0 launched on executor multi with executor id 3282 parsl.dataflow.dflow INFO: Standard output for task 3281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f34ef32e-1329-4ee0-b7ef-1f21490fdb2d_isrForCrosstalkSources_997020_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f34ef32e-1329-4ee0-b7ef-1f21490fdb2d_isrForCrosstalkSources_997020_44.stderr parsl.dataflow.dflow DEBUG: Task 3282 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3282 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3282 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3282 has memoization hash 50df77904d7ac6eda55b57194f548243 parsl.dataflow.memoization INFO: Task 3282 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3282 try 0 launched on executor multi with executor id 3283 parsl.dataflow.dflow INFO: Standard output for task 3282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/236080cc-ae97-46b8-819a-436b4f191cb3_isrForCrosstalkSources_996959_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/236080cc-ae97-46b8-819a-436b4f191cb3_isrForCrosstalkSources_996959_28.stderr parsl.dataflow.dflow DEBUG: Task 3283 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3283 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3283 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3283 has memoization hash 7dad7c1a40a47b2b80b66fc4ee816a6c parsl.dataflow.memoization INFO: Task 3283 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3283 try 0 launched on executor multi with executor id 3284 parsl.dataflow.dflow INFO: Standard output for task 3283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/6334bec6-7cdd-40f0-9a23-6ae2c6954ae2_isrForCrosstalkSources_997028_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/6334bec6-7cdd-40f0-9a23-6ae2c6954ae2_isrForCrosstalkSources_997028_31.stderr parsl.dataflow.dflow DEBUG: Task 3284 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3284 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3284 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3284 has memoization hash 86b6a54ae86725f5b23d53201c3842c1 parsl.dataflow.memoization INFO: Task 3284 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5991c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3284 try 0 launched on executor multi with executor id 3285 parsl.dataflow.dflow INFO: Standard output for task 3284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7cbecc84-08d9-4ba7-8595-f45e402ff526_isrForCrosstalkSources_997029_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7cbecc84-08d9-4ba7-8595-f45e402ff526_isrForCrosstalkSources_997029_9.stderr parsl.dataflow.dflow DEBUG: Task 3285 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3285 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3285 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3285 has memoization hash 38e98ac3b4f48d23a9890efcc01e30d5 parsl.dataflow.memoization INFO: Task 3285 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3285 try 0 launched on executor multi with executor id 3286 parsl.dataflow.dflow INFO: Standard output for task 3285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/76093ac7-8de8-488a-9c5b-93b3e2fe8e0d_isrForCrosstalkSources_996976_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/76093ac7-8de8-488a-9c5b-93b3e2fe8e0d_isrForCrosstalkSources_996976_55.stderr parsl.dataflow.dflow DEBUG: Task 3286 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3286 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3286 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3286 has memoization hash 550ff7cc4125c83ecdc2e431f0c545b3 parsl.dataflow.memoization INFO: Task 3286 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3286 try 0 launched on executor multi with executor id 3287 parsl.dataflow.dflow INFO: Standard output for task 3286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/13b33dd8-d3a2-4edb-aa61-4519d6ad5ba2_isrForCrosstalkSources_996999_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/13b33dd8-d3a2-4edb-aa61-4519d6ad5ba2_isrForCrosstalkSources_996999_45.stderr parsl.dataflow.dflow DEBUG: Task 3287 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3287 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3287 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3287 has memoization hash 8a301f1c1eb7db261ec8c70d83008118 parsl.dataflow.memoization INFO: Task 3287 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3287 try 0 launched on executor multi with executor id 3288 parsl.dataflow.dflow INFO: Standard output for task 3287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ab728d8c-98c9-4650-9e37-a2e9ddcd9fd6_isrForCrosstalkSources_996976_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ab728d8c-98c9-4650-9e37-a2e9ddcd9fd6_isrForCrosstalkSources_996976_37.stderr parsl.dataflow.dflow DEBUG: Task 3288 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3288 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3288 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3288 has memoization hash 992e9afc89cce4251bb210b26406283b parsl.dataflow.memoization INFO: Task 3288 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5993a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3288 try 0 launched on executor multi with executor id 3289 parsl.dataflow.dflow INFO: Standard output for task 3288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b4931aff-0ff0-446b-8994-f35bfd9b2efd_isrForCrosstalkSources_996975_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b4931aff-0ff0-446b-8994-f35bfd9b2efd_isrForCrosstalkSources_996975_17.stderr parsl.dataflow.dflow DEBUG: Task 3289 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3289 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3289 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3289 has memoization hash 23cbed4e8f04657fb3fe92484ca8bffc parsl.dataflow.memoization INFO: Task 3289 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5994e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3289 try 0 launched on executor multi with executor id 3290 parsl.dataflow.dflow INFO: Standard output for task 3289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3b31aee7-3145-4e17-a781-778abddfe1be_isrForCrosstalkSources_997014_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3b31aee7-3145-4e17-a781-778abddfe1be_isrForCrosstalkSources_997014_15.stderr parsl.dataflow.dflow DEBUG: Task 3290 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3290 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3290 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3290 has memoization hash 7e84230010fa86d01ab91b87185eb782 parsl.dataflow.memoization INFO: Task 3290 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3290 try 0 launched on executor multi with executor id 3291 parsl.dataflow.dflow INFO: Standard output for task 3290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/8e8eeae7-f312-4152-b85b-2816f54347d4_isrForCrosstalkSources_996972_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/8e8eeae7-f312-4152-b85b-2816f54347d4_isrForCrosstalkSources_996972_59.stderr parsl.dataflow.dflow DEBUG: Task 3291 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3291 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3291 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3291 has memoization hash c07bc559f5a37b927906b9a3a96252a0 parsl.dataflow.memoization INFO: Task 3291 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3291 try 0 launched on executor multi with executor id 3292 parsl.dataflow.dflow INFO: Standard output for task 3291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/855ac11b-788b-49be-b3ee-72d246680481_isrForCrosstalkSources_997022_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/855ac11b-788b-49be-b3ee-72d246680481_isrForCrosstalkSources_997022_19.stderr parsl.dataflow.dflow DEBUG: Task 3292 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3292 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3292 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3292 has memoization hash 356576c0828076e7e4e9255ea0dec653 parsl.dataflow.memoization INFO: Task 3292 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3292 try 0 launched on executor multi with executor id 3293 parsl.dataflow.dflow INFO: Standard output for task 3292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/dcecf8aa-9a34-463d-96ce-4885c3a8a07a_isrForCrosstalkSources_996990_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/dcecf8aa-9a34-463d-96ce-4885c3a8a07a_isrForCrosstalkSources_996990_6.stderr parsl.dataflow.dflow DEBUG: Task 3293 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3293 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3293 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3293 has memoization hash 5a2ed3b5f561830d3bc369bbf35c310f parsl.dataflow.memoization INFO: Task 3293 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5996c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3293 try 0 launched on executor multi with executor id 3294 parsl.dataflow.dflow INFO: Standard output for task 3293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/746c9a11-6d8e-4576-b1fd-3f3bcfafa6e1_isrForCrosstalkSources_996976_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/746c9a11-6d8e-4576-b1fd-3f3bcfafa6e1_isrForCrosstalkSources_996976_12.stderr parsl.dataflow.dflow DEBUG: Task 3294 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3294 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3294 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3294 has memoization hash 4976fbcd4b543227ed4268c0c50e30b9 parsl.dataflow.memoization INFO: Task 3294 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3294 try 0 launched on executor multi with executor id 3295 parsl.dataflow.dflow INFO: Standard output for task 3294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c817db70-217e-4880-b3b1-1685ebbd8e81_isrForCrosstalkSources_996975_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c817db70-217e-4880-b3b1-1685ebbd8e81_isrForCrosstalkSources_996975_8.stderr parsl.dataflow.dflow DEBUG: Task 3295 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3295 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3295 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3295 has memoization hash 369ee59c955255efee832bbf4c72e9bb parsl.dataflow.memoization INFO: Task 3295 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3295 try 0 launched on executor multi with executor id 3296 parsl.dataflow.dflow INFO: Standard output for task 3295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1d5ecacb-5e75-4bfb-927a-b1aa8d49f2d7_isrForCrosstalkSources_997014_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1d5ecacb-5e75-4bfb-927a-b1aa8d49f2d7_isrForCrosstalkSources_997014_46.stderr parsl.dataflow.dflow DEBUG: Task 3296 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3296 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3296 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3296 has memoization hash 9e63a561da99bdbd979142751de24bb1 parsl.dataflow.memoization INFO: Task 3296 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5998a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3296 try 0 launched on executor multi with executor id 3297 parsl.dataflow.dflow INFO: Standard output for task 3296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b69c37a3-6008-4b74-988f-dfb495a5f6f0_isrForCrosstalkSources_996986_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b69c37a3-6008-4b74-988f-dfb495a5f6f0_isrForCrosstalkSources_996986_10.stderr parsl.dataflow.dflow DEBUG: Task 3297 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3297 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3297 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3297 has memoization hash ac6a8004c9c70237a5877947e4f2e2a7 parsl.dataflow.memoization INFO: Task 3297 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3297 try 0 launched on executor multi with executor id 3298 parsl.dataflow.dflow INFO: Standard output for task 3297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e46034a9-74c3-47f8-ba97-a69ce97512cf_isrForCrosstalkSources_996993_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e46034a9-74c3-47f8-ba97-a69ce97512cf_isrForCrosstalkSources_996993_10.stderr parsl.dataflow.dflow DEBUG: Task 3298 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3298 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3298 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3298 has memoization hash f749086426f233d295d0d90e7c13c838 parsl.dataflow.memoization INFO: Task 3298 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c5999e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3298 try 0 launched on executor multi with executor id 3299 parsl.dataflow.dflow INFO: Standard output for task 3298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/2e8008a3-8764-4d10-ade6-bb0189c76a41_isrForCrosstalkSources_996953_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/2e8008a3-8764-4d10-ade6-bb0189c76a41_isrForCrosstalkSources_996953_31.stderr parsl.dataflow.dflow DEBUG: Task 3299 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3299 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3299 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3299 has memoization hash b4b071e7a92d54392ade3b55d9b3831d parsl.dataflow.memoization INFO: Task 3299 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3299 try 0 launched on executor multi with executor id 3300 parsl.dataflow.dflow INFO: Standard output for task 3299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6fa383af-1ddd-46ea-a88e-f796b3101b6a_isrForCrosstalkSources_996963_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6fa383af-1ddd-46ea-a88e-f796b3101b6a_isrForCrosstalkSources_996963_15.stderr parsl.dataflow.dflow DEBUG: Task 3300 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3300 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3300 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3300 has memoization hash 7d9c316b217bc68cff40ba7d4c4dfc5c parsl.dataflow.memoization INFO: Task 3300 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3300 try 0 launched on executor multi with executor id 3301 parsl.dataflow.dflow INFO: Standard output for task 3300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/70e25ea8-bc83-4603-aa97-d02d58bc22ae_isrForCrosstalkSources_996993_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/70e25ea8-bc83-4603-aa97-d02d58bc22ae_isrForCrosstalkSources_996993_19.stderr parsl.dataflow.dflow DEBUG: Task 3301 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3301 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3301 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3301 has memoization hash 600d412999185bdf5a2d030fc464e07d parsl.dataflow.memoization INFO: Task 3301 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3301 try 0 launched on executor multi with executor id 3302 parsl.dataflow.dflow INFO: Standard output for task 3301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/5f57e9ee-5c7b-4583-8124-022f78a626ba_isrForCrosstalkSources_996977_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/5f57e9ee-5c7b-4583-8124-022f78a626ba_isrForCrosstalkSources_996977_13.stderr parsl.dataflow.dflow DEBUG: Task 3302 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3302 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3302 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3302 has memoization hash 2aeba134118602fd5414ee0811ea16a5 parsl.dataflow.memoization INFO: Task 3302 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:57 proc_lsst.multi:170 [INFO] [multi] job_id 17074888 proc_lsst.multi INFO: [multi] job_id 17074888 2024-03-15 13:49:57 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 4 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 4 2024-03-15 13:49:57 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:57 proc_lsst.multi:178 [INFO] [multi] provider escience_rtx6k accepted submit and returned 17074888 proc_lsst.multi INFO: [multi] provider escience_rtx6k accepted submit and returned 17074888 parsl.executors.status_handling DEBUG: Launched block 34 on executor multi with job ID 17074888 parsl.executors.status_handling INFO: Allocated block ID 35 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-35 2024-03-15 13:49:57 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-35 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-35 2024-03-15 13:49:57 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:57 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:49:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:57 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 4 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 4 2024-03-15 13:49:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:57 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=35 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_rtx6k parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 3302 try 0 launched on executor multi with executor id 3303 parsl.dataflow.dflow INFO: Standard output for task 3302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/0e21ecd1-c0a3-4de0-af5c-9a3a3bd9e3f8_isrForCrosstalkSources_997029_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/0e21ecd1-c0a3-4de0-af5c-9a3a3bd9e3f8_isrForCrosstalkSources_997029_49.stderr parsl.dataflow.dflow DEBUG: Task 3303 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3303 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3303 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3303 has memoization hash 4a2f6f324aa8cb27b7219346063a24f2 parsl.dataflow.memoization INFO: Task 3303 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3303 try 0 launched on executor multi with executor id 3304 parsl.dataflow.dflow INFO: Standard output for task 3303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5e317365-03ae-475b-b687-b7794f361817_isrForCrosstalkSources_997021_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5e317365-03ae-475b-b687-b7794f361817_isrForCrosstalkSources_997021_13.stderr parsl.dataflow.dflow DEBUG: Task 3304 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3304 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3304 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3304 has memoization hash 3d750a88aa5b26e79d3edbea5533e7b2 parsl.dataflow.memoization INFO: Task 3304 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59bf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3304 try 0 launched on executor multi with executor id 3305 parsl.dataflow.dflow INFO: Standard output for task 3304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/cccfbba1-186b-4574-9166-d6651374a742_isrForCrosstalkSources_996968_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/cccfbba1-186b-4574-9166-d6651374a742_isrForCrosstalkSources_996968_60.stderr parsl.dataflow.dflow DEBUG: Task 3305 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3305 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3305 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3305 has memoization hash 5bbda126a080b829fa64fd193088fe60 parsl.dataflow.memoization INFO: Task 3305 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3305 try 0 launched on executor multi with executor id 3306 parsl.dataflow.dflow INFO: Standard output for task 3305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ef95ef8e-5eeb-4a8d-b25d-ba2f7f77fe7a_isrForCrosstalkSources_997005_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ef95ef8e-5eeb-4a8d-b25d-ba2f7f77fe7a_isrForCrosstalkSources_997005_58.stderr parsl.dataflow.dflow DEBUG: Task 3306 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3306 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3306 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3306 has memoization hash 681c5f2d8a0ac1804f34d6f5727b3a4a parsl.dataflow.memoization INFO: Task 3306 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59be20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3306 try 0 launched on executor multi with executor id 3307 parsl.dataflow.dflow INFO: Standard output for task 3306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/9bafbe9c-e8d4-474e-846d-f0e98506188b_isrForCrosstalkSources_996994_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/9bafbe9c-e8d4-474e-846d-f0e98506188b_isrForCrosstalkSources_996994_19.stderr parsl.dataflow.dflow DEBUG: Task 3307 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3307 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3307 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3307 has memoization hash 84b34b3de46b2c196c8faef2c58016b8 parsl.dataflow.memoization INFO: Task 3307 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59bec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3307 try 0 launched on executor multi with executor id 3308 parsl.dataflow.dflow INFO: Standard output for task 3307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b46e73bb-e872-4fb9-954f-1c441d285544_isrForCrosstalkSources_996957_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b46e73bb-e872-4fb9-954f-1c441d285544_isrForCrosstalkSources_996957_8.stderr parsl.dataflow.dflow DEBUG: Task 3308 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3308 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3308 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3308 has memoization hash 8c146010644b3dfcf83e556d1cd949d4 parsl.dataflow.memoization INFO: Task 3308 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3308 try 0 launched on executor multi with executor id 3309 parsl.dataflow.dflow INFO: Standard output for task 3308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fbea0fbd-c2db-43ce-a598-11ad613eb48c_isrForCrosstalkSources_996990_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fbea0fbd-c2db-43ce-a598-11ad613eb48c_isrForCrosstalkSources_996990_35.stderr parsl.dataflow.dflow DEBUG: Task 3309 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3309 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3309 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3309 has memoization hash 18e6c1354e8dddf8a73461a6d3ac17d9 parsl.dataflow.memoization INFO: Task 3309 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3309 try 0 launched on executor multi with executor id 3310 parsl.dataflow.dflow INFO: Standard output for task 3309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ef2e4274-d671-478c-9cd8-bfec861316a2_isrForCrosstalkSources_996960_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ef2e4274-d671-478c-9cd8-bfec861316a2_isrForCrosstalkSources_996960_27.stderr parsl.dataflow.dflow DEBUG: Task 3310 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3310 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3310 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3310 has memoization hash d7aed51fa4eb49e1d1564578abf0ffcc parsl.dataflow.memoization INFO: Task 3310 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3310 try 0 launched on executor multi with executor id 3311 parsl.dataflow.dflow INFO: Standard output for task 3310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/73673270-259e-41d4-bb05-8083cc8cc483_isrForCrosstalkSources_996955_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/73673270-259e-41d4-bb05-8083cc8cc483_isrForCrosstalkSources_996955_41.stderr parsl.dataflow.dflow DEBUG: Task 3311 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3311 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3311 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3311 has memoization hash 8268db8a767400a899ca5bfb62f9b38c parsl.dataflow.memoization INFO: Task 3311 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3311 try 0 launched on executor multi with executor id 3312 parsl.dataflow.dflow INFO: Standard output for task 3311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2a590b61-1471-4bd8-aaa8-43b22c92f47e_isrForCrosstalkSources_996981_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2a590b61-1471-4bd8-aaa8-43b22c92f47e_isrForCrosstalkSources_996981_18.stderr parsl.dataflow.dflow DEBUG: Task 3312 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3312 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3312 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3312 has memoization hash 1f33c14b7c0cf595b37b928c8d244186 parsl.dataflow.memoization INFO: Task 3312 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3312 try 0 launched on executor multi with executor id 3313 parsl.dataflow.dflow INFO: Standard output for task 3312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/6696a742-de7d-4d1d-9eb0-dffbbc06ea48_isrForCrosstalkSources_996962_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/6696a742-de7d-4d1d-9eb0-dffbbc06ea48_isrForCrosstalkSources_996962_30.stderr parsl.dataflow.dflow DEBUG: Task 3313 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3313 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3313 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3313 has memoization hash fa7e64375073b09a565cd5e74e3d5619 parsl.dataflow.memoization INFO: Task 3313 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3313 try 0 launched on executor multi with executor id 3314 parsl.dataflow.dflow INFO: Standard output for task 3313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/310a60ab-e0ff-4194-be9d-fd54375fc6b5_isrForCrosstalkSources_997025_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/310a60ab-e0ff-4194-be9d-fd54375fc6b5_isrForCrosstalkSources_997025_27.stderr parsl.dataflow.dflow DEBUG: Task 3314 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3314 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3314 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3314 has memoization hash 4d400125de48dcdd517e50e1f8cb1ed1 parsl.dataflow.memoization INFO: Task 3314 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3314 try 0 launched on executor multi with executor id 3315 parsl.dataflow.dflow INFO: Standard output for task 3314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/509fd5b4-41d1-4871-8f6b-1a19d5eb8fc7_isrForCrosstalkSources_996979_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/509fd5b4-41d1-4871-8f6b-1a19d5eb8fc7_isrForCrosstalkSources_996979_49.stderr parsl.dataflow.dflow DEBUG: Task 3315 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3315 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3315 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3315 has memoization hash 8e91cadd7013959b00e68ae1bae97d44 parsl.dataflow.memoization INFO: Task 3315 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3315 try 0 launched on executor multi with executor id 3316 parsl.dataflow.dflow INFO: Standard output for task 3315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/9736c035-d4b3-46e3-9c48-447392d39c2a_isrForCrosstalkSources_996978_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/9736c035-d4b3-46e3-9c48-447392d39c2a_isrForCrosstalkSources_996978_8.stderr parsl.dataflow.dflow DEBUG: Task 3316 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3316 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3316 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3316 has memoization hash 703868cbb295d5f13f913930b2e4e97c parsl.dataflow.memoization INFO: Task 3316 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3316 try 0 launched on executor multi with executor id 3317 parsl.dataflow.dflow INFO: Standard output for task 3316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/245462cd-89ba-4499-a057-98b45c6a2e96_isrForCrosstalkSources_996988_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/245462cd-89ba-4499-a057-98b45c6a2e96_isrForCrosstalkSources_996988_46.stderr parsl.dataflow.dflow DEBUG: Task 3317 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3317 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3317 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3317 has memoization hash 150333336384370241acdde71eaa2429 parsl.dataflow.memoization INFO: Task 3317 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3317 try 0 launched on executor multi with executor id 3318 parsl.dataflow.dflow INFO: Standard output for task 3317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/1ec48d5a-259f-42a6-9fae-5144174cbe3f_isrForCrosstalkSources_997031_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/1ec48d5a-259f-42a6-9fae-5144174cbe3f_isrForCrosstalkSources_997031_23.stderr parsl.dataflow.dflow DEBUG: Task 3318 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3318 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3318 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3318 has memoization hash e2ac0042f30f8644b5b2cfab3e7df79e parsl.dataflow.memoization INFO: Task 3318 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3318 try 0 launched on executor multi with executor id 3319 parsl.dataflow.dflow INFO: Standard output for task 3318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/7605bbfc-6713-492e-afb9-33076fc809fa_isrForCrosstalkSources_996987_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/7605bbfc-6713-492e-afb9-33076fc809fa_isrForCrosstalkSources_996987_31.stderr parsl.dataflow.dflow DEBUG: Task 3319 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3319 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3319 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3319 has memoization hash d0378419447cd3487bee17bd17934f05 parsl.dataflow.memoization INFO: Task 3319 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3319 try 0 launched on executor multi with executor id 3320 parsl.dataflow.dflow INFO: Standard output for task 3319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/107515ed-ae0e-4689-9671-2025e9649f4e_isrForCrosstalkSources_996952_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/107515ed-ae0e-4689-9671-2025e9649f4e_isrForCrosstalkSources_996952_13.stderr parsl.dataflow.dflow DEBUG: Task 3320 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3320 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3320 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3320 has memoization hash cf06d857c48759d28337b2da9b8fbd06 parsl.dataflow.memoization INFO: Task 3320 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3320 try 0 launched on executor multi with executor id 3321 parsl.dataflow.dflow INFO: Standard output for task 3320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/aab54a7d-d7ad-40a9-a8e5-11616f74ce29_isrForCrosstalkSources_997000_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/aab54a7d-d7ad-40a9-a8e5-11616f74ce29_isrForCrosstalkSources_997000_21.stderr parsl.dataflow.dflow DEBUG: Task 3321 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3321 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3321 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3321 has memoization hash 91e54edf2854d79174ac990377bb91e0 parsl.dataflow.memoization INFO: Task 3321 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3321 try 0 launched on executor multi with executor id 3322 parsl.dataflow.dflow INFO: Standard output for task 3321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f77adb2c-8fcb-4ff1-ba05-8356979a53e8_isrForCrosstalkSources_996985_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f77adb2c-8fcb-4ff1-ba05-8356979a53e8_isrForCrosstalkSources_996985_6.stderr parsl.dataflow.dflow DEBUG: Task 3322 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3322 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3322 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3322 has memoization hash cf197dee65b4333a58fb5cf5bbff688b parsl.dataflow.memoization INFO: Task 3322 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3322 try 0 launched on executor multi with executor id 3323 parsl.dataflow.dflow INFO: Standard output for task 3322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6d1aaa6e-6c88-4c9b-b290-1fb390b5eebb_isrForCrosstalkSources_996952_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6d1aaa6e-6c88-4c9b-b290-1fb390b5eebb_isrForCrosstalkSources_996952_23.stderr parsl.dataflow.dflow DEBUG: Task 3323 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3323 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3323 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3323 has memoization hash 1fd7f2eefaabb2c43769e58edfa032be parsl.dataflow.memoization INFO: Task 3323 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3323 try 0 launched on executor multi with executor id 3324 parsl.dataflow.dflow INFO: Standard output for task 3323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e379d7a5-8612-4825-bff4-7e84ce8a1257_isrForCrosstalkSources_997011_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e379d7a5-8612-4825-bff4-7e84ce8a1257_isrForCrosstalkSources_997011_45.stderr parsl.dataflow.dflow DEBUG: Task 3324 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3324 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3324 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3324 has memoization hash b8df22a1139fafa21f4a82620cce5ec8 parsl.dataflow.memoization INFO: Task 3324 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3324 try 0 launched on executor multi with executor id 3325 parsl.dataflow.dflow INFO: Standard output for task 3324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/a4cf3674-29da-4562-991b-99eb67f2a881_isrForCrosstalkSources_996971_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/a4cf3674-29da-4562-991b-99eb67f2a881_isrForCrosstalkSources_996971_59.stderr parsl.dataflow.dflow DEBUG: Task 3325 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3325 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3325 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3325 has memoization hash 399f2241d387554b14a536d94346089c parsl.dataflow.memoization INFO: Task 3325 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3325 try 0 launched on executor multi with executor id 3326 parsl.dataflow.dflow INFO: Standard output for task 3325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1ec8a115-f790-445e-a5f4-a552942c30cd_isrForCrosstalkSources_996998_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1ec8a115-f790-445e-a5f4-a552942c30cd_isrForCrosstalkSources_996998_43.stderr parsl.dataflow.dflow DEBUG: Task 3326 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3326 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3326 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3326 has memoization hash 0a1505606c22590421a439efce2da837 parsl.dataflow.memoization INFO: Task 3326 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59a7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3326 try 0 launched on executor multi with executor id 3327 parsl.dataflow.dflow INFO: Standard output for task 3326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/ee36e191-dbda-4592-aae2-06e2ecf726fd_isrForCrosstalkSources_996997_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/ee36e191-dbda-4592-aae2-06e2ecf726fd_isrForCrosstalkSources_996997_7.stderr parsl.dataflow.dflow DEBUG: Task 3327 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3327 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3327 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3327 has memoization hash 168213b9ef246816087e3d56e3760ff7 parsl.dataflow.memoization INFO: Task 3327 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59aa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3327 try 0 launched on executor multi with executor id 3328 parsl.dataflow.dflow INFO: Standard output for task 3327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c3886520-c91a-4187-81fd-000ac4a6c762_isrForCrosstalkSources_996997_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c3886520-c91a-4187-81fd-000ac4a6c762_isrForCrosstalkSources_996997_1.stderr parsl.dataflow.dflow DEBUG: Task 3328 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3328 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3328 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3328 has memoization hash 5d13416d8985102c4e7785b89bb7a7ae parsl.dataflow.memoization INFO: Task 3328 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59ab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3328 try 0 launched on executor multi with executor id 3329 parsl.dataflow.dflow INFO: Standard output for task 3328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/e5ebf769-0558-4758-9466-8997ade498d2_isrForCrosstalkSources_997013_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/e5ebf769-0558-4758-9466-8997ade498d2_isrForCrosstalkSources_997013_35.stderr parsl.dataflow.dflow DEBUG: Task 3329 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3329 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3329 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3329 has memoization hash 21aa4c4bdb7a07222ed4f5c6cfd02027 parsl.dataflow.memoization INFO: Task 3329 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59aca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3329 try 0 launched on executor multi with executor id 3330 parsl.dataflow.dflow INFO: Standard output for task 3329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/6aeebf76-d2ac-44c0-8d5a-41f13f452db8_isrForCrosstalkSources_996976_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/6aeebf76-d2ac-44c0-8d5a-41f13f452db8_isrForCrosstalkSources_996976_36.stderr parsl.dataflow.dflow DEBUG: Task 3330 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3330 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3330 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3330 has memoization hash 6f1d9064e7547b06e2986ead6d89c74e parsl.dataflow.memoization INFO: Task 3330 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59ac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3330 try 0 launched on executor multi with executor id 3331 parsl.dataflow.dflow INFO: Standard output for task 3330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/ef686f83-df77-477d-a1b2-1daf50c197ba_isrForCrosstalkSources_997003_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/ef686f83-df77-477d-a1b2-1daf50c197ba_isrForCrosstalkSources_997003_33.stderr parsl.dataflow.dflow DEBUG: Task 3331 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3331 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3331 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3331 has memoization hash 5e3ffd83702a68a9d1220cf905af3825 parsl.dataflow.memoization INFO: Task 3331 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59aac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3331 try 0 launched on executor multi with executor id 3332 parsl.dataflow.dflow INFO: Standard output for task 3331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/84f229fe-e4c5-422f-b320-0391b8bb4248_isrForCrosstalkSources_996978_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/84f229fe-e4c5-422f-b320-0391b8bb4248_isrForCrosstalkSources_996978_27.stderr parsl.dataflow.dflow DEBUG: Task 3332 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3332 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3332 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3332 has memoization hash 699fad3e59cb6cdeb5aa1099becc6092 parsl.dataflow.memoization INFO: Task 3332 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59ade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3332 try 0 launched on executor multi with executor id 3333 parsl.dataflow.dflow INFO: Standard output for task 3332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/81cf0bc5-7028-47af-a4da-e25a9a30164c_isrForCrosstalkSources_997011_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/81cf0bc5-7028-47af-a4da-e25a9a30164c_isrForCrosstalkSources_997011_18.stderr parsl.dataflow.dflow DEBUG: Task 3333 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3333 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3333 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3333 has memoization hash 264db0278f5896390e43db38781f1218 parsl.dataflow.memoization INFO: Task 3333 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59ad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3333 try 0 launched on executor multi with executor id 3334 parsl.dataflow.dflow INFO: Standard output for task 3333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/1ca5f5d9-3a32-4e75-9fdf-7aedace0604c_isrForCrosstalkSources_997008_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/1ca5f5d9-3a32-4e75-9fdf-7aedace0604c_isrForCrosstalkSources_997008_39.stderr parsl.dataflow.dflow DEBUG: Task 3334 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3334 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3334 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3334 has memoization hash 44250622eafd68cf993b3733838c0ddf parsl.dataflow.memoization INFO: Task 3334 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59afc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3334 try 0 launched on executor multi with executor id 3335 parsl.dataflow.dflow INFO: Standard output for task 3334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/ab8d72fb-1a36-4940-a25c-15ede7da89de_isrForCrosstalkSources_996952_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/ab8d72fb-1a36-4940-a25c-15ede7da89de_isrForCrosstalkSources_996952_18.stderr parsl.dataflow.dflow DEBUG: Task 3335 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3335 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3335 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3335 has memoization hash 63e0364df08627eab65ae4bf994aa8a7 parsl.dataflow.memoization INFO: Task 3335 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59ae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3335 try 0 launched on executor multi with executor id 3336 parsl.dataflow.dflow INFO: Standard output for task 3335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/6e4e2002-5fb8-4fc9-a4e0-5cfe0461d5bc_isrForCrosstalkSources_997012_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/6e4e2002-5fb8-4fc9-a4e0-5cfe0461d5bc_isrForCrosstalkSources_997012_44.stderr parsl.dataflow.dflow DEBUG: Task 3336 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3336 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3336 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3336 has memoization hash 57ef1f2679dd71aa854417e10a92cc11 parsl.dataflow.memoization INFO: Task 3336 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59af20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3336 try 0 launched on executor multi with executor id 3337 parsl.dataflow.dflow INFO: Standard output for task 3336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5becfe46-9d18-4669-bfa1-a1c8478a20c0_isrForCrosstalkSources_997010_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5becfe46-9d18-4669-bfa1-a1c8478a20c0_isrForCrosstalkSources_997010_28.stderr parsl.dataflow.dflow DEBUG: Task 3337 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3337 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3337 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3337 has memoization hash e651e4297ec1be550935d00ac8cab83b parsl.dataflow.memoization INFO: Task 3337 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3337 try 0 launched on executor multi with executor id 3338 parsl.dataflow.dflow INFO: Standard output for task 3337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0d214ce9-2d00-4fec-9d70-e3f2b18bd4c1_isrForCrosstalkSources_997007_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0d214ce9-2d00-4fec-9d70-e3f2b18bd4c1_isrForCrosstalkSources_997007_39.stderr parsl.dataflow.dflow DEBUG: Task 3338 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3338 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3338 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3338 has memoization hash 4a258aaf9eb12242663e6a52bbcfcd6e parsl.dataflow.memoization INFO: Task 3338 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3338 try 0 launched on executor multi with executor id 3339 parsl.dataflow.dflow INFO: Standard output for task 3338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cd4c47bd-a869-4853-ac1a-8e09079c915d_isrForCrosstalkSources_996987_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cd4c47bd-a869-4853-ac1a-8e09079c915d_isrForCrosstalkSources_996987_33.stderr parsl.dataflow.dflow DEBUG: Task 3339 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3339 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3339 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3339 has memoization hash af3639d78d5e42ecb9372a34979efe28 parsl.dataflow.memoization INFO: Task 3339 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3339 try 0 launched on executor multi with executor id 3340 parsl.dataflow.dflow INFO: Standard output for task 3339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/77ef21c1-98f6-4b7d-9ba8-6421392af82f_isrForCrosstalkSources_996990_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/77ef21c1-98f6-4b7d-9ba8-6421392af82f_isrForCrosstalkSources_996990_27.stderr parsl.dataflow.dflow DEBUG: Task 3340 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3340 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3340 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3340 has memoization hash d076d50f2182bd336454065a47ad4308 parsl.dataflow.memoization INFO: Task 3340 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3340 try 0 launched on executor multi with executor id 3341 parsl.dataflow.dflow INFO: Standard output for task 3340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/60220f04-daa0-414f-b3b6-0582b28dc9a9_isrForCrosstalkSources_996984_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/60220f04-daa0-414f-b3b6-0582b28dc9a9_isrForCrosstalkSources_996984_30.stderr parsl.dataflow.dflow DEBUG: Task 3341 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3341 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3341 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3341 has memoization hash c751be7b58635c77be296302d70e5b9e parsl.dataflow.memoization INFO: Task 3341 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3341 try 0 launched on executor multi with executor id 3342 parsl.dataflow.dflow INFO: Standard output for task 3341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/9fedbcc1-87e8-4f4c-baea-5a18209d59b4_isrForCrosstalkSources_997031_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/9fedbcc1-87e8-4f4c-baea-5a18209d59b4_isrForCrosstalkSources_997031_39.stderr parsl.dataflow.dflow DEBUG: Task 3342 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3342 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3342 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3342 has memoization hash c2164d58952afc75208052cc61340db0 parsl.dataflow.memoization INFO: Task 3342 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3342 try 0 launched on executor multi with executor id 3343 parsl.dataflow.dflow INFO: Standard output for task 3342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e665c0ab-4fb2-4bd9-820a-c815c621f54b_isrForCrosstalkSources_996964_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e665c0ab-4fb2-4bd9-820a-c815c621f54b_isrForCrosstalkSources_996964_37.stderr parsl.dataflow.dflow DEBUG: Task 3343 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3343 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3343 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3343 has memoization hash ead4e5dc29c815a60cbf34278829dccb parsl.dataflow.memoization INFO: Task 3343 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3343 try 0 launched on executor multi with executor id 3344 parsl.dataflow.dflow INFO: Standard output for task 3343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e383a5ef-3842-4d82-ac03-66c5bb15e805_isrForCrosstalkSources_997018_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e383a5ef-3842-4d82-ac03-66c5bb15e805_isrForCrosstalkSources_997018_8.stderr parsl.dataflow.dflow DEBUG: Task 3344 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3344 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3344 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3344 has memoization hash 48a57850dea74c2e507475f997f1de43 parsl.dataflow.memoization INFO: Task 3344 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3344 try 0 launched on executor multi with executor id 3345 parsl.dataflow.dflow INFO: Standard output for task 3344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ec180685-56d5-4af3-8a98-fcd30059fc96_isrForCrosstalkSources_996976_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ec180685-56d5-4af3-8a98-fcd30059fc96_isrForCrosstalkSources_996976_48.stderr parsl.dataflow.dflow DEBUG: Task 3345 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3345 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3345 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3345 has memoization hash c03dbcdf130bb3f734946e2495923f23 parsl.dataflow.memoization INFO: Task 3345 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3345 try 0 launched on executor multi with executor id 3346 parsl.dataflow.dflow INFO: Standard output for task 3345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b43841d9-063e-4347-9dfb-67ed4ad33a05_isrForCrosstalkSources_997008_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b43841d9-063e-4347-9dfb-67ed4ad33a05_isrForCrosstalkSources_997008_57.stderr parsl.dataflow.dflow DEBUG: Task 3346 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3346 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3346 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3346 has memoization hash 1d76b233befff78d0cb57fd3c84f2a4f parsl.dataflow.memoization INFO: Task 3346 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3346 try 0 launched on executor multi with executor id 3347 parsl.dataflow.dflow INFO: Standard output for task 3346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f3a0e0cc-a8a8-401e-acf3-5955143e5357_isrForCrosstalkSources_996988_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f3a0e0cc-a8a8-401e-acf3-5955143e5357_isrForCrosstalkSources_996988_23.stderr parsl.dataflow.dflow DEBUG: Task 3347 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3347 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3347 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3347 has memoization hash 1de16c2130fb4cf082ec942f4a9f84ac parsl.dataflow.memoization INFO: Task 3347 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3347 try 0 launched on executor multi with executor id 3348 parsl.dataflow.dflow INFO: Standard output for task 3347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/fa8738a7-ef97-4c7f-8832-5ea1d54bf055_isrForCrosstalkSources_996963_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/fa8738a7-ef97-4c7f-8832-5ea1d54bf055_isrForCrosstalkSources_996963_60.stderr parsl.dataflow.dflow DEBUG: Task 3348 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3348 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3348 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3348 has memoization hash a334b7b9ed495ad849a2ac366d22eb17 parsl.dataflow.memoization INFO: Task 3348 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3348 try 0 launched on executor multi with executor id 3349 parsl.dataflow.dflow INFO: Standard output for task 3348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/11fb70b6-883e-4790-b4d2-1327ef67980f_isrForCrosstalkSources_997000_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/11fb70b6-883e-4790-b4d2-1327ef67980f_isrForCrosstalkSources_997000_42.stderr parsl.dataflow.dflow DEBUG: Task 3349 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3349 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3349 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3349 has memoization hash e4f6c444b577ef79cb2a96d69bd51f2d parsl.dataflow.memoization INFO: Task 3349 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3349 try 0 launched on executor multi with executor id 3350 parsl.dataflow.dflow INFO: Standard output for task 3349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/09504192-8cf6-4289-b384-abe37101c052_isrForCrosstalkSources_996998_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/09504192-8cf6-4289-b384-abe37101c052_isrForCrosstalkSources_996998_18.stderr parsl.dataflow.dflow DEBUG: Task 3350 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3350 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3350 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3350 has memoization hash e41c20e7e8e191f9d5b4b52f2ee39097 parsl.dataflow.memoization INFO: Task 3350 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3350 try 0 launched on executor multi with executor id 3351 parsl.dataflow.dflow INFO: Standard output for task 3350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/53a1f3c4-7dc1-4a69-91bc-3af16d6a0145_isrForCrosstalkSources_996961_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/53a1f3c4-7dc1-4a69-91bc-3af16d6a0145_isrForCrosstalkSources_996961_50.stderr parsl.dataflow.dflow DEBUG: Task 3351 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3351 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3351 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3351 has memoization hash 8bab6d99bf4e2135332c2cc9c2dbdd66 parsl.dataflow.memoization INFO: Task 3351 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3351 try 0 launched on executor multi with executor id 3352 parsl.dataflow.dflow INFO: Standard output for task 3351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/e0f659b9-733d-4367-a798-5da26facb931_isrForCrosstalkSources_996979_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/e0f659b9-733d-4367-a798-5da26facb931_isrForCrosstalkSources_996979_36.stderr parsl.dataflow.dflow DEBUG: Task 3352 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3352 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3352 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3352 has memoization hash f1067d5feba108c90de6b37a63386576 parsl.dataflow.memoization INFO: Task 3352 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59ba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3352 try 0 launched on executor multi with executor id 3353 parsl.dataflow.dflow INFO: Standard output for task 3352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/04ce60a4-2d7a-4f62-8abf-d82cd32d59a1_isrForCrosstalkSources_996961_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/04ce60a4-2d7a-4f62-8abf-d82cd32d59a1_isrForCrosstalkSources_996961_49.stderr parsl.dataflow.dflow DEBUG: Task 3353 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3353 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3353 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3353 has memoization hash de4841c6c6f13cda9a839257d994e230 parsl.dataflow.memoization INFO: Task 3353 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59bb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3353 try 0 launched on executor multi with executor id 3354 parsl.dataflow.dflow INFO: Standard output for task 3353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/acbc46c5-1707-4bc1-abb3-6464df5df683_isrForCrosstalkSources_996954_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/acbc46c5-1707-4bc1-abb3-6464df5df683_isrForCrosstalkSources_996954_40.stderr parsl.dataflow.dflow DEBUG: Task 3354 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3354 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3354 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3354 has memoization hash 30de106bdc8bc5339640bf1f76959631 parsl.dataflow.memoization INFO: Task 3354 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59bba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3354 try 0 launched on executor multi with executor id 3355 parsl.dataflow.dflow INFO: Standard output for task 3354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a5482471-416a-40fa-a27c-f2ead467ccbe_isrForCrosstalkSources_996957_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a5482471-416a-40fa-a27c-f2ead467ccbe_isrForCrosstalkSources_996957_17.stderr parsl.dataflow.dflow DEBUG: Task 3355 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3355 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3355 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3355 has memoization hash a62646faeae1e4385ae08fbdcf3a1865 parsl.dataflow.memoization INFO: Task 3355 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59b920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3355 try 0 launched on executor multi with executor id 3356 parsl.dataflow.dflow INFO: Standard output for task 3355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/18b90d5a-c335-4ef6-a03e-2d8a4accdd04_isrForCrosstalkSources_996957_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/18b90d5a-c335-4ef6-a03e-2d8a4accdd04_isrForCrosstalkSources_996957_1.stderr parsl.dataflow.dflow DEBUG: Task 3356 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3356 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3356 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3356 has memoization hash 83b53ff5f1210ebc89d6a1d413e7b586 parsl.dataflow.memoization INFO: Task 3356 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59bc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3356 try 0 launched on executor multi with executor id 3357 parsl.dataflow.dflow INFO: Standard output for task 3356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/7eaaa837-a482-4a75-a09f-afa2b54d00f6_isrForCrosstalkSources_997009_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/7eaaa837-a482-4a75-a09f-afa2b54d00f6_isrForCrosstalkSources_997009_3.stderr parsl.dataflow.dflow DEBUG: Task 3357 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3357 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3357 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3357 has memoization hash d1f1ee61f254494134709f5ecdd6e102 parsl.dataflow.memoization INFO: Task 3357 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59bd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3357 try 0 launched on executor multi with executor id 3358 parsl.dataflow.dflow INFO: Standard output for task 3357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/4a332746-74e0-4446-890c-21c4a75b327e_isrForCrosstalkSources_996953_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/4a332746-74e0-4446-890c-21c4a75b327e_isrForCrosstalkSources_996953_22.stderr parsl.dataflow.dflow DEBUG: Task 3358 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3358 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3358 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3358 has memoization hash 0598d5533319f80f1f4822f0cb7a3dbf parsl.dataflow.memoization INFO: Task 3358 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c59bce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3358 try 0 launched on executor multi with executor id 3359 parsl.dataflow.dflow INFO: Standard output for task 3358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4332c31e-211f-4b1b-8506-7090894ffbe8_isrForCrosstalkSources_996963_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4332c31e-211f-4b1b-8506-7090894ffbe8_isrForCrosstalkSources_996963_5.stderr parsl.dataflow.dflow DEBUG: Task 3359 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3359 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3359 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3359 has memoization hash 675e9a68ad0a3e587861c4494a6f135c parsl.dataflow.memoization INFO: Task 3359 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c599d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3359 try 0 launched on executor multi with executor id 3360 parsl.dataflow.dflow INFO: Standard output for task 3359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e877cdea-354a-4101-b11f-254839a74c95_isrForCrosstalkSources_997032_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e877cdea-354a-4101-b11f-254839a74c95_isrForCrosstalkSources_997032_54.stderr parsl.dataflow.dflow DEBUG: Task 3360 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3360 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3360 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3360 has memoization hash 9946edddfbfb9fee1925f74b4a0e5142 parsl.dataflow.memoization INFO: Task 3360 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3360 try 0 launched on executor multi with executor id 3361 parsl.dataflow.dflow INFO: Standard output for task 3360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e3209367-4906-45fa-9bf9-047c0802a4d2_isrForCrosstalkSources_997018_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e3209367-4906-45fa-9bf9-047c0802a4d2_isrForCrosstalkSources_997018_6.stderr parsl.dataflow.dflow DEBUG: Task 3361 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3361 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3361 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3361 has memoization hash 084a2face063f59f22b1c57b8b692813 parsl.dataflow.memoization INFO: Task 3361 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3361 try 0 launched on executor multi with executor id 3362 parsl.dataflow.dflow INFO: Standard output for task 3361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/82b8a0c2-8e51-4674-88e0-7892005e211f_isrForCrosstalkSources_996980_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/82b8a0c2-8e51-4674-88e0-7892005e211f_isrForCrosstalkSources_996980_36.stderr parsl.dataflow.dflow DEBUG: Task 3362 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3362 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3362 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3362 has memoization hash 274620d3b4aca23275cdab1fb96aa3ce parsl.dataflow.memoization INFO: Task 3362 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3362 try 0 launched on executor multi with executor id 3363 parsl.dataflow.dflow INFO: Standard output for task 3362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/78940c5a-e5df-4314-9b27-a8650fb63c23_isrForCrosstalkSources_996956_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/78940c5a-e5df-4314-9b27-a8650fb63c23_isrForCrosstalkSources_996956_4.stderr parsl.dataflow.dflow DEBUG: Task 3363 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3363 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3363 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3363 has memoization hash d265fb55a7cdde5edf1eac08df7c746a parsl.dataflow.memoization INFO: Task 3363 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3363 try 0 launched on executor multi with executor id 3364 parsl.dataflow.dflow INFO: Standard output for task 3363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/628e2c84-3ef4-4a64-8f1d-5d966b1381a9_isrForCrosstalkSources_996954_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/628e2c84-3ef4-4a64-8f1d-5d966b1381a9_isrForCrosstalkSources_996954_56.stderr parsl.dataflow.dflow DEBUG: Task 3364 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3364 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3364 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3364 has memoization hash 1b97eedbfe7c7d3ec56d9247b734c9f1 parsl.dataflow.memoization INFO: Task 3364 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3364 try 0 launched on executor multi with executor id 3365 parsl.dataflow.dflow INFO: Standard output for task 3364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/8530d2b9-49be-4a20-b1f8-7a37bde303d2_isrForCrosstalkSources_997003_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/8530d2b9-49be-4a20-b1f8-7a37bde303d2_isrForCrosstalkSources_997003_11.stderr parsl.dataflow.dflow DEBUG: Task 3365 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3365 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3365 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3365 has memoization hash 5f7ae1f46edd671df7e7f52b694d7a40 parsl.dataflow.memoization INFO: Task 3365 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3365 try 0 launched on executor multi with executor id 3366 parsl.dataflow.dflow INFO: Standard output for task 3365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/1134b7ab-9193-428a-b7c9-89f6ede4409f_isrForCrosstalkSources_996966_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/1134b7ab-9193-428a-b7c9-89f6ede4409f_isrForCrosstalkSources_996966_39.stderr parsl.dataflow.dflow DEBUG: Task 3366 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3366 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3366 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3366 has memoization hash 399375e2118e7c28b52e7e766cb50569 parsl.dataflow.memoization INFO: Task 3366 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3366 try 0 launched on executor multi with executor id 3367 parsl.dataflow.dflow INFO: Standard output for task 3366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7c531fb4-3fa4-4dec-8556-4b744814e57b_isrForCrosstalkSources_996953_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7c531fb4-3fa4-4dec-8556-4b744814e57b_isrForCrosstalkSources_996953_26.stderr parsl.dataflow.dflow DEBUG: Task 3367 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3367 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3367 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3367 has memoization hash 3fdf3172e27f37a2324e04cfb89d1c01 parsl.dataflow.memoization INFO: Task 3367 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3367 try 0 launched on executor multi with executor id 3368 parsl.dataflow.dflow INFO: Standard output for task 3367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6d5caa47-b85b-4b7f-96a5-806c9b6078e9_isrForCrosstalkSources_996963_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6d5caa47-b85b-4b7f-96a5-806c9b6078e9_isrForCrosstalkSources_996963_59.stderr parsl.dataflow.dflow DEBUG: Task 3368 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3368 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3368 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3368 has memoization hash a7f92b487fc873930a77454e967508a5 parsl.dataflow.memoization INFO: Task 3368 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3368 try 0 launched on executor multi with executor id 3369 parsl.dataflow.dflow INFO: Standard output for task 3368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/89d4232d-8c16-4bb1-9347-136ebaa87759_isrForCrosstalkSources_996974_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/89d4232d-8c16-4bb1-9347-136ebaa87759_isrForCrosstalkSources_996974_54.stderr parsl.dataflow.dflow DEBUG: Task 3369 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3369 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3369 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3369 has memoization hash e32fe76600c979c27a90cc4f41f0f494 parsl.dataflow.memoization INFO: Task 3369 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3369 try 0 launched on executor multi with executor id 3370 parsl.dataflow.dflow INFO: Standard output for task 3369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/27b09f8a-0f14-45ca-803e-f151d7b926b2_isrForCrosstalkSources_996996_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/27b09f8a-0f14-45ca-803e-f151d7b926b2_isrForCrosstalkSources_996996_35.stderr parsl.dataflow.dflow DEBUG: Task 3370 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3370 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3370 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3370 has memoization hash 94e41ad5915f4e45594174af1b1a20ad parsl.dataflow.memoization INFO: Task 3370 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3370 try 0 launched on executor multi with executor id 3371 parsl.dataflow.dflow INFO: Standard output for task 3370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8aa5c0f6-bc18-42a4-b6da-4d3fb5743fcb_isrForCrosstalkSources_996976_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8aa5c0f6-bc18-42a4-b6da-4d3fb5743fcb_isrForCrosstalkSources_996976_8.stderr parsl.dataflow.dflow DEBUG: Task 3371 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3371 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3371 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3371 has memoization hash b8101d38150d3ca17044231661cd9b2f parsl.dataflow.memoization INFO: Task 3371 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3371 try 0 launched on executor multi with executor id 3372 parsl.dataflow.dflow INFO: Standard output for task 3371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/e71e36df-908e-4b65-be55-1b379992f9ba_isrForCrosstalkSources_996959_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/e71e36df-908e-4b65-be55-1b379992f9ba_isrForCrosstalkSources_996959_35.stderr parsl.dataflow.dflow DEBUG: Task 3372 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3372 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3372 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3372 has memoization hash 7f8247e367210e2303c5a8bbd186703f parsl.dataflow.memoization INFO: Task 3372 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3372 try 0 launched on executor multi with executor id 3373 parsl.dataflow.dflow INFO: Standard output for task 3372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/f153a0eb-13ef-47da-8ba4-23ecb0f49620_isrForCrosstalkSources_996963_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/f153a0eb-13ef-47da-8ba4-23ecb0f49620_isrForCrosstalkSources_996963_27.stderr parsl.dataflow.dflow DEBUG: Task 3373 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3373 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3373 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3373 has memoization hash 273c9bab42854162aa3111dd5a0a395e parsl.dataflow.memoization INFO: Task 3373 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3373 try 0 launched on executor multi with executor id 3374 parsl.dataflow.dflow INFO: Standard output for task 3373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ecbaf3de-a168-4e95-8044-ac26a2a9604e_isrForCrosstalkSources_996991_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ecbaf3de-a168-4e95-8044-ac26a2a9604e_isrForCrosstalkSources_996991_23.stderr parsl.dataflow.dflow DEBUG: Task 3374 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3374 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3374 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3374 has memoization hash 58cdc5405e944f6a5d9743eccc4ad318 parsl.dataflow.memoization INFO: Task 3374 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3374 try 0 launched on executor multi with executor id 3375 parsl.dataflow.dflow INFO: Standard output for task 3374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2d5eb532-bf59-4db7-93c5-98a1b4efe775_isrForCrosstalkSources_996961_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2d5eb532-bf59-4db7-93c5-98a1b4efe775_isrForCrosstalkSources_996961_56.stderr parsl.dataflow.dflow DEBUG: Task 3375 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3375 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3375 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3375 has memoization hash 02238c9ab4af668166506a0e3d071296 parsl.dataflow.memoization INFO: Task 3375 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3375 try 0 launched on executor multi with executor id 3376 parsl.dataflow.dflow INFO: Standard output for task 3375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d9080eb4-5eb1-4d4e-a3ae-50c7ad2b6a95_isrForCrosstalkSources_996971_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d9080eb4-5eb1-4d4e-a3ae-50c7ad2b6a95_isrForCrosstalkSources_996971_60.stderr parsl.dataflow.dflow DEBUG: Task 3376 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3376 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3376 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3376 has memoization hash 542ae74c985359448c38f32491420067 parsl.dataflow.memoization INFO: Task 3376 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3376 try 0 launched on executor multi with executor id 3377 parsl.dataflow.dflow INFO: Standard output for task 3376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/6fffe458-1d95-4fc7-a443-5b8e555856ed_isrForCrosstalkSources_996987_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/6fffe458-1d95-4fc7-a443-5b8e555856ed_isrForCrosstalkSources_996987_35.stderr parsl.dataflow.dflow DEBUG: Task 3377 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3377 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3377 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3377 has memoization hash b9962b4f74fc54f88a2cb584f0da8a04 parsl.dataflow.memoization INFO: Task 3377 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3377 try 0 launched on executor multi with executor id 3378 parsl.dataflow.dflow INFO: Standard output for task 3377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d7c705f8-d3b0-40dc-9ee1-857dd65c4e33_isrForCrosstalkSources_996989_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d7c705f8-d3b0-40dc-9ee1-857dd65c4e33_isrForCrosstalkSources_996989_10.stderr parsl.dataflow.dflow DEBUG: Task 3378 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3378 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3378 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3378 has memoization hash 46875b0687a5cda71eda8b70e38d2fbb parsl.dataflow.memoization INFO: Task 3378 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3378 try 0 launched on executor multi with executor id 3379 parsl.dataflow.dflow INFO: Standard output for task 3378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/994eb883-6d32-46cf-9fd1-fb8639c39ec0_isrForCrosstalkSources_997016_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/994eb883-6d32-46cf-9fd1-fb8639c39ec0_isrForCrosstalkSources_997016_29.stderr parsl.dataflow.dflow DEBUG: Task 3379 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3379 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3379 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3379 has memoization hash 25ff1f7bd1987dc0b377829081bb88d8 parsl.dataflow.memoization INFO: Task 3379 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3379 try 0 launched on executor multi with executor id 3380 parsl.dataflow.dflow INFO: Standard output for task 3379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/ea4aa46a-b108-4336-a6e0-9d0680b93b76_isrForCrosstalkSources_996971_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/ea4aa46a-b108-4336-a6e0-9d0680b93b76_isrForCrosstalkSources_996971_13.stderr parsl.dataflow.dflow DEBUG: Task 3380 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3380 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3380 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3380 has memoization hash 3f0fe3f1fc5b2e13ad78307fc903c8f9 parsl.dataflow.memoization INFO: Task 3380 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3380 try 0 launched on executor multi with executor id 3381 parsl.dataflow.dflow INFO: Standard output for task 3380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/dfe09c84-79e6-4160-b47d-bb140c90ca2b_isrForCrosstalkSources_997004_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/dfe09c84-79e6-4160-b47d-bb140c90ca2b_isrForCrosstalkSources_997004_49.stderr parsl.dataflow.dflow DEBUG: Task 3381 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3381 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3381 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3381 has memoization hash 864f7187d0cb9a13cd6f382c9069b933 parsl.dataflow.memoization INFO: Task 3381 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3381 try 0 launched on executor multi with executor id 3382 parsl.dataflow.dflow INFO: Standard output for task 3381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5aefc0bf-7f28-467f-9fbc-f68140cc2159_isrForCrosstalkSources_997003_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5aefc0bf-7f28-467f-9fbc-f68140cc2159_isrForCrosstalkSources_997003_23.stderr parsl.dataflow.dflow DEBUG: Task 3382 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3382 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3382 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3382 has memoization hash 2c42ce1a4d1ca8f6e5e2c2fb92f7ff98 parsl.dataflow.memoization INFO: Task 3382 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3382 try 0 launched on executor multi with executor id 3383 parsl.dataflow.dflow INFO: Standard output for task 3382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/bdfc3769-4e30-4e1d-a736-ccbf9515e45b_isrForCrosstalkSources_997010_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/bdfc3769-4e30-4e1d-a736-ccbf9515e45b_isrForCrosstalkSources_997010_13.stderr parsl.dataflow.dflow DEBUG: Task 3383 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3383 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3383 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3383 has memoization hash 342a373d6b4e3573c6f636f2861be76e parsl.dataflow.memoization INFO: Task 3383 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3383 try 0 launched on executor multi with executor id 3384 parsl.dataflow.dflow INFO: Standard output for task 3383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e02921c5-e531-46b5-a08c-e8a605289f98_isrForCrosstalkSources_997010_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e02921c5-e531-46b5-a08c-e8a605289f98_isrForCrosstalkSources_997010_59.stderr parsl.dataflow.dflow DEBUG: Task 3384 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3384 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3384 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3384 has memoization hash 5e1411c0a9a612bcc67ad104cf5509fb parsl.dataflow.memoization INFO: Task 3384 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3384 try 0 launched on executor multi with executor id 3385 parsl.dataflow.dflow INFO: Standard output for task 3384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a26340fa-4881-44e5-980c-d60129f32e29_isrForCrosstalkSources_996976_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a26340fa-4881-44e5-980c-d60129f32e29_isrForCrosstalkSources_996976_34.stderr parsl.dataflow.dflow DEBUG: Task 3385 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3385 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3385 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3385 has memoization hash 45d35d39ca9fc853e45a95bead8bbb36 parsl.dataflow.memoization INFO: Task 3385 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3385 try 0 launched on executor multi with executor id 3386 parsl.dataflow.dflow INFO: Standard output for task 3385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/c50df46b-4af6-4106-8caf-ad9d3fa9cd1b_isrForCrosstalkSources_997017_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/c50df46b-4af6-4106-8caf-ad9d3fa9cd1b_isrForCrosstalkSources_997017_5.stderr parsl.dataflow.dflow DEBUG: Task 3386 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3386 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3386 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3386 has memoization hash 53c7540c3ee6d2e4b595c76f14c96126 parsl.dataflow.memoization INFO: Task 3386 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3386 try 0 launched on executor multi with executor id 3387 parsl.dataflow.dflow INFO: Standard output for task 3386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2aba0980-47f4-4d06-aa58-38716613ed43_isrForCrosstalkSources_996974_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2aba0980-47f4-4d06-aa58-38716613ed43_isrForCrosstalkSources_996974_6.stderr parsl.dataflow.dflow DEBUG: Task 3387 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3387 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3387 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3387 has memoization hash bdbe061218724f134745eab250f2e8b6 parsl.dataflow.memoization INFO: Task 3387 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3387 try 0 launched on executor multi with executor id 3388 parsl.dataflow.dflow INFO: Standard output for task 3387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/2242a6ab-eb15-429c-8c0b-dec8c474409e_isrForCrosstalkSources_997024_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/2242a6ab-eb15-429c-8c0b-dec8c474409e_isrForCrosstalkSources_997024_29.stderr parsl.dataflow.dflow DEBUG: Task 3388 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3388 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3388 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3388 has memoization hash fbede176979991054d0399ab67322ce8 parsl.dataflow.memoization INFO: Task 3388 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3388 try 0 launched on executor multi with executor id 3389 parsl.dataflow.dflow INFO: Standard output for task 3388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/9e223c22-b2c6-4fee-bb9e-286b672cd5f0_isrForCrosstalkSources_996962_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/9e223c22-b2c6-4fee-bb9e-286b672cd5f0_isrForCrosstalkSources_996962_21.stderr parsl.dataflow.dflow DEBUG: Task 3389 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3389 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3389 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3389 has memoization hash a9303cf07e899d2549da98bbdbba037c parsl.dataflow.memoization INFO: Task 3389 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3389 try 0 launched on executor multi with executor id 3390 parsl.dataflow.dflow INFO: Standard output for task 3389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/c442c9bf-b8f4-417f-8584-5d812d9cc969_isrForCrosstalkSources_996981_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/c442c9bf-b8f4-417f-8584-5d812d9cc969_isrForCrosstalkSources_996981_45.stderr parsl.dataflow.dflow DEBUG: Task 3390 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3390 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3390 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3390 has memoization hash 725515e7eb684b127298814070698f92 parsl.dataflow.memoization INFO: Task 3390 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3390 try 0 launched on executor multi with executor id 3391 parsl.dataflow.dflow INFO: Standard output for task 3390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f8856572-3789-44de-bbb9-e2a8dc122e66_isrForCrosstalkSources_997024_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f8856572-3789-44de-bbb9-e2a8dc122e66_isrForCrosstalkSources_997024_3.stderr parsl.dataflow.dflow DEBUG: Task 3391 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3391 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3391 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3391 has memoization hash 5fd6a381179c818e8f088a03cd9ea929 parsl.dataflow.memoization INFO: Task 3391 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3391 try 0 launched on executor multi with executor id 3392 parsl.dataflow.dflow INFO: Standard output for task 3391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0f5283ee-6cb1-4055-9448-a573edeb52eb_isrForCrosstalkSources_996991_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0f5283ee-6cb1-4055-9448-a573edeb52eb_isrForCrosstalkSources_996991_2.stderr parsl.dataflow.dflow DEBUG: Task 3392 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3392 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3392 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3392 has memoization hash 99750f95562203f89f782d0d62ce7770 parsl.dataflow.memoization INFO: Task 3392 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3392 try 0 launched on executor multi with executor id 3393 parsl.dataflow.dflow INFO: Standard output for task 3392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/f49c050b-3a7e-42e5-a0cd-54556367be34_isrForCrosstalkSources_997010_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/f49c050b-3a7e-42e5-a0cd-54556367be34_isrForCrosstalkSources_997010_23.stderr parsl.dataflow.dflow DEBUG: Task 3393 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3393 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3393 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3393 has memoization hash 717c7d6692e03ae1a4471743ece28bcc parsl.dataflow.memoization INFO: Task 3393 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3393 try 0 launched on executor multi with executor id 3394 parsl.dataflow.dflow INFO: Standard output for task 3393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/30638c61-f947-4f02-b599-7e1998cbdd83_isrForCrosstalkSources_996979_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/30638c61-f947-4f02-b599-7e1998cbdd83_isrForCrosstalkSources_996979_8.stderr parsl.dataflow.dflow DEBUG: Task 3394 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3394 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3394 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3394 has memoization hash 233a2196436e9fb2d2f4db56e8f2f36c parsl.dataflow.memoization INFO: Task 3394 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3394 try 0 launched on executor multi with executor id 3395 parsl.dataflow.dflow INFO: Standard output for task 3394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/608055f6-bd7f-4990-ba87-60a29de2890e_isrForCrosstalkSources_996952_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/608055f6-bd7f-4990-ba87-60a29de2890e_isrForCrosstalkSources_996952_5.stderr parsl.dataflow.dflow DEBUG: Task 3395 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3395 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3395 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3395 has memoization hash ee468ab2a4a9f4ebb96d670c18255dc3 parsl.dataflow.memoization INFO: Task 3395 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3395 try 0 launched on executor multi with executor id 3396 parsl.dataflow.dflow INFO: Standard output for task 3395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f975e241-a990-4a16-86e1-60b26e6c84c2_isrForCrosstalkSources_996986_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f975e241-a990-4a16-86e1-60b26e6c84c2_isrForCrosstalkSources_996986_27.stderr parsl.dataflow.dflow DEBUG: Task 3396 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3396 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3396 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3396 has memoization hash eb1fd83edf2f881e336db6bbece21e69 parsl.dataflow.memoization INFO: Task 3396 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3396 try 0 launched on executor multi with executor id 3397 parsl.dataflow.dflow INFO: Standard output for task 3396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/bb07408a-e7e5-4acd-b8fd-2918efaa7f1a_isrForCrosstalkSources_997021_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/bb07408a-e7e5-4acd-b8fd-2918efaa7f1a_isrForCrosstalkSources_997021_38.stderr parsl.dataflow.dflow DEBUG: Task 3397 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3397 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3397 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3397 has memoization hash e2171c33be8c03a22a77cc93eda9c82f parsl.dataflow.memoization INFO: Task 3397 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3397 try 0 launched on executor multi with executor id 3398 parsl.dataflow.dflow INFO: Standard output for task 3397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/6fc3be7f-4296-4a60-aa43-24aaba93029f_isrForCrosstalkSources_996998_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/6fc3be7f-4296-4a60-aa43-24aaba93029f_isrForCrosstalkSources_996998_10.stderr parsl.dataflow.dflow DEBUG: Task 3398 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3398 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3398 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3398 has memoization hash 3c52c2ea20128849a8baf84b26953d5c parsl.dataflow.memoization INFO: Task 3398 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3398 try 0 launched on executor multi with executor id 3399 parsl.dataflow.dflow INFO: Standard output for task 3398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/d0c14004-4991-481d-b8b7-98f77c6814a0_isrForCrosstalkSources_996968_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/d0c14004-4991-481d-b8b7-98f77c6814a0_isrForCrosstalkSources_996968_43.stderr parsl.dataflow.dflow DEBUG: Task 3399 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3399 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3399 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3399 has memoization hash 015b77a7949b858709521a061384f279 parsl.dataflow.memoization INFO: Task 3399 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3399 try 0 launched on executor multi with executor id 3400 parsl.dataflow.dflow INFO: Standard output for task 3399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/53402ed0-e9a5-4701-8865-e90e3c6f9d9f_isrForCrosstalkSources_996989_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/53402ed0-e9a5-4701-8865-e90e3c6f9d9f_isrForCrosstalkSources_996989_35.stderr parsl.dataflow.dflow DEBUG: Task 3400 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3400 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3400 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3400 has memoization hash 7c58b831a1fd9c00e3c5efbaf344fba5 parsl.dataflow.memoization INFO: Task 3400 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3400 try 0 launched on executor multi with executor id 3401 parsl.dataflow.dflow INFO: Standard output for task 3400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0a4c5f7b-dce1-47e7-b755-51be50781688_isrForCrosstalkSources_996980_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0a4c5f7b-dce1-47e7-b755-51be50781688_isrForCrosstalkSources_996980_49.stderr parsl.dataflow.dflow DEBUG: Task 3401 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3401 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3401 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3401 has memoization hash 17c6d2b7baf241fa2a1563ca7fb61898 parsl.dataflow.memoization INFO: Task 3401 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3401 try 0 launched on executor multi with executor id 3402 parsl.dataflow.dflow INFO: Standard output for task 3401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ea901c4b-bfe3-4357-97d4-3305c16ebadf_isrForCrosstalkSources_996990_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ea901c4b-bfe3-4357-97d4-3305c16ebadf_isrForCrosstalkSources_996990_28.stderr parsl.dataflow.dflow DEBUG: Task 3402 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3402 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3402 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3402 has memoization hash c33307f9ccf020662bb91ca0eaf4df46 parsl.dataflow.memoization INFO: Task 3402 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3402 try 0 launched on executor multi with executor id 3403 parsl.dataflow.dflow INFO: Standard output for task 3402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/5cd2723f-4d0a-4bdd-b90e-8e029c1bf6d4_isrForCrosstalkSources_997019_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/5cd2723f-4d0a-4bdd-b90e-8e029c1bf6d4_isrForCrosstalkSources_997019_19.stderr parsl.dataflow.dflow DEBUG: Task 3403 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3403 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3403 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3403 has memoization hash 5b49c07c89e0211813309a7147ea04a6 parsl.dataflow.memoization INFO: Task 3403 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3403 try 0 launched on executor multi with executor id 3404 parsl.dataflow.dflow INFO: Standard output for task 3403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/d3007ea3-e31e-480a-951f-f30980c39554_isrForCrosstalkSources_996999_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/d3007ea3-e31e-480a-951f-f30980c39554_isrForCrosstalkSources_996999_44.stderr parsl.dataflow.dflow DEBUG: Task 3404 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3404 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3404 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3404 has memoization hash 1110ce86c38861054e947ecd71141b50 parsl.dataflow.memoization INFO: Task 3404 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3404 try 0 launched on executor multi with executor id 3405 parsl.dataflow.dflow INFO: Standard output for task 3404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/88693045-6570-4c5f-8371-a00c0025f2c0_isrForCrosstalkSources_996973_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/88693045-6570-4c5f-8371-a00c0025f2c0_isrForCrosstalkSources_996973_53.stderr parsl.dataflow.dflow DEBUG: Task 3405 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3405 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3405 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3405 has memoization hash 53f5a580779ea783b40e38d0bc4a5dbe parsl.dataflow.memoization INFO: Task 3405 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3405 try 0 launched on executor multi with executor id 3406 parsl.dataflow.dflow INFO: Standard output for task 3405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/43585f3f-6469-48bb-af38-7d62fb510ff7_isrForCrosstalkSources_996987_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/43585f3f-6469-48bb-af38-7d62fb510ff7_isrForCrosstalkSources_996987_43.stderr parsl.dataflow.dflow DEBUG: Task 3406 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3406 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3406 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3406 has memoization hash 59d213eac960ebf72c57a7d1873c1ba5 parsl.dataflow.memoization INFO: Task 3406 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3406 try 0 launched on executor multi with executor id 3407 parsl.dataflow.dflow INFO: Standard output for task 3406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7e93d477-87e6-4556-88f8-e331ae5dab14_isrForCrosstalkSources_997008_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7e93d477-87e6-4556-88f8-e331ae5dab14_isrForCrosstalkSources_997008_34.stderr parsl.dataflow.dflow DEBUG: Task 3407 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3407 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3407 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3407 has memoization hash 6e5fe00faa5d8517f91f4746859605f2 parsl.dataflow.memoization INFO: Task 3407 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3407 try 0 launched on executor multi with executor id 3408 parsl.dataflow.dflow INFO: Standard output for task 3407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/02749f72-e999-4198-b245-58753521c92c_isrForCrosstalkSources_996968_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/02749f72-e999-4198-b245-58753521c92c_isrForCrosstalkSources_996968_36.stderr parsl.dataflow.dflow DEBUG: Task 3408 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3408 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3408 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3408 has memoization hash 455191ba451b9bc2717cb12acd15cdb5 parsl.dataflow.memoization INFO: Task 3408 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3408 try 0 launched on executor multi with executor id 3409 parsl.dataflow.dflow INFO: Standard output for task 3408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a976bb45-4871-4a3b-acc8-325f0ca683f1_isrForCrosstalkSources_996995_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a976bb45-4871-4a3b-acc8-325f0ca683f1_isrForCrosstalkSources_996995_33.stderr parsl.dataflow.dflow DEBUG: Task 3409 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3409 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3409 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3409 has memoization hash 6544706719a190fa0cab04dc9d6fd6ad parsl.dataflow.memoization INFO: Task 3409 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3409 try 0 launched on executor multi with executor id 3410 parsl.dataflow.dflow INFO: Standard output for task 3409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/71afd1ab-7cdd-4402-8a17-a83a72e2b5c8_isrForCrosstalkSources_996982_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/71afd1ab-7cdd-4402-8a17-a83a72e2b5c8_isrForCrosstalkSources_996982_32.stderr parsl.dataflow.dflow DEBUG: Task 3410 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3410 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3410 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3410 has memoization hash 57f91c529bfe72f4bb0367af80fff992 parsl.dataflow.memoization INFO: Task 3410 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3410 try 0 launched on executor multi with executor id 3411 parsl.dataflow.dflow INFO: Standard output for task 3410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/5c7011ea-eb04-4b6f-9fa0-3b4ef2d1fa67_isrForCrosstalkSources_996961_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/5c7011ea-eb04-4b6f-9fa0-3b4ef2d1fa67_isrForCrosstalkSources_996961_14.stderr parsl.dataflow.dflow DEBUG: Task 3411 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3411 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3411 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3411 has memoization hash 161d519ac909075758d194946708317e parsl.dataflow.memoization INFO: Task 3411 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3411 try 0 launched on executor multi with executor id 3412 parsl.dataflow.dflow INFO: Standard output for task 3411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b7bdce11-d3a6-47bc-a18a-7be65d4e17ee_isrForCrosstalkSources_997022_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b7bdce11-d3a6-47bc-a18a-7be65d4e17ee_isrForCrosstalkSources_997022_31.stderr parsl.dataflow.dflow DEBUG: Task 3412 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3412 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3412 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3412 has memoization hash 697809287c1382e9c520f83928ab6961 parsl.dataflow.memoization INFO: Task 3412 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3412 try 0 launched on executor multi with executor id 3413 parsl.dataflow.dflow INFO: Standard output for task 3412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1a7c52e0-ca0e-4ec1-b94c-d188e9d45ff5_isrForCrosstalkSources_996967_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1a7c52e0-ca0e-4ec1-b94c-d188e9d45ff5_isrForCrosstalkSources_996967_57.stderr parsl.dataflow.dflow DEBUG: Task 3413 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3413 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3413 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3413 has memoization hash 9ee82fd8b2b45bf2b97c3b0d5269bf82 parsl.dataflow.memoization INFO: Task 3413 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3413 try 0 launched on executor multi with executor id 3414 parsl.dataflow.dflow INFO: Standard output for task 3413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/22a1a4a4-e31d-47dc-8640-6e886d08171d_isrForCrosstalkSources_997005_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/22a1a4a4-e31d-47dc-8640-6e886d08171d_isrForCrosstalkSources_997005_36.stderr parsl.dataflow.dflow DEBUG: Task 3414 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3414 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3414 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3414 has memoization hash cd6979ccedaf9514869d996bd8c32466 parsl.dataflow.memoization INFO: Task 3414 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3414 try 0 launched on executor multi with executor id 3415 parsl.dataflow.dflow INFO: Standard output for task 3414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dd02294e-bbff-489a-80ad-41030ccd9114_isrForCrosstalkSources_996971_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dd02294e-bbff-489a-80ad-41030ccd9114_isrForCrosstalkSources_996971_11.stderr parsl.dataflow.dflow DEBUG: Task 3415 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3415 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3415 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3415 has memoization hash 53c9b1a68429f968ec4e51fc8d146760 parsl.dataflow.memoization INFO: Task 3415 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3415 try 0 launched on executor multi with executor id 3416 parsl.dataflow.dflow INFO: Standard output for task 3415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4a1ded38-d194-41e5-be36-3ef3e09f46e9_isrForCrosstalkSources_996999_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4a1ded38-d194-41e5-be36-3ef3e09f46e9_isrForCrosstalkSources_996999_48.stderr parsl.dataflow.dflow DEBUG: Task 3416 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3416 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3416 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3416 has memoization hash 765501d10c9abfe0352f783514208b1d parsl.dataflow.memoization INFO: Task 3416 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3416 try 0 launched on executor multi with executor id 3417 parsl.dataflow.dflow INFO: Standard output for task 3416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/76ecd649-c5a7-4446-956e-e2436949b3d7_isrForCrosstalkSources_996978_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/76ecd649-c5a7-4446-956e-e2436949b3d7_isrForCrosstalkSources_996978_30.stderr parsl.dataflow.dflow DEBUG: Task 3417 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3417 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3417 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3417 has memoization hash e232ff6eeacfeb1cbeaf70e0694c133e parsl.dataflow.memoization INFO: Task 3417 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3417 try 0 launched on executor multi with executor id 3418 parsl.dataflow.dflow INFO: Standard output for task 3417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/10e87278-2ffd-4dbc-8eea-51c30207e84a_isrForCrosstalkSources_997018_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/10e87278-2ffd-4dbc-8eea-51c30207e84a_isrForCrosstalkSources_997018_53.stderr parsl.dataflow.dflow DEBUG: Task 3418 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3418 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3418 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3418 has memoization hash 1321c117adac975fb604fba278fe74c2 parsl.dataflow.memoization INFO: Task 3418 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3418 try 0 launched on executor multi with executor id 3419 parsl.dataflow.dflow INFO: Standard output for task 3418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/4d740d17-2d7d-4495-bc1b-dd29102ab174_isrForCrosstalkSources_996967_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/4d740d17-2d7d-4495-bc1b-dd29102ab174_isrForCrosstalkSources_996967_55.stderr parsl.dataflow.dflow DEBUG: Task 3419 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3419 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3419 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3419 has memoization hash 3c2e453791448e9d279d63b6836e6c49 parsl.dataflow.memoization INFO: Task 3419 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3419 try 0 launched on executor multi with executor id 3420 parsl.dataflow.dflow INFO: Standard output for task 3419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/4b3f8ff8-8bd7-4ea1-ace8-235acc09611a_isrForCrosstalkSources_996973_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/4b3f8ff8-8bd7-4ea1-ace8-235acc09611a_isrForCrosstalkSources_996973_50.stderr parsl.dataflow.dflow DEBUG: Task 3420 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3420 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3420 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3420 has memoization hash e7734830d06f5ea507667cb51650713d parsl.dataflow.memoization INFO: Task 3420 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3420 try 0 launched on executor multi with executor id 3421 parsl.dataflow.dflow INFO: Standard output for task 3420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dcbc0ec3-e999-4901-bc93-fa91d14f8df6_isrForCrosstalkSources_996971_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dcbc0ec3-e999-4901-bc93-fa91d14f8df6_isrForCrosstalkSources_996971_5.stderr parsl.dataflow.dflow DEBUG: Task 3421 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3421 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3421 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3421 has memoization hash 50e5e1101bfc30e576ccbd60c50e50d4 parsl.dataflow.memoization INFO: Task 3421 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3421 try 0 launched on executor multi with executor id 3422 parsl.dataflow.dflow INFO: Standard output for task 3421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c51e5a64-16cb-48b5-8561-a1381805905e_isrForCrosstalkSources_996995_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c51e5a64-16cb-48b5-8561-a1381805905e_isrForCrosstalkSources_996995_28.stderr parsl.dataflow.dflow DEBUG: Task 3422 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3422 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3422 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3422 has memoization hash c2c21557a9d6a48883e3df85996d0072 parsl.dataflow.memoization INFO: Task 3422 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3422 try 0 launched on executor multi with executor id 3423 parsl.dataflow.dflow INFO: Standard output for task 3422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d3a43845-8066-4471-b377-f4d1c6e9b280_isrForCrosstalkSources_997024_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d3a43845-8066-4471-b377-f4d1c6e9b280_isrForCrosstalkSources_997024_42.stderr parsl.dataflow.dflow DEBUG: Task 3423 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3423 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3423 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3423 has memoization hash b59b1cf2fed0fb6f62da164450f023d4 parsl.dataflow.memoization INFO: Task 3423 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3423 try 0 launched on executor multi with executor id 3424 parsl.dataflow.dflow INFO: Standard output for task 3423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/15a35360-2e8c-4cca-acf6-367c1e78efee_isrForCrosstalkSources_996997_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/15a35360-2e8c-4cca-acf6-367c1e78efee_isrForCrosstalkSources_996997_47.stderr parsl.dataflow.dflow DEBUG: Task 3424 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3424 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3424 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3424 has memoization hash f19a8a2e85faf1921af8cb120fa2c878 parsl.dataflow.memoization INFO: Task 3424 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3424 try 0 launched on executor multi with executor id 3425 parsl.dataflow.dflow INFO: Standard output for task 3424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d5e6461-e67f-45cf-b571-90605e2b27bb_isrForCrosstalkSources_996995_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d5e6461-e67f-45cf-b571-90605e2b27bb_isrForCrosstalkSources_996995_17.stderr parsl.dataflow.dflow DEBUG: Task 3425 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3425 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3425 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3425 has memoization hash 13bf401c44a48ce87ab23ad52627b9ca parsl.dataflow.memoization INFO: Task 3425 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3425 try 0 launched on executor multi with executor id 3426 parsl.dataflow.dflow INFO: Standard output for task 3425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d700b0a4-af78-4382-80db-0dced0b43a05_isrForCrosstalkSources_997025_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d700b0a4-af78-4382-80db-0dced0b43a05_isrForCrosstalkSources_997025_48.stderr parsl.dataflow.dflow DEBUG: Task 3426 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3426 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3426 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3426 has memoization hash 40d5833d06bea9ca176217568eae1851 parsl.dataflow.memoization INFO: Task 3426 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3426 try 0 launched on executor multi with executor id 3427 parsl.dataflow.dflow INFO: Standard output for task 3426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/1b539583-ce73-4777-b5c9-336f44951a95_isrForCrosstalkSources_997010_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/1b539583-ce73-4777-b5c9-336f44951a95_isrForCrosstalkSources_997010_56.stderr parsl.dataflow.dflow DEBUG: Task 3427 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3427 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3427 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3427 has memoization hash 7abedd7395c24718eee71f7430434f6c parsl.dataflow.memoization INFO: Task 3427 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3427 try 0 launched on executor multi with executor id 3428 parsl.dataflow.dflow INFO: Standard output for task 3427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/bc168d8d-20b4-40c8-bdb3-755f48c44fe8_isrForCrosstalkSources_996991_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/bc168d8d-20b4-40c8-bdb3-755f48c44fe8_isrForCrosstalkSources_996991_59.stderr parsl.dataflow.dflow DEBUG: Task 3428 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3428 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3428 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3428 has memoization hash 2caa10b849116fb551b93806e1e9da8a parsl.dataflow.memoization INFO: Task 3428 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3428 try 0 launched on executor multi with executor id 3429 parsl.dataflow.dflow INFO: Standard output for task 3428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/de450d25-eec3-497a-bcd8-90e0dfab0a53_isrForCrosstalkSources_997010_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/de450d25-eec3-497a-bcd8-90e0dfab0a53_isrForCrosstalkSources_997010_52.stderr parsl.dataflow.dflow DEBUG: Task 3429 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3429 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3429 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3429 has memoization hash 51d637d11a18fd564792db983f091922 parsl.dataflow.memoization INFO: Task 3429 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3429 try 0 launched on executor multi with executor id 3430 parsl.dataflow.dflow INFO: Standard output for task 3429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/9a0f1848-334d-4771-9496-65b91e251aad_isrForCrosstalkSources_996956_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/9a0f1848-334d-4771-9496-65b91e251aad_isrForCrosstalkSources_996956_44.stderr parsl.dataflow.dflow DEBUG: Task 3430 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3430 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3430 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3430 has memoization hash b6f3f151ea9aa59372df46c3a82639cb parsl.dataflow.memoization INFO: Task 3430 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3430 try 0 launched on executor multi with executor id 3431 parsl.dataflow.dflow INFO: Standard output for task 3430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6a429c0-d948-4b90-ad94-603ac4bef9b1_isrForCrosstalkSources_997026_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6a429c0-d948-4b90-ad94-603ac4bef9b1_isrForCrosstalkSources_997026_17.stderr parsl.dataflow.dflow DEBUG: Task 3431 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3431 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3431 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3431 has memoization hash 4a9b02745a2a0efe8b604f87de8e0fcc parsl.dataflow.memoization INFO: Task 3431 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3431 try 0 launched on executor multi with executor id 3432 parsl.dataflow.dflow INFO: Standard output for task 3431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fbf30e7a-33ff-4d6c-a7df-cca2c9cb6ad1_isrForCrosstalkSources_997012_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fbf30e7a-33ff-4d6c-a7df-cca2c9cb6ad1_isrForCrosstalkSources_997012_3.stderr parsl.dataflow.dflow DEBUG: Task 3432 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3432 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3432 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3432 has memoization hash 1f399ace812ad7b42592fa2eabcaecea parsl.dataflow.memoization INFO: Task 3432 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3432 try 0 launched on executor multi with executor id 3433 parsl.dataflow.dflow INFO: Standard output for task 3432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/543bf4c4-0263-4ddf-a716-8a96d8f56767_isrForCrosstalkSources_997022_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/543bf4c4-0263-4ddf-a716-8a96d8f56767_isrForCrosstalkSources_997022_47.stderr parsl.dataflow.dflow DEBUG: Task 3433 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3433 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3433 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3433 has memoization hash 07f39a87abfcd476a746a708f05653bb parsl.dataflow.memoization INFO: Task 3433 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3433 try 0 launched on executor multi with executor id 3434 parsl.dataflow.dflow INFO: Standard output for task 3433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/86363ce8-5b90-4701-a0c6-86c90b179e51_isrForCrosstalkSources_996979_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/86363ce8-5b90-4701-a0c6-86c90b179e51_isrForCrosstalkSources_996979_26.stderr parsl.dataflow.dflow DEBUG: Task 3434 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3434 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3434 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3434 has memoization hash 8a628f2d7e6b9f30edb9736485b53ffc parsl.dataflow.memoization INFO: Task 3434 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3434 try 0 launched on executor multi with executor id 3435 parsl.dataflow.dflow INFO: Standard output for task 3434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/8a66645d-1737-4bce-a959-cb5127f9f7ec_isrForCrosstalkSources_996964_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/8a66645d-1737-4bce-a959-cb5127f9f7ec_isrForCrosstalkSources_996964_46.stderr parsl.dataflow.dflow DEBUG: Task 3435 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3435 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3435 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3435 has memoization hash 64c1478f0eff796fd62cbdb5d27ea385 parsl.dataflow.memoization INFO: Task 3435 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3435 try 0 launched on executor multi with executor id 3436 parsl.dataflow.dflow INFO: Standard output for task 3435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f67905c3-9db9-489e-8d45-01498aa81155_isrForCrosstalkSources_997028_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f67905c3-9db9-489e-8d45-01498aa81155_isrForCrosstalkSources_997028_32.stderr parsl.dataflow.dflow DEBUG: Task 3436 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3436 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3436 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3436 has memoization hash 50e094efbb700e8d42aa5a82fff6868a parsl.dataflow.memoization INFO: Task 3436 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3436 try 0 launched on executor multi with executor id 3437 parsl.dataflow.dflow INFO: Standard output for task 3436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/6a0163d0-3b32-4028-be7b-4c7f049f5528_isrForCrosstalkSources_996977_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/6a0163d0-3b32-4028-be7b-4c7f049f5528_isrForCrosstalkSources_996977_32.stderr parsl.dataflow.dflow DEBUG: Task 3437 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3437 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3437 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3437 has memoization hash 026f90986f40e49e14b789c9dec943a2 parsl.dataflow.memoization INFO: Task 3437 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3437 try 0 launched on executor multi with executor id 3438 parsl.dataflow.dflow INFO: Standard output for task 3437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5bd51ae6-7e5f-4537-8c5f-31cf7ca52cb7_isrForCrosstalkSources_996999_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5bd51ae6-7e5f-4537-8c5f-31cf7ca52cb7_isrForCrosstalkSources_996999_24.stderr parsl.dataflow.dflow DEBUG: Task 3438 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3438 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3438 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3438 has memoization hash adf317de200dddd2e57fb33009353f6d parsl.dataflow.memoization INFO: Task 3438 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3438 try 0 launched on executor multi with executor id 3439 parsl.dataflow.dflow INFO: Standard output for task 3438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/91219d6d-f817-4221-942b-7a2890e9455a_isrForCrosstalkSources_997006_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/91219d6d-f817-4221-942b-7a2890e9455a_isrForCrosstalkSources_997006_13.stderr parsl.dataflow.dflow DEBUG: Task 3439 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3439 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3439 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3439 has memoization hash 1c88e1c055c24db6dc76e91d3f1d102d parsl.dataflow.memoization INFO: Task 3439 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3439 try 0 launched on executor multi with executor id 3440 parsl.dataflow.dflow INFO: Standard output for task 3439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/a32a6610-1f28-4c98-a5b6-4ff768fe71a6_isrForCrosstalkSources_996969_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/a32a6610-1f28-4c98-a5b6-4ff768fe71a6_isrForCrosstalkSources_996969_43.stderr parsl.dataflow.dflow DEBUG: Task 3440 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3440 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3440 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3440 has memoization hash 4f6a3cf72e6a2b7f44c179d0e815ff0d parsl.dataflow.memoization INFO: Task 3440 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3440 try 0 launched on executor multi with executor id 3441 parsl.dataflow.dflow INFO: Standard output for task 3440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a22a2f17-f4e1-4c18-af5e-8d1e2bb675aa_isrForCrosstalkSources_996954_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a22a2f17-f4e1-4c18-af5e-8d1e2bb675aa_isrForCrosstalkSources_996954_31.stderr parsl.dataflow.dflow DEBUG: Task 3441 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3441 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3441 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3441 has memoization hash d58d401c63024d8a6c088f9b0de47780 parsl.dataflow.memoization INFO: Task 3441 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3441 try 0 launched on executor multi with executor id 3442 parsl.dataflow.dflow INFO: Standard output for task 3441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1202e874-d9eb-41cd-b938-dbb875924832_isrForCrosstalkSources_996998_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1202e874-d9eb-41cd-b938-dbb875924832_isrForCrosstalkSources_996998_40.stderr parsl.dataflow.dflow DEBUG: Task 3442 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3442 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3442 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3442 has memoization hash e00af5c12445827d1cd2180daef97181 parsl.dataflow.memoization INFO: Task 3442 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3442 try 0 launched on executor multi with executor id 3443 parsl.dataflow.dflow INFO: Standard output for task 3442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/13a8e6d6-8a3b-4ec8-b456-4447de7ee05a_isrForCrosstalkSources_996958_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/13a8e6d6-8a3b-4ec8-b456-4447de7ee05a_isrForCrosstalkSources_996958_6.stderr parsl.dataflow.dflow DEBUG: Task 3443 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3443 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3443 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3443 has memoization hash b5203972c3411e38cd6c1a3b5a21888b parsl.dataflow.memoization INFO: Task 3443 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3443 try 0 launched on executor multi with executor id 3444 parsl.dataflow.dflow INFO: Standard output for task 3443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/917e82ff-5aac-46dd-9bcd-2156ab971f9c_isrForCrosstalkSources_996983_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/917e82ff-5aac-46dd-9bcd-2156ab971f9c_isrForCrosstalkSources_996983_8.stderr parsl.dataflow.dflow DEBUG: Task 3444 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3444 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3444 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3444 has memoization hash e13395a491da1efdda97ac2948bddb6f parsl.dataflow.memoization INFO: Task 3444 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3444 try 0 launched on executor multi with executor id 3445 parsl.dataflow.dflow INFO: Standard output for task 3444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/74029931-c4ca-4628-b14e-664e7c0ca552_isrForCrosstalkSources_996965_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/74029931-c4ca-4628-b14e-664e7c0ca552_isrForCrosstalkSources_996965_29.stderr parsl.dataflow.dflow DEBUG: Task 3445 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3445 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3445 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3445 has memoization hash e784cd9bedfaa24f751e4b33d0cc92fc parsl.dataflow.memoization INFO: Task 3445 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3445 try 0 launched on executor multi with executor id 3446 parsl.dataflow.dflow INFO: Standard output for task 3445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3bbd8d9b-dbdc-41d0-bdfd-13b3080cc825_isrForCrosstalkSources_996988_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3bbd8d9b-dbdc-41d0-bdfd-13b3080cc825_isrForCrosstalkSources_996988_48.stderr parsl.dataflow.dflow DEBUG: Task 3446 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3446 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3446 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3446 has memoization hash 07c0b5a854f398361babbd436a92b1fc parsl.dataflow.memoization INFO: Task 3446 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3446 try 0 launched on executor multi with executor id 3447 parsl.dataflow.dflow INFO: Standard output for task 3446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e84c5fd7-97b0-4700-afbd-5d7c1b719e7b_isrForCrosstalkSources_996957_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e84c5fd7-97b0-4700-afbd-5d7c1b719e7b_isrForCrosstalkSources_996957_56.stderr parsl.dataflow.dflow DEBUG: Task 3447 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3447 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3447 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3447 has memoization hash 890684813896afa4f000ec84587f64ea parsl.dataflow.memoization INFO: Task 3447 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:49:58 proc_lsst.multi:170 [INFO] [multi] job_id 17074889 proc_lsst.multi INFO: [multi] job_id 17074889 2024-03-15 13:49:58 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:49:58 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:58 proc_lsst.multi:178 [INFO] [multi] provider escience_rtx6k accepted submit and returned 17074889 proc_lsst.multi INFO: [multi] provider escience_rtx6k accepted submit and returned 17074889 parsl.executors.status_handling DEBUG: Launched block 35 on executor multi with job ID 17074889 parsl.executors.status_handling INFO: Allocated block ID 36 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-36 2024-03-15 13:49:58 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-36 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-36 2024-03-15 13:49:58 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:58 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:49:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:58 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:49:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:58 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 0 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 0 2024-03-15 13:49:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:49:58 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=36 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 3447 try 0 launched on executor multi with executor id 3448 parsl.dataflow.dflow INFO: Standard output for task 3447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/edcfcf67-bb0f-405a-9564-d264455753b6_isrForCrosstalkSources_997017_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/edcfcf67-bb0f-405a-9564-d264455753b6_isrForCrosstalkSources_997017_22.stderr parsl.dataflow.dflow DEBUG: Task 3448 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3448 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3448 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3448 has memoization hash 91fc18488ebec5afc218f7d29a47e5ff parsl.dataflow.memoization INFO: Task 3448 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c598040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3448 try 0 launched on executor multi with executor id 3449 parsl.dataflow.dflow INFO: Standard output for task 3448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/65cb4df0-a860-4658-8b56-2bf4b7e723f0_isrForCrosstalkSources_997019_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/65cb4df0-a860-4658-8b56-2bf4b7e723f0_isrForCrosstalkSources_997019_48.stderr parsl.dataflow.dflow DEBUG: Task 3449 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3449 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3449 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3449 has memoization hash 37f5bbeece8c2f301ca115de57fab019 parsl.dataflow.memoization INFO: Task 3449 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 3449 try 0 launched on executor multi with executor id 3450 parsl.dataflow.dflow INFO: Standard output for task 3449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/e8a57b92-16bc-45ad-b88c-e48c989ba930_isrForCrosstalkSources_996977_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/e8a57b92-16bc-45ad-b88c-e48c989ba930_isrForCrosstalkSources_996977_7.stderr parsl.dataflow.dflow DEBUG: Task 3450 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3450 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3450 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3450 has memoization hash 9f3da8f270677c1afb43f0bc8330889d parsl.dataflow.memoization INFO: Task 3450 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3450 try 0 launched on executor multi with executor id 3451 parsl.dataflow.dflow INFO: Standard output for task 3450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/210d0ada-7e18-4f6a-81e0-fb9bc08cdb81_isrForCrosstalkSources_997017_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/210d0ada-7e18-4f6a-81e0-fb9bc08cdb81_isrForCrosstalkSources_997017_23.stderr parsl.dataflow.dflow DEBUG: Task 3451 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3451 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3451 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3451 has memoization hash 9e3decdaaf91e1f6d009e953a482e894 parsl.dataflow.memoization INFO: Task 3451 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3451 try 0 launched on executor multi with executor id 3452 parsl.dataflow.dflow INFO: Standard output for task 3451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f972d9e2-fee1-4b50-b664-f71b25dd7c1a_isrForCrosstalkSources_997019_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f972d9e2-fee1-4b50-b664-f71b25dd7c1a_isrForCrosstalkSources_997019_16.stderr parsl.dataflow.dflow DEBUG: Task 3452 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3452 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3452 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3452 has memoization hash 44911f678e4383fe6fd618872f00c191 parsl.dataflow.memoization INFO: Task 3452 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ca3eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3452 try 0 launched on executor multi with executor id 3453 parsl.dataflow.dflow INFO: Standard output for task 3452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/6ac1f232-52cd-461a-97dc-0b8b01a8a605_isrForCrosstalkSources_996994_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/6ac1f232-52cd-461a-97dc-0b8b01a8a605_isrForCrosstalkSources_996994_61.stderr parsl.dataflow.dflow DEBUG: Task 3453 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3453 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3453 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3453 has memoization hash bc5069a38deb9ce53631959da953493b parsl.dataflow.memoization INFO: Task 3453 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3453 try 0 launched on executor multi with executor id 3454 parsl.dataflow.dflow INFO: Standard output for task 3453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/873ff00d-98ef-4c0c-b3a0-b90de102f67d_isrForCrosstalkSources_997019_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/873ff00d-98ef-4c0c-b3a0-b90de102f67d_isrForCrosstalkSources_997019_33.stderr parsl.dataflow.dflow DEBUG: Task 3454 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3454 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3454 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3454 has memoization hash 6925a09b823f1daa42553d9ad4b0f135 parsl.dataflow.memoization INFO: Task 3454 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3454 try 0 launched on executor multi with executor id 3455 parsl.dataflow.dflow INFO: Standard output for task 3454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/870ff890-9133-44d8-b329-11acad383020_isrForCrosstalkSources_997000_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/870ff890-9133-44d8-b329-11acad383020_isrForCrosstalkSources_997000_46.stderr parsl.dataflow.dflow DEBUG: Task 3455 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3455 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3455 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3455 has memoization hash d38763688d6c65bf2532e7f37146241e parsl.dataflow.memoization INFO: Task 3455 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3455 try 0 launched on executor multi with executor id 3456 parsl.dataflow.dflow INFO: Standard output for task 3455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/8bb8aa8c-c6a5-435e-b0b1-092e956a3bfc_isrForCrosstalkSources_996958_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/8bb8aa8c-c6a5-435e-b0b1-092e956a3bfc_isrForCrosstalkSources_996958_22.stderr parsl.dataflow.dflow DEBUG: Task 3456 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3456 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3456 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3456 has memoization hash 524b9a6c4c4a01ce1a9eed48522c8628 parsl.dataflow.memoization INFO: Task 3456 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3456 try 0 launched on executor multi with executor id 3457 parsl.dataflow.dflow INFO: Standard output for task 3456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/848193c5-f066-4bd7-8d90-7c4c69387839_isrForCrosstalkSources_997002_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/848193c5-f066-4bd7-8d90-7c4c69387839_isrForCrosstalkSources_997002_36.stderr parsl.dataflow.dflow DEBUG: Task 3457 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3457 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3457 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3457 has memoization hash a3ad30f8ce19c538bf9c4b2fc472b5a2 parsl.dataflow.memoization INFO: Task 3457 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3457 try 0 launched on executor multi with executor id 3458 parsl.dataflow.dflow INFO: Standard output for task 3457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/553718d7-5f26-4030-90de-3a4089435583_isrForCrosstalkSources_996962_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/553718d7-5f26-4030-90de-3a4089435583_isrForCrosstalkSources_996962_19.stderr parsl.dataflow.dflow DEBUG: Task 3458 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3458 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3458 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3458 has memoization hash bca0ed57bc2575720943ca341c6bac34 parsl.dataflow.memoization INFO: Task 3458 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3458 try 0 launched on executor multi with executor id 3459 parsl.dataflow.dflow INFO: Standard output for task 3458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1b6eea7d-722b-497a-8b8a-719466968b60_isrForCrosstalkSources_997022_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1b6eea7d-722b-497a-8b8a-719466968b60_isrForCrosstalkSources_997022_49.stderr parsl.dataflow.dflow DEBUG: Task 3459 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3459 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3459 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3459 has memoization hash c3eb7f53cae66290e774793be8cfb62e parsl.dataflow.memoization INFO: Task 3459 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3459 try 0 launched on executor multi with executor id 3460 parsl.dataflow.dflow INFO: Standard output for task 3459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/70b048ee-da56-43bf-ad5c-9c53cbfb3923_isrForCrosstalkSources_997007_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/70b048ee-da56-43bf-ad5c-9c53cbfb3923_isrForCrosstalkSources_997007_43.stderr parsl.dataflow.dflow DEBUG: Task 3460 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3460 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3460 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3460 has memoization hash 9aa5deb3e0095d4515453ec930a009bf parsl.dataflow.memoization INFO: Task 3460 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3460 try 0 launched on executor multi with executor id 3461 parsl.dataflow.dflow INFO: Standard output for task 3460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/699c1ec7-574e-4f61-be27-fbbef3605c17_isrForCrosstalkSources_996987_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/699c1ec7-574e-4f61-be27-fbbef3605c17_isrForCrosstalkSources_996987_28.stderr parsl.dataflow.dflow DEBUG: Task 3461 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3461 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3461 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3461 has memoization hash 50fccc24b0fe1cea257ae38ff7823450 parsl.dataflow.memoization INFO: Task 3461 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3461 try 0 launched on executor multi with executor id 3462 parsl.dataflow.dflow INFO: Standard output for task 3461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/deea36e6-f0e8-4cb9-80a2-6ae44fb6f13d_isrForCrosstalkSources_996953_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/deea36e6-f0e8-4cb9-80a2-6ae44fb6f13d_isrForCrosstalkSources_996953_5.stderr parsl.dataflow.dflow DEBUG: Task 3462 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3462 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3462 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3462 has memoization hash df29b9318c5c52fc428b8a25024a67dc parsl.dataflow.memoization INFO: Task 3462 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3462 try 0 launched on executor multi with executor id 3463 parsl.dataflow.dflow INFO: Standard output for task 3462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/3e5e0ca1-f03c-4694-9bbf-3bed1f0c2290_isrForCrosstalkSources_997030_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/3e5e0ca1-f03c-4694-9bbf-3bed1f0c2290_isrForCrosstalkSources_997030_51.stderr parsl.dataflow.dflow DEBUG: Task 3463 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3463 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3463 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3463 has memoization hash 09371ce5e028a3fc38db0a6574e998af parsl.dataflow.memoization INFO: Task 3463 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3463 try 0 launched on executor multi with executor id 3464 parsl.dataflow.dflow INFO: Standard output for task 3463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d9f51d59-61f0-4f15-bc0f-5cb8624a5e6c_isrForCrosstalkSources_996990_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d9f51d59-61f0-4f15-bc0f-5cb8624a5e6c_isrForCrosstalkSources_996990_38.stderr parsl.dataflow.dflow DEBUG: Task 3464 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3464 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3464 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3464 has memoization hash 6df07daeee43340deb36fc0bf8ebe52e parsl.dataflow.memoization INFO: Task 3464 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3464 try 0 launched on executor multi with executor id 3465 parsl.dataflow.dflow INFO: Standard output for task 3464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/197ee9f3-bd84-4ef8-a9be-6ba11992df91_isrForCrosstalkSources_997024_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/197ee9f3-bd84-4ef8-a9be-6ba11992df91_isrForCrosstalkSources_997024_60.stderr parsl.dataflow.dflow DEBUG: Task 3465 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3465 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3465 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3465 has memoization hash 7e908a3c1615d625bdeb9e112342ab50 parsl.dataflow.memoization INFO: Task 3465 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3465 try 0 launched on executor multi with executor id 3466 parsl.dataflow.dflow INFO: Standard output for task 3465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7c439e7a-bae5-4a3e-a8cc-5c3b987c054b_isrForCrosstalkSources_997025_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7c439e7a-bae5-4a3e-a8cc-5c3b987c054b_isrForCrosstalkSources_997025_60.stderr parsl.dataflow.dflow DEBUG: Task 3466 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3466 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3466 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3466 has memoization hash 00ddfd865aeb23f41a0e3b0355aa7244 parsl.dataflow.memoization INFO: Task 3466 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3466 try 0 launched on executor multi with executor id 3467 parsl.dataflow.dflow INFO: Standard output for task 3466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/6ba13cb1-ca2c-47cd-8d20-2a5ca7b798e9_isrForCrosstalkSources_996969_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/6ba13cb1-ca2c-47cd-8d20-2a5ca7b798e9_isrForCrosstalkSources_996969_15.stderr parsl.dataflow.dflow DEBUG: Task 3467 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3467 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3467 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3467 has memoization hash 757e8fd39c4223fad66f7e2c2790c9ae parsl.dataflow.memoization INFO: Task 3467 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3467 try 0 launched on executor multi with executor id 3468 parsl.dataflow.dflow INFO: Standard output for task 3467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/ef667500-5712-403d-ad28-e17e613d5548_isrForCrosstalkSources_996999_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/ef667500-5712-403d-ad28-e17e613d5548_isrForCrosstalkSources_996999_34.stderr parsl.dataflow.dflow DEBUG: Task 3468 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3468 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3468 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3468 has memoization hash 0c9db8bf5d4a12eaf26ca5752e5f9f88 parsl.dataflow.memoization INFO: Task 3468 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3468 try 0 launched on executor multi with executor id 3469 parsl.dataflow.dflow INFO: Standard output for task 3468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1936ac38-8749-458a-88dc-923d63422b1c_isrForCrosstalkSources_996968_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1936ac38-8749-458a-88dc-923d63422b1c_isrForCrosstalkSources_996968_59.stderr parsl.dataflow.dflow DEBUG: Task 3469 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3469 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3469 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3469 has memoization hash 04a807705ffafa45c10bfa2794f8e506 parsl.dataflow.memoization INFO: Task 3469 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3469 try 0 launched on executor multi with executor id 3470 parsl.dataflow.dflow INFO: Standard output for task 3469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/af3f0835-3c40-4240-9791-fb488bd0454c_isrForCrosstalkSources_997020_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/af3f0835-3c40-4240-9791-fb488bd0454c_isrForCrosstalkSources_997020_36.stderr parsl.dataflow.dflow DEBUG: Task 3470 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3470 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3470 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3470 has memoization hash 59faab8666c07d6ddf7949396b74c604 parsl.dataflow.memoization INFO: Task 3470 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3470 try 0 launched on executor multi with executor id 3471 parsl.dataflow.dflow INFO: Standard output for task 3470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/694e9c62-cafb-4cf1-94b5-434072166fdb_isrForCrosstalkSources_996976_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/694e9c62-cafb-4cf1-94b5-434072166fdb_isrForCrosstalkSources_996976_31.stderr parsl.dataflow.dflow DEBUG: Task 3471 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3471 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3471 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3471 has memoization hash 208cc31f54358f74d6d7af7d30a43e49 parsl.dataflow.memoization INFO: Task 3471 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3471 try 0 launched on executor multi with executor id 3472 parsl.dataflow.dflow INFO: Standard output for task 3471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/80cc9b78-9007-43fa-9b5e-c62d141bb795_isrForCrosstalkSources_996979_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/80cc9b78-9007-43fa-9b5e-c62d141bb795_isrForCrosstalkSources_996979_31.stderr parsl.dataflow.dflow DEBUG: Task 3472 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3472 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3472 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3472 has memoization hash 64e64de99db88130dd801489d487f8d0 parsl.dataflow.memoization INFO: Task 3472 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3472 try 0 launched on executor multi with executor id 3473 parsl.dataflow.dflow INFO: Standard output for task 3472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a28ee8cf-a4ab-4c70-ab88-52a1349bf3df_isrForCrosstalkSources_997006_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a28ee8cf-a4ab-4c70-ab88-52a1349bf3df_isrForCrosstalkSources_997006_31.stderr parsl.dataflow.dflow DEBUG: Task 3473 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3473 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3473 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3473 has memoization hash f527e265f409ff93230dd3b1ef555338 parsl.dataflow.memoization INFO: Task 3473 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3473 try 0 launched on executor multi with executor id 3474 parsl.dataflow.dflow INFO: Standard output for task 3473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3685e924-829c-44ec-8eec-9a81b863301c_isrForCrosstalkSources_996968_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3685e924-829c-44ec-8eec-9a81b863301c_isrForCrosstalkSources_996968_13.stderr parsl.dataflow.dflow DEBUG: Task 3474 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3474 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3474 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3474 has memoization hash 21ecc75c289c3317f66e6ff666110f20 parsl.dataflow.memoization INFO: Task 3474 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3474 try 0 launched on executor multi with executor id 3475 parsl.dataflow.dflow INFO: Standard output for task 3474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/261b67ca-8d97-4e11-9835-0b492d93e6fc_isrForCrosstalkSources_996962_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/261b67ca-8d97-4e11-9835-0b492d93e6fc_isrForCrosstalkSources_996962_9.stderr parsl.dataflow.dflow DEBUG: Task 3475 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3475 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3475 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3475 has memoization hash 3f12ba40306ff64548577a382f245c28 parsl.dataflow.memoization INFO: Task 3475 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3475 try 0 launched on executor multi with executor id 3476 parsl.dataflow.dflow INFO: Standard output for task 3475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/06e0bbdc-8fd1-42e1-a8db-90319fa54086_isrForCrosstalkSources_996954_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/06e0bbdc-8fd1-42e1-a8db-90319fa54086_isrForCrosstalkSources_996954_55.stderr parsl.dataflow.dflow DEBUG: Task 3476 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3476 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3476 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3476 has memoization hash 31b055863ad2b3601dc3e3dd96b715de parsl.dataflow.memoization INFO: Task 3476 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3476 try 0 launched on executor multi with executor id 3477 parsl.dataflow.dflow INFO: Standard output for task 3476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/afde6ba9-853a-48b6-9771-7b05b00611b6_isrForCrosstalkSources_996964_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/afde6ba9-853a-48b6-9771-7b05b00611b6_isrForCrosstalkSources_996964_18.stderr parsl.dataflow.dflow DEBUG: Task 3477 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3477 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3477 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3477 has memoization hash 5f81906bc0477fdc3920ad60fe0350c6 parsl.dataflow.memoization INFO: Task 3477 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3477 try 0 launched on executor multi with executor id 3478 parsl.dataflow.dflow INFO: Standard output for task 3477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/84d9a0a0-9010-4c05-85e0-777bda43f74f_isrForCrosstalkSources_997001_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/84d9a0a0-9010-4c05-85e0-777bda43f74f_isrForCrosstalkSources_997001_26.stderr parsl.dataflow.dflow DEBUG: Task 3478 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3478 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3478 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3478 has memoization hash cae9d0b270465027ef228120aa4700dc parsl.dataflow.memoization INFO: Task 3478 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3478 try 0 launched on executor multi with executor id 3479 parsl.dataflow.dflow INFO: Standard output for task 3478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/404cb793-1dd5-46c8-a428-a817a6b3a14e_isrForCrosstalkSources_997024_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/404cb793-1dd5-46c8-a428-a817a6b3a14e_isrForCrosstalkSources_997024_34.stderr parsl.dataflow.dflow DEBUG: Task 3479 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3479 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3479 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3479 has memoization hash 30f87cc617132b9cae79459c9280a741 parsl.dataflow.memoization INFO: Task 3479 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bcae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3479 try 0 launched on executor multi with executor id 3480 parsl.dataflow.dflow INFO: Standard output for task 3479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/7cc6aba9-cf4e-450f-b23f-a7a97e24af12_isrForCrosstalkSources_996960_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/7cc6aba9-cf4e-450f-b23f-a7a97e24af12_isrForCrosstalkSources_996960_53.stderr parsl.dataflow.dflow DEBUG: Task 3480 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3480 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3480 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3480 has memoization hash 16e5debaa2eb9de209b818a04ee94510 parsl.dataflow.memoization INFO: Task 3480 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bc7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3480 try 0 launched on executor multi with executor id 3481 parsl.dataflow.dflow INFO: Standard output for task 3480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/ad709dad-6e62-4876-be3f-c089383593cf_isrForCrosstalkSources_996957_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/ad709dad-6e62-4876-be3f-c089383593cf_isrForCrosstalkSources_996957_18.stderr parsl.dataflow.dflow DEBUG: Task 3481 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3481 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3481 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3481 has memoization hash e6684978cf5749e1808e87e47a6363b6 parsl.dataflow.memoization INFO: Task 3481 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3481 try 0 launched on executor multi with executor id 3482 parsl.dataflow.dflow INFO: Standard output for task 3481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/04b0865c-6197-422a-bf53-16c13d43c675_isrForCrosstalkSources_997018_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/04b0865c-6197-422a-bf53-16c13d43c675_isrForCrosstalkSources_997018_30.stderr parsl.dataflow.dflow DEBUG: Task 3482 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3482 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3482 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3482 has memoization hash 398177264d407c3692a3f86d28fa167f parsl.dataflow.memoization INFO: Task 3482 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bcb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3482 try 0 launched on executor multi with executor id 3483 parsl.dataflow.dflow INFO: Standard output for task 3482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/5929babd-06f4-4b66-9096-d8af9fdf7c85_isrForCrosstalkSources_997005_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/5929babd-06f4-4b66-9096-d8af9fdf7c85_isrForCrosstalkSources_997005_32.stderr parsl.dataflow.dflow DEBUG: Task 3483 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3483 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3483 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3483 has memoization hash 015db32225d6c4c7d2487e587fcd457b parsl.dataflow.memoization INFO: Task 3483 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3483 try 0 launched on executor multi with executor id 3484 parsl.dataflow.dflow INFO: Standard output for task 3483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a8a82568-1bc0-4c74-b899-98ec2364f401_isrForCrosstalkSources_997028_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a8a82568-1bc0-4c74-b899-98ec2364f401_isrForCrosstalkSources_997028_46.stderr parsl.dataflow.dflow DEBUG: Task 3484 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3484 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3484 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3484 has memoization hash b544286fb9c63adb5ee47518ea71e512 parsl.dataflow.memoization INFO: Task 3484 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3484 try 0 launched on executor multi with executor id 3485 parsl.dataflow.dflow INFO: Standard output for task 3484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/f9611eaa-e670-4e84-92d1-eb0c1b70e372_isrForCrosstalkSources_997027_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/f9611eaa-e670-4e84-92d1-eb0c1b70e372_isrForCrosstalkSources_997027_38.stderr parsl.dataflow.dflow DEBUG: Task 3485 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3485 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3485 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3485 has memoization hash 7bbe8d1a4d5e7d4cb100e81b5dc50afe parsl.dataflow.memoization INFO: Task 3485 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bcc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3485 try 0 launched on executor multi with executor id 3486 parsl.dataflow.dflow INFO: Standard output for task 3485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/f8a4e0e9-4fbf-4f91-8107-b72b90405b0d_isrForCrosstalkSources_997014_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/f8a4e0e9-4fbf-4f91-8107-b72b90405b0d_isrForCrosstalkSources_997014_27.stderr parsl.dataflow.dflow DEBUG: Task 3486 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3486 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3486 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3486 has memoization hash 3408eaa7f3474c15c760f380f47453ce parsl.dataflow.memoization INFO: Task 3486 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bcd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3486 try 0 launched on executor multi with executor id 3487 parsl.dataflow.dflow INFO: Standard output for task 3486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3d47a5b2-8a96-4c84-8099-47d0831cf3af_isrForCrosstalkSources_996995_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3d47a5b2-8a96-4c84-8099-47d0831cf3af_isrForCrosstalkSources_996995_43.stderr parsl.dataflow.dflow DEBUG: Task 3487 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3487 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3487 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3487 has memoization hash 402a8064aa3b9af709b702bd99a8ca21 parsl.dataflow.memoization INFO: Task 3487 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bcea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3487 try 0 launched on executor multi with executor id 3488 parsl.dataflow.dflow INFO: Standard output for task 3487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/fa1a5a92-5a92-4e26-9c4e-c3805e64e673_isrForCrosstalkSources_997033_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/fa1a5a92-5a92-4e26-9c4e-c3805e64e673_isrForCrosstalkSources_997033_16.stderr parsl.dataflow.dflow DEBUG: Task 3488 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3488 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3488 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3488 has memoization hash 19fffa7edf8874a280633007cd4aab40 parsl.dataflow.memoization INFO: Task 3488 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bcfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3488 try 0 launched on executor multi with executor id 3489 parsl.dataflow.dflow INFO: Standard output for task 3488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/6b6b9c6d-7675-418f-a277-f490fa6fcb55_isrForCrosstalkSources_997032_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/6b6b9c6d-7675-418f-a277-f490fa6fcb55_isrForCrosstalkSources_997032_50.stderr parsl.dataflow.dflow DEBUG: Task 3489 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3489 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3489 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3489 has memoization hash 0790b75ad387735d4cf96cd4680d7a1a parsl.dataflow.memoization INFO: Task 3489 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3489 try 0 launched on executor multi with executor id 3490 parsl.dataflow.dflow INFO: Standard output for task 3489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5ecf746b-d469-4710-b6ff-9f88a6399f52_isrForCrosstalkSources_997009_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5ecf746b-d469-4710-b6ff-9f88a6399f52_isrForCrosstalkSources_997009_43.stderr parsl.dataflow.dflow DEBUG: Task 3490 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3490 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3490 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3490 has memoization hash 49457e87eb797cc13b1bded4a11f4de0 parsl.dataflow.memoization INFO: Task 3490 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bcf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3490 try 0 launched on executor multi with executor id 3491 parsl.dataflow.dflow INFO: Standard output for task 3490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/e69c9811-51a5-4152-9e7c-83c9de0dcd29_isrForCrosstalkSources_997017_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/e69c9811-51a5-4152-9e7c-83c9de0dcd29_isrForCrosstalkSources_997017_39.stderr parsl.dataflow.dflow DEBUG: Task 3491 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3491 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3491 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3491 has memoization hash e7d7727abc1c1fb994a06bda65aab88d parsl.dataflow.memoization INFO: Task 3491 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3491 try 0 launched on executor multi with executor id 3492 parsl.dataflow.dflow INFO: Standard output for task 3491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/5a8fc2ae-0277-42b4-84e5-5407a86fc23f_isrForCrosstalkSources_996980_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/5a8fc2ae-0277-42b4-84e5-5407a86fc23f_isrForCrosstalkSources_996980_32.stderr parsl.dataflow.dflow DEBUG: Task 3492 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3492 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3492 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3492 has memoization hash 63151faec408be8e1fe735f5e9a6d78d parsl.dataflow.memoization INFO: Task 3492 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3492 try 0 launched on executor multi with executor id 3493 parsl.dataflow.dflow INFO: Standard output for task 3492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ff6cb520-595e-4f35-9566-4d8055d3de0e_isrForCrosstalkSources_996962_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ff6cb520-595e-4f35-9566-4d8055d3de0e_isrForCrosstalkSources_996962_1.stderr parsl.dataflow.dflow DEBUG: Task 3493 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3493 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3493 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3493 has memoization hash 9c5249f64dfc706a658014160ee2af71 parsl.dataflow.memoization INFO: Task 3493 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3493 try 0 launched on executor multi with executor id 3494 parsl.dataflow.dflow INFO: Standard output for task 3493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/e68ac325-ead5-45c5-a0eb-1fea7a492996_isrForCrosstalkSources_997003_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/e68ac325-ead5-45c5-a0eb-1fea7a492996_isrForCrosstalkSources_997003_35.stderr parsl.dataflow.dflow DEBUG: Task 3494 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3494 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3494 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3494 has memoization hash ff7bd414eecf0164856316f50702f01f parsl.dataflow.memoization INFO: Task 3494 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3494 try 0 launched on executor multi with executor id 3495 parsl.dataflow.dflow INFO: Standard output for task 3494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d9a9b125-d697-4435-99b1-1a54fe62b620_isrForCrosstalkSources_997030_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d9a9b125-d697-4435-99b1-1a54fe62b620_isrForCrosstalkSources_997030_61.stderr parsl.dataflow.dflow DEBUG: Task 3495 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3495 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3495 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3495 has memoization hash 1b6b98d491b6aa01893c28d15748d854 parsl.dataflow.memoization INFO: Task 3495 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3495 try 0 launched on executor multi with executor id 3496 parsl.dataflow.dflow INFO: Standard output for task 3495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/75e52507-f1a6-46ef-afd0-0edb2605a070_isrForCrosstalkSources_996993_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/75e52507-f1a6-46ef-afd0-0edb2605a070_isrForCrosstalkSources_996993_5.stderr parsl.dataflow.dflow DEBUG: Task 3496 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3496 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3496 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3496 has memoization hash 95d541c1ffdad9db572bf1a8218a8180 parsl.dataflow.memoization INFO: Task 3496 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3496 try 0 launched on executor multi with executor id 3497 parsl.dataflow.dflow INFO: Standard output for task 3496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/99779995-a737-4312-bec3-639c32003f54_isrForCrosstalkSources_996980_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/99779995-a737-4312-bec3-639c32003f54_isrForCrosstalkSources_996980_35.stderr parsl.dataflow.dflow DEBUG: Task 3497 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3497 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3497 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3497 has memoization hash e6685194f859ed4ae81a26445870671a parsl.dataflow.memoization INFO: Task 3497 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3497 try 0 launched on executor multi with executor id 3498 parsl.dataflow.dflow INFO: Standard output for task 3497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c6f378d2-d4bb-4da2-8e11-df7e29afead1_isrForCrosstalkSources_996977_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c6f378d2-d4bb-4da2-8e11-df7e29afead1_isrForCrosstalkSources_996977_20.stderr parsl.dataflow.dflow DEBUG: Task 3498 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3498 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3498 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3498 has memoization hash 49834bbd560d7b675db6f9c392ee15d7 parsl.dataflow.memoization INFO: Task 3498 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3498 try 0 launched on executor multi with executor id 3499 parsl.dataflow.dflow INFO: Standard output for task 3498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9ce4c65d-f957-4bfa-ac75-46c2efd8781c_isrForCrosstalkSources_997030_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9ce4c65d-f957-4bfa-ac75-46c2efd8781c_isrForCrosstalkSources_997030_12.stderr parsl.dataflow.dflow DEBUG: Task 3499 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3499 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3499 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3499 has memoization hash dce505dc0dd6c39754902b3ecdf8b75f parsl.dataflow.memoization INFO: Task 3499 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3499 try 0 launched on executor multi with executor id 3500 parsl.dataflow.dflow INFO: Standard output for task 3499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/39971b06-55e9-4655-9a10-441f9e792561_isrForCrosstalkSources_996987_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/39971b06-55e9-4655-9a10-441f9e792561_isrForCrosstalkSources_996987_5.stderr parsl.dataflow.dflow DEBUG: Task 3500 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3500 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3500 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3500 has memoization hash f9c357bf533e67f1eb88f5cd2af9ae58 parsl.dataflow.memoization INFO: Task 3500 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3500 try 0 launched on executor multi with executor id 3501 parsl.dataflow.dflow INFO: Standard output for task 3500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/5682efc4-08b8-4495-8f47-6b7f1b8db364_isrForCrosstalkSources_997030_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/5682efc4-08b8-4495-8f47-6b7f1b8db364_isrForCrosstalkSources_997030_39.stderr parsl.dataflow.dflow DEBUG: Task 3501 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3501 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3501 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3501 has memoization hash 1f4ef89e2cb2a29ccdb968a3e6e7cf21 parsl.dataflow.memoization INFO: Task 3501 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3501 try 0 launched on executor multi with executor id 3502 parsl.dataflow.dflow INFO: Standard output for task 3501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/46102891-0ea5-4b82-bcf4-5db0cf197c06_isrForCrosstalkSources_997007_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/46102891-0ea5-4b82-bcf4-5db0cf197c06_isrForCrosstalkSources_997007_35.stderr parsl.dataflow.dflow DEBUG: Task 3502 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3502 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3502 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3502 has memoization hash b91adabe8244f5257863d594cf4ad367 parsl.dataflow.memoization INFO: Task 3502 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3502 try 0 launched on executor multi with executor id 3503 parsl.dataflow.dflow INFO: Standard output for task 3502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/4b377648-2447-45cf-9000-5e35b81e29a3_isrForCrosstalkSources_997012_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/4b377648-2447-45cf-9000-5e35b81e29a3_isrForCrosstalkSources_997012_40.stderr parsl.dataflow.dflow DEBUG: Task 3503 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3503 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3503 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3503 has memoization hash 92b39385e7b26c0b306f220825e7dc4b parsl.dataflow.memoization INFO: Task 3503 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3503 try 0 launched on executor multi with executor id 3504 parsl.dataflow.dflow INFO: Standard output for task 3503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/016780c3-4164-4dfc-98ab-f3a9d520684d_isrForCrosstalkSources_996957_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/016780c3-4164-4dfc-98ab-f3a9d520684d_isrForCrosstalkSources_996957_21.stderr parsl.dataflow.dflow DEBUG: Task 3504 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3504 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3504 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3504 has memoization hash 2583e4540cbae205b456d0b977de0287 parsl.dataflow.memoization INFO: Task 3504 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bda80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3504 try 0 launched on executor multi with executor id 3505 parsl.dataflow.dflow INFO: Standard output for task 3504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/199d9928-26d4-41e9-b282-c1b89353c8e1_isrForCrosstalkSources_997027_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/199d9928-26d4-41e9-b282-c1b89353c8e1_isrForCrosstalkSources_997027_8.stderr parsl.dataflow.dflow DEBUG: Task 3505 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3505 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3505 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3505 has memoization hash 6a2a53ce7544ce7f6870b5511f09786b parsl.dataflow.memoization INFO: Task 3505 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3505 try 0 launched on executor multi with executor id 3506 parsl.dataflow.dflow INFO: Standard output for task 3505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a0f7df20-1751-4372-9c3f-fc52d4bcbc7b_isrForCrosstalkSources_997026_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a0f7df20-1751-4372-9c3f-fc52d4bcbc7b_isrForCrosstalkSources_997026_40.stderr parsl.dataflow.dflow DEBUG: Task 3506 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3506 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3506 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3506 has memoization hash 54d4a2a3acf75a81efbb6c358c5bafb6 parsl.dataflow.memoization INFO: Task 3506 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3506 try 0 launched on executor multi with executor id 3507 parsl.dataflow.dflow INFO: Standard output for task 3506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c37b4abc-242d-464f-b9a9-749c45502593_isrForCrosstalkSources_996970_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c37b4abc-242d-464f-b9a9-749c45502593_isrForCrosstalkSources_996970_5.stderr parsl.dataflow.dflow DEBUG: Task 3507 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3507 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3507 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3507 has memoization hash 8157bfe6498f7f7d02693414f96eb18a parsl.dataflow.memoization INFO: Task 3507 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3507 try 0 launched on executor multi with executor id 3508 parsl.dataflow.dflow INFO: Standard output for task 3507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/450f6c53-79b3-4543-a5f4-23033fa7ae1c_isrForCrosstalkSources_996981_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/450f6c53-79b3-4543-a5f4-23033fa7ae1c_isrForCrosstalkSources_996981_36.stderr parsl.dataflow.dflow DEBUG: Task 3508 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3508 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3508 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3508 has memoization hash 91104064c2baf295b424405a0daeff35 parsl.dataflow.memoization INFO: Task 3508 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4beb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3508 try 0 launched on executor multi with executor id 3509 parsl.dataflow.dflow INFO: Standard output for task 3508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/43467481-0a11-41a3-8f8d-5cc9d8e0cb32_isrForCrosstalkSources_997012_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/43467481-0a11-41a3-8f8d-5cc9d8e0cb32_isrForCrosstalkSources_997012_32.stderr parsl.dataflow.dflow DEBUG: Task 3509 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3509 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3509 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3509 has memoization hash 990d434121d514a82fe1a37cfccefef3 parsl.dataflow.memoization INFO: Task 3509 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3509 try 0 launched on executor multi with executor id 3510 parsl.dataflow.dflow INFO: Standard output for task 3509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/75f0d663-7b42-44a6-82aa-a50e21853311_isrForCrosstalkSources_997028_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/75f0d663-7b42-44a6-82aa-a50e21853311_isrForCrosstalkSources_997028_52.stderr parsl.dataflow.dflow DEBUG: Task 3510 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3510 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3510 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3510 has memoization hash 24851d79b5677bf6438c4922077a667b parsl.dataflow.memoization INFO: Task 3510 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4beac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3510 try 0 launched on executor multi with executor id 3511 parsl.dataflow.dflow INFO: Standard output for task 3510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/4d253210-81b6-42db-85ab-d23fe5dfc17e_isrForCrosstalkSources_997031_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/4d253210-81b6-42db-85ab-d23fe5dfc17e_isrForCrosstalkSources_997031_40.stderr parsl.dataflow.dflow DEBUG: Task 3511 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3511 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3511 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3511 has memoization hash 24fc4d4c5e41ad17cf1d932f20261017 parsl.dataflow.memoization INFO: Task 3511 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3511 try 0 launched on executor multi with executor id 3512 parsl.dataflow.dflow INFO: Standard output for task 3511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/62516faa-fee6-4eb9-ab29-52eebaa76293_isrForCrosstalkSources_997010_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/62516faa-fee6-4eb9-ab29-52eebaa76293_isrForCrosstalkSources_997010_55.stderr parsl.dataflow.dflow DEBUG: Task 3512 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3512 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3512 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3512 has memoization hash d33d8150e76fac8ec3612d26492da7a2 parsl.dataflow.memoization INFO: Task 3512 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3512 try 0 launched on executor multi with executor id 3513 parsl.dataflow.dflow INFO: Standard output for task 3512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/df3b0177-59ad-44b8-8ca5-c2e40a2f3282_isrForCrosstalkSources_997007_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/df3b0177-59ad-44b8-8ca5-c2e40a2f3282_isrForCrosstalkSources_997007_61.stderr parsl.dataflow.dflow DEBUG: Task 3513 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3513 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3513 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3513 has memoization hash 0d830f1203c1af66038dee7d7364fd1a parsl.dataflow.memoization INFO: Task 3513 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3513 try 0 launched on executor multi with executor id 3514 parsl.dataflow.dflow INFO: Standard output for task 3513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/3bc6fcb2-2f3a-4188-9c51-f222cb51cc39_isrForCrosstalkSources_996991_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/3bc6fcb2-2f3a-4188-9c51-f222cb51cc39_isrForCrosstalkSources_996991_12.stderr parsl.dataflow.dflow DEBUG: Task 3514 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3514 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3514 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3514 has memoization hash 936a051c013da4eb203363a1bf83c1cb parsl.dataflow.memoization INFO: Task 3514 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3514 try 0 launched on executor multi with executor id 3515 parsl.dataflow.dflow INFO: Standard output for task 3514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c0a8e29b-d29f-46ca-8d9a-bfba687ea20a_isrForCrosstalkSources_997012_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c0a8e29b-d29f-46ca-8d9a-bfba687ea20a_isrForCrosstalkSources_997012_14.stderr parsl.dataflow.dflow DEBUG: Task 3515 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3515 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3515 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3515 has memoization hash 541f2da213e309ebb54dab9d07369841 parsl.dataflow.memoization INFO: Task 3515 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3515 try 0 launched on executor multi with executor id 3516 parsl.dataflow.dflow INFO: Standard output for task 3515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/64bce767-b592-4941-a994-e98901910e71_isrForCrosstalkSources_997021_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/64bce767-b592-4941-a994-e98901910e71_isrForCrosstalkSources_997021_54.stderr parsl.dataflow.dflow DEBUG: Task 3516 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3516 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3516 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3516 has memoization hash 18792f8fb8c85f9e8c8a1098ff0cd845 parsl.dataflow.memoization INFO: Task 3516 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3516 try 0 launched on executor multi with executor id 3517 parsl.dataflow.dflow INFO: Standard output for task 3516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/588873bf-b8d3-4da3-97ef-0d9a3a1d6b6a_isrForCrosstalkSources_997030_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/588873bf-b8d3-4da3-97ef-0d9a3a1d6b6a_isrForCrosstalkSources_997030_18.stderr parsl.dataflow.dflow DEBUG: Task 3517 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3517 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3517 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3517 has memoization hash f7a3ecf5c8e7f4893e897cbda7c06039 parsl.dataflow.memoization INFO: Task 3517 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3517 try 0 launched on executor multi with executor id 3518 parsl.dataflow.dflow INFO: Standard output for task 3517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/23945fba-0dd8-479b-a54b-08bfd9a69bb8_isrForCrosstalkSources_996971_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/23945fba-0dd8-479b-a54b-08bfd9a69bb8_isrForCrosstalkSources_996971_17.stderr parsl.dataflow.dflow DEBUG: Task 3518 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3518 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3518 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3518 has memoization hash 307637eacd4b8f1450fea4153aff052f parsl.dataflow.memoization INFO: Task 3518 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3518 try 0 launched on executor multi with executor id 3519 parsl.dataflow.dflow INFO: Standard output for task 3518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/dec44a4b-6544-4285-bca7-4eea9a8f443d_isrForCrosstalkSources_996998_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/dec44a4b-6544-4285-bca7-4eea9a8f443d_isrForCrosstalkSources_996998_33.stderr parsl.dataflow.dflow DEBUG: Task 3519 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3519 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3519 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3519 has memoization hash 6e92a77486e0c1b21e7d9ec32cf26192 parsl.dataflow.memoization INFO: Task 3519 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3519 try 0 launched on executor multi with executor id 3520 parsl.dataflow.dflow INFO: Standard output for task 3519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/8d9c467a-8aea-4370-a5f8-cbb0ba693d0e_isrForCrosstalkSources_997016_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/8d9c467a-8aea-4370-a5f8-cbb0ba693d0e_isrForCrosstalkSources_997016_59.stderr parsl.dataflow.dflow DEBUG: Task 3520 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3520 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3520 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3520 has memoization hash 1a4bd04f178b22323d3f9d62c76dd01d parsl.dataflow.memoization INFO: Task 3520 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3520 try 0 launched on executor multi with executor id 3521 parsl.dataflow.dflow INFO: Standard output for task 3520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/9f9bfee8-d276-4154-a472-e83edfc44dec_isrForCrosstalkSources_996982_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/9f9bfee8-d276-4154-a472-e83edfc44dec_isrForCrosstalkSources_996982_62.stderr parsl.dataflow.dflow DEBUG: Task 3521 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3521 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3521 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3521 has memoization hash e8cc2cc4c226174d83a228de881ee8e3 parsl.dataflow.memoization INFO: Task 3521 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3521 try 0 launched on executor multi with executor id 3522 parsl.dataflow.dflow INFO: Standard output for task 3521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bd555028-1229-4b3b-b131-e2f712a57eb0_isrForCrosstalkSources_997026_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bd555028-1229-4b3b-b131-e2f712a57eb0_isrForCrosstalkSources_997026_7.stderr parsl.dataflow.dflow DEBUG: Task 3522 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3522 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3522 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3522 has memoization hash 19c39e793125c825d5ab7287d028d572 parsl.dataflow.memoization INFO: Task 3522 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3522 try 0 launched on executor multi with executor id 3523 parsl.dataflow.dflow INFO: Standard output for task 3522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/bfd0f07c-dd98-4238-88a1-b4b0dac846cb_isrForCrosstalkSources_996957_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/bfd0f07c-dd98-4238-88a1-b4b0dac846cb_isrForCrosstalkSources_996957_29.stderr parsl.dataflow.dflow DEBUG: Task 3523 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3523 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3523 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3523 has memoization hash 929d59082d75d38f7c7ef34511107727 parsl.dataflow.memoization INFO: Task 3523 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3523 try 0 launched on executor multi with executor id 3524 parsl.dataflow.dflow INFO: Standard output for task 3523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/dbbb1d65-34ae-4ef0-8883-fc9649476104_isrForCrosstalkSources_997018_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/dbbb1d65-34ae-4ef0-8883-fc9649476104_isrForCrosstalkSources_997018_54.stderr parsl.dataflow.dflow DEBUG: Task 3524 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3524 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3524 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3524 has memoization hash ab7c478b73d7dc457e58a2b06ddb67d4 parsl.dataflow.memoization INFO: Task 3524 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3524 try 0 launched on executor multi with executor id 3525 parsl.dataflow.dflow INFO: Standard output for task 3524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1477a841-fa28-4ffc-8f9e-c1276e22d8da_isrForCrosstalkSources_997003_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1477a841-fa28-4ffc-8f9e-c1276e22d8da_isrForCrosstalkSources_997003_17.stderr parsl.dataflow.dflow DEBUG: Task 3525 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3525 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3525 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3525 has memoization hash 8114984d15c80e823a6bc2ad704e4cf9 parsl.dataflow.memoization INFO: Task 3525 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3525 try 0 launched on executor multi with executor id 3526 parsl.dataflow.dflow INFO: Standard output for task 3525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f09a5771-e29b-42be-96dc-d9245fd3c2a0_isrForCrosstalkSources_997031_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f09a5771-e29b-42be-96dc-d9245fd3c2a0_isrForCrosstalkSources_997031_43.stderr parsl.dataflow.dflow DEBUG: Task 3526 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3526 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3526 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3526 has memoization hash 5a0a04055dc110eeeee5b193fbf89de1 parsl.dataflow.memoization INFO: Task 3526 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4be020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3526 try 0 launched on executor multi with executor id 3527 parsl.dataflow.dflow INFO: Standard output for task 3526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e2625eaf-a3b5-4f58-b514-4d2beae5ddd6_isrForCrosstalkSources_997008_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e2625eaf-a3b5-4f58-b514-4d2beae5ddd6_isrForCrosstalkSources_997008_40.stderr parsl.dataflow.dflow DEBUG: Task 3527 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3527 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3527 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3527 has memoization hash 16d0064496f0d19d92f71d8bc3aa182f parsl.dataflow.memoization INFO: Task 3527 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bdf80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3527 try 0 launched on executor multi with executor id 3528 parsl.dataflow.dflow INFO: Standard output for task 3527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c083ba35-6f6c-45cb-97f9-3136f5252a4d_isrForCrosstalkSources_997003_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c083ba35-6f6c-45cb-97f9-3136f5252a4d_isrForCrosstalkSources_997003_3.stderr parsl.dataflow.dflow DEBUG: Task 3528 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3528 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3528 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3528 has memoization hash 1fed7c461593df09bf5c7c24bc05450e parsl.dataflow.memoization INFO: Task 3528 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bdee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3528 try 0 launched on executor multi with executor id 3529 parsl.dataflow.dflow INFO: Standard output for task 3528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0c79b297-d127-41f2-8e59-15835308c943_isrForCrosstalkSources_996985_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0c79b297-d127-41f2-8e59-15835308c943_isrForCrosstalkSources_996985_24.stderr parsl.dataflow.dflow DEBUG: Task 3529 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3529 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3529 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3529 has memoization hash 9ef150a35f63f6d9785aef9a9d29a338 parsl.dataflow.memoization INFO: Task 3529 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bdda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3529 try 0 launched on executor multi with executor id 3530 parsl.dataflow.dflow INFO: Standard output for task 3529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d6cd2d3d-847e-4de5-a7aa-22a2c518e986_isrForCrosstalkSources_996966_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d6cd2d3d-847e-4de5-a7aa-22a2c518e986_isrForCrosstalkSources_996966_30.stderr parsl.dataflow.dflow DEBUG: Task 3530 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3530 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3530 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3530 has memoization hash 3e1187baa059238aefb93aa80d39386e parsl.dataflow.memoization INFO: Task 3530 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bde40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3530 try 0 launched on executor multi with executor id 3531 parsl.dataflow.dflow INFO: Standard output for task 3530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9604bd4-6226-49e2-a657-0c5410e709ac_isrForCrosstalkSources_997009_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9604bd4-6226-49e2-a657-0c5410e709ac_isrForCrosstalkSources_997009_58.stderr parsl.dataflow.dflow DEBUG: Task 3531 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3531 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3531 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3531 has memoization hash b8a69d3893dd924c2e594169ef7aba49 parsl.dataflow.memoization INFO: Task 3531 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bdd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3531 try 0 launched on executor multi with executor id 3532 parsl.dataflow.dflow INFO: Standard output for task 3531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6b2ecbb5-a9fb-4a4a-9adc-c326536d06a8_isrForCrosstalkSources_996996_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6b2ecbb5-a9fb-4a4a-9adc-c326536d06a8_isrForCrosstalkSources_996996_51.stderr parsl.dataflow.dflow DEBUG: Task 3532 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3532 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3532 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3532 has memoization hash 88f7febffb127df7e66029f58c10b47d parsl.dataflow.memoization INFO: Task 3532 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bdc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3532 try 0 launched on executor multi with executor id 3533 parsl.dataflow.dflow INFO: Standard output for task 3532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/003d66fe-dbd5-4ee6-aa2e-0ff93efce1e6_isrForCrosstalkSources_996953_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/003d66fe-dbd5-4ee6-aa2e-0ff93efce1e6_isrForCrosstalkSources_996953_42.stderr parsl.dataflow.dflow DEBUG: Task 3533 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3533 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3533 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3533 has memoization hash fa74ea0098e999bc6320b32cfc270eef parsl.dataflow.memoization INFO: Task 3533 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bdbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3533 try 0 launched on executor multi with executor id 3534 parsl.dataflow.dflow INFO: Standard output for task 3533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/30905291-635d-440b-a542-ca59b0a238ba_isrForCrosstalkSources_997029_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/30905291-635d-440b-a542-ca59b0a238ba_isrForCrosstalkSources_997029_39.stderr parsl.dataflow.dflow DEBUG: Task 3534 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3534 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3534 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3534 has memoization hash e7be7965ce0608cad8c8bc69da981ab9 parsl.dataflow.memoization INFO: Task 3534 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bd9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3534 try 0 launched on executor multi with executor id 3535 parsl.dataflow.dflow INFO: Standard output for task 3534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/6a6fdc83-2378-4c9e-ba77-8063cadb7416_isrForCrosstalkSources_997008_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/6a6fdc83-2378-4c9e-ba77-8063cadb7416_isrForCrosstalkSources_997008_46.stderr parsl.dataflow.dflow DEBUG: Task 3535 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3535 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3535 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3535 has memoization hash 4adb67237f03484ab2446e788284be9a parsl.dataflow.memoization INFO: Task 3535 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bdb20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3535 try 0 launched on executor multi with executor id 3536 parsl.dataflow.dflow INFO: Standard output for task 3535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ecdccfd9-ee16-425e-ab26-d70fa15a4616_isrForCrosstalkSources_996994_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ecdccfd9-ee16-425e-ab26-d70fa15a4616_isrForCrosstalkSources_996994_37.stderr parsl.dataflow.dflow DEBUG: Task 3536 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3536 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3536 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3536 has memoization hash 1032370705cb3b26edfec9f6a7f0b1d8 parsl.dataflow.memoization INFO: Task 3536 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3536 try 0 launched on executor multi with executor id 3537 parsl.dataflow.dflow INFO: Standard output for task 3536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/8a12c7de-ac37-48fa-a23d-2e2dc9c0114f_isrForCrosstalkSources_996979_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/8a12c7de-ac37-48fa-a23d-2e2dc9c0114f_isrForCrosstalkSources_996979_41.stderr parsl.dataflow.dflow DEBUG: Task 3537 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3537 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3537 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3537 has memoization hash 9409387715b54c9788664b1135116b5f parsl.dataflow.memoization INFO: Task 3537 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4befc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3537 try 0 launched on executor multi with executor id 3538 parsl.dataflow.dflow INFO: Standard output for task 3537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/a2ce60f4-8fe6-44bb-a535-bce37781b350_isrForCrosstalkSources_996959_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/a2ce60f4-8fe6-44bb-a535-bce37781b350_isrForCrosstalkSources_996959_22.stderr parsl.dataflow.dflow DEBUG: Task 3538 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3538 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3538 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3538 has memoization hash 4e168aa77fc2773e7c9d6d2c1ac37abb parsl.dataflow.memoization INFO: Task 3538 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3538 try 0 launched on executor multi with executor id 3539 parsl.dataflow.dflow INFO: Standard output for task 3538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/583648eb-bb24-4e21-8f6b-6efa0a22e842_isrForCrosstalkSources_997006_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/583648eb-bb24-4e21-8f6b-6efa0a22e842_isrForCrosstalkSources_997006_26.stderr parsl.dataflow.dflow DEBUG: Task 3539 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3539 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3539 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3539 has memoization hash 7f0770d1ec72a89f65fb0f5e88a8bb40 parsl.dataflow.memoization INFO: Task 3539 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3539 try 0 launched on executor multi with executor id 3540 parsl.dataflow.dflow INFO: Standard output for task 3539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/713d38ed-d64e-49e7-814a-722675f38a1c_isrForCrosstalkSources_997026_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/713d38ed-d64e-49e7-814a-722675f38a1c_isrForCrosstalkSources_997026_12.stderr parsl.dataflow.dflow DEBUG: Task 3540 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3540 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3540 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3540 has memoization hash cdf0d1920d935c90da8ae490f453c8f6 parsl.dataflow.memoization INFO: Task 3540 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3540 try 0 launched on executor multi with executor id 3541 parsl.dataflow.dflow INFO: Standard output for task 3540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/c7c1b434-11b5-4cc9-966c-1ae6525f72fe_isrForCrosstalkSources_996974_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/c7c1b434-11b5-4cc9-966c-1ae6525f72fe_isrForCrosstalkSources_996974_21.stderr parsl.dataflow.dflow DEBUG: Task 3541 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3541 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3541 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3541 has memoization hash 7fc42594a28091963f1da043fd156269 parsl.dataflow.memoization INFO: Task 3541 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3541 try 0 launched on executor multi with executor id 3542 parsl.dataflow.dflow INFO: Standard output for task 3541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e8e5196c-1c0e-4e56-9773-52b5e22a6698_isrForCrosstalkSources_996956_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e8e5196c-1c0e-4e56-9773-52b5e22a6698_isrForCrosstalkSources_996956_35.stderr parsl.dataflow.dflow DEBUG: Task 3542 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3542 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3542 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3542 has memoization hash 767dbe8557b7a71ce9813ee7c59e04a1 parsl.dataflow.memoization INFO: Task 3542 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3542 try 0 launched on executor multi with executor id 3543 parsl.dataflow.dflow INFO: Standard output for task 3542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/50059f01-1ddb-48f0-b05b-72d170b811ed_isrForCrosstalkSources_997033_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/50059f01-1ddb-48f0-b05b-72d170b811ed_isrForCrosstalkSources_997033_40.stderr parsl.dataflow.dflow DEBUG: Task 3543 will be sent to executor multi 2024-03-15 13:49:59 proc_lsst.multi:170 [INFO] [multi] job_id 17074894 proc_lsst.multi INFO: [multi] job_id 17074894 2024-03-15 13:49:59 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:59 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:49:59 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074894 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074894 parsl.executors.status_handling DEBUG: Launched block 36 on executor multi with job ID 17074894 parsl.executors.status_handling INFO: Allocated block ID 37 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-37 2024-03-15 13:49:59 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-37 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-37 2024-03-15 13:49:59 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:49:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:49:59 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:49:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:49:59 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:49:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:49:59 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:49:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:49:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:49:59 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=37 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3543 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3543 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3543 has memoization hash 5b62e39e7d66b2a22d07ec56a5d9ec69 parsl.dataflow.memoization INFO: Task 3543 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3543 try 0 launched on executor multi with executor id 3544 parsl.dataflow.dflow INFO: Standard output for task 3543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/ad588edf-1131-4be6-b969-90472daffa27_isrForCrosstalkSources_996974_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/ad588edf-1131-4be6-b969-90472daffa27_isrForCrosstalkSources_996974_24.stderr parsl.dataflow.dflow DEBUG: Task 3544 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3544 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3544 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3544 has memoization hash 97e3df9031eab74363831e9c453e334e parsl.dataflow.memoization INFO: Task 3544 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4beca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3544 try 0 launched on executor multi with executor id 3545 parsl.dataflow.dflow INFO: Standard output for task 3544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/4fc8f5d5-7d0d-4739-b115-342f63bb1fad_isrForCrosstalkSources_996982_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/4fc8f5d5-7d0d-4739-b115-342f63bb1fad_isrForCrosstalkSources_996982_61.stderr parsl.dataflow.dflow DEBUG: Task 3545 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3545 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3545 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3545 has memoization hash 676b85437b1c1f7d36bc4b8c67438abd parsl.dataflow.memoization INFO: Task 3545 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3545 try 0 launched on executor multi with executor id 3546 parsl.dataflow.dflow INFO: Standard output for task 3545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f2ffdee1-688d-45da-bb79-ea02b280aa8d_isrForCrosstalkSources_997028_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f2ffdee1-688d-45da-bb79-ea02b280aa8d_isrForCrosstalkSources_997028_47.stderr parsl.dataflow.dflow DEBUG: Task 3546 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3546 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3546 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3546 has memoization hash 6243df00cea3b88651723e78bcf11cea parsl.dataflow.memoization INFO: Task 3546 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3546 try 0 launched on executor multi with executor id 3547 parsl.dataflow.dflow INFO: Standard output for task 3546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d2d6eb16-7fcc-46f5-bcd2-1a583ff0a1bf_isrForCrosstalkSources_997009_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d2d6eb16-7fcc-46f5-bcd2-1a583ff0a1bf_isrForCrosstalkSources_997009_6.stderr parsl.dataflow.dflow DEBUG: Task 3547 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3547 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3547 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3547 has memoization hash b2bea1a1e01f247efcd422b3d38093a0 parsl.dataflow.memoization INFO: Task 3547 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3547 try 0 launched on executor multi with executor id 3548 parsl.dataflow.dflow INFO: Standard output for task 3547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/4c73455a-44f1-4c82-a33b-c870f579f57c_isrForCrosstalkSources_997011_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/4c73455a-44f1-4c82-a33b-c870f579f57c_isrForCrosstalkSources_997011_5.stderr parsl.dataflow.dflow DEBUG: Task 3548 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3548 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3548 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3548 has memoization hash c3f34fa4a1b108f48e50eaeb0f0423de parsl.dataflow.memoization INFO: Task 3548 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3548 try 0 launched on executor multi with executor id 3549 parsl.dataflow.dflow INFO: Standard output for task 3548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/7e37b2da-4b3b-4123-b231-e3a508bdd068_isrForCrosstalkSources_996967_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/7e37b2da-4b3b-4123-b231-e3a508bdd068_isrForCrosstalkSources_996967_7.stderr parsl.dataflow.dflow DEBUG: Task 3549 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3549 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3549 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3549 has memoization hash baf9cb3e504d8fe3863afe71bd84ad4b parsl.dataflow.memoization INFO: Task 3549 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3549 try 0 launched on executor multi with executor id 3550 parsl.dataflow.dflow INFO: Standard output for task 3549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/891effa2-dac9-4e5e-bcbe-cb9aa20ce692_isrForCrosstalkSources_997008_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/891effa2-dac9-4e5e-bcbe-cb9aa20ce692_isrForCrosstalkSources_997008_45.stderr parsl.dataflow.dflow DEBUG: Task 3550 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3550 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3550 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3550 has memoization hash 5a69086b5a3f9a9dfb7af797d3ab3aaf parsl.dataflow.memoization INFO: Task 3550 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3550 try 0 launched on executor multi with executor id 3551 parsl.dataflow.dflow INFO: Standard output for task 3550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/903eef68-8311-4f60-9737-8c1a2ee25e9e_isrForCrosstalkSources_997026_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/903eef68-8311-4f60-9737-8c1a2ee25e9e_isrForCrosstalkSources_997026_52.stderr parsl.dataflow.dflow DEBUG: Task 3551 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3551 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3551 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3551 has memoization hash 2e8b89838ab7adca08636ccb648a1fad parsl.dataflow.memoization INFO: Task 3551 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3551 try 0 launched on executor multi with executor id 3552 parsl.dataflow.dflow INFO: Standard output for task 3551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/18f4f06b-f499-4726-aaef-67ad1816208c_isrForCrosstalkSources_996953_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/18f4f06b-f499-4726-aaef-67ad1816208c_isrForCrosstalkSources_996953_41.stderr parsl.dataflow.dflow DEBUG: Task 3552 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3552 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3552 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3552 has memoization hash 33ed8db261a8d04c86d51cbe3171add8 parsl.dataflow.memoization INFO: Task 3552 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bfa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3552 try 0 launched on executor multi with executor id 3553 parsl.dataflow.dflow INFO: Standard output for task 3552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fa9a6e44-2d50-443e-9077-62260c8eb794_isrForCrosstalkSources_996968_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fa9a6e44-2d50-443e-9077-62260c8eb794_isrForCrosstalkSources_996968_44.stderr parsl.dataflow.dflow DEBUG: Task 3553 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3553 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3553 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3553 has memoization hash 97a008c7b51f0ec6c5ffccae2d10ef4d parsl.dataflow.memoization INFO: Task 3553 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3553 try 0 launched on executor multi with executor id 3554 parsl.dataflow.dflow INFO: Standard output for task 3553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/c613ac2f-e14b-4f72-a8fc-059ffb9ede01_isrForCrosstalkSources_997026_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/c613ac2f-e14b-4f72-a8fc-059ffb9ede01_isrForCrosstalkSources_997026_34.stderr parsl.dataflow.dflow DEBUG: Task 3554 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3554 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3554 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3554 has memoization hash 7a8813ce160f0c62f2979a89ec8ac543 parsl.dataflow.memoization INFO: Task 3554 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bfba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3554 try 0 launched on executor multi with executor id 3555 parsl.dataflow.dflow INFO: Standard output for task 3554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/9c45b11c-9e0e-4074-9930-469f9f35d604_isrForCrosstalkSources_996991_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/9c45b11c-9e0e-4074-9930-469f9f35d604_isrForCrosstalkSources_996991_21.stderr parsl.dataflow.dflow DEBUG: Task 3555 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3555 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3555 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3555 has memoization hash 805a7704a64d690ed50228693ce9620a parsl.dataflow.memoization INFO: Task 3555 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bfb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3555 try 0 launched on executor multi with executor id 3556 parsl.dataflow.dflow INFO: Standard output for task 3555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/92ea1f2a-f1af-4104-b589-31653d1bd198_isrForCrosstalkSources_997032_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/92ea1f2a-f1af-4104-b589-31653d1bd198_isrForCrosstalkSources_997032_33.stderr parsl.dataflow.dflow DEBUG: Task 3556 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3556 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3556 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3556 has memoization hash f776dda90bfacd054f2b819e464ced90 parsl.dataflow.memoization INFO: Task 3556 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bfc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3556 try 0 launched on executor multi with executor id 3557 parsl.dataflow.dflow INFO: Standard output for task 3556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f98bae2d-0ef5-437f-ae87-9ab020c1bdb1_isrForCrosstalkSources_997015_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f98bae2d-0ef5-437f-ae87-9ab020c1bdb1_isrForCrosstalkSources_997015_6.stderr parsl.dataflow.dflow DEBUG: Task 3557 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3557 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3557 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3557 has memoization hash 225c8139a81a89e71c3504b37f824af6 parsl.dataflow.memoization INFO: Task 3557 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bfd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3557 try 0 launched on executor multi with executor id 3558 parsl.dataflow.dflow INFO: Standard output for task 3557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/77b7050f-af95-495a-be7e-b7cf05a78efb_isrForCrosstalkSources_996986_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/77b7050f-af95-495a-be7e-b7cf05a78efb_isrForCrosstalkSources_996986_24.stderr parsl.dataflow.dflow DEBUG: Task 3558 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3558 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3558 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3558 has memoization hash efd902f06e5d6b7c0c738aff11dc1669 parsl.dataflow.memoization INFO: Task 3558 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bfe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3558 try 0 launched on executor multi with executor id 3559 parsl.dataflow.dflow INFO: Standard output for task 3558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/20b46eaa-c899-4480-a5a2-6dab69b7b5e5_isrForCrosstalkSources_997016_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/20b46eaa-c899-4480-a5a2-6dab69b7b5e5_isrForCrosstalkSources_997016_37.stderr parsl.dataflow.dflow DEBUG: Task 3559 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3559 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3559 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3559 has memoization hash 53d850cb15ef159eb65cd4b8232ed3e9 parsl.dataflow.memoization INFO: Task 3559 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bfce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3559 try 0 launched on executor multi with executor id 3560 parsl.dataflow.dflow INFO: Standard output for task 3559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e0c244c9-3d33-40bf-ae6c-20ac6eeb53c8_isrForCrosstalkSources_997018_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e0c244c9-3d33-40bf-ae6c-20ac6eeb53c8_isrForCrosstalkSources_997018_56.stderr parsl.dataflow.dflow DEBUG: Task 3560 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3560 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3560 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3560 has memoization hash 15e288ffbe1093705e140cb25ed098c1 parsl.dataflow.memoization INFO: Task 3560 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bfec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3560 try 0 launched on executor multi with executor id 3561 parsl.dataflow.dflow INFO: Standard output for task 3560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f34f7fce-ef9a-4a0a-b93b-b1fc1b2b096f_isrForCrosstalkSources_996978_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f34f7fce-ef9a-4a0a-b93b-b1fc1b2b096f_isrForCrosstalkSources_996978_17.stderr parsl.dataflow.dflow DEBUG: Task 3561 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3561 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3561 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3561 has memoization hash 327f5f47acc380fe2b22afcff6813a57 parsl.dataflow.memoization INFO: Task 3561 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3561 try 0 launched on executor multi with executor id 3562 parsl.dataflow.dflow INFO: Standard output for task 3561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/c6c1c150-e46a-4064-9841-5bd0d989d6ba_isrForCrosstalkSources_997023_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/c6c1c150-e46a-4064-9841-5bd0d989d6ba_isrForCrosstalkSources_997023_46.stderr parsl.dataflow.dflow DEBUG: Task 3562 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3562 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3562 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3562 has memoization hash 9d62dc554345f0178b9fec2fec60f615 parsl.dataflow.memoization INFO: Task 3562 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3562 try 0 launched on executor multi with executor id 3563 parsl.dataflow.dflow INFO: Standard output for task 3562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a22cd30f-f4a0-46a7-930a-c88e5609a4ad_isrForCrosstalkSources_996964_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a22cd30f-f4a0-46a7-930a-c88e5609a4ad_isrForCrosstalkSources_996964_62.stderr parsl.dataflow.dflow DEBUG: Task 3563 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3563 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3563 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3563 has memoization hash 5f1e7e008f968408b14b02dd578d6216 parsl.dataflow.memoization INFO: Task 3563 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c4bf560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3563 try 0 launched on executor multi with executor id 3564 parsl.dataflow.dflow INFO: Standard output for task 3563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/9edc8f01-dd3f-468b-83cc-9487523da59b_isrForCrosstalkSources_996974_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/9edc8f01-dd3f-468b-83cc-9487523da59b_isrForCrosstalkSources_996974_30.stderr parsl.dataflow.dflow DEBUG: Task 3564 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3564 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3564 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3564 has memoization hash 96d3190a4a5d969542ab53c8a3ed11e0 parsl.dataflow.memoization INFO: Task 3564 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3564 try 0 launched on executor multi with executor id 3565 parsl.dataflow.dflow INFO: Standard output for task 3564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/4d681975-6371-4323-a149-001514e24109_isrForCrosstalkSources_996968_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/4d681975-6371-4323-a149-001514e24109_isrForCrosstalkSources_996968_31.stderr parsl.dataflow.dflow DEBUG: Task 3565 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3565 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3565 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3565 has memoization hash 05729c0f985263d258d60b12c5e4af24 parsl.dataflow.memoization INFO: Task 3565 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3565 try 0 launched on executor multi with executor id 3566 parsl.dataflow.dflow INFO: Standard output for task 3565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a14c3d8e-3d64-4090-a11c-d4a770c0559c_isrForCrosstalkSources_997022_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a14c3d8e-3d64-4090-a11c-d4a770c0559c_isrForCrosstalkSources_997022_10.stderr parsl.dataflow.dflow DEBUG: Task 3566 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3566 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3566 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3566 has memoization hash e85ecb01d6013852e1c5b447fc7a6503 parsl.dataflow.memoization INFO: Task 3566 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3566 try 0 launched on executor multi with executor id 3567 parsl.dataflow.dflow INFO: Standard output for task 3566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/5ff32d6b-eca8-4559-8f1c-f625f238af65_isrForCrosstalkSources_996974_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/5ff32d6b-eca8-4559-8f1c-f625f238af65_isrForCrosstalkSources_996974_60.stderr parsl.dataflow.dflow DEBUG: Task 3567 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3567 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3567 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3567 has memoization hash 847e6c9b33f08609081aa33ec4eb717b parsl.dataflow.memoization INFO: Task 3567 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3567 try 0 launched on executor multi with executor id 3568 parsl.dataflow.dflow INFO: Standard output for task 3567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6c2adae9-97a9-4242-b097-8173aa9b481c_isrForCrosstalkSources_996970_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6c2adae9-97a9-4242-b097-8173aa9b481c_isrForCrosstalkSources_996970_23.stderr parsl.dataflow.dflow DEBUG: Task 3568 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3568 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3568 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3568 has memoization hash d701ec350319a556faa44cf2c8f02250 parsl.dataflow.memoization INFO: Task 3568 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3568 try 0 launched on executor multi with executor id 3569 parsl.dataflow.dflow INFO: Standard output for task 3568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/074ce75a-919f-4d94-bc24-cd0f6d8eedab_isrForCrosstalkSources_996978_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/074ce75a-919f-4d94-bc24-cd0f6d8eedab_isrForCrosstalkSources_996978_28.stderr parsl.dataflow.dflow DEBUG: Task 3569 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3569 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3569 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3569 has memoization hash ac10b6425373f631a90c3e7926168437 parsl.dataflow.memoization INFO: Task 3569 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3569 try 0 launched on executor multi with executor id 3570 parsl.dataflow.dflow INFO: Standard output for task 3569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/375fd162-f343-4d4e-b78b-0ccec21d6fd2_isrForCrosstalkSources_997023_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/375fd162-f343-4d4e-b78b-0ccec21d6fd2_isrForCrosstalkSources_997023_62.stderr parsl.dataflow.dflow DEBUG: Task 3570 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3570 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3570 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3570 has memoization hash 281a4955f675255b6ffdd263a60a3e7b parsl.dataflow.memoization INFO: Task 3570 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3570 try 0 launched on executor multi with executor id 3571 parsl.dataflow.dflow INFO: Standard output for task 3570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/419648c4-580e-4354-88d8-a9dcb2e73fa4_isrForCrosstalkSources_996993_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/419648c4-580e-4354-88d8-a9dcb2e73fa4_isrForCrosstalkSources_996993_26.stderr parsl.dataflow.dflow DEBUG: Task 3571 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3571 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3571 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3571 has memoization hash f423a0cc7a9cd5f95cb6dc3ee65f3212 parsl.dataflow.memoization INFO: Task 3571 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3571 try 0 launched on executor multi with executor id 3572 parsl.dataflow.dflow INFO: Standard output for task 3571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/d63b0f0b-8f57-4a37-b51c-0a137a318880_isrForCrosstalkSources_997014_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/d63b0f0b-8f57-4a37-b51c-0a137a318880_isrForCrosstalkSources_997014_42.stderr parsl.dataflow.dflow DEBUG: Task 3572 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3572 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3572 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3572 has memoization hash 8414c074353edb3819db36af85621ec8 parsl.dataflow.memoization INFO: Task 3572 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3572 try 0 launched on executor multi with executor id 3573 parsl.dataflow.dflow INFO: Standard output for task 3572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/66cd768b-b7d5-4289-b1bb-7d4bc9e94611_isrForCrosstalkSources_997013_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/66cd768b-b7d5-4289-b1bb-7d4bc9e94611_isrForCrosstalkSources_997013_28.stderr parsl.dataflow.dflow DEBUG: Task 3573 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3573 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3573 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3573 has memoization hash 4fbd61cfc1f31a638290876b58a14db2 parsl.dataflow.memoization INFO: Task 3573 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3573 try 0 launched on executor multi with executor id 3574 parsl.dataflow.dflow INFO: Standard output for task 3573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/208afc97-32fe-4622-b552-06e43127a962_isrForCrosstalkSources_996990_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/208afc97-32fe-4622-b552-06e43127a962_isrForCrosstalkSources_996990_13.stderr parsl.dataflow.dflow DEBUG: Task 3574 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3574 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3574 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3574 has memoization hash e813982fd92d2d0233720ad724619ee4 parsl.dataflow.memoization INFO: Task 3574 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3574 try 0 launched on executor multi with executor id 3575 parsl.dataflow.dflow INFO: Standard output for task 3574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/3c471814-55d2-41a2-8a63-92005e074d60_isrForCrosstalkSources_996961_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/3c471814-55d2-41a2-8a63-92005e074d60_isrForCrosstalkSources_996961_9.stderr parsl.dataflow.dflow DEBUG: Task 3575 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3575 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3575 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3575 has memoization hash e2b456f249ff0c540c095afbab9dbf51 parsl.dataflow.memoization INFO: Task 3575 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3575 try 0 launched on executor multi with executor id 3576 parsl.dataflow.dflow INFO: Standard output for task 3575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/a59caae6-2065-42d1-bd15-30edaa63c3d2_isrForCrosstalkSources_996958_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/a59caae6-2065-42d1-bd15-30edaa63c3d2_isrForCrosstalkSources_996958_17.stderr parsl.dataflow.dflow DEBUG: Task 3576 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3576 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3576 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3576 has memoization hash 2bbe9eb3cc5d6a3ee69ca583210220a9 parsl.dataflow.memoization INFO: Task 3576 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3576 try 0 launched on executor multi with executor id 3577 parsl.dataflow.dflow INFO: Standard output for task 3576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/89529e65-0972-4e11-9156-5f9988d6e9dd_isrForCrosstalkSources_997009_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/89529e65-0972-4e11-9156-5f9988d6e9dd_isrForCrosstalkSources_997009_50.stderr parsl.dataflow.dflow DEBUG: Task 3577 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3577 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3577 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3577 has memoization hash 287aaec04477ce086d7a547402dd06b5 parsl.dataflow.memoization INFO: Task 3577 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3577 try 0 launched on executor multi with executor id 3578 parsl.dataflow.dflow INFO: Standard output for task 3577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ae2300f7-193d-4e27-9570-62aa6211e92c_isrForCrosstalkSources_997019_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ae2300f7-193d-4e27-9570-62aa6211e92c_isrForCrosstalkSources_997019_12.stderr parsl.dataflow.dflow DEBUG: Task 3578 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3578 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3578 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3578 has memoization hash df87bf3c518b1438db9a75a018d0a5df parsl.dataflow.memoization INFO: Task 3578 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3578 try 0 launched on executor multi with executor id 3579 parsl.dataflow.dflow INFO: Standard output for task 3578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/168709c3-a032-461b-a3ea-6d475f3429c4_isrForCrosstalkSources_997030_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/168709c3-a032-461b-a3ea-6d475f3429c4_isrForCrosstalkSources_997030_9.stderr parsl.dataflow.dflow DEBUG: Task 3579 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3579 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3579 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3579 has memoization hash ff6561eb21f6770acc21e44ef71a116c parsl.dataflow.memoization INFO: Task 3579 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3579 try 0 launched on executor multi with executor id 3580 parsl.dataflow.dflow INFO: Standard output for task 3579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/957dfbc4-cc06-4e46-b139-b62034d06850_isrForCrosstalkSources_996968_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/957dfbc4-cc06-4e46-b139-b62034d06850_isrForCrosstalkSources_996968_1.stderr parsl.dataflow.dflow DEBUG: Task 3580 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3580 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3580 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3580 has memoization hash 62e8fee018dcda6b8802a7673802a5eb parsl.dataflow.memoization INFO: Task 3580 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3580 try 0 launched on executor multi with executor id 3581 parsl.dataflow.dflow INFO: Standard output for task 3580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1a76a4d7-43ad-4670-9035-c9a267122720_isrForCrosstalkSources_997017_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1a76a4d7-43ad-4670-9035-c9a267122720_isrForCrosstalkSources_997017_32.stderr parsl.dataflow.dflow DEBUG: Task 3581 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3581 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3581 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3581 has memoization hash 5f214e5a12962b7c43043efbf8b6d8a1 parsl.dataflow.memoization INFO: Task 3581 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3581 try 0 launched on executor multi with executor id 3582 parsl.dataflow.dflow INFO: Standard output for task 3581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/188e5004-858a-4f53-b4d4-fcd1999fe01d_isrForCrosstalkSources_996953_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/188e5004-858a-4f53-b4d4-fcd1999fe01d_isrForCrosstalkSources_996953_29.stderr parsl.dataflow.dflow DEBUG: Task 3582 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3582 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3582 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3582 has memoization hash e1630c0fe46dad596cc92cfb22cbf904 parsl.dataflow.memoization INFO: Task 3582 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3582 try 0 launched on executor multi with executor id 3583 parsl.dataflow.dflow INFO: Standard output for task 3582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/262bfd13-e763-4610-8ce4-7c95fe45aa44_isrForCrosstalkSources_996981_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/262bfd13-e763-4610-8ce4-7c95fe45aa44_isrForCrosstalkSources_996981_52.stderr parsl.dataflow.dflow DEBUG: Task 3583 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3583 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3583 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3583 has memoization hash 15bd1a40a72bad6a3408d44e3c9cc77d parsl.dataflow.memoization INFO: Task 3583 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3583 try 0 launched on executor multi with executor id 3584 parsl.dataflow.dflow INFO: Standard output for task 3583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/1da408b4-d5fc-4288-a745-fe21c420c687_isrForCrosstalkSources_996975_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/1da408b4-d5fc-4288-a745-fe21c420c687_isrForCrosstalkSources_996975_14.stderr parsl.dataflow.dflow DEBUG: Task 3584 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3584 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3584 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3584 has memoization hash dfb2a8685ec99093a5d444242e1aa204 parsl.dataflow.memoization INFO: Task 3584 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3584 try 0 launched on executor multi with executor id 3585 parsl.dataflow.dflow INFO: Standard output for task 3584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c43455dc-344c-42c8-9105-a6057001c465_isrForCrosstalkSources_997021_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c43455dc-344c-42c8-9105-a6057001c465_isrForCrosstalkSources_997021_22.stderr parsl.dataflow.dflow DEBUG: Task 3585 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3585 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3585 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3585 has memoization hash 3f7d072b3b66232ebfda3db220d2c4a5 parsl.dataflow.memoization INFO: Task 3585 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3585 try 0 launched on executor multi with executor id 3586 parsl.dataflow.dflow INFO: Standard output for task 3585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/d31eb312-5e71-40f0-8376-2da8fccbd323_isrForCrosstalkSources_996953_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/d31eb312-5e71-40f0-8376-2da8fccbd323_isrForCrosstalkSources_996953_28.stderr parsl.dataflow.dflow DEBUG: Task 3586 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3586 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3586 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3586 has memoization hash 4090067cc504eb3cbacf89cd53b616ca parsl.dataflow.memoization INFO: Task 3586 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3586 try 0 launched on executor multi with executor id 3587 parsl.dataflow.dflow INFO: Standard output for task 3586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ffb4b679-a9e0-4a44-95a2-30d2f9788886_isrForCrosstalkSources_996992_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ffb4b679-a9e0-4a44-95a2-30d2f9788886_isrForCrosstalkSources_996992_41.stderr parsl.dataflow.dflow DEBUG: Task 3587 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3587 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3587 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3587 has memoization hash 5e9347829a435955f8c06d157dc8e2c9 parsl.dataflow.memoization INFO: Task 3587 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3587 try 0 launched on executor multi with executor id 3588 parsl.dataflow.dflow INFO: Standard output for task 3587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/329bfa40-c8d8-4ff5-84ea-47b804e2db33_isrForCrosstalkSources_996966_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/329bfa40-c8d8-4ff5-84ea-47b804e2db33_isrForCrosstalkSources_996966_48.stderr parsl.dataflow.dflow DEBUG: Task 3588 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3588 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3588 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3588 has memoization hash c119b8c2fc5c76558071dda5f775bcd2 parsl.dataflow.memoization INFO: Task 3588 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3588 try 0 launched on executor multi with executor id 3589 parsl.dataflow.dflow INFO: Standard output for task 3588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e896f523-2444-49aa-b289-b618dfa10d5d_isrForCrosstalkSources_996992_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e896f523-2444-49aa-b289-b618dfa10d5d_isrForCrosstalkSources_996992_10.stderr parsl.dataflow.dflow DEBUG: Task 3589 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3589 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3589 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3589 has memoization hash 7d7ec5cb8de12ff20e63d152cc80e449 parsl.dataflow.memoization INFO: Task 3589 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3589 try 0 launched on executor multi with executor id 3590 parsl.dataflow.dflow INFO: Standard output for task 3589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8b98ca34-a73f-4627-8bfa-774941f6a98d_isrForCrosstalkSources_996971_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8b98ca34-a73f-4627-8bfa-774941f6a98d_isrForCrosstalkSources_996971_35.stderr parsl.dataflow.dflow DEBUG: Task 3590 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3590 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3590 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3590 has memoization hash 5c06ce6ceec0735f5c098facd129ba0e parsl.dataflow.memoization INFO: Task 3590 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3590 try 0 launched on executor multi with executor id 3591 parsl.dataflow.dflow INFO: Standard output for task 3590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a5c039cb-a8ae-4452-a558-9406ca09b830_isrForCrosstalkSources_996999_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a5c039cb-a8ae-4452-a558-9406ca09b830_isrForCrosstalkSources_996999_59.stderr parsl.dataflow.dflow DEBUG: Task 3591 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3591 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3591 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3591 has memoization hash 46b1ecd96afa4a176bfac01815696d69 parsl.dataflow.memoization INFO: Task 3591 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3591 try 0 launched on executor multi with executor id 3592 parsl.dataflow.dflow INFO: Standard output for task 3591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/87254e1f-88bf-4880-a487-4139899716c8_isrForCrosstalkSources_996966_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/87254e1f-88bf-4880-a487-4139899716c8_isrForCrosstalkSources_996966_62.stderr parsl.dataflow.dflow DEBUG: Task 3592 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3592 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3592 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3592 has memoization hash 78f57e3ec7d1b55826a32fcc99053ccc parsl.dataflow.memoization INFO: Task 3592 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3592 try 0 launched on executor multi with executor id 3593 parsl.dataflow.dflow INFO: Standard output for task 3592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/cc242ef1-78ee-412e-9c03-12b01f6f5154_isrForCrosstalkSources_996998_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/cc242ef1-78ee-412e-9c03-12b01f6f5154_isrForCrosstalkSources_996998_44.stderr parsl.dataflow.dflow DEBUG: Task 3593 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3593 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3593 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3593 has memoization hash 55d7e20839b25ef118098893bbc4175f parsl.dataflow.memoization INFO: Task 3593 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3593 try 0 launched on executor multi with executor id 3594 parsl.dataflow.dflow INFO: Standard output for task 3593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7a976478-8e9e-4dd9-8557-29b16ca1ec12_isrForCrosstalkSources_996986_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7a976478-8e9e-4dd9-8557-29b16ca1ec12_isrForCrosstalkSources_996986_16.stderr parsl.dataflow.dflow DEBUG: Task 3594 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3594 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3594 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3594 has memoization hash 3e8b5dd9fba18b4e69abe8f1e2ffb05c parsl.dataflow.memoization INFO: Task 3594 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3594 try 0 launched on executor multi with executor id 3595 parsl.dataflow.dflow INFO: Standard output for task 3594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cc774f27-a6b6-4edf-9ed2-6dfa3ae063d3_isrForCrosstalkSources_996978_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cc774f27-a6b6-4edf-9ed2-6dfa3ae063d3_isrForCrosstalkSources_996978_41.stderr parsl.dataflow.dflow DEBUG: Task 3595 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3595 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3595 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3595 has memoization hash 0658f057610abf387abd14ced4888aad parsl.dataflow.memoization INFO: Task 3595 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3595 try 0 launched on executor multi with executor id 3596 parsl.dataflow.dflow INFO: Standard output for task 3595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/34c68e4f-74b6-4f2a-8bba-0c593e04e1ab_isrForCrosstalkSources_996961_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/34c68e4f-74b6-4f2a-8bba-0c593e04e1ab_isrForCrosstalkSources_996961_54.stderr parsl.dataflow.dflow DEBUG: Task 3596 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3596 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3596 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3596 has memoization hash 0ebf23dc91416628fb33767a567bfcee parsl.dataflow.memoization INFO: Task 3596 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3596 try 0 launched on executor multi with executor id 3597 parsl.dataflow.dflow INFO: Standard output for task 3596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ca1ebdb6-e451-4e16-ae46-93cafb72ea29_isrForCrosstalkSources_996984_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ca1ebdb6-e451-4e16-ae46-93cafb72ea29_isrForCrosstalkSources_996984_11.stderr parsl.dataflow.dflow DEBUG: Task 3597 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3597 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3597 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3597 has memoization hash 1fe2f6e0399ea60e0cd205ee42db4c61 parsl.dataflow.memoization INFO: Task 3597 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3597 try 0 launched on executor multi with executor id 3598 parsl.dataflow.dflow INFO: Standard output for task 3597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/f7e36e8c-98c4-4c4e-8fad-8af40d117864_isrForCrosstalkSources_997012_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/f7e36e8c-98c4-4c4e-8fad-8af40d117864_isrForCrosstalkSources_997012_38.stderr parsl.dataflow.dflow DEBUG: Task 3598 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3598 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3598 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3598 has memoization hash 6db9dce2c9418d6ec0d27e7b889f73a0 parsl.dataflow.memoization INFO: Task 3598 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3598 try 0 launched on executor multi with executor id 3599 parsl.dataflow.dflow INFO: Standard output for task 3598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/11a9f932-4ea5-46f3-a0ef-594f4892237b_isrForCrosstalkSources_996963_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/11a9f932-4ea5-46f3-a0ef-594f4892237b_isrForCrosstalkSources_996963_52.stderr parsl.dataflow.dflow DEBUG: Task 3599 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3599 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3599 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3599 has memoization hash 12f2ef811b60acdfb56bd09e4d09ebea parsl.dataflow.memoization INFO: Task 3599 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3599 try 0 launched on executor multi with executor id 3600 parsl.dataflow.dflow INFO: Standard output for task 3599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e8bd96ed-6a24-41c5-b20a-091f3d1bccb7_isrForCrosstalkSources_996989_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e8bd96ed-6a24-41c5-b20a-091f3d1bccb7_isrForCrosstalkSources_996989_47.stderr parsl.dataflow.dflow DEBUG: Task 3600 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3600 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3600 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3600 has memoization hash b2822fd28dc99bf19a8f28b6d60ad57e parsl.dataflow.memoization INFO: Task 3600 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3600 try 0 launched on executor multi with executor id 3601 parsl.dataflow.dflow INFO: Standard output for task 3600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/cdf1c10f-91c8-472a-aefc-3a0e4c5aa197_isrForCrosstalkSources_997009_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/cdf1c10f-91c8-472a-aefc-3a0e4c5aa197_isrForCrosstalkSources_997009_19.stderr parsl.dataflow.dflow DEBUG: Task 3601 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3601 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3601 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3601 has memoization hash 8e0f0859f8ce06af19d6a1e3cd8fecb9 parsl.dataflow.memoization INFO: Task 3601 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3601 try 0 launched on executor multi with executor id 3602 parsl.dataflow.dflow INFO: Standard output for task 3601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e35254a4-549e-4710-9c0a-2cc5f7802265_isrForCrosstalkSources_997010_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e35254a4-549e-4710-9c0a-2cc5f7802265_isrForCrosstalkSources_997010_39.stderr parsl.dataflow.dflow DEBUG: Task 3602 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3602 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3602 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3602 has memoization hash 92bbbdc8c6cf2c8053d040ca18b642cd parsl.dataflow.memoization INFO: Task 3602 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3602 try 0 launched on executor multi with executor id 3603 parsl.dataflow.dflow INFO: Standard output for task 3602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/e5df4e1d-dc37-4a62-b508-069c8bb807df_isrForCrosstalkSources_997003_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/e5df4e1d-dc37-4a62-b508-069c8bb807df_isrForCrosstalkSources_997003_12.stderr parsl.dataflow.dflow DEBUG: Task 3603 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3603 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3603 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3603 has memoization hash 266e504ad1a860684a7763abf32f9802 parsl.dataflow.memoization INFO: Task 3603 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3603 try 0 launched on executor multi with executor id 3604 parsl.dataflow.dflow INFO: Standard output for task 3603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f562c574-e098-4ce0-9880-405c187cf3f0_isrForCrosstalkSources_996961_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f562c574-e098-4ce0-9880-405c187cf3f0_isrForCrosstalkSources_996961_62.stderr parsl.dataflow.dflow DEBUG: Task 3604 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3604 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3604 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3604 has memoization hash 7e2409e88020e917b3048517913639c6 parsl.dataflow.memoization INFO: Task 3604 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3604 try 0 launched on executor multi with executor id 3605 parsl.dataflow.dflow INFO: Standard output for task 3604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8671b32e-596f-46e4-81aa-e67afaefa3fc_isrForCrosstalkSources_996992_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8671b32e-596f-46e4-81aa-e67afaefa3fc_isrForCrosstalkSources_996992_20.stderr parsl.dataflow.dflow DEBUG: Task 3605 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3605 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3605 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3605 has memoization hash c3fb0e85080dd4e0c0335a58aedc80e4 parsl.dataflow.memoization INFO: Task 3605 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3605 try 0 launched on executor multi with executor id 3606 parsl.dataflow.dflow INFO: Standard output for task 3605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ef38add7-517f-4cdc-8aa6-5799be8d7423_isrForCrosstalkSources_997019_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ef38add7-517f-4cdc-8aa6-5799be8d7423_isrForCrosstalkSources_997019_40.stderr parsl.dataflow.dflow DEBUG: Task 3606 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3606 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3606 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3606 has memoization hash 61cb73783fd704098901d8e1adb30821 parsl.dataflow.memoization INFO: Task 3606 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3606 try 0 launched on executor multi with executor id 3607 parsl.dataflow.dflow INFO: Standard output for task 3606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/321b89c8-740e-4263-b858-0aec21f3c81e_isrForCrosstalkSources_997002_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/321b89c8-740e-4263-b858-0aec21f3c81e_isrForCrosstalkSources_997002_20.stderr parsl.dataflow.dflow DEBUG: Task 3607 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3607 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3607 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3607 has memoization hash 841100a9aa51762cedfff8fb60ab2052 parsl.dataflow.memoization INFO: Task 3607 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3607 try 0 launched on executor multi with executor id 3608 parsl.dataflow.dflow INFO: Standard output for task 3607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/43030cd1-d15a-434f-8e9f-9fb9ee71c0c2_isrForCrosstalkSources_996988_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/43030cd1-d15a-434f-8e9f-9fb9ee71c0c2_isrForCrosstalkSources_996988_43.stderr parsl.dataflow.dflow DEBUG: Task 3608 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3608 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3608 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3608 has memoization hash ddae82b4ef9b50cc17d3f544bca534b3 parsl.dataflow.memoization INFO: Task 3608 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3608 try 0 launched on executor multi with executor id 3609 parsl.dataflow.dflow INFO: Standard output for task 3608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/4b87311a-8412-4dc6-816b-0446682ded0a_isrForCrosstalkSources_997029_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/4b87311a-8412-4dc6-816b-0446682ded0a_isrForCrosstalkSources_997029_24.stderr parsl.dataflow.dflow DEBUG: Task 3609 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3609 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3609 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3609 has memoization hash 96593686e1154e80d8335419af2f92a2 parsl.dataflow.memoization INFO: Task 3609 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3609 try 0 launched on executor multi with executor id 3610 parsl.dataflow.dflow INFO: Standard output for task 3609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/47d5f179-83a7-4f4f-a754-f088724160d5_isrForCrosstalkSources_996956_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/47d5f179-83a7-4f4f-a754-f088724160d5_isrForCrosstalkSources_996956_6.stderr parsl.dataflow.dflow DEBUG: Task 3610 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3610 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3610 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3610 has memoization hash 272382b3f1a6e19cdaa08dbdbc70cfd5 parsl.dataflow.memoization INFO: Task 3610 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3610 try 0 launched on executor multi with executor id 3611 parsl.dataflow.dflow INFO: Standard output for task 3610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/7f265e9d-0d8e-4119-bd9f-3287842cd279_isrForCrosstalkSources_996956_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/7f265e9d-0d8e-4119-bd9f-3287842cd279_isrForCrosstalkSources_996956_55.stderr parsl.dataflow.dflow DEBUG: Task 3611 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3611 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3611 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3611 has memoization hash 5d481f55d00b3787a4da1bc50a33457a parsl.dataflow.memoization INFO: Task 3611 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3611 try 0 launched on executor multi with executor id 3612 parsl.dataflow.dflow INFO: Standard output for task 3611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/bd91a66c-da8f-4e60-8fdd-ef57b744e83c_isrForCrosstalkSources_996961_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/bd91a66c-da8f-4e60-8fdd-ef57b744e83c_isrForCrosstalkSources_996961_25.stderr parsl.dataflow.dflow DEBUG: Task 3612 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3612 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3612 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3612 has memoization hash 0cebed5315b7a909a6d9ff6fb7f6f252 parsl.dataflow.memoization INFO: Task 3612 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3612 try 0 launched on executor multi with executor id 3613 parsl.dataflow.dflow INFO: Standard output for task 3612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/38d5455b-bae9-43a6-b393-eab9a31ebe31_isrForCrosstalkSources_996977_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/38d5455b-bae9-43a6-b393-eab9a31ebe31_isrForCrosstalkSources_996977_35.stderr parsl.dataflow.dflow DEBUG: Task 3613 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3613 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3613 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3613 has memoization hash eacde9ffc29ea3d6a85b30f31061df45 parsl.dataflow.memoization INFO: Task 3613 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3613 try 0 launched on executor multi with executor id 3614 parsl.dataflow.dflow INFO: Standard output for task 3613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1e392b2e-9d3a-45da-89fc-f69df52daca1_isrForCrosstalkSources_996960_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1e392b2e-9d3a-45da-89fc-f69df52daca1_isrForCrosstalkSources_996960_54.stderr parsl.dataflow.dflow DEBUG: Task 3614 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3614 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3614 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3614 has memoization hash 175e96bfc66f87dc17028dbaa1361dd9 parsl.dataflow.memoization INFO: Task 3614 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3614 try 0 launched on executor multi with executor id 3615 parsl.dataflow.dflow INFO: Standard output for task 3614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/36be853a-11a1-4ea9-b475-240def996622_isrForCrosstalkSources_996975_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/36be853a-11a1-4ea9-b475-240def996622_isrForCrosstalkSources_996975_53.stderr parsl.dataflow.dflow DEBUG: Task 3615 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3615 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3615 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3615 has memoization hash ed5abfcc18d5713f8526fe45bfc611d0 parsl.dataflow.memoization INFO: Task 3615 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3615 try 0 launched on executor multi with executor id 3616 parsl.dataflow.dflow INFO: Standard output for task 3615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/67db2923-b436-478c-9e33-945171411f03_isrForCrosstalkSources_997003_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/67db2923-b436-478c-9e33-945171411f03_isrForCrosstalkSources_997003_48.stderr parsl.dataflow.dflow DEBUG: Task 3616 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3616 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3616 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3616 has memoization hash 9510d50fd7c744270b616473c26e29f6 parsl.dataflow.memoization INFO: Task 3616 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3616 try 0 launched on executor multi with executor id 3617 parsl.dataflow.dflow INFO: Standard output for task 3616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/1d6ecadf-9368-4595-b275-e13c2d3e2ef8_isrForCrosstalkSources_997021_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/1d6ecadf-9368-4595-b275-e13c2d3e2ef8_isrForCrosstalkSources_997021_55.stderr parsl.dataflow.dflow DEBUG: Task 3617 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3617 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3617 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3617 has memoization hash de2070448298fd1ff833aa099d270d4a parsl.dataflow.memoization INFO: Task 3617 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3617 try 0 launched on executor multi with executor id 3618 parsl.dataflow.dflow INFO: Standard output for task 3617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/b1b9303a-082f-4987-bfd8-9ec125960f47_isrForCrosstalkSources_996967_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/b1b9303a-082f-4987-bfd8-9ec125960f47_isrForCrosstalkSources_996967_11.stderr parsl.dataflow.dflow DEBUG: Task 3618 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3618 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3618 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3618 has memoization hash 020f579762f27abd02af57372bd5721f parsl.dataflow.memoization INFO: Task 3618 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3618 try 0 launched on executor multi with executor id 3619 parsl.dataflow.dflow INFO: Standard output for task 3618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/60acf02d-3de7-4760-ab38-9e26d9ff20af_isrForCrosstalkSources_997015_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/60acf02d-3de7-4760-ab38-9e26d9ff20af_isrForCrosstalkSources_997015_52.stderr parsl.dataflow.dflow DEBUG: Task 3619 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3619 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3619 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3619 has memoization hash 5d78245ed56e36f6b146c472f37ef5cd parsl.dataflow.memoization INFO: Task 3619 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3619 try 0 launched on executor multi with executor id 3620 parsl.dataflow.dflow INFO: Standard output for task 3619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b4c10945-e679-4d34-83ce-3f7f732971fe_isrForCrosstalkSources_996998_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b4c10945-e679-4d34-83ce-3f7f732971fe_isrForCrosstalkSources_996998_52.stderr parsl.dataflow.dflow DEBUG: Task 3620 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3620 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3620 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3620 has memoization hash 0a79ad3dfaa514bf9882f3deff588acf parsl.dataflow.memoization INFO: Task 3620 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3620 try 0 launched on executor multi with executor id 3621 parsl.dataflow.dflow INFO: Standard output for task 3620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e9097ce8-8eeb-4dc5-b89a-17313312ed6f_isrForCrosstalkSources_996961_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e9097ce8-8eeb-4dc5-b89a-17313312ed6f_isrForCrosstalkSources_996961_30.stderr parsl.dataflow.dflow DEBUG: Task 3621 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3621 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3621 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3621 has memoization hash 8002e1ecc3fdf98f2a60a1fceb195bdb parsl.dataflow.memoization INFO: Task 3621 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3621 try 0 launched on executor multi with executor id 3622 parsl.dataflow.dflow INFO: Standard output for task 3621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/500b9fd6-4cad-475f-8a9d-734f5d3720cc_isrForCrosstalkSources_997006_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/500b9fd6-4cad-475f-8a9d-734f5d3720cc_isrForCrosstalkSources_997006_37.stderr parsl.dataflow.dflow DEBUG: Task 3622 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3622 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3622 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3622 has memoization hash 56096dbc2715850a4cee5bb3b38cc5d1 parsl.dataflow.memoization INFO: Task 3622 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3622 try 0 launched on executor multi with executor id 3623 parsl.dataflow.dflow INFO: Standard output for task 3622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c32149ed-2163-4ee2-bf44-d997f5426496_isrForCrosstalkSources_997003_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c32149ed-2163-4ee2-bf44-d997f5426496_isrForCrosstalkSources_997003_44.stderr parsl.dataflow.dflow DEBUG: Task 3623 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3623 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3623 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3623 has memoization hash dd2abca199be81e8fb76c8694f98e26e parsl.dataflow.memoization INFO: Task 3623 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3623 try 0 launched on executor multi with executor id 3624 parsl.dataflow.dflow INFO: Standard output for task 3623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/32faea51-5282-4903-b9ba-93221fa793cc_isrForCrosstalkSources_997025_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/32faea51-5282-4903-b9ba-93221fa793cc_isrForCrosstalkSources_997025_61.stderr parsl.dataflow.dflow DEBUG: Task 3624 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3624 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3624 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3624 has memoization hash 67712dd9a07a0bbe464692932b798947 parsl.dataflow.memoization INFO: Task 3624 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3624 try 0 launched on executor multi with executor id 3625 parsl.dataflow.dflow INFO: Standard output for task 3624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/feabd467-5067-4cac-9c1d-9a337a017660_isrForCrosstalkSources_996978_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/feabd467-5067-4cac-9c1d-9a337a017660_isrForCrosstalkSources_996978_52.stderr parsl.dataflow.dflow DEBUG: Task 3625 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3625 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3625 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3625 has memoization hash c7e9b1fa0fd4b4f2a53105b556f99873 parsl.dataflow.memoization INFO: Task 3625 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3625 try 0 launched on executor multi with executor id 3626 parsl.dataflow.dflow INFO: Standard output for task 3625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7087d550-d870-4c11-9330-e832f3bffb6c_isrForCrosstalkSources_996974_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7087d550-d870-4c11-9330-e832f3bffb6c_isrForCrosstalkSources_996974_7.stderr parsl.dataflow.dflow DEBUG: Task 3626 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3626 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3626 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3626 has memoization hash 9b15c5404671dcf05d99992bcee63821 parsl.dataflow.memoization INFO: Task 3626 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3626 try 0 launched on executor multi with executor id 3627 parsl.dataflow.dflow INFO: Standard output for task 3626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/6338a5b1-7544-4084-95ae-5f9c94e8d7d8_isrForCrosstalkSources_996977_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/6338a5b1-7544-4084-95ae-5f9c94e8d7d8_isrForCrosstalkSources_996977_45.stderr parsl.dataflow.dflow DEBUG: Task 3627 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3627 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3627 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3627 has memoization hash a0a49410a3509305ed43e3e3c8042bc3 parsl.dataflow.memoization INFO: Task 3627 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3627 try 0 launched on executor multi with executor id 3628 parsl.dataflow.dflow INFO: Standard output for task 3627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7da84138-18e2-44c6-9916-2c21162835c0_isrForCrosstalkSources_997027_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7da84138-18e2-44c6-9916-2c21162835c0_isrForCrosstalkSources_997027_12.stderr parsl.dataflow.dflow DEBUG: Task 3628 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3628 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3628 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3628 has memoization hash f094742a7fd9be697ccc3dd91eaa68c5 parsl.dataflow.memoization INFO: Task 3628 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3628 try 0 launched on executor multi with executor id 3629 parsl.dataflow.dflow INFO: Standard output for task 3628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/da8ed39a-d1cf-4414-9a1d-650e10e3bb52_isrForCrosstalkSources_997009_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/da8ed39a-d1cf-4414-9a1d-650e10e3bb52_isrForCrosstalkSources_997009_5.stderr parsl.dataflow.dflow DEBUG: Task 3629 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3629 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3629 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3629 has memoization hash f9407015c3e56ce99f1f4d02819d6b60 parsl.dataflow.memoization INFO: Task 3629 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3629 try 0 launched on executor multi with executor id 3630 parsl.dataflow.dflow INFO: Standard output for task 3629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/29c58776-1b31-48dd-948d-ebb7f88a153a_isrForCrosstalkSources_996983_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/29c58776-1b31-48dd-948d-ebb7f88a153a_isrForCrosstalkSources_996983_4.stderr parsl.dataflow.dflow DEBUG: Task 3630 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3630 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3630 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3630 has memoization hash 5fcf30e2de78c6aa0dda01f35279b244 parsl.dataflow.memoization INFO: Task 3630 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3630 try 0 launched on executor multi with executor id 3631 parsl.dataflow.dflow INFO: Standard output for task 3630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/57f30ef9-a085-4b9f-8111-61f071177f26_isrForCrosstalkSources_996989_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/57f30ef9-a085-4b9f-8111-61f071177f26_isrForCrosstalkSources_996989_22.stderr parsl.dataflow.dflow DEBUG: Task 3631 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3631 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3631 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3631 has memoization hash d735972e84f7ea3362fa2b50240b24f7 parsl.dataflow.memoization INFO: Task 3631 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3631 try 0 launched on executor multi with executor id 3632 parsl.dataflow.dflow INFO: Standard output for task 3631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/2eb3e35d-12d9-4233-a755-bd762ddf4a94_isrForCrosstalkSources_997033_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/2eb3e35d-12d9-4233-a755-bd762ddf4a94_isrForCrosstalkSources_997033_1.stderr parsl.dataflow.dflow DEBUG: Task 3632 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3632 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3632 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3632 has memoization hash 544f2526a60031b5e8b3fcd6736288d4 parsl.dataflow.memoization INFO: Task 3632 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3632 try 0 launched on executor multi with executor id 3633 parsl.dataflow.dflow INFO: Standard output for task 3632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b689fdd8-8878-439c-ae2c-fa39dfc6bf2b_isrForCrosstalkSources_996964_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b689fdd8-8878-439c-ae2c-fa39dfc6bf2b_isrForCrosstalkSources_996964_41.stderr parsl.dataflow.dflow DEBUG: Task 3633 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3633 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3633 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3633 has memoization hash f4e6a0a5bbb1bac0cd260f5e82fb89de parsl.dataflow.memoization INFO: Task 3633 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3633 try 0 launched on executor multi with executor id 3634 parsl.dataflow.dflow INFO: Standard output for task 3633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/182c2d24-5493-466f-a162-81330283a8bb_isrForCrosstalkSources_996959_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/182c2d24-5493-466f-a162-81330283a8bb_isrForCrosstalkSources_996959_4.stderr parsl.dataflow.dflow DEBUG: Task 3634 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3634 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3634 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3634 has memoization hash af981772b1b47b58dd6c9308eba209f2 parsl.dataflow.memoization INFO: Task 3634 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3634 try 0 launched on executor multi with executor id 3635 parsl.dataflow.dflow INFO: Standard output for task 3634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/4a032281-d50e-4d98-b4a1-da1012f7ef67_isrForCrosstalkSources_997029_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/4a032281-d50e-4d98-b4a1-da1012f7ef67_isrForCrosstalkSources_997029_27.stderr parsl.dataflow.dflow DEBUG: Task 3635 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3635 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3635 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3635 has memoization hash 9b500e6beabc7dc157093c5c5d7a318a parsl.dataflow.memoization INFO: Task 3635 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3635 try 0 launched on executor multi with executor id 3636 parsl.dataflow.dflow INFO: Standard output for task 3635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/1b47a674-d48e-4ddc-8393-3ab6f6cfed58_isrForCrosstalkSources_997018_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/1b47a674-d48e-4ddc-8393-3ab6f6cfed58_isrForCrosstalkSources_997018_27.stderr parsl.dataflow.dflow DEBUG: Task 3636 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3636 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3636 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3636 has memoization hash b6bc5413443b4edad9deb977a35fdb42 parsl.dataflow.memoization INFO: Task 3636 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3636 try 0 launched on executor multi with executor id 3637 parsl.dataflow.dflow INFO: Standard output for task 3636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/9df6fcda-9dda-49e9-ac24-13f261d68cda_isrForCrosstalkSources_996989_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/9df6fcda-9dda-49e9-ac24-13f261d68cda_isrForCrosstalkSources_996989_37.stderr parsl.dataflow.dflow DEBUG: Task 3637 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3637 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3637 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3637 has memoization hash a8f1b982dfb5d176915c087b27021f02 parsl.dataflow.memoization INFO: Task 3637 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3637 try 0 launched on executor multi with executor id 3638 parsl.dataflow.dflow INFO: Standard output for task 3637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/e871a168-01fb-4c48-b5e8-6d67dc9af9e9_isrForCrosstalkSources_996958_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/e871a168-01fb-4c48-b5e8-6d67dc9af9e9_isrForCrosstalkSources_996958_3.stderr parsl.dataflow.dflow DEBUG: Task 3638 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3638 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3638 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3638 has memoization hash 786769ab257d52921f809d9c40edb5e6 parsl.dataflow.memoization INFO: Task 3638 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3638 try 0 launched on executor multi with executor id 3639 parsl.dataflow.dflow INFO: Standard output for task 3638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f1d2ea8a-13a3-4c82-bc09-a370ead4f81c_isrForCrosstalkSources_997024_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f1d2ea8a-13a3-4c82-bc09-a370ead4f81c_isrForCrosstalkSources_997024_5.stderr parsl.dataflow.dflow DEBUG: Task 3639 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3639 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3639 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3639 has memoization hash a79df5a5817a3129eacffd2757f54ab3 parsl.dataflow.memoization INFO: Task 3639 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3639 try 0 launched on executor multi with executor id 3640 parsl.dataflow.dflow INFO: Standard output for task 3639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7b6d7d7e-c5c1-4315-8533-c92ce34843ac_isrForCrosstalkSources_996968_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7b6d7d7e-c5c1-4315-8533-c92ce34843ac_isrForCrosstalkSources_996968_6.stderr parsl.dataflow.dflow DEBUG: Task 3640 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3640 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3640 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3640 has memoization hash 454542a1b8ac86d21b34606fc526e663 parsl.dataflow.memoization INFO: Task 3640 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3640 try 0 launched on executor multi with executor id 3641 parsl.dataflow.dflow INFO: Standard output for task 3640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b7cee98a-ae23-40d4-ac2a-a478dc17d5a9_isrForCrosstalkSources_997022_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b7cee98a-ae23-40d4-ac2a-a478dc17d5a9_isrForCrosstalkSources_997022_4.stderr parsl.dataflow.dflow DEBUG: Task 3641 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3641 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3641 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3641 has memoization hash 57c0063dbd46d7197607b0da12bc82ce parsl.dataflow.memoization INFO: Task 3641 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3641 try 0 launched on executor multi with executor id 3642 parsl.dataflow.dflow INFO: Standard output for task 3641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b2b32dae-f197-4493-8e0d-85d5ce1eb7c3_isrForCrosstalkSources_996959_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b2b32dae-f197-4493-8e0d-85d5ce1eb7c3_isrForCrosstalkSources_996959_49.stderr parsl.dataflow.dflow DEBUG: Task 3642 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3642 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3642 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3642 has memoization hash 913063731b38c10c691cdaf360522440 parsl.dataflow.memoization INFO: Task 3642 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3642 try 0 launched on executor multi with executor id 3643 parsl.dataflow.dflow INFO: Standard output for task 3642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fb458501-9768-454f-94dd-0b712bccb1db_isrForCrosstalkSources_996990_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fb458501-9768-454f-94dd-0b712bccb1db_isrForCrosstalkSources_996990_18.stderr parsl.dataflow.dflow DEBUG: Task 3643 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3643 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3643 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3643 has memoization hash 9828eb72809da65e3a363d4f7771dbea parsl.dataflow.memoization INFO: Task 3643 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3643 try 0 launched on executor multi with executor id 3644 parsl.dataflow.dflow INFO: Standard output for task 3643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/9fd0d4a8-2b9b-42eb-9151-501a14fb4514_isrForCrosstalkSources_996998_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/9fd0d4a8-2b9b-42eb-9151-501a14fb4514_isrForCrosstalkSources_996998_51.stderr parsl.dataflow.dflow DEBUG: Task 3644 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3644 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3644 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3644 has memoization hash 4789bfaa84c7ce844553fa36a17c993b parsl.dataflow.memoization INFO: Task 3644 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3644 try 0 launched on executor multi with executor id 3645 parsl.dataflow.dflow INFO: Standard output for task 3644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/2c8a2c44-22a8-4337-ae3e-aed256b5c2d9_isrForCrosstalkSources_996978_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/2c8a2c44-22a8-4337-ae3e-aed256b5c2d9_isrForCrosstalkSources_996978_45.stderr parsl.dataflow.dflow DEBUG: Task 3645 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3645 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3645 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3645 has memoization hash 46796cb9fbecc0e2ae7c066a7df51be2 parsl.dataflow.memoization INFO: Task 3645 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3645 try 0 launched on executor multi with executor id 3646 parsl.dataflow.dflow INFO: Standard output for task 3645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d58562ef-3cf2-4f1d-8cb5-357d96776384_isrForCrosstalkSources_996973_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d58562ef-3cf2-4f1d-8cb5-357d96776384_isrForCrosstalkSources_996973_16.stderr parsl.dataflow.dflow DEBUG: Task 3646 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3646 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3646 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3646 has memoization hash 4e00c51d98a1c19848156071101e2374 parsl.dataflow.memoization INFO: Task 3646 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3646 try 0 launched on executor multi with executor id 3647 parsl.dataflow.dflow INFO: Standard output for task 3646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/bce40ee1-93d4-4e10-9b4a-3a1e00726c04_isrForCrosstalkSources_996985_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/bce40ee1-93d4-4e10-9b4a-3a1e00726c04_isrForCrosstalkSources_996985_43.stderr parsl.dataflow.dflow DEBUG: Task 3647 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3647 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3647 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3647 has memoization hash 49cb588faccd3a56c67493fd91102dac parsl.dataflow.memoization INFO: Task 3647 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3647 try 0 launched on executor multi with executor id 3648 parsl.dataflow.dflow INFO: Standard output for task 3647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0b3ed7b1-b92c-480d-8fac-18fba4f79c55_isrForCrosstalkSources_997028_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0b3ed7b1-b92c-480d-8fac-18fba4f79c55_isrForCrosstalkSources_997028_35.stderr parsl.dataflow.dflow DEBUG: Task 3648 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3648 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3648 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3648 has memoization hash 1487ad9d7cce7ec59f82ce27b5605a73 parsl.dataflow.memoization INFO: Task 3648 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3648 try 0 launched on executor multi with executor id 3649 parsl.dataflow.dflow INFO: Standard output for task 3648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/90e6167a-d74e-4468-bc31-4a91a8dfb96a_isrForCrosstalkSources_996972_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/90e6167a-d74e-4468-bc31-4a91a8dfb96a_isrForCrosstalkSources_996972_33.stderr parsl.dataflow.dflow DEBUG: Task 3649 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3649 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3649 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3649 has memoization hash 1bdc228b261b03c7711408fd3ad1a69b parsl.dataflow.memoization INFO: Task 3649 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3649 try 0 launched on executor multi with executor id 3650 parsl.dataflow.dflow INFO: Standard output for task 3649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0e59bb08-4589-4e75-800f-bc090e04692b_isrForCrosstalkSources_996956_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0e59bb08-4589-4e75-800f-bc090e04692b_isrForCrosstalkSources_996956_18.stderr parsl.dataflow.dflow DEBUG: Task 3650 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3650 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3650 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3650 has memoization hash 9a12f0e8eee17859866d8f795344c572 parsl.dataflow.memoization INFO: Task 3650 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3650 try 0 launched on executor multi with executor id 3651 parsl.dataflow.dflow INFO: Standard output for task 3650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/984b97a0-7fe9-46d9-a124-4fe80f7525e9_isrForCrosstalkSources_996979_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/984b97a0-7fe9-46d9-a124-4fe80f7525e9_isrForCrosstalkSources_996979_12.stderr parsl.dataflow.dflow DEBUG: Task 3651 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3651 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3651 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3651 has memoization hash aeac6e874fa2cd0d078f9f20a6514466 parsl.dataflow.memoization INFO: Task 3651 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3651 try 0 launched on executor multi with executor id 3652 parsl.dataflow.dflow INFO: Standard output for task 3651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d0dff02d-082e-4aaf-bc95-9bc80354072d_isrForCrosstalkSources_997024_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d0dff02d-082e-4aaf-bc95-9bc80354072d_isrForCrosstalkSources_997024_10.stderr parsl.dataflow.dflow DEBUG: Task 3652 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3652 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3652 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3652 has memoization hash 7ae3a41875e6582424752b63846b6dc3 parsl.dataflow.memoization INFO: Task 3652 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3652 try 0 launched on executor multi with executor id 3653 parsl.dataflow.dflow INFO: Standard output for task 3652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/1d8f2348-2599-472c-8c2c-614a5587237b_isrForCrosstalkSources_997029_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/1d8f2348-2599-472c-8c2c-614a5587237b_isrForCrosstalkSources_997029_45.stderr parsl.dataflow.dflow DEBUG: Task 3653 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3653 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3653 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3653 has memoization hash 643463f12b31df691ecb5537dde25028 parsl.dataflow.memoization INFO: Task 3653 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3653 try 0 launched on executor multi with executor id 3654 parsl.dataflow.dflow INFO: Standard output for task 3653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5ab34ba9-0a45-443d-9c16-f7f7e7fc3ae7_isrForCrosstalkSources_996999_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5ab34ba9-0a45-443d-9c16-f7f7e7fc3ae7_isrForCrosstalkSources_996999_6.stderr parsl.dataflow.dflow DEBUG: Task 3654 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3654 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3654 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3654 has memoization hash 46c96f44fcb4a39602734a04dea750a8 parsl.dataflow.memoization INFO: Task 3654 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3654 try 0 launched on executor multi with executor id 3655 parsl.dataflow.dflow INFO: Standard output for task 3654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e69dc506-ea7c-4a64-9713-04bfdfd60993_isrForCrosstalkSources_997002_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e69dc506-ea7c-4a64-9713-04bfdfd60993_isrForCrosstalkSources_997002_13.stderr parsl.dataflow.dflow DEBUG: Task 3655 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3655 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3655 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3655 has memoization hash 3e80dcd5174c7be321f89f689095578f parsl.dataflow.memoization INFO: Task 3655 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3655 try 0 launched on executor multi with executor id 3656 parsl.dataflow.dflow INFO: Standard output for task 3655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/77c537fd-e0ae-4873-9500-5258832f1939_isrForCrosstalkSources_997020_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/77c537fd-e0ae-4873-9500-5258832f1939_isrForCrosstalkSources_997020_27.stderr parsl.dataflow.dflow DEBUG: Task 3656 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3656 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3656 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3656 has memoization hash f90be0dec7bfd04b891e9dac7f03a1ed parsl.dataflow.memoization INFO: Task 3656 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3656 try 0 launched on executor multi with executor id 3657 parsl.dataflow.dflow INFO: Standard output for task 3656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/649c5b86-9165-490d-8cd6-fed9d01a9475_isrForCrosstalkSources_996985_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/649c5b86-9165-490d-8cd6-fed9d01a9475_isrForCrosstalkSources_996985_30.stderr parsl.dataflow.dflow DEBUG: Task 3657 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3657 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3657 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3657 has memoization hash 45c6f96d701687bffbb3656b1781d7c7 parsl.dataflow.memoization INFO: Task 3657 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3657 try 0 launched on executor multi with executor id 3658 parsl.dataflow.dflow INFO: Standard output for task 3657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/44b6f5cc-5748-4fef-95f7-f9b6c05edd6f_isrForCrosstalkSources_996975_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/44b6f5cc-5748-4fef-95f7-f9b6c05edd6f_isrForCrosstalkSources_996975_30.stderr parsl.dataflow.dflow DEBUG: Task 3658 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3658 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3658 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3658 has memoization hash 354f10c9b2529841726d15406c4dc0ab parsl.dataflow.memoization INFO: Task 3658 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3658 try 0 launched on executor multi with executor id 3659 parsl.dataflow.dflow INFO: Standard output for task 3658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/95a4050b-fa21-40c8-acb2-98e117165694_isrForCrosstalkSources_996957_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/95a4050b-fa21-40c8-acb2-98e117165694_isrForCrosstalkSources_996957_38.stderr parsl.dataflow.dflow DEBUG: Task 3659 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3659 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3659 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3659 has memoization hash 13ef57900ca4de8b8b0a9ec8db7104d0 parsl.dataflow.memoization INFO: Task 3659 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3659 try 0 launched on executor multi with executor id 3660 parsl.dataflow.dflow INFO: Standard output for task 3659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5af08bc9-4bc7-419d-9c9b-c92ad1cbcffc_isrForCrosstalkSources_996964_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5af08bc9-4bc7-419d-9c9b-c92ad1cbcffc_isrForCrosstalkSources_996964_9.stderr parsl.dataflow.dflow DEBUG: Task 3660 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3660 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3660 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3660 has memoization hash 6514f82c30d9bbf6f34a9da6653ed462 parsl.dataflow.memoization INFO: Task 3660 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3660 try 0 launched on executor multi with executor id 3661 parsl.dataflow.dflow INFO: Standard output for task 3660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9f8182e1-99bd-401a-9156-4d3c50cb5c13_isrForCrosstalkSources_996958_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9f8182e1-99bd-401a-9156-4d3c50cb5c13_isrForCrosstalkSources_996958_8.stderr parsl.dataflow.dflow DEBUG: Task 3661 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3661 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3661 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3661 has memoization hash 0608e61585efd716ebf180d187c83109 parsl.dataflow.memoization INFO: Task 3661 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3661 try 0 launched on executor multi with executor id 3662 parsl.dataflow.dflow INFO: Standard output for task 3661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/bf4924e5-647c-4db9-9678-2918f71e6f17_isrForCrosstalkSources_996957_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/bf4924e5-647c-4db9-9678-2918f71e6f17_isrForCrosstalkSources_996957_10.stderr parsl.dataflow.dflow DEBUG: Task 3662 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3662 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3662 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3662 has memoization hash b7df2bd4d90da64b04451e058daeca3b parsl.dataflow.memoization INFO: Task 3662 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3662 try 0 launched on executor multi with executor id 3663 parsl.dataflow.dflow INFO: Standard output for task 3662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d8591211-ce92-462c-af03-8eb82cf8ab7d_isrForCrosstalkSources_997003_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d8591211-ce92-462c-af03-8eb82cf8ab7d_isrForCrosstalkSources_997003_50.stderr parsl.dataflow.dflow DEBUG: Task 3663 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3663 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3663 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3663 has memoization hash 6fb024bcecbc6d59bae8c678293fa754 parsl.dataflow.memoization INFO: Task 3663 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3663 try 0 launched on executor multi with executor id 3664 parsl.dataflow.dflow INFO: Standard output for task 3663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e4456fdf-34c4-4cbc-ba12-4cd4893076fd_isrForCrosstalkSources_996986_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e4456fdf-34c4-4cbc-ba12-4cd4893076fd_isrForCrosstalkSources_996986_2.stderr parsl.dataflow.dflow DEBUG: Task 3664 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3664 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3664 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3664 has memoization hash 7457f3e9959b9158b644247813f0589d parsl.dataflow.memoization INFO: Task 3664 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3664 try 0 launched on executor multi with executor id 3665 parsl.dataflow.dflow INFO: Standard output for task 3664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/8f5e13c3-86e2-4e6f-97b9-3b728d87e98b_isrForCrosstalkSources_996996_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/8f5e13c3-86e2-4e6f-97b9-3b728d87e98b_isrForCrosstalkSources_996996_25.stderr parsl.dataflow.dflow DEBUG: Task 3665 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3665 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3665 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3665 has memoization hash 6f587f6fd3eceab9d18e82aa37800f12 parsl.dataflow.memoization INFO: Task 3665 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c55ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3665 try 0 launched on executor multi with executor id 3666 parsl.dataflow.dflow INFO: Standard output for task 3665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/47cd431a-444c-48f9-b6f9-39e89644954f_isrForCrosstalkSources_997014_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/47cd431a-444c-48f9-b6f9-39e89644954f_isrForCrosstalkSources_997014_11.stderr parsl.dataflow.dflow DEBUG: Task 3666 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3666 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3666 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3666 has memoization hash 3683bfc7b8d4bc1e61b013780dd8508e parsl.dataflow.memoization INFO: Task 3666 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f00e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3666 try 0 launched on executor multi with executor id 3667 parsl.dataflow.dflow INFO: Standard output for task 3666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c5b0207d-a17e-4aa6-9c01-b761663b87d7_isrForCrosstalkSources_997021_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c5b0207d-a17e-4aa6-9c01-b761663b87d7_isrForCrosstalkSources_997021_25.stderr parsl.dataflow.dflow DEBUG: Task 3667 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3667 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3667 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3667 has memoization hash f6e5c18ab322682d434b610d23f4e4ce parsl.dataflow.memoization INFO: Task 3667 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3667 try 0 launched on executor multi with executor id 3668 parsl.dataflow.dflow INFO: Standard output for task 3667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2dd4b338-c8b8-4543-98e6-47e34235331a_isrForCrosstalkSources_997015_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2dd4b338-c8b8-4543-98e6-47e34235331a_isrForCrosstalkSources_997015_8.stderr parsl.dataflow.dflow DEBUG: Task 3668 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3668 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3668 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3668 has memoization hash 92ca56ca52ae2c6792e91430bdd0b85e parsl.dataflow.memoization INFO: Task 3668 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3668 try 0 launched on executor multi with executor id 3669 parsl.dataflow.dflow INFO: Standard output for task 3668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9026d6b7-d744-407c-babd-99892866593d_isrForCrosstalkSources_996959_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9026d6b7-d744-407c-babd-99892866593d_isrForCrosstalkSources_996959_17.stderr parsl.dataflow.dflow DEBUG: Task 3669 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3669 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3669 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3669 has memoization hash 19d187ca90a2325840f5d0f1c3d6e43e parsl.dataflow.memoization INFO: Task 3669 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3669 try 0 launched on executor multi with executor id 3670 parsl.dataflow.dflow INFO: Standard output for task 3669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/a8ddd2e1-4dd9-4fc9-b3d4-a4d107fa5ecf_isrForCrosstalkSources_997024_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/a8ddd2e1-4dd9-4fc9-b3d4-a4d107fa5ecf_isrForCrosstalkSources_997024_20.stderr parsl.dataflow.dflow DEBUG: Task 3670 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3670 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3670 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3670 has memoization hash 9d00a040fb61a22610b45002cb774497 parsl.dataflow.memoization INFO: Task 3670 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3670 try 0 launched on executor multi with executor id 3671 parsl.dataflow.dflow INFO: Standard output for task 3670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/8176f043-cda2-4028-b237-6e6044795be5_isrForCrosstalkSources_996980_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/8176f043-cda2-4028-b237-6e6044795be5_isrForCrosstalkSources_996980_39.stderr parsl.dataflow.dflow DEBUG: Task 3671 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3671 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3671 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3671 has memoization hash 15d435ec682d0fcfad51d018b052b4c6 parsl.dataflow.memoization INFO: Task 3671 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3671 try 0 launched on executor multi with executor id 3672 parsl.dataflow.dflow INFO: Standard output for task 3671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/7fc3e582-e2f2-4a94-8c60-576d84f5088d_isrForCrosstalkSources_997022_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/7fc3e582-e2f2-4a94-8c60-576d84f5088d_isrForCrosstalkSources_997022_35.stderr parsl.dataflow.dflow DEBUG: Task 3672 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3672 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3672 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3672 has memoization hash af3214e1afcaa7f59db6ecdb53aae036 parsl.dataflow.memoization INFO: Task 3672 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f04a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3672 try 0 launched on executor multi with executor id 3673 parsl.dataflow.dflow INFO: Standard output for task 3672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/da413efb-6f21-4035-8e7c-26678bb3db6e_isrForCrosstalkSources_996961_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/da413efb-6f21-4035-8e7c-26678bb3db6e_isrForCrosstalkSources_996961_18.stderr parsl.dataflow.dflow DEBUG: Task 3673 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3673 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3673 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3673 has memoization hash a24e4f91e3f05d0d006283fab0e361d1 parsl.dataflow.memoization INFO: Task 3673 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3673 try 0 launched on executor multi with executor id 3674 parsl.dataflow.dflow INFO: Standard output for task 3673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a7ef39e-9c58-46d7-ab0a-093c83ba2676_isrForCrosstalkSources_996994_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a7ef39e-9c58-46d7-ab0a-093c83ba2676_isrForCrosstalkSources_996994_14.stderr parsl.dataflow.dflow DEBUG: Task 3674 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3674 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3674 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3674 has memoization hash 676ee5da8d217de82713cf90aa19b5b7 parsl.dataflow.memoization INFO: Task 3674 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3674 try 0 launched on executor multi with executor id 3675 parsl.dataflow.dflow INFO: Standard output for task 3674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f4430a0a-97d1-4fe1-9905-6db944edde0e_isrForCrosstalkSources_996980_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f4430a0a-97d1-4fe1-9905-6db944edde0e_isrForCrosstalkSources_996980_18.stderr parsl.dataflow.dflow DEBUG: Task 3675 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3675 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3675 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3675 has memoization hash ca62b9b3ce5d6a083b8e569227d04b7c parsl.dataflow.memoization INFO: Task 3675 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f05e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3675 try 0 launched on executor multi with executor id 3676 parsl.dataflow.dflow INFO: Standard output for task 3675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/83083d77-1904-4602-a46b-6305113c2d2f_isrForCrosstalkSources_997016_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/83083d77-1904-4602-a46b-6305113c2d2f_isrForCrosstalkSources_997016_46.stderr parsl.dataflow.dflow DEBUG: Task 3676 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3676 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3676 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3676 has memoization hash ded71e1032a9348440353db6436ff905 parsl.dataflow.memoization INFO: Task 3676 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3676 try 0 launched on executor multi with executor id 3677 parsl.dataflow.dflow INFO: Standard output for task 3676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c6fe5391-d2e2-4978-9f43-860a850a1173_isrForCrosstalkSources_996980_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c6fe5391-d2e2-4978-9f43-860a850a1173_isrForCrosstalkSources_996980_13.stderr parsl.dataflow.dflow DEBUG: Task 3677 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3677 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3677 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3677 has memoization hash 6b309da98a9c7c8755b3a281afaee898 parsl.dataflow.memoization INFO: Task 3677 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f07c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3677 try 0 launched on executor multi with executor id 3678 parsl.dataflow.dflow INFO: Standard output for task 3677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f2b9b519-4005-442e-b457-27a8b8a2bbbc_isrForCrosstalkSources_996967_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f2b9b519-4005-442e-b457-27a8b8a2bbbc_isrForCrosstalkSources_996967_41.stderr parsl.dataflow.dflow DEBUG: Task 3678 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3678 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3678 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3678 has memoization hash 9b3d25c690adff44b6981033c4c548c9 parsl.dataflow.memoization INFO: Task 3678 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3678 try 0 launched on executor multi with executor id 3679 parsl.dataflow.dflow INFO: Standard output for task 3678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/8b5e9975-8415-4261-809c-d032c9344a40_isrForCrosstalkSources_996957_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/8b5e9975-8415-4261-809c-d032c9344a40_isrForCrosstalkSources_996957_55.stderr parsl.dataflow.dflow DEBUG: Task 3679 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3679 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3679 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3679 has memoization hash 551c2f3739709baaaa3f2a0aa696950a parsl.dataflow.memoization INFO: Task 3679 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f09a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3679 try 0 launched on executor multi with executor id 3680 parsl.dataflow.dflow INFO: Standard output for task 3679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/45a452f3-d5e6-470c-83a2-e66ad83ba501_isrForCrosstalkSources_996987_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/45a452f3-d5e6-470c-83a2-e66ad83ba501_isrForCrosstalkSources_996987_18.stderr parsl.dataflow.dflow DEBUG: Task 3680 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3680 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3680 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3680 has memoization hash bed04b42431f534156a2cdd795c6b4c9 parsl.dataflow.memoization INFO: Task 3680 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3680 try 0 launched on executor multi with executor id 3681 parsl.dataflow.dflow INFO: Standard output for task 3680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e5dc8360-1ed7-4092-96e7-312c3ad3d8aa_isrForCrosstalkSources_996995_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e5dc8360-1ed7-4092-96e7-312c3ad3d8aa_isrForCrosstalkSources_996995_7.stderr parsl.dataflow.dflow DEBUG: Task 3681 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3681 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3681 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3681 has memoization hash 123ca87e96cfe9c4ec0b9c92d5d757a7 parsl.dataflow.memoization INFO: Task 3681 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3681 try 0 launched on executor multi with executor id 3682 parsl.dataflow.dflow INFO: Standard output for task 3681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a5d82d10-ef42-4e79-92c3-8818a4bde98c_isrForCrosstalkSources_996981_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a5d82d10-ef42-4e79-92c3-8818a4bde98c_isrForCrosstalkSources_996981_2.stderr parsl.dataflow.dflow DEBUG: Task 3682 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3682 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3682 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3682 has memoization hash f53cd54abe69c869ff983a0935b614fb parsl.dataflow.memoization INFO: Task 3682 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3682 try 0 launched on executor multi with executor id 3683 parsl.dataflow.dflow INFO: Standard output for task 3682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9709dd43-c140-4ad8-9804-afba73e7069a_isrForCrosstalkSources_997015_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9709dd43-c140-4ad8-9804-afba73e7069a_isrForCrosstalkSources_997015_11.stderr parsl.dataflow.dflow DEBUG: Task 3683 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3683 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3683 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3683 has memoization hash 18f2c1bfb370178162b1319824d0ee87 parsl.dataflow.memoization INFO: Task 3683 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3683 try 0 launched on executor multi with executor id 3684 parsl.dataflow.dflow INFO: Standard output for task 3683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/334ab2b7-84d0-43dc-b4ed-5de52e2a3e96_isrForCrosstalkSources_997021_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/334ab2b7-84d0-43dc-b4ed-5de52e2a3e96_isrForCrosstalkSources_997021_9.stderr parsl.dataflow.dflow DEBUG: Task 3684 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3684 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3684 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3684 has memoization hash d3500f42be19124ce21617ad5dd4f3c9 parsl.dataflow.memoization INFO: Task 3684 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3684 try 0 launched on executor multi with executor id 3685 parsl.dataflow.dflow INFO: Standard output for task 3684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/4fbe933d-2663-4f0c-a68c-2b4bbf28e517_isrForCrosstalkSources_996974_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/4fbe933d-2663-4f0c-a68c-2b4bbf28e517_isrForCrosstalkSources_996974_18.stderr parsl.dataflow.dflow DEBUG: Task 3685 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3685 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3685 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3685 has memoization hash 286267a45c9efdb39ab272bc15bc1caa parsl.dataflow.memoization INFO: Task 3685 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3685 try 0 launched on executor multi with executor id 3686 parsl.dataflow.dflow INFO: Standard output for task 3685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4317e33e-a28a-4d8d-9a94-172adce0f11a_isrForCrosstalkSources_997018_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4317e33e-a28a-4d8d-9a94-172adce0f11a_isrForCrosstalkSources_997018_51.stderr parsl.dataflow.dflow DEBUG: Task 3686 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3686 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3686 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3686 has memoization hash 9a4d34b75bb8032e1a0c3a270ed61b6d parsl.dataflow.memoization INFO: Task 3686 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3686 try 0 launched on executor multi with executor id 3687 parsl.dataflow.dflow INFO: Standard output for task 3686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/400d7374-f54b-462c-bd03-daa8a176ac24_isrForCrosstalkSources_997029_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/400d7374-f54b-462c-bd03-daa8a176ac24_isrForCrosstalkSources_997029_61.stderr parsl.dataflow.dflow DEBUG: Task 3687 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3687 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3687 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3687 has memoization hash 8684b818a85220c108997cc5e0919bdd parsl.dataflow.memoization INFO: Task 3687 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3687 try 0 launched on executor multi with executor id 3688 parsl.dataflow.dflow INFO: Standard output for task 3687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0c8759cf-7399-4b89-9735-66a195ec014f_isrForCrosstalkSources_997007_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0c8759cf-7399-4b89-9735-66a195ec014f_isrForCrosstalkSources_997007_41.stderr parsl.dataflow.dflow DEBUG: Task 3688 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3688 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3688 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3688 has memoization hash 050331c4d52e4ff3f335423552b457ca parsl.dataflow.memoization INFO: Task 3688 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77ccd3ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:50:01 proc_lsst.multi:170 [INFO] [multi] job_id 17074895 parsl.dataflow.dflow INFO: Parsl task 3688 try 0 launched on executor multi with executor id 3689 parsl.dataflow.dflow INFO: Standard output for task 3688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/7265c119-4e98-4a1a-8a3b-3805a8ff8cc2_isrForCrosstalkSources_997010_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/7265c119-4e98-4a1a-8a3b-3805a8ff8cc2_isrForCrosstalkSources_997010_7.stderr parsl.dataflow.dflow DEBUG: Task 3689 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3689 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3689 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3689 has memoization hash 60ca1297efd3c7a4557a9b5febe4f1c9 parsl.dataflow.memoization INFO: Task 3689 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) proc_lsst.multi INFO: [multi] job_id 17074895 2024-03-15 13:50:01 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 2 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 2 2024-03-15 13:50:01 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:01 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074895 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074895 parsl.executors.status_handling DEBUG: Launched block 37 on executor multi with job ID 17074895 parsl.executors.status_handling INFO: Allocated block ID 38 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-38 2024-03-15 13:50:01 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-38 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-38 2024-03-15 13:50:01 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:01 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:01 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:01 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 2 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 2 2024-03-15 13:50:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:01 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=38 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 3689 try 0 launched on executor multi with executor id 3690 parsl.dataflow.dflow INFO: Standard output for task 3689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/1148f132-8353-4881-9949-e05e58e099a5_isrForCrosstalkSources_996975_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/1148f132-8353-4881-9949-e05e58e099a5_isrForCrosstalkSources_996975_15.stderr parsl.dataflow.dflow DEBUG: Task 3690 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3690 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3690 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3690 has memoization hash 32b3e8ab6b61ccdf976d3142f9fdecc3 parsl.dataflow.memoization INFO: Task 3690 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3690 try 0 launched on executor multi with executor id 3691 parsl.dataflow.dflow INFO: Standard output for task 3690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/2c546be9-5f30-4d76-8867-a3965627f205_isrForCrosstalkSources_997031_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/2c546be9-5f30-4d76-8867-a3965627f205_isrForCrosstalkSources_997031_25.stderr parsl.dataflow.dflow DEBUG: Task 3691 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3691 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3691 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3691 has memoization hash e56cfad57ef80735752ee8d0189adee2 parsl.dataflow.memoization INFO: Task 3691 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3691 try 0 launched on executor multi with executor id 3692 parsl.dataflow.dflow INFO: Standard output for task 3691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/62a743e9-9c91-47a6-b5a5-674bb536207f_isrForCrosstalkSources_997009_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/62a743e9-9c91-47a6-b5a5-674bb536207f_isrForCrosstalkSources_997009_62.stderr parsl.dataflow.dflow DEBUG: Task 3692 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3692 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3692 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3692 has memoization hash e3f62affc5f4aee305904fde8cfac4e7 parsl.dataflow.memoization INFO: Task 3692 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c62f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3692 try 0 launched on executor multi with executor id 3693 parsl.dataflow.dflow INFO: Standard output for task 3692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/da98c1c5-2705-4aa8-826d-8c967fdd8855_isrForCrosstalkSources_996977_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/da98c1c5-2705-4aa8-826d-8c967fdd8855_isrForCrosstalkSources_996977_22.stderr parsl.dataflow.dflow DEBUG: Task 3693 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3693 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3693 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3693 has memoization hash 7363d1102f5375c2f7c25287cba0c5e6 parsl.dataflow.memoization INFO: Task 3693 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3693 try 0 launched on executor multi with executor id 3694 parsl.dataflow.dflow INFO: Standard output for task 3693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1a146776-5280-4686-90c6-09f09c4e299e_isrForCrosstalkSources_997017_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1a146776-5280-4686-90c6-09f09c4e299e_isrForCrosstalkSources_997017_62.stderr parsl.dataflow.dflow DEBUG: Task 3694 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3694 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3694 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3694 has memoization hash 8d6d1a56fab764d02b4d7bd616ab22f5 parsl.dataflow.memoization INFO: Task 3694 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3694 try 0 launched on executor multi with executor id 3695 parsl.dataflow.dflow INFO: Standard output for task 3694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/fbbc96b4-ed3b-4c66-a4b9-fd74a0f77a6c_isrForCrosstalkSources_997022_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/fbbc96b4-ed3b-4c66-a4b9-fd74a0f77a6c_isrForCrosstalkSources_997022_17.stderr parsl.dataflow.dflow DEBUG: Task 3695 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3695 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3695 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3695 has memoization hash cae5edd3f2741217430974497bdbfc33 parsl.dataflow.memoization INFO: Task 3695 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3695 try 0 launched on executor multi with executor id 3696 parsl.dataflow.dflow INFO: Standard output for task 3695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2327d6c6-d374-428c-8f7f-eadb0389497b_isrForCrosstalkSources_997007_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2327d6c6-d374-428c-8f7f-eadb0389497b_isrForCrosstalkSources_997007_42.stderr parsl.dataflow.dflow DEBUG: Task 3696 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3696 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3696 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3696 has memoization hash 5c9e655b94c75ce2fe97e50be9f7e606 parsl.dataflow.memoization INFO: Task 3696 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f11c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3696 try 0 launched on executor multi with executor id 3697 parsl.dataflow.dflow INFO: Standard output for task 3696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/f5855bb2-9e45-43d6-bdfb-c2df0db7bef8_isrForCrosstalkSources_997009_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/f5855bb2-9e45-43d6-bdfb-c2df0db7bef8_isrForCrosstalkSources_997009_56.stderr parsl.dataflow.dflow DEBUG: Task 3697 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3697 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3697 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3697 has memoization hash 09edf2602ed563b57177fee109dc8ae9 parsl.dataflow.memoization INFO: Task 3697 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 3697 try 0 launched on executor multi with executor id 3698 parsl.dataflow.dflow INFO: Standard output for task 3697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/5603cc04-f8bb-474a-93fb-5df56f10a398_isrForCrosstalkSources_996992_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/5603cc04-f8bb-474a-93fb-5df56f10a398_isrForCrosstalkSources_996992_22.stderr parsl.dataflow.dflow DEBUG: Task 3698 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3698 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3698 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3698 has memoization hash 150421785737e417f3442aa685f56944 parsl.dataflow.memoization INFO: Task 3698 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3698 try 0 launched on executor multi with executor id 3699 parsl.dataflow.dflow INFO: Standard output for task 3698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/718b4f64-70b6-448e-ae53-d782692cb982_isrForCrosstalkSources_997019_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/718b4f64-70b6-448e-ae53-d782692cb982_isrForCrosstalkSources_997019_32.stderr parsl.dataflow.dflow DEBUG: Task 3699 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3699 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3699 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3699 has memoization hash 7532e3d3d2fca9778f2bc85dfd8f99eb parsl.dataflow.memoization INFO: Task 3699 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f14e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3699 try 0 launched on executor multi with executor id 3700 parsl.dataflow.dflow INFO: Standard output for task 3699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/48bfc154-c1ea-4491-94c8-ab0444abb97d_isrForCrosstalkSources_996985_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/48bfc154-c1ea-4491-94c8-ab0444abb97d_isrForCrosstalkSources_996985_23.stderr parsl.dataflow.dflow DEBUG: Task 3700 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3700 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3700 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3700 has memoization hash db51895b1947e4ff53c5ba3f263bd2ed parsl.dataflow.memoization INFO: Task 3700 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3700 try 0 launched on executor multi with executor id 3701 parsl.dataflow.dflow INFO: Standard output for task 3700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e8e27880-ee43-450d-90cc-0ba956a8b58c_isrForCrosstalkSources_997028_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e8e27880-ee43-450d-90cc-0ba956a8b58c_isrForCrosstalkSources_997028_37.stderr parsl.dataflow.dflow DEBUG: Task 3701 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3701 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3701 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3701 has memoization hash b76220e70d7a29bfcb6ac164b5ca0cf3 parsl.dataflow.memoization INFO: Task 3701 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3701 try 0 launched on executor multi with executor id 3702 parsl.dataflow.dflow INFO: Standard output for task 3701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/dce83fe1-58c8-4ec4-b84d-a3f904df67df_isrForCrosstalkSources_996991_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/dce83fe1-58c8-4ec4-b84d-a3f904df67df_isrForCrosstalkSources_996991_55.stderr parsl.dataflow.dflow DEBUG: Task 3702 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3702 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3702 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3702 has memoization hash f4a5d7d0ce3e489995a2fd71ceb92c21 parsl.dataflow.memoization INFO: Task 3702 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3702 try 0 launched on executor multi with executor id 3703 parsl.dataflow.dflow INFO: Standard output for task 3702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/0081d364-559e-4141-b06d-35a30386d431_isrForCrosstalkSources_996961_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/0081d364-559e-4141-b06d-35a30386d431_isrForCrosstalkSources_996961_47.stderr parsl.dataflow.dflow DEBUG: Task 3703 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3703 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3703 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3703 has memoization hash 92d4ac0e4d54bfe5bb4c677614dff73c parsl.dataflow.memoization INFO: Task 3703 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3703 try 0 launched on executor multi with executor id 3704 parsl.dataflow.dflow INFO: Standard output for task 3703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/23d59f5c-cf14-4fa0-806f-8dbe177de133_isrForCrosstalkSources_997008_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/23d59f5c-cf14-4fa0-806f-8dbe177de133_isrForCrosstalkSources_997008_51.stderr parsl.dataflow.dflow DEBUG: Task 3704 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3704 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3704 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3704 has memoization hash c73ad2c0d0a630c9c672c1500397fc22 parsl.dataflow.memoization INFO: Task 3704 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3704 try 0 launched on executor multi with executor id 3705 parsl.dataflow.dflow INFO: Standard output for task 3704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5d675980-c86e-4eb8-91cd-d9f8edd3e86b_isrForCrosstalkSources_996999_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5d675980-c86e-4eb8-91cd-d9f8edd3e86b_isrForCrosstalkSources_996999_15.stderr parsl.dataflow.dflow DEBUG: Task 3705 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3705 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3705 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3705 has memoization hash bc0afec8363b68467aa1f9d9bcbce1ea parsl.dataflow.memoization INFO: Task 3705 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3705 try 0 launched on executor multi with executor id 3706 parsl.dataflow.dflow INFO: Standard output for task 3705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/134a2c5d-04a4-4e70-a25c-bb272625eac0_isrForCrosstalkSources_996993_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/134a2c5d-04a4-4e70-a25c-bb272625eac0_isrForCrosstalkSources_996993_50.stderr parsl.dataflow.dflow DEBUG: Task 3706 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3706 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3706 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3706 has memoization hash 714f6b6b6e15650e65243a5698d83b77 parsl.dataflow.memoization INFO: Task 3706 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3706 try 0 launched on executor multi with executor id 3707 parsl.dataflow.dflow INFO: Standard output for task 3706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ecbfa483-dac5-4a42-a916-73e39ef20bfa_isrForCrosstalkSources_997015_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ecbfa483-dac5-4a42-a916-73e39ef20bfa_isrForCrosstalkSources_997015_28.stderr parsl.dataflow.dflow DEBUG: Task 3707 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3707 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3707 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3707 has memoization hash aaa7af4cf893014f3de69a7b9412e67d parsl.dataflow.memoization INFO: Task 3707 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3707 try 0 launched on executor multi with executor id 3708 parsl.dataflow.dflow INFO: Standard output for task 3707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/0d61aace-95c7-4e4a-8613-e7fa90acebf3_isrForCrosstalkSources_997016_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/0d61aace-95c7-4e4a-8613-e7fa90acebf3_isrForCrosstalkSources_997016_24.stderr parsl.dataflow.dflow DEBUG: Task 3708 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3708 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3708 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3708 has memoization hash 56d030ca3b8aba396b0e859eedfa3d36 parsl.dataflow.memoization INFO: Task 3708 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f19e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3708 try 0 launched on executor multi with executor id 3709 parsl.dataflow.dflow INFO: Standard output for task 3708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/76f4406c-ecbc-4c9e-ab43-93797eb1f93d_isrForCrosstalkSources_996982_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/76f4406c-ecbc-4c9e-ab43-93797eb1f93d_isrForCrosstalkSources_996982_56.stderr parsl.dataflow.dflow DEBUG: Task 3709 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3709 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3709 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3709 has memoization hash 7fab7867272fe3b5d2f93f376f3e805b parsl.dataflow.memoization INFO: Task 3709 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f18a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3709 try 0 launched on executor multi with executor id 3710 parsl.dataflow.dflow INFO: Standard output for task 3709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/eb4fae1a-85c6-4c24-af87-6f7cf81b7cf3_isrForCrosstalkSources_997000_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/eb4fae1a-85c6-4c24-af87-6f7cf81b7cf3_isrForCrosstalkSources_997000_7.stderr parsl.dataflow.dflow DEBUG: Task 3710 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3710 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3710 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3710 has memoization hash 31afd4611ee84c858050ee2d0c431488 parsl.dataflow.memoization INFO: Task 3710 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3710 try 0 launched on executor multi with executor id 3711 parsl.dataflow.dflow INFO: Standard output for task 3710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1f305f05-3e4d-41f6-a3ed-3ca45433ce9d_isrForCrosstalkSources_997019_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1f305f05-3e4d-41f6-a3ed-3ca45433ce9d_isrForCrosstalkSources_997019_13.stderr parsl.dataflow.dflow DEBUG: Task 3711 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3711 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3711 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3711 has memoization hash e45892eb1ee173448821f4b8bce80a7b parsl.dataflow.memoization INFO: Task 3711 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3711 try 0 launched on executor multi with executor id 3712 parsl.dataflow.dflow INFO: Standard output for task 3711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/52b1310d-3a59-43b9-83cd-1da29040c07d_isrForCrosstalkSources_997024_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/52b1310d-3a59-43b9-83cd-1da29040c07d_isrForCrosstalkSources_997024_4.stderr parsl.dataflow.dflow DEBUG: Task 3712 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3712 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3712 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3712 has memoization hash 7498b480758f618d806241ba06e32d71 parsl.dataflow.memoization INFO: Task 3712 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3712 try 0 launched on executor multi with executor id 3713 parsl.dataflow.dflow INFO: Standard output for task 3712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/9d16245e-8e34-4866-9094-3b59f5179b2a_isrForCrosstalkSources_997021_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/9d16245e-8e34-4866-9094-3b59f5179b2a_isrForCrosstalkSources_997021_11.stderr parsl.dataflow.dflow DEBUG: Task 3713 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3713 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3713 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3713 has memoization hash 1017c02e8d38787ccb1373a1fa249e9a parsl.dataflow.memoization INFO: Task 3713 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3713 try 0 launched on executor multi with executor id 3714 parsl.dataflow.dflow INFO: Standard output for task 3713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c40d3465-acdc-4353-ad03-d542be9dc3c0_isrForCrosstalkSources_996968_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c40d3465-acdc-4353-ad03-d542be9dc3c0_isrForCrosstalkSources_996968_3.stderr parsl.dataflow.dflow DEBUG: Task 3714 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3714 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3714 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3714 has memoization hash ab28f2e8d51b6d91fe1b7063a53af01d parsl.dataflow.memoization INFO: Task 3714 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3714 try 0 launched on executor multi with executor id 3715 parsl.dataflow.dflow INFO: Standard output for task 3714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/8ee68f0e-f4c4-4f22-8f64-cceae620e566_isrForCrosstalkSources_996991_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/8ee68f0e-f4c4-4f22-8f64-cceae620e566_isrForCrosstalkSources_996991_1.stderr parsl.dataflow.dflow DEBUG: Task 3715 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3715 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3715 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3715 has memoization hash 823997420231aa4053d5ffba2c8a9361 parsl.dataflow.memoization INFO: Task 3715 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3715 try 0 launched on executor multi with executor id 3716 parsl.dataflow.dflow INFO: Standard output for task 3715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0d1f8ebf-7b2c-40f2-8ea0-2be002955714_isrForCrosstalkSources_997028_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0d1f8ebf-7b2c-40f2-8ea0-2be002955714_isrForCrosstalkSources_997028_55.stderr parsl.dataflow.dflow DEBUG: Task 3716 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3716 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3716 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3716 has memoization hash fcb386ce35b9964b39fc6a1c2c90cda8 parsl.dataflow.memoization INFO: Task 3716 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3716 try 0 launched on executor multi with executor id 3717 parsl.dataflow.dflow INFO: Standard output for task 3716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/323a7aa6-573a-4515-84de-3c6f271f35f7_isrForCrosstalkSources_997010_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/323a7aa6-573a-4515-84de-3c6f271f35f7_isrForCrosstalkSources_997010_50.stderr parsl.dataflow.dflow DEBUG: Task 3717 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3717 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3717 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3717 has memoization hash a3f050919b83fd36b6c7f4ab9ec384d9 parsl.dataflow.memoization INFO: Task 3717 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3717 try 0 launched on executor multi with executor id 3718 parsl.dataflow.dflow INFO: Standard output for task 3717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a400f9c7-f692-41b6-bddd-e2038e3b50d2_isrForCrosstalkSources_997033_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a400f9c7-f692-41b6-bddd-e2038e3b50d2_isrForCrosstalkSources_997033_10.stderr parsl.dataflow.dflow DEBUG: Task 3718 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3718 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3718 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3718 has memoization hash cbf06d0d2f314a5a555e2e9ad3ff6a7c parsl.dataflow.memoization INFO: Task 3718 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3718 try 0 launched on executor multi with executor id 3719 parsl.dataflow.dflow INFO: Standard output for task 3718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/932d186c-aed7-4ea6-bf5b-d9c16960e962_isrForCrosstalkSources_997012_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/932d186c-aed7-4ea6-bf5b-d9c16960e962_isrForCrosstalkSources_997012_8.stderr parsl.dataflow.dflow DEBUG: Task 3719 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3719 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3719 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3719 has memoization hash b3e418b4aa875e704f8d721d5615f91a parsl.dataflow.memoization INFO: Task 3719 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3719 try 0 launched on executor multi with executor id 3720 parsl.dataflow.dflow INFO: Standard output for task 3719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/7e3b0276-b573-4d40-8f24-d785ab1bf499_isrForCrosstalkSources_997030_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/7e3b0276-b573-4d40-8f24-d785ab1bf499_isrForCrosstalkSources_997030_4.stderr parsl.dataflow.dflow DEBUG: Task 3720 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3720 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3720 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3720 has memoization hash 049355f8feb4f17e987b0cc7a6572a6e parsl.dataflow.memoization INFO: Task 3720 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3720 try 0 launched on executor multi with executor id 3721 parsl.dataflow.dflow INFO: Standard output for task 3720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f5aca3f1-7cb7-40ed-b07e-e95ae096d715_isrForCrosstalkSources_996966_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f5aca3f1-7cb7-40ed-b07e-e95ae096d715_isrForCrosstalkSources_996966_34.stderr parsl.dataflow.dflow DEBUG: Task 3721 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3721 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3721 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3721 has memoization hash bb7e2ffaf4e46e18f2639eb68c2cce50 parsl.dataflow.memoization INFO: Task 3721 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3721 try 0 launched on executor multi with executor id 3722 parsl.dataflow.dflow INFO: Standard output for task 3721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d2dcd442-7a96-49a4-8d73-05f113a29169_isrForCrosstalkSources_996966_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d2dcd442-7a96-49a4-8d73-05f113a29169_isrForCrosstalkSources_996966_3.stderr parsl.dataflow.dflow DEBUG: Task 3722 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3722 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3722 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3722 has memoization hash 69056c6367ba00e4ce0a156e28a4c061 parsl.dataflow.memoization INFO: Task 3722 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f22a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3722 try 0 launched on executor multi with executor id 3723 parsl.dataflow.dflow INFO: Standard output for task 3722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/7b452528-41d5-444f-b8a9-58b32a143c27_isrForCrosstalkSources_996957_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/7b452528-41d5-444f-b8a9-58b32a143c27_isrForCrosstalkSources_996957_22.stderr parsl.dataflow.dflow DEBUG: Task 3723 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3723 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3723 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3723 has memoization hash c7168be07a6a94de27419af7a95ca607 parsl.dataflow.memoization INFO: Task 3723 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f20c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3723 try 0 launched on executor multi with executor id 3724 parsl.dataflow.dflow INFO: Standard output for task 3723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5f8a89ce-932c-4b7b-87c0-b0e216f502d0_isrForCrosstalkSources_996987_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5f8a89ce-932c-4b7b-87c0-b0e216f502d0_isrForCrosstalkSources_996987_37.stderr parsl.dataflow.dflow DEBUG: Task 3724 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3724 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3724 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3724 has memoization hash 60a0a63f98d7e3d6a5c9fca931efbb0d parsl.dataflow.memoization INFO: Task 3724 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3724 try 0 launched on executor multi with executor id 3725 parsl.dataflow.dflow INFO: Standard output for task 3724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e1dda7fb-400c-44fc-b562-c4c282d1b6a9_isrForCrosstalkSources_997030_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e1dda7fb-400c-44fc-b562-c4c282d1b6a9_isrForCrosstalkSources_997030_44.stderr parsl.dataflow.dflow DEBUG: Task 3725 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3725 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3725 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3725 has memoization hash f7274c5fec0782fe390dd69b863590ea parsl.dataflow.memoization INFO: Task 3725 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3725 try 0 launched on executor multi with executor id 3726 parsl.dataflow.dflow INFO: Standard output for task 3725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/659e4f22-f777-48ff-8048-9f3b0a6096b4_isrForCrosstalkSources_997006_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/659e4f22-f777-48ff-8048-9f3b0a6096b4_isrForCrosstalkSources_997006_5.stderr parsl.dataflow.dflow DEBUG: Task 3726 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3726 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3726 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3726 has memoization hash f72fb81403036fd0f04d35b15caea64f parsl.dataflow.memoization INFO: Task 3726 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3726 try 0 launched on executor multi with executor id 3727 parsl.dataflow.dflow INFO: Standard output for task 3726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c930329b-c53b-4952-b898-ec49b9081403_isrForCrosstalkSources_996969_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c930329b-c53b-4952-b898-ec49b9081403_isrForCrosstalkSources_996969_33.stderr parsl.dataflow.dflow DEBUG: Task 3727 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3727 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3727 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3727 has memoization hash 8a7498a01275316ab230973914f7aae3 parsl.dataflow.memoization INFO: Task 3727 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f25c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3727 try 0 launched on executor multi with executor id 3728 parsl.dataflow.dflow INFO: Standard output for task 3727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/2f1b3777-1c8b-43b4-9dc0-2aa64c8bc122_isrForCrosstalkSources_997004_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/2f1b3777-1c8b-43b4-9dc0-2aa64c8bc122_isrForCrosstalkSources_997004_7.stderr parsl.dataflow.dflow DEBUG: Task 3728 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3728 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3728 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3728 has memoization hash b0bafbcbf7dedc4004bee0b853ddc4d2 parsl.dataflow.memoization INFO: Task 3728 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f23e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3728 try 0 launched on executor multi with executor id 3729 parsl.dataflow.dflow INFO: Standard output for task 3728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/f5080978-a0f6-4589-b2ac-1282100e3252_isrForCrosstalkSources_996982_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/f5080978-a0f6-4589-b2ac-1282100e3252_isrForCrosstalkSources_996982_23.stderr parsl.dataflow.dflow DEBUG: Task 3729 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3729 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3729 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3729 has memoization hash b4395e4ca91650d3fcd3479e57baf662 parsl.dataflow.memoization INFO: Task 3729 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3729 try 0 launched on executor multi with executor id 3730 parsl.dataflow.dflow INFO: Standard output for task 3729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d01be70b-c25f-420d-b893-4b533bc0ce3e_isrForCrosstalkSources_996991_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d01be70b-c25f-420d-b893-4b533bc0ce3e_isrForCrosstalkSources_996991_35.stderr parsl.dataflow.dflow DEBUG: Task 3730 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3730 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3730 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3730 has memoization hash 76cec014d56d0ea8d35f02af791d8a4c parsl.dataflow.memoization INFO: Task 3730 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3730 try 0 launched on executor multi with executor id 3731 parsl.dataflow.dflow INFO: Standard output for task 3730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4ed67aef-94dc-4ae2-a09c-d1a4535407ba_isrForCrosstalkSources_997023_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4ed67aef-94dc-4ae2-a09c-d1a4535407ba_isrForCrosstalkSources_997023_3.stderr parsl.dataflow.dflow DEBUG: Task 3731 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3731 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3731 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3731 has memoization hash 4cd6e9ec58555b29189e757f1e3ff2e2 parsl.dataflow.memoization INFO: Task 3731 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f27a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3731 try 0 launched on executor multi with executor id 3732 parsl.dataflow.dflow INFO: Standard output for task 3731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ea48cd38-d2fc-4cda-8fb9-3795caa7f7d3_isrForCrosstalkSources_996984_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ea48cd38-d2fc-4cda-8fb9-3795caa7f7d3_isrForCrosstalkSources_996984_32.stderr parsl.dataflow.dflow DEBUG: Task 3732 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3732 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3732 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3732 has memoization hash c4254d3ed2d01912a29c3ca2447ee62b parsl.dataflow.memoization INFO: Task 3732 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f28e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3732 try 0 launched on executor multi with executor id 3733 parsl.dataflow.dflow INFO: Standard output for task 3732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/b62bf01b-e9ba-4b12-b246-0504c4de8622_isrForCrosstalkSources_996978_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/b62bf01b-e9ba-4b12-b246-0504c4de8622_isrForCrosstalkSources_996978_29.stderr parsl.dataflow.dflow DEBUG: Task 3733 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3733 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3733 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3733 has memoization hash ec700dc87ab3e9a0d5936859694a976e parsl.dataflow.memoization INFO: Task 3733 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3733 try 0 launched on executor multi with executor id 3734 parsl.dataflow.dflow INFO: Standard output for task 3733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/93137248-c716-40b1-a1bb-c0b01650d945_isrForCrosstalkSources_996963_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/93137248-c716-40b1-a1bb-c0b01650d945_isrForCrosstalkSources_996963_1.stderr parsl.dataflow.dflow DEBUG: Task 3734 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3734 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3734 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3734 has memoization hash dd9cca8468a67b73048cc7faf0b27084 parsl.dataflow.memoization INFO: Task 3734 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3734 try 0 launched on executor multi with executor id 3735 parsl.dataflow.dflow INFO: Standard output for task 3734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/dbc8c93a-5a74-4651-a675-b8d42c2aac34_isrForCrosstalkSources_996964_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/dbc8c93a-5a74-4651-a675-b8d42c2aac34_isrForCrosstalkSources_996964_58.stderr parsl.dataflow.dflow DEBUG: Task 3735 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3735 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3735 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3735 has memoization hash 9ea9ed1f0591562849116d92dc8e4eb2 parsl.dataflow.memoization INFO: Task 3735 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3735 try 0 launched on executor multi with executor id 3736 parsl.dataflow.dflow INFO: Standard output for task 3735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/25a01807-d7a9-41a0-8766-aa89a9c051f6_isrForCrosstalkSources_996977_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/25a01807-d7a9-41a0-8766-aa89a9c051f6_isrForCrosstalkSources_996977_53.stderr parsl.dataflow.dflow DEBUG: Task 3736 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3736 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3736 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3736 has memoization hash 2fe464e9777bcb77eac27bc134498ce2 parsl.dataflow.memoization INFO: Task 3736 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3736 try 0 launched on executor multi with executor id 3737 parsl.dataflow.dflow INFO: Standard output for task 3736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/059d0ee5-81a5-4de4-b926-efbd2ff5efde_isrForCrosstalkSources_997014_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/059d0ee5-81a5-4de4-b926-efbd2ff5efde_isrForCrosstalkSources_997014_37.stderr parsl.dataflow.dflow DEBUG: Task 3737 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3737 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3737 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3737 has memoization hash f5927c2f1767dd7a1e1f1c05d6f83a4c parsl.dataflow.memoization INFO: Task 3737 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3737 try 0 launched on executor multi with executor id 3738 parsl.dataflow.dflow INFO: Standard output for task 3737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/da4a612d-f7ff-44d7-9ea6-a2a025f23815_isrForCrosstalkSources_997027_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/da4a612d-f7ff-44d7-9ea6-a2a025f23815_isrForCrosstalkSources_997027_31.stderr parsl.dataflow.dflow DEBUG: Task 3738 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3738 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3738 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3738 has memoization hash 0795a3955f7242055e4606b198718b89 parsl.dataflow.memoization INFO: Task 3738 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3738 try 0 launched on executor multi with executor id 3739 parsl.dataflow.dflow INFO: Standard output for task 3738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/aa507461-fd6d-4226-ae18-6d72f1a64dac_isrForCrosstalkSources_996961_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/aa507461-fd6d-4226-ae18-6d72f1a64dac_isrForCrosstalkSources_996961_51.stderr parsl.dataflow.dflow DEBUG: Task 3739 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3739 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3739 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3739 has memoization hash 01fb332833704d8c29320f93023df9a7 parsl.dataflow.memoization INFO: Task 3739 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3739 try 0 launched on executor multi with executor id 3740 parsl.dataflow.dflow INFO: Standard output for task 3739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/035f9578-79e6-4ee4-9f23-aa7cefa167ea_isrForCrosstalkSources_997026_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/035f9578-79e6-4ee4-9f23-aa7cefa167ea_isrForCrosstalkSources_997026_45.stderr parsl.dataflow.dflow DEBUG: Task 3740 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3740 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3740 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3740 has memoization hash 93fdc263c7085f55bf4489ab189019b4 parsl.dataflow.memoization INFO: Task 3740 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3740 try 0 launched on executor multi with executor id 3741 parsl.dataflow.dflow INFO: Standard output for task 3740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7d5af3fe-f3cb-4db7-b847-361257ae7133_isrForCrosstalkSources_996993_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7d5af3fe-f3cb-4db7-b847-361257ae7133_isrForCrosstalkSources_996993_4.stderr parsl.dataflow.dflow DEBUG: Task 3741 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3741 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3741 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3741 has memoization hash 548138c8ba93e19746d94ed1c272b83a parsl.dataflow.memoization INFO: Task 3741 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3741 try 0 launched on executor multi with executor id 3742 parsl.dataflow.dflow INFO: Standard output for task 3741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/dd2f46a9-bf73-4d25-a0ad-0bb7cf5675b2_isrForCrosstalkSources_996969_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/dd2f46a9-bf73-4d25-a0ad-0bb7cf5675b2_isrForCrosstalkSources_996969_23.stderr parsl.dataflow.dflow DEBUG: Task 3742 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3742 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3742 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3742 has memoization hash 9fc049ef12821a25964bf9ddfae3ae0b parsl.dataflow.memoization INFO: Task 3742 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3742 try 0 launched on executor multi with executor id 3743 parsl.dataflow.dflow INFO: Standard output for task 3742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ca98e3b3-2b73-4048-9663-de6993f25ca6_isrForCrosstalkSources_997019_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ca98e3b3-2b73-4048-9663-de6993f25ca6_isrForCrosstalkSources_997019_42.stderr parsl.dataflow.dflow DEBUG: Task 3743 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3743 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3743 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3743 has memoization hash 167b88a450c1ae38756150a66a98ab39 parsl.dataflow.memoization INFO: Task 3743 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3743 try 0 launched on executor multi with executor id 3744 parsl.dataflow.dflow INFO: Standard output for task 3743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0a08623f-10ee-4367-81c7-6eafb994c3f3_isrForCrosstalkSources_996980_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0a08623f-10ee-4367-81c7-6eafb994c3f3_isrForCrosstalkSources_996980_23.stderr parsl.dataflow.dflow DEBUG: Task 3744 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3744 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3744 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3744 has memoization hash 5b21a429dc9c668db746af84f8286089 parsl.dataflow.memoization INFO: Task 3744 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3744 try 0 launched on executor multi with executor id 3745 parsl.dataflow.dflow INFO: Standard output for task 3744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/44b17767-4c91-4f69-bd6f-d359202563e9_isrForCrosstalkSources_997012_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/44b17767-4c91-4f69-bd6f-d359202563e9_isrForCrosstalkSources_997012_59.stderr parsl.dataflow.dflow DEBUG: Task 3745 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3745 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3745 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3745 has memoization hash fa44b3260be4058d4f484b70685e8362 parsl.dataflow.memoization INFO: Task 3745 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3745 try 0 launched on executor multi with executor id 3746 parsl.dataflow.dflow INFO: Standard output for task 3745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/de07963c-008f-4d30-8734-7a04cc30f58f_isrForCrosstalkSources_996954_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/de07963c-008f-4d30-8734-7a04cc30f58f_isrForCrosstalkSources_996954_49.stderr parsl.dataflow.dflow DEBUG: Task 3746 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3746 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3746 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3746 has memoization hash 64643ffe90ed6fc13a4e0a91439ad175 parsl.dataflow.memoization INFO: Task 3746 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f31a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3746 try 0 launched on executor multi with executor id 3747 parsl.dataflow.dflow INFO: Standard output for task 3746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/0ea9c1ce-7137-4c46-8dfa-ee9a6b3efd13_isrForCrosstalkSources_996955_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/0ea9c1ce-7137-4c46-8dfa-ee9a6b3efd13_isrForCrosstalkSources_996955_44.stderr parsl.dataflow.dflow DEBUG: Task 3747 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3747 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3747 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3747 has memoization hash e7014b2d6ea6a9c95181486196eabcf8 parsl.dataflow.memoization INFO: Task 3747 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f2fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3747 try 0 launched on executor multi with executor id 3748 parsl.dataflow.dflow INFO: Standard output for task 3747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6399ad81-c9d3-4c30-8bd0-23efb976fdbc_isrForCrosstalkSources_996991_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6399ad81-c9d3-4c30-8bd0-23efb976fdbc_isrForCrosstalkSources_996991_31.stderr parsl.dataflow.dflow DEBUG: Task 3748 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3748 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3748 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3748 has memoization hash 1a96b11d6562270bb24fcc79bbce5ad6 parsl.dataflow.memoization INFO: Task 3748 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f32e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3748 try 0 launched on executor multi with executor id 3749 parsl.dataflow.dflow INFO: Standard output for task 3748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b8f97ad0-11b3-4d6c-acd3-1d77e7a473cd_isrForCrosstalkSources_996962_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b8f97ad0-11b3-4d6c-acd3-1d77e7a473cd_isrForCrosstalkSources_996962_61.stderr parsl.dataflow.dflow DEBUG: Task 3749 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3749 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3749 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3749 has memoization hash e2f89d47a50829af8701884d8576a82c parsl.dataflow.memoization INFO: Task 3749 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3749 try 0 launched on executor multi with executor id 3750 parsl.dataflow.dflow INFO: Standard output for task 3749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/eda6619a-aba7-49ef-a873-36b6956a0fa9_isrForCrosstalkSources_996960_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/eda6619a-aba7-49ef-a873-36b6956a0fa9_isrForCrosstalkSources_996960_19.stderr parsl.dataflow.dflow DEBUG: Task 3750 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3750 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3750 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3750 has memoization hash 6404e86d24d028c843133cdf604bf9f9 parsl.dataflow.memoization INFO: Task 3750 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3750 try 0 launched on executor multi with executor id 3751 parsl.dataflow.dflow INFO: Standard output for task 3750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/35d3dc6d-90ba-44d5-90b6-88ffe9a138a9_isrForCrosstalkSources_997001_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/35d3dc6d-90ba-44d5-90b6-88ffe9a138a9_isrForCrosstalkSources_997001_52.stderr parsl.dataflow.dflow DEBUG: Task 3751 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3751 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3751 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3751 has memoization hash c4dd724a6e2f0900dc98509e530bc565 parsl.dataflow.memoization INFO: Task 3751 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3751 try 0 launched on executor multi with executor id 3752 parsl.dataflow.dflow INFO: Standard output for task 3751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4387a87e-f040-4415-a0af-c692a8b1bea5_isrForCrosstalkSources_996962_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4387a87e-f040-4415-a0af-c692a8b1bea5_isrForCrosstalkSources_996962_22.stderr parsl.dataflow.dflow DEBUG: Task 3752 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3752 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3752 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3752 has memoization hash 45c1170b3e77c1452af8ba1d68fcd21b parsl.dataflow.memoization INFO: Task 3752 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f34c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3752 try 0 launched on executor multi with executor id 3753 parsl.dataflow.dflow INFO: Standard output for task 3752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/8e64767d-bfd6-48e5-bb78-48525b3e9813_isrForCrosstalkSources_997021_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/8e64767d-bfd6-48e5-bb78-48525b3e9813_isrForCrosstalkSources_997021_60.stderr parsl.dataflow.dflow DEBUG: Task 3753 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3753 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3753 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3753 has memoization hash 79524ef136fe15ae45618b12b9206397 parsl.dataflow.memoization INFO: Task 3753 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3753 try 0 launched on executor multi with executor id 3754 parsl.dataflow.dflow INFO: Standard output for task 3753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e9df7005-33b9-43c0-a5b5-6a982406c126_isrForCrosstalkSources_996983_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e9df7005-33b9-43c0-a5b5-6a982406c126_isrForCrosstalkSources_996983_41.stderr parsl.dataflow.dflow DEBUG: Task 3754 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3754 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3754 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3754 has memoization hash cf595e8776bdb4f30fdb05ab35ff914a parsl.dataflow.memoization INFO: Task 3754 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3754 try 0 launched on executor multi with executor id 3755 parsl.dataflow.dflow INFO: Standard output for task 3754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e4155eb4-31bf-464a-bad9-20324c182cf0_isrForCrosstalkSources_996998_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e4155eb4-31bf-464a-bad9-20324c182cf0_isrForCrosstalkSources_996998_16.stderr parsl.dataflow.dflow DEBUG: Task 3755 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3755 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3755 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3755 has memoization hash 9b1235a197480f0689b472feda164b21 parsl.dataflow.memoization INFO: Task 3755 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f36a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3755 try 0 launched on executor multi with executor id 3756 parsl.dataflow.dflow INFO: Standard output for task 3755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/d3e9d7d4-aa47-43c8-8571-826fcf554107_isrForCrosstalkSources_996983_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/d3e9d7d4-aa47-43c8-8571-826fcf554107_isrForCrosstalkSources_996983_30.stderr parsl.dataflow.dflow DEBUG: Task 3756 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3756 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3756 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3756 has memoization hash b40dc79f787966ed4ab74dcff4b24e93 parsl.dataflow.memoization INFO: Task 3756 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3756 try 0 launched on executor multi with executor id 3757 parsl.dataflow.dflow INFO: Standard output for task 3756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5b2d0ac3-5253-440c-a9b7-8209be24bf10_isrForCrosstalkSources_996972_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5b2d0ac3-5253-440c-a9b7-8209be24bf10_isrForCrosstalkSources_996972_49.stderr parsl.dataflow.dflow DEBUG: Task 3757 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3757 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3757 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3757 has memoization hash 70a922d3797f5ebedc8ca3fd9b4b5801 parsl.dataflow.memoization INFO: Task 3757 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3757 try 0 launched on executor multi with executor id 3758 parsl.dataflow.dflow INFO: Standard output for task 3757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/00e8d836-1fa1-4b20-9bfe-3a88ae9f0688_isrForCrosstalkSources_996980_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/00e8d836-1fa1-4b20-9bfe-3a88ae9f0688_isrForCrosstalkSources_996980_59.stderr parsl.dataflow.dflow DEBUG: Task 3758 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3758 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3758 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3758 has memoization hash 33af6aea1808ee31ebdf5c645c84d908 parsl.dataflow.memoization INFO: Task 3758 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3758 try 0 launched on executor multi with executor id 3759 parsl.dataflow.dflow INFO: Standard output for task 3758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/f49a8758-02d5-4a73-90dd-563a150a8bb6_isrForCrosstalkSources_997011_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/f49a8758-02d5-4a73-90dd-563a150a8bb6_isrForCrosstalkSources_997011_12.stderr parsl.dataflow.dflow DEBUG: Task 3759 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3759 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3759 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3759 has memoization hash 9aee7b37ebf59ec4a370a8acff438f8d parsl.dataflow.memoization INFO: Task 3759 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3759 try 0 launched on executor multi with executor id 3760 parsl.dataflow.dflow INFO: Standard output for task 3759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/44fdc416-6f56-4960-900e-5fc4fb8e947e_isrForCrosstalkSources_996954_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/44fdc416-6f56-4960-900e-5fc4fb8e947e_isrForCrosstalkSources_996954_53.stderr parsl.dataflow.dflow DEBUG: Task 3760 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3760 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3760 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3760 has memoization hash 51834262c567eb146d1988b49c5e2443 parsl.dataflow.memoization INFO: Task 3760 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f39c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3760 try 0 launched on executor multi with executor id 3761 parsl.dataflow.dflow INFO: Standard output for task 3760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/97bf9157-5b87-4b47-8eff-92bb74720532_isrForCrosstalkSources_996974_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/97bf9157-5b87-4b47-8eff-92bb74720532_isrForCrosstalkSources_996974_28.stderr parsl.dataflow.dflow DEBUG: Task 3761 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3761 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3761 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3761 has memoization hash 134dfde1d6073078c19bec55732dc0ef parsl.dataflow.memoization INFO: Task 3761 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3761 try 0 launched on executor multi with executor id 3762 parsl.dataflow.dflow INFO: Standard output for task 3761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cad81c87-f1a2-4403-a833-f5ded003bf4b_isrForCrosstalkSources_997024_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cad81c87-f1a2-4403-a833-f5ded003bf4b_isrForCrosstalkSources_997024_62.stderr parsl.dataflow.dflow DEBUG: Task 3762 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3762 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3762 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3762 has memoization hash 24c5dc22a1982dd8dcbb8092d7caf412 parsl.dataflow.memoization INFO: Task 3762 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f37e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3762 try 0 launched on executor multi with executor id 3763 parsl.dataflow.dflow INFO: Standard output for task 3762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d3f4a938-ac1e-4bdb-aeb1-7519026d8089_isrForCrosstalkSources_997008_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d3f4a938-ac1e-4bdb-aeb1-7519026d8089_isrForCrosstalkSources_997008_32.stderr parsl.dataflow.dflow DEBUG: Task 3763 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3763 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3763 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3763 has memoization hash f38a453f4c0fd14ae308e43fdd490636 parsl.dataflow.memoization INFO: Task 3763 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3763 try 0 launched on executor multi with executor id 3764 parsl.dataflow.dflow INFO: Standard output for task 3763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2c2cc430-62f2-49d9-b8b9-733e27cb10c5_isrForCrosstalkSources_996962_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2c2cc430-62f2-49d9-b8b9-733e27cb10c5_isrForCrosstalkSources_996962_24.stderr parsl.dataflow.dflow DEBUG: Task 3764 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3764 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3764 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3764 has memoization hash 6edc21787f24d925ca45e4673ba2ae92 parsl.dataflow.memoization INFO: Task 3764 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3764 try 0 launched on executor multi with executor id 3765 parsl.dataflow.dflow INFO: Standard output for task 3764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c82dfea5-d8ce-454a-9ceb-35ea3736b518_isrForCrosstalkSources_997001_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c82dfea5-d8ce-454a-9ceb-35ea3736b518_isrForCrosstalkSources_997001_33.stderr parsl.dataflow.dflow DEBUG: Task 3765 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3765 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3765 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3765 has memoization hash 38917fce167c22f244f8a125cf74a5f0 parsl.dataflow.memoization INFO: Task 3765 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3765 try 0 launched on executor multi with executor id 3766 parsl.dataflow.dflow INFO: Standard output for task 3765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/bf3e4baf-bcb7-46fd-8fe2-85836409a792_isrForCrosstalkSources_997000_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/bf3e4baf-bcb7-46fd-8fe2-85836409a792_isrForCrosstalkSources_997000_3.stderr parsl.dataflow.dflow DEBUG: Task 3766 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3766 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3766 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3766 has memoization hash bdbbfc2be277c52243daebace3bd19de parsl.dataflow.memoization INFO: Task 3766 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f13a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3766 try 0 launched on executor multi with executor id 3767 parsl.dataflow.dflow INFO: Standard output for task 3766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/141eef14-1a62-4c1b-82ff-7bfe9075fecf_isrForCrosstalkSources_996978_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/141eef14-1a62-4c1b-82ff-7bfe9075fecf_isrForCrosstalkSources_996978_59.stderr parsl.dataflow.dflow DEBUG: Task 3767 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3767 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3767 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3767 has memoization hash 5d9fe48f54df4ee1456afa71f987a338 parsl.dataflow.memoization INFO: Task 3767 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f3f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3767 try 0 launched on executor multi with executor id 3768 parsl.dataflow.dflow INFO: Standard output for task 3767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e0cdb5f4-65d2-45e6-baa8-9750dbf33044_isrForCrosstalkSources_996964_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e0cdb5f4-65d2-45e6-baa8-9750dbf33044_isrForCrosstalkSources_996964_53.stderr parsl.dataflow.dflow DEBUG: Task 3768 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3768 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3768 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3768 has memoization hash 09b534ad5c898c2e51700e9d5846e215 parsl.dataflow.memoization INFO: Task 3768 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3768 try 0 launched on executor multi with executor id 3769 parsl.dataflow.dflow INFO: Standard output for task 3768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/96b4f9bb-d6a9-47a2-bede-67492eeda27f_isrForCrosstalkSources_997028_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/96b4f9bb-d6a9-47a2-bede-67492eeda27f_isrForCrosstalkSources_997028_28.stderr parsl.dataflow.dflow DEBUG: Task 3769 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3769 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3769 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3769 has memoization hash 636c92794854fadd98a88236ec1c214e parsl.dataflow.memoization INFO: Task 3769 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3769 try 0 launched on executor multi with executor id 3770 parsl.dataflow.dflow INFO: Standard output for task 3769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b9c305d9-77ec-449b-be55-e91851c3babb_isrForCrosstalkSources_996986_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b9c305d9-77ec-449b-be55-e91851c3babb_isrForCrosstalkSources_996986_11.stderr parsl.dataflow.dflow DEBUG: Task 3770 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3770 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3770 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3770 has memoization hash 3cc6ee0908b4dd68dc096d1b022fc1f2 parsl.dataflow.memoization INFO: Task 3770 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3770 try 0 launched on executor multi with executor id 3771 parsl.dataflow.dflow INFO: Standard output for task 3770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/2d031450-a403-4123-97df-fbfb48bfd6a3_isrForCrosstalkSources_997013_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/2d031450-a403-4123-97df-fbfb48bfd6a3_isrForCrosstalkSources_997013_4.stderr parsl.dataflow.dflow DEBUG: Task 3771 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3771 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3771 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3771 has memoization hash 2fe2968c772c53850d753c9395abd153 parsl.dataflow.memoization INFO: Task 3771 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3771 try 0 launched on executor multi with executor id 3772 parsl.dataflow.dflow INFO: Standard output for task 3771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3ea19d43-d8ff-49dc-ba7a-233c110410d5_isrForCrosstalkSources_997019_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3ea19d43-d8ff-49dc-ba7a-233c110410d5_isrForCrosstalkSources_997019_25.stderr parsl.dataflow.dflow DEBUG: Task 3772 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3772 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3772 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3772 has memoization hash 374294ca90fe288d02917d42834f1fcd parsl.dataflow.memoization INFO: Task 3772 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3772 try 0 launched on executor multi with executor id 3773 parsl.dataflow.dflow INFO: Standard output for task 3772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f4162314-9760-47d5-93f4-4b226a9f20e4_isrForCrosstalkSources_996999_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f4162314-9760-47d5-93f4-4b226a9f20e4_isrForCrosstalkSources_996999_56.stderr parsl.dataflow.dflow DEBUG: Task 3773 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3773 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3773 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3773 has memoization hash 137ca423108cd39a146420ebddc99feb parsl.dataflow.memoization INFO: Task 3773 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3773 try 0 launched on executor multi with executor id 3774 parsl.dataflow.dflow INFO: Standard output for task 3773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/42bef535-11f7-4ca9-b7b7-dafd0837fa75_isrForCrosstalkSources_996999_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/42bef535-11f7-4ca9-b7b7-dafd0837fa75_isrForCrosstalkSources_996999_49.stderr parsl.dataflow.dflow DEBUG: Task 3774 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3774 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3774 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3774 has memoization hash 0f4f6848aceb727fd7a176ca6acdfc0a parsl.dataflow.memoization INFO: Task 3774 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3774 try 0 launched on executor multi with executor id 3775 parsl.dataflow.dflow INFO: Standard output for task 3774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1963cef7-b361-45ba-84e7-0181b570b1fc_isrForCrosstalkSources_996977_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1963cef7-b361-45ba-84e7-0181b570b1fc_isrForCrosstalkSources_996977_26.stderr parsl.dataflow.dflow DEBUG: Task 3775 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3775 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3775 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3775 has memoization hash 6b5eb75a8596d27cb0405a02c2e57f67 parsl.dataflow.memoization INFO: Task 3775 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3775 try 0 launched on executor multi with executor id 3776 parsl.dataflow.dflow INFO: Standard output for task 3775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/689e85b4-b3f5-45aa-a939-40dfd49d3267_isrForCrosstalkSources_996971_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/689e85b4-b3f5-45aa-a939-40dfd49d3267_isrForCrosstalkSources_996971_49.stderr parsl.dataflow.dflow DEBUG: Task 3776 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3776 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3776 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3776 has memoization hash 37e6a53099ef20aca640dcdaab6ec8e8 parsl.dataflow.memoization INFO: Task 3776 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3776 try 0 launched on executor multi with executor id 3777 parsl.dataflow.dflow INFO: Standard output for task 3776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e06d7f00-d09a-4541-b5a7-89d50eeab592_isrForCrosstalkSources_996983_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e06d7f00-d09a-4541-b5a7-89d50eeab592_isrForCrosstalkSources_996983_47.stderr parsl.dataflow.dflow DEBUG: Task 3777 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3777 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3777 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3777 has memoization hash e53ed00693044e81c42d3b15600f9a21 parsl.dataflow.memoization INFO: Task 3777 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3777 try 0 launched on executor multi with executor id 3778 parsl.dataflow.dflow INFO: Standard output for task 3777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/86a9525a-0f29-4fdb-bad9-d1182f8be0de_isrForCrosstalkSources_997002_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/86a9525a-0f29-4fdb-bad9-d1182f8be0de_isrForCrosstalkSources_997002_46.stderr parsl.dataflow.dflow DEBUG: Task 3778 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3778 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3778 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3778 has memoization hash 1ce7f251b8bb78ec241bb71c1295ccba parsl.dataflow.memoization INFO: Task 3778 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3778 try 0 launched on executor multi with executor id 3779 parsl.dataflow.dflow INFO: Standard output for task 3778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a11a0ec0-f0cd-473b-a507-4d06f326a5d9_isrForCrosstalkSources_997008_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a11a0ec0-f0cd-473b-a507-4d06f326a5d9_isrForCrosstalkSources_997008_21.stderr parsl.dataflow.dflow DEBUG: Task 3779 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3779 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3779 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3779 has memoization hash a0b12474824eafadc3ef775f82695e70 parsl.dataflow.memoization INFO: Task 3779 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3779 try 0 launched on executor multi with executor id 3780 parsl.dataflow.dflow INFO: Standard output for task 3779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/26dbba2b-fb37-46ba-825a-74afaeebaa0e_isrForCrosstalkSources_997007_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/26dbba2b-fb37-46ba-825a-74afaeebaa0e_isrForCrosstalkSources_997007_15.stderr parsl.dataflow.dflow DEBUG: Task 3780 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3780 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3780 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3780 has memoization hash 4c0661237b3637c0497d91621670abdf parsl.dataflow.memoization INFO: Task 3780 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3780 try 0 launched on executor multi with executor id 3781 parsl.dataflow.dflow INFO: Standard output for task 3780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/ff56f6a5-5368-4015-b891-70380f0cdf01_isrForCrosstalkSources_996968_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/ff56f6a5-5368-4015-b891-70380f0cdf01_isrForCrosstalkSources_996968_4.stderr parsl.dataflow.dflow DEBUG: Task 3781 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3781 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3781 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3781 has memoization hash 6101d543b1e0c43db8dbeea67c69774a parsl.dataflow.memoization INFO: Task 3781 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3781 try 0 launched on executor multi with executor id 3782 parsl.dataflow.dflow INFO: Standard output for task 3781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/08d9a9a2-5c0a-4317-9655-0107f74bc733_isrForCrosstalkSources_996975_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/08d9a9a2-5c0a-4317-9655-0107f74bc733_isrForCrosstalkSources_996975_42.stderr parsl.dataflow.dflow DEBUG: Task 3782 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3782 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3782 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3782 has memoization hash d4ac6dcc4ec0d2cad398769fff1f0cc9 parsl.dataflow.memoization INFO: Task 3782 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3782 try 0 launched on executor multi with executor id 3783 parsl.dataflow.dflow INFO: Standard output for task 3782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/7424823e-4925-4558-8b1d-b07fb8ff3518_isrForCrosstalkSources_997019_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/7424823e-4925-4558-8b1d-b07fb8ff3518_isrForCrosstalkSources_997019_8.stderr parsl.dataflow.dflow DEBUG: Task 3783 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3783 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3783 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3783 has memoization hash 528189fd0053423eb32e54bd1941e3b8 parsl.dataflow.memoization INFO: Task 3783 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3783 try 0 launched on executor multi with executor id 3784 parsl.dataflow.dflow INFO: Standard output for task 3783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/52a697d3-2731-4039-87b6-53cf4552de25_isrForCrosstalkSources_997013_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/52a697d3-2731-4039-87b6-53cf4552de25_isrForCrosstalkSources_997013_51.stderr parsl.dataflow.dflow DEBUG: Task 3784 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3784 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3784 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3784 has memoization hash 1d0888377dc12b6a586247c1ced07647 parsl.dataflow.memoization INFO: Task 3784 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3784 try 0 launched on executor multi with executor id 3785 parsl.dataflow.dflow INFO: Standard output for task 3784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/39789347-3276-4af7-9cf8-a92099708856_isrForCrosstalkSources_997026_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/39789347-3276-4af7-9cf8-a92099708856_isrForCrosstalkSources_997026_35.stderr parsl.dataflow.dflow DEBUG: Task 3785 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3785 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3785 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3785 has memoization hash e437174d292f93a80f1631e652694f52 parsl.dataflow.memoization INFO: Task 3785 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3785 try 0 launched on executor multi with executor id 3786 parsl.dataflow.dflow INFO: Standard output for task 3785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3fa0b926-17b6-426f-b38c-6ae23d44cc99_isrForCrosstalkSources_996963_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3fa0b926-17b6-426f-b38c-6ae23d44cc99_isrForCrosstalkSources_996963_58.stderr parsl.dataflow.dflow DEBUG: Task 3786 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3786 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3786 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3786 has memoization hash c2b4ccf874c6a4b399652002101b1601 parsl.dataflow.memoization INFO: Task 3786 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3786 try 0 launched on executor multi with executor id 3787 parsl.dataflow.dflow INFO: Standard output for task 3786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e0e6870d-fb5d-4fc6-896c-f0c2cd1ae1bc_isrForCrosstalkSources_997005_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e0e6870d-fb5d-4fc6-896c-f0c2cd1ae1bc_isrForCrosstalkSources_997005_46.stderr parsl.dataflow.dflow DEBUG: Task 3787 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3787 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3787 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3787 has memoization hash 6b36a78c84a60ecd1ce2f9ea42d80416 parsl.dataflow.memoization INFO: Task 3787 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3787 try 0 launched on executor multi with executor id 3788 parsl.dataflow.dflow INFO: Standard output for task 3787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6517670c-8319-4a01-9ad7-9d55f41e9004_isrForCrosstalkSources_996996_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6517670c-8319-4a01-9ad7-9d55f41e9004_isrForCrosstalkSources_996996_12.stderr parsl.dataflow.dflow DEBUG: Task 3788 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3788 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3788 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3788 has memoization hash 1422d93f6a58b3912a47aa6a24cdc9dd parsl.dataflow.memoization INFO: Task 3788 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3788 try 0 launched on executor multi with executor id 3789 parsl.dataflow.dflow INFO: Standard output for task 3788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/e3abf641-c857-49d9-a9a8-62147cbc1125_isrForCrosstalkSources_997006_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/e3abf641-c857-49d9-a9a8-62147cbc1125_isrForCrosstalkSources_997006_2.stderr parsl.dataflow.dflow DEBUG: Task 3789 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3789 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3789 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3789 has memoization hash 80e13eb1aeb969ab1c0fe9bedb807547 parsl.dataflow.memoization INFO: Task 3789 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3789 try 0 launched on executor multi with executor id 3790 parsl.dataflow.dflow INFO: Standard output for task 3789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/213ff86c-362c-41cb-a126-7c9bff43d1df_isrForCrosstalkSources_996975_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/213ff86c-362c-41cb-a126-7c9bff43d1df_isrForCrosstalkSources_996975_45.stderr parsl.dataflow.dflow DEBUG: Task 3790 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3790 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3790 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3790 has memoization hash e5c90e85f67e2d6f82548de25cb98fea parsl.dataflow.memoization INFO: Task 3790 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3790 try 0 launched on executor multi with executor id 3791 parsl.dataflow.dflow INFO: Standard output for task 3790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d6c13469-2ee2-4e3b-b3fb-6c59120dee38_isrForCrosstalkSources_996958_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d6c13469-2ee2-4e3b-b3fb-6c59120dee38_isrForCrosstalkSources_996958_37.stderr parsl.dataflow.dflow DEBUG: Task 3791 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3791 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3791 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3791 has memoization hash 9d85169c8d37c8d812dde1c00993a5b0 parsl.dataflow.memoization INFO: Task 3791 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3791 try 0 launched on executor multi with executor id 3792 parsl.dataflow.dflow INFO: Standard output for task 3791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/83b26efd-57fe-4242-b5cd-e3e2f3e54e2e_isrForCrosstalkSources_997020_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/83b26efd-57fe-4242-b5cd-e3e2f3e54e2e_isrForCrosstalkSources_997020_48.stderr parsl.dataflow.dflow DEBUG: Task 3792 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3792 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3792 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3792 has memoization hash 9eace6bc6483b71f2ce423291ab4c631 parsl.dataflow.memoization INFO: Task 3792 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3792 try 0 launched on executor multi with executor id 3793 parsl.dataflow.dflow INFO: Standard output for task 3792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e5aa79fe-d14c-46d4-b32d-3aaa5e88a995_isrForCrosstalkSources_997008_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e5aa79fe-d14c-46d4-b32d-3aaa5e88a995_isrForCrosstalkSources_997008_24.stderr parsl.dataflow.dflow DEBUG: Task 3793 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3793 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3793 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3793 has memoization hash b76012fb752f99a3c357f1f01b1187f5 parsl.dataflow.memoization INFO: Task 3793 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3793 try 0 launched on executor multi with executor id 3794 parsl.dataflow.dflow INFO: Standard output for task 3793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/0c930d80-4d31-4ef3-9f67-128b940f68d9_isrForCrosstalkSources_996996_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/0c930d80-4d31-4ef3-9f67-128b940f68d9_isrForCrosstalkSources_996996_40.stderr parsl.dataflow.dflow DEBUG: Task 3794 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3794 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3794 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3794 has memoization hash 84382b9913c458c8edd2d753c4fd2ca6 parsl.dataflow.memoization INFO: Task 3794 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3794 try 0 launched on executor multi with executor id 3795 parsl.dataflow.dflow INFO: Standard output for task 3794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e0e04b72-8966-4cbc-b723-0f9a9963fa64_isrForCrosstalkSources_996995_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e0e04b72-8966-4cbc-b723-0f9a9963fa64_isrForCrosstalkSources_996995_57.stderr parsl.dataflow.dflow DEBUG: Task 3795 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3795 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3795 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3795 has memoization hash c93eb64ebbf1916e955c94158d7817c8 parsl.dataflow.memoization INFO: Task 3795 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3795 try 0 launched on executor multi with executor id 3796 parsl.dataflow.dflow INFO: Standard output for task 3795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0d32b070-c6f8-472a-8ccd-6002be0ea72e_isrForCrosstalkSources_997023_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0d32b070-c6f8-472a-8ccd-6002be0ea72e_isrForCrosstalkSources_997023_55.stderr parsl.dataflow.dflow DEBUG: Task 3796 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3796 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3796 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3796 has memoization hash b0f152f74278fc93ded96a5256c61838 parsl.dataflow.memoization INFO: Task 3796 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3796 try 0 launched on executor multi with executor id 3797 parsl.dataflow.dflow INFO: Standard output for task 3796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/41250f54-cc37-4c05-93d6-ae08c5eb004a_isrForCrosstalkSources_997027_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/41250f54-cc37-4c05-93d6-ae08c5eb004a_isrForCrosstalkSources_997027_11.stderr parsl.dataflow.dflow DEBUG: Task 3797 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3797 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3797 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3797 has memoization hash 3f2343115503c14d9262dee5018aeb8d parsl.dataflow.memoization INFO: Task 3797 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3797 try 0 launched on executor multi with executor id 3798 parsl.dataflow.dflow INFO: Standard output for task 3797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/24d1276e-9655-4668-a656-ad603693ad42_isrForCrosstalkSources_997033_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/24d1276e-9655-4668-a656-ad603693ad42_isrForCrosstalkSources_997033_6.stderr parsl.dataflow.dflow DEBUG: Task 3798 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3798 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3798 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3798 has memoization hash ab61efe065809faf4d58c3bc9b780f24 parsl.dataflow.memoization INFO: Task 3798 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3798 try 0 launched on executor multi with executor id 3799 parsl.dataflow.dflow INFO: Standard output for task 3798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/fb27cb9c-4d11-4b16-8704-25aa082307a2_isrForCrosstalkSources_996973_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/fb27cb9c-4d11-4b16-8704-25aa082307a2_isrForCrosstalkSources_996973_29.stderr parsl.dataflow.dflow DEBUG: Task 3799 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3799 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3799 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3799 has memoization hash b5730ad896e39d3778733ef1f65da220 parsl.dataflow.memoization INFO: Task 3799 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3799 try 0 launched on executor multi with executor id 3800 parsl.dataflow.dflow INFO: Standard output for task 3799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/3656be07-bbba-4d08-9581-99ce5e9d7846_isrForCrosstalkSources_997008_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/3656be07-bbba-4d08-9581-99ce5e9d7846_isrForCrosstalkSources_997008_25.stderr parsl.dataflow.dflow DEBUG: Task 3800 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3800 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3800 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3800 has memoization hash e0d28aeea6724d3c354774b4a2f29e5d parsl.dataflow.memoization INFO: Task 3800 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3800 try 0 launched on executor multi with executor id 3801 parsl.dataflow.dflow INFO: Standard output for task 3800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eafc093-107a-48a0-8a63-4847c0581b97_isrForCrosstalkSources_996985_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eafc093-107a-48a0-8a63-4847c0581b97_isrForCrosstalkSources_996985_36.stderr parsl.dataflow.dflow DEBUG: Task 3801 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3801 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3801 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3801 has memoization hash 46024288cb3cae0c3124b12d55fc43b8 parsl.dataflow.memoization INFO: Task 3801 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3801 try 0 launched on executor multi with executor id 3802 parsl.dataflow.dflow INFO: Standard output for task 3801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/8e79ef15-110f-4eb2-a94c-973be8dd6aea_isrForCrosstalkSources_997009_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/8e79ef15-110f-4eb2-a94c-973be8dd6aea_isrForCrosstalkSources_997009_45.stderr parsl.dataflow.dflow DEBUG: Task 3802 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3802 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3802 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3802 has memoization hash 88cadbd5592b792d760b9909ca2d9327 parsl.dataflow.memoization INFO: Task 3802 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3802 try 0 launched on executor multi with executor id 3803 parsl.dataflow.dflow INFO: Standard output for task 3802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/927c031d-f18c-4195-baa7-bfcefd8653d4_isrForCrosstalkSources_996964_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/927c031d-f18c-4195-baa7-bfcefd8653d4_isrForCrosstalkSources_996964_16.stderr parsl.dataflow.dflow DEBUG: Task 3803 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3803 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3803 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3803 has memoization hash cddd114aa0d78c60c2deb2f313bb76dd parsl.dataflow.memoization INFO: Task 3803 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3803 try 0 launched on executor multi with executor id 3804 parsl.dataflow.dflow INFO: Standard output for task 3803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fe870f27-3f76-4e69-a28d-75f4484e6d1e_isrForCrosstalkSources_997001_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fe870f27-3f76-4e69-a28d-75f4484e6d1e_isrForCrosstalkSources_997001_56.stderr parsl.dataflow.dflow DEBUG: Task 3804 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3804 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3804 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3804 has memoization hash b238d3377af727d892bab61a355b31ad parsl.dataflow.memoization INFO: Task 3804 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3804 try 0 launched on executor multi with executor id 3805 parsl.dataflow.dflow INFO: Standard output for task 3804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/591be311-4487-4480-adab-b4c239d43c92_isrForCrosstalkSources_997008_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/591be311-4487-4480-adab-b4c239d43c92_isrForCrosstalkSources_997008_43.stderr parsl.dataflow.dflow DEBUG: Task 3805 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3805 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3805 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3805 has memoization hash 58c8ce2cb50b023c5c251608d92f06ec parsl.dataflow.memoization INFO: Task 3805 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3805 try 0 launched on executor multi with executor id 3806 parsl.dataflow.dflow INFO: Standard output for task 3805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/323875c8-f946-49ce-9ba8-8784d8559232_isrForCrosstalkSources_997003_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/323875c8-f946-49ce-9ba8-8784d8559232_isrForCrosstalkSources_997003_58.stderr parsl.dataflow.dflow DEBUG: Task 3806 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3806 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3806 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3806 has memoization hash b28908f601c5cf5b363b341f09530ef4 parsl.dataflow.memoization INFO: Task 3806 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3806 try 0 launched on executor multi with executor id 3807 parsl.dataflow.dflow INFO: Standard output for task 3806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/1ba0243b-5a7a-4352-91fc-f04c8f50461e_isrForCrosstalkSources_996962_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/1ba0243b-5a7a-4352-91fc-f04c8f50461e_isrForCrosstalkSources_996962_40.stderr parsl.dataflow.dflow DEBUG: Task 3807 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3807 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3807 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3807 has memoization hash 04ecd9cfd80f7cdec42011cde4ede68c parsl.dataflow.memoization INFO: Task 3807 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3807 try 0 launched on executor multi with executor id 3808 parsl.dataflow.dflow INFO: Standard output for task 3807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0362cb2c-d119-4051-a085-2526ae9c5e2d_isrForCrosstalkSources_997007_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0362cb2c-d119-4051-a085-2526ae9c5e2d_isrForCrosstalkSources_997007_57.stderr parsl.dataflow.dflow DEBUG: Task 3808 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3808 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3808 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3808 has memoization hash 86771a2a85d968aa6691db052aefbab2 parsl.dataflow.memoization INFO: Task 3808 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3808 try 0 launched on executor multi with executor id 3809 parsl.dataflow.dflow INFO: Standard output for task 3808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/01680928-b54b-417f-9ad8-c27fa46354be_isrForCrosstalkSources_997008_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/01680928-b54b-417f-9ad8-c27fa46354be_isrForCrosstalkSources_997008_47.stderr parsl.dataflow.dflow DEBUG: Task 3809 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3809 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3809 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3809 has memoization hash a21d8e1329d626c4516bb287ed40759b parsl.dataflow.memoization INFO: Task 3809 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3809 try 0 launched on executor multi with executor id 3810 parsl.dataflow.dflow INFO: Standard output for task 3809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/7f2d2d58-70b9-4a39-bc13-4604b7400ed0_isrForCrosstalkSources_996972_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/7f2d2d58-70b9-4a39-bc13-4604b7400ed0_isrForCrosstalkSources_996972_34.stderr parsl.dataflow.dflow DEBUG: Task 3810 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3810 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3810 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3810 has memoization hash ae7e8ed0c7dc5f247c76c6d8a97657b0 parsl.dataflow.memoization INFO: Task 3810 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3810 try 0 launched on executor multi with executor id 3811 parsl.dataflow.dflow INFO: Standard output for task 3810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/35be3402-f11e-49c7-9801-138a518d9273_isrForCrosstalkSources_997031_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/35be3402-f11e-49c7-9801-138a518d9273_isrForCrosstalkSources_997031_7.stderr parsl.dataflow.dflow DEBUG: Task 3811 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3811 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3811 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3811 has memoization hash 6c2b2ef4b9b906c7b70b5ec8a3a6aa13 parsl.dataflow.memoization INFO: Task 3811 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3811 try 0 launched on executor multi with executor id 3812 parsl.dataflow.dflow INFO: Standard output for task 3811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0b5d3bf9-857a-481a-a28a-ef0e2aa91feb_isrForCrosstalkSources_996980_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0b5d3bf9-857a-481a-a28a-ef0e2aa91feb_isrForCrosstalkSources_996980_38.stderr parsl.dataflow.dflow DEBUG: Task 3812 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3812 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3812 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3812 has memoization hash 48a61e06515780146909239bc2a0c80f parsl.dataflow.memoization INFO: Task 3812 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3812 try 0 launched on executor multi with executor id 3813 parsl.dataflow.dflow INFO: Standard output for task 3812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/e53a23d1-cc8b-41e2-af08-7e2d9e3a5187_isrForCrosstalkSources_997006_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/e53a23d1-cc8b-41e2-af08-7e2d9e3a5187_isrForCrosstalkSources_997006_46.stderr parsl.dataflow.dflow DEBUG: Task 3813 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3813 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3813 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3813 has memoization hash b74d96ab8f68982d47c5effdafa93e10 parsl.dataflow.memoization INFO: Task 3813 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3813 try 0 launched on executor multi with executor id 3814 parsl.dataflow.dflow INFO: Standard output for task 3813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/1d2e5425-5680-4b97-a6d1-36d447448412_isrForCrosstalkSources_996985_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/1d2e5425-5680-4b97-a6d1-36d447448412_isrForCrosstalkSources_996985_9.stderr parsl.dataflow.dflow DEBUG: Task 3814 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3814 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3814 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3814 has memoization hash a1f8a9757ff00c799c093c20fa9400ae parsl.dataflow.memoization INFO: Task 3814 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3814 try 0 launched on executor multi with executor id 3815 parsl.dataflow.dflow INFO: Standard output for task 3814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/7eb8e8c6-cde1-462f-b63f-76aa1042af3e_isrForCrosstalkSources_996992_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/7eb8e8c6-cde1-462f-b63f-76aa1042af3e_isrForCrosstalkSources_996992_30.stderr parsl.dataflow.dflow DEBUG: Task 3815 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3815 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3815 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3815 has memoization hash f25fee91ba1f91b90a5d5bc8d736a214 parsl.dataflow.memoization INFO: Task 3815 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3815 try 0 launched on executor multi with executor id 3816 parsl.dataflow.dflow INFO: Standard output for task 3815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e2dfc619-3ef0-4668-8434-14aba3d1ad61_isrForCrosstalkSources_996965_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e2dfc619-3ef0-4668-8434-14aba3d1ad61_isrForCrosstalkSources_996965_37.stderr parsl.dataflow.dflow DEBUG: Task 3816 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3816 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3816 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3816 has memoization hash 6c1e1fa7e9947d327d947bf7176cf992 parsl.dataflow.memoization INFO: Task 3816 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3816 try 0 launched on executor multi with executor id 3817 parsl.dataflow.dflow INFO: Standard output for task 3816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/f5678b82-a1fc-489b-913d-380f424e1bd6_isrForCrosstalkSources_996968_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/f5678b82-a1fc-489b-913d-380f424e1bd6_isrForCrosstalkSources_996968_54.stderr parsl.dataflow.dflow DEBUG: Task 3817 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3817 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3817 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3817 has memoization hash 9c87c7e0b611a87f15285ed26ff2a6d2 parsl.dataflow.memoization INFO: Task 3817 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3817 try 0 launched on executor multi with executor id 3818 parsl.dataflow.dflow INFO: Standard output for task 3817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1b3164fd-931a-4685-b19f-6c4c32d3acc0_isrForCrosstalkSources_997004_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1b3164fd-931a-4685-b19f-6c4c32d3acc0_isrForCrosstalkSources_997004_11.stderr parsl.dataflow.dflow DEBUG: Task 3818 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3818 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3818 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3818 has memoization hash 2d8f0358eeacc2026f8658f7020190e9 parsl.dataflow.memoization INFO: Task 3818 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3818 try 0 launched on executor multi with executor id 3819 parsl.dataflow.dflow INFO: Standard output for task 3818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d3e5c72a-7900-4be6-9c24-759dc29b018d_isrForCrosstalkSources_996998_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d3e5c72a-7900-4be6-9c24-759dc29b018d_isrForCrosstalkSources_996998_7.stderr parsl.dataflow.dflow DEBUG: Task 3819 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3819 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3819 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3819 has memoization hash cf1b717bd2ec6fb323fd1a5cf1498a76 parsl.dataflow.memoization INFO: Task 3819 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3819 try 0 launched on executor multi with executor id 3820 parsl.dataflow.dflow INFO: Standard output for task 3819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/12d28997-a2fd-40cf-a3e5-f8fb8fcf5a77_isrForCrosstalkSources_996977_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/12d28997-a2fd-40cf-a3e5-f8fb8fcf5a77_isrForCrosstalkSources_996977_38.stderr parsl.dataflow.dflow DEBUG: Task 3820 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3820 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3820 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3820 has memoization hash bf014002684b3515ae178f25f6a002b4 parsl.dataflow.memoization INFO: Task 3820 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3820 try 0 launched on executor multi with executor id 3821 parsl.dataflow.dflow INFO: Standard output for task 3820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/5af224e0-3a82-44f9-bc25-dee78b83b925_isrForCrosstalkSources_996962_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/5af224e0-3a82-44f9-bc25-dee78b83b925_isrForCrosstalkSources_996962_18.stderr parsl.dataflow.dflow DEBUG: Task 3821 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3821 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3821 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3821 has memoization hash 3da7838e6a6d7d1fe73cdc0bb40c6c48 parsl.dataflow.memoization INFO: Task 3821 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3821 try 0 launched on executor multi with executor id 3822 parsl.dataflow.dflow INFO: Standard output for task 3821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/5941fbef-6797-4286-b7c1-854125544cc6_isrForCrosstalkSources_997013_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/5941fbef-6797-4286-b7c1-854125544cc6_isrForCrosstalkSources_997013_57.stderr parsl.dataflow.dflow DEBUG: Task 3822 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3822 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3822 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3822 has memoization hash 5a60e7571dbbbf5550f9287f1831c0cc parsl.dataflow.memoization INFO: Task 3822 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3822 try 0 launched on executor multi with executor id 3823 parsl.dataflow.dflow INFO: Standard output for task 3822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/7e4f1cb4-2f2b-467b-a7b1-7de547b52d1b_isrForCrosstalkSources_996996_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/7e4f1cb4-2f2b-467b-a7b1-7de547b52d1b_isrForCrosstalkSources_996996_7.stderr parsl.dataflow.dflow DEBUG: Task 3823 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3823 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3823 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3823 has memoization hash 9b75a500870f6064c1b0bc6dc1f01cc7 parsl.dataflow.memoization INFO: Task 3823 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3823 try 0 launched on executor multi with executor id 3824 parsl.dataflow.dflow INFO: Standard output for task 3823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/d8826375-7278-4f34-b754-21a584b8e748_isrForCrosstalkSources_997013_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/d8826375-7278-4f34-b754-21a584b8e748_isrForCrosstalkSources_997013_55.stderr parsl.dataflow.dflow DEBUG: Task 3824 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3824 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3824 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3824 has memoization hash a7f516d6c873d75d0a3271f9145ef622 parsl.dataflow.memoization INFO: Task 3824 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3824 try 0 launched on executor multi with executor id 3825 parsl.dataflow.dflow INFO: Standard output for task 3824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/57e027f6-d58b-4cbc-8078-4a80f54b7b31_isrForCrosstalkSources_996959_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/57e027f6-d58b-4cbc-8078-4a80f54b7b31_isrForCrosstalkSources_996959_55.stderr parsl.dataflow.dflow DEBUG: Task 3825 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3825 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3825 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3825 has memoization hash 6ee3e9610bf1e93680c4f17cd6813fe0 parsl.dataflow.memoization INFO: Task 3825 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3825 try 0 launched on executor multi with executor id 3826 parsl.dataflow.dflow INFO: Standard output for task 3825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/827f780b-5ff7-48c0-b192-b1e11fba9b1c_isrForCrosstalkSources_996992_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/827f780b-5ff7-48c0-b192-b1e11fba9b1c_isrForCrosstalkSources_996992_51.stderr parsl.dataflow.dflow DEBUG: Task 3826 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3826 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3826 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3826 has memoization hash 01945313d5e71ef30e0c1499927352b4 parsl.dataflow.memoization INFO: Task 3826 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3826 try 0 launched on executor multi with executor id 3827 parsl.dataflow.dflow INFO: Standard output for task 3826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/71d909f1-04ef-4a4b-b885-120abc4f0ecb_isrForCrosstalkSources_996976_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/71d909f1-04ef-4a4b-b885-120abc4f0ecb_isrForCrosstalkSources_996976_5.stderr parsl.dataflow.dflow DEBUG: Task 3827 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3827 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3827 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3827 has memoization hash 89a0b9739e5aad5a34494a03be0372c8 parsl.dataflow.memoization INFO: Task 3827 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3827 try 0 launched on executor multi with executor id 3828 parsl.dataflow.dflow INFO: Standard output for task 3827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/80925e69-b017-4ca9-939a-65faf40d7be6_isrForCrosstalkSources_996964_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/80925e69-b017-4ca9-939a-65faf40d7be6_isrForCrosstalkSources_996964_38.stderr parsl.dataflow.dflow DEBUG: Task 3828 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3828 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3828 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3828 has memoization hash 191cf0b249932b9c7fd131821143bd65 parsl.dataflow.memoization INFO: Task 3828 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3828 try 0 launched on executor multi with executor id 3829 parsl.dataflow.dflow INFO: Standard output for task 3828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/dced2144-9e6a-4584-8545-8cde8f6434fe_isrForCrosstalkSources_996965_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/dced2144-9e6a-4584-8545-8cde8f6434fe_isrForCrosstalkSources_996965_3.stderr parsl.dataflow.dflow DEBUG: Task 3829 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3829 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3829 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3829 has memoization hash 035615b29b8a1406a5a5646dfeb854bb parsl.dataflow.memoization INFO: Task 3829 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3829 try 0 launched on executor multi with executor id 3830 parsl.dataflow.dflow INFO: Standard output for task 3829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/937e1bf0-f3ea-4c2d-8b51-ba39de4b7fca_isrForCrosstalkSources_997028_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/937e1bf0-f3ea-4c2d-8b51-ba39de4b7fca_isrForCrosstalkSources_997028_6.stderr parsl.dataflow.dflow DEBUG: Task 3830 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3830 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3830 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3830 has memoization hash 18bbac4383bdae16cd537a388b0032b0 parsl.dataflow.memoization INFO: Task 3830 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3830 try 0 launched on executor multi with executor id 3831 parsl.dataflow.dflow INFO: Standard output for task 3830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4c47c570-4681-461e-80c8-200c9858b068_isrForCrosstalkSources_997030_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4c47c570-4681-461e-80c8-200c9858b068_isrForCrosstalkSources_997030_57.stderr parsl.dataflow.dflow DEBUG: Task 3831 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3831 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3831 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3831 has memoization hash c9246168935f833a29502c1b280e36cc parsl.dataflow.memoization INFO: Task 3831 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3831 try 0 launched on executor multi with executor id 3832 parsl.dataflow.dflow INFO: Standard output for task 3831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bc9a7ef4-ea4b-4db1-8e7d-497fd436f3c6_isrForCrosstalkSources_997014_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bc9a7ef4-ea4b-4db1-8e7d-497fd436f3c6_isrForCrosstalkSources_997014_29.stderr parsl.dataflow.dflow DEBUG: Task 3832 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3832 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3832 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3832 has memoization hash c955743e2243800a70ac09fc77c7583f parsl.dataflow.memoization INFO: Task 3832 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3832 try 0 launched on executor multi with executor id 3833 parsl.dataflow.dflow INFO: Standard output for task 3832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a84dfd6f-7f3c-497b-bb41-263134819584_isrForCrosstalkSources_996972_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a84dfd6f-7f3c-497b-bb41-263134819584_isrForCrosstalkSources_996972_54.stderr parsl.dataflow.dflow DEBUG: Task 3833 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3833 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3833 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3833 has memoization hash 127bb696abc32d50954980027795f0fe parsl.dataflow.memoization INFO: Task 3833 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3833 try 0 launched on executor multi with executor id 3834 parsl.dataflow.dflow INFO: Standard output for task 3833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/000728c9-a311-4fa4-b096-00eef2eb6467_isrForCrosstalkSources_996998_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/000728c9-a311-4fa4-b096-00eef2eb6467_isrForCrosstalkSources_996998_30.stderr parsl.dataflow.dflow DEBUG: Task 3834 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3834 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3834 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3834 has memoization hash 483f1e2818dd56d15681379acea57e54 parsl.dataflow.memoization INFO: Task 3834 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3834 try 0 launched on executor multi with executor id 3835 parsl.dataflow.dflow INFO: Standard output for task 3834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/56e40733-20f4-4802-9f3d-38511541216e_isrForCrosstalkSources_996956_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/56e40733-20f4-4802-9f3d-38511541216e_isrForCrosstalkSources_996956_28.stderr parsl.dataflow.dflow DEBUG: Task 3835 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3835 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3835 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3835 has memoization hash ff4442c9ba59f3a3daf5b9fdb9e3108d parsl.dataflow.memoization INFO: Task 3835 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3835 try 0 launched on executor multi with executor id 3836 parsl.dataflow.dflow INFO: Standard output for task 3835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3ba178bd-5aff-472a-a38b-d1b0b53b2838_isrForCrosstalkSources_996988_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3ba178bd-5aff-472a-a38b-d1b0b53b2838_isrForCrosstalkSources_996988_15.stderr parsl.dataflow.dflow DEBUG: Task 3836 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3836 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3836 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3836 has memoization hash d3b0dcd1edcafdcce020ab4705e70826 parsl.dataflow.memoization INFO: Task 3836 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3836 try 0 launched on executor multi with executor id 3837 parsl.dataflow.dflow INFO: Standard output for task 3836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/0886d232-6538-4a8e-b215-8c69d5390d43_isrForCrosstalkSources_996968_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/0886d232-6538-4a8e-b215-8c69d5390d43_isrForCrosstalkSources_996968_38.stderr parsl.dataflow.dflow DEBUG: Task 3837 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3837 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3837 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3837 has memoization hash 0bd5edbbc00a80664b28029f9c2f6021 parsl.dataflow.memoization INFO: Task 3837 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3837 try 0 launched on executor multi with executor id 3838 parsl.dataflow.dflow INFO: Standard output for task 3837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/6153715b-c994-4779-a2ee-1feb40e725d5_isrForCrosstalkSources_996969_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/6153715b-c994-4779-a2ee-1feb40e725d5_isrForCrosstalkSources_996969_38.stderr parsl.dataflow.dflow DEBUG: Task 3838 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3838 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3838 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3838 has memoization hash f5f96e3ea7e0da78bf063e818033a3f4 parsl.dataflow.memoization INFO: Task 3838 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3838 try 0 launched on executor multi with executor id 3839 parsl.dataflow.dflow INFO: Standard output for task 3838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fd6d775d-86a0-4713-a715-cce6faa3a8e1_isrForCrosstalkSources_997012_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fd6d775d-86a0-4713-a715-cce6faa3a8e1_isrForCrosstalkSources_997012_26.stderr parsl.dataflow.dflow DEBUG: Task 3839 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3839 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3839 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3839 has memoization hash 8a61dac0cf5b210a456a56eb3a2ff6e6 parsl.dataflow.memoization INFO: Task 3839 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3839 try 0 launched on executor multi with executor id 3840 parsl.dataflow.dflow INFO: Standard output for task 3839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/11768f6c-dcdf-4c95-8eee-f1e943d17d12_isrForCrosstalkSources_997008_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/11768f6c-dcdf-4c95-8eee-f1e943d17d12_isrForCrosstalkSources_997008_3.stderr parsl.dataflow.dflow DEBUG: Task 3840 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3840 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3840 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3840 has memoization hash 4af146f70e953da2d919304203fec83d parsl.dataflow.memoization INFO: Task 3840 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3840 try 0 launched on executor multi with executor id 3841 parsl.dataflow.dflow INFO: Standard output for task 3840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/fd37037d-1848-4f4a-a322-37f7348fe732_isrForCrosstalkSources_996999_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/fd37037d-1848-4f4a-a322-37f7348fe732_isrForCrosstalkSources_996999_61.stderr parsl.dataflow.dflow DEBUG: Task 3841 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3841 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3841 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3841 has memoization hash 8e4259350ccf8e27935e1788982608c4 parsl.dataflow.memoization INFO: Task 3841 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3841 try 0 launched on executor multi with executor id 3842 parsl.dataflow.dflow INFO: Standard output for task 3841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/93e9586b-9d77-4c19-8964-c3fc5d3600d9_isrForCrosstalkSources_997021_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/93e9586b-9d77-4c19-8964-c3fc5d3600d9_isrForCrosstalkSources_997021_52.stderr parsl.dataflow.dflow DEBUG: Task 3842 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3842 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3842 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3842 has memoization hash 86644f25bbc54e36a87899a704341338 parsl.dataflow.memoization INFO: Task 3842 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3842 try 0 launched on executor multi with executor id 3843 parsl.dataflow.dflow INFO: Standard output for task 3842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/b7380c84-a6ff-406d-90bf-44f7d7f7de38_isrForCrosstalkSources_996982_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/b7380c84-a6ff-406d-90bf-44f7d7f7de38_isrForCrosstalkSources_996982_35.stderr parsl.dataflow.dflow DEBUG: Task 3843 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3843 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3843 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3843 has memoization hash 475828ff7ece545c127e158ab98abda8 parsl.dataflow.memoization INFO: Task 3843 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3843 try 0 launched on executor multi with executor id 3844 parsl.dataflow.dflow INFO: Standard output for task 3843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1954f380-0ea4-46f1-b50d-27e3dffb2a43_isrForCrosstalkSources_997027_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1954f380-0ea4-46f1-b50d-27e3dffb2a43_isrForCrosstalkSources_997027_26.stderr parsl.dataflow.dflow DEBUG: Task 3844 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3844 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3844 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3844 has memoization hash e53ca6ec478f74bc70fd1cddbb8752c5 parsl.dataflow.memoization INFO: Task 3844 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3844 try 0 launched on executor multi with executor id 3845 parsl.dataflow.dflow INFO: Standard output for task 3844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/1f35c49c-46f8-4156-8200-9c3f1180d71c_isrForCrosstalkSources_996992_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/1f35c49c-46f8-4156-8200-9c3f1180d71c_isrForCrosstalkSources_996992_50.stderr parsl.dataflow.dflow DEBUG: Task 3845 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3845 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3845 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3845 has memoization hash 4a0ae8cbea7acf2b9ae8c1029358b506 parsl.dataflow.memoization INFO: Task 3845 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3845 try 0 launched on executor multi with executor id 3846 parsl.dataflow.dflow INFO: Standard output for task 3845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/5b1c7bb4-8a37-4331-b56a-39eb959e4852_isrForCrosstalkSources_997007_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/5b1c7bb4-8a37-4331-b56a-39eb959e4852_isrForCrosstalkSources_997007_54.stderr parsl.dataflow.dflow DEBUG: Task 3846 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3846 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3846 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3846 has memoization hash 8a53c7b01cc4a58b381c70992f037667 parsl.dataflow.memoization INFO: Task 3846 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3846 try 0 launched on executor multi with executor id 3847 parsl.dataflow.dflow INFO: Standard output for task 3846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b03edc3e-7770-4e6e-a5d5-85114aa2ee55_isrForCrosstalkSources_996969_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b03edc3e-7770-4e6e-a5d5-85114aa2ee55_isrForCrosstalkSources_996969_11.stderr parsl.dataflow.dflow DEBUG: Task 3847 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3847 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3847 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3847 has memoization hash 4a4b5b61e0f421701128a794ac02992f parsl.dataflow.memoization INFO: Task 3847 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3847 try 0 launched on executor multi with executor id 3848 parsl.dataflow.dflow INFO: Standard output for task 3847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/aea4d690-a160-4863-9fd6-e1cf91b11eb3_isrForCrosstalkSources_996973_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/aea4d690-a160-4863-9fd6-e1cf91b11eb3_isrForCrosstalkSources_996973_25.stderr parsl.dataflow.dflow DEBUG: Task 3848 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3848 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3848 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3848 has memoization hash 44bef34bc7164ffaad624f9ed18f45cb parsl.dataflow.memoization INFO: Task 3848 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3848 try 0 launched on executor multi with executor id 3849 parsl.dataflow.dflow INFO: Standard output for task 3848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/1000b7ce-33b7-4c2a-9d6c-9c5a866300b1_isrForCrosstalkSources_996992_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/1000b7ce-33b7-4c2a-9d6c-9c5a866300b1_isrForCrosstalkSources_996992_2.stderr parsl.dataflow.dflow DEBUG: Task 3849 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3849 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3849 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3849 has memoization hash f1ae735e781737e2aa2d811a14d8c226 parsl.dataflow.memoization INFO: Task 3849 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:50:02 proc_lsst.multi:170 [INFO] [multi] job_id 17074897 proc_lsst.multi INFO: [multi] job_id 17074897 2024-03-15 13:50:02 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 3 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 3 2024-03-15 13:50:02 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:02 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074897 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074897 parsl.executors.status_handling DEBUG: Launched block 38 on executor multi with job ID 17074897 parsl.executors.status_handling INFO: Allocated block ID 39 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-39 2024-03-15 13:50:02 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-39 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-39 2024-03-15 13:50:02 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:02 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:02 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:02 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 3 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 3 2024-03-15 13:50:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:02 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=39 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Parsl task 3849 try 0 launched on executor multi with executor id 3850 parsl.dataflow.dflow INFO: Standard output for task 3849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e144f878-be4d-419c-abca-1eb44c0454f0_isrForCrosstalkSources_996998_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e144f878-be4d-419c-abca-1eb44c0454f0_isrForCrosstalkSources_996998_11.stderr parsl.dataflow.dflow DEBUG: Task 3850 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3850 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3850 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3850 has memoization hash ac361a51b01ef0c00193d52f49645c23 parsl.dataflow.memoization INFO: Task 3850 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3850 try 0 launched on executor multi with executor id 3851 parsl.dataflow.dflow INFO: Standard output for task 3850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/4ecc1f8a-6555-4eae-b73a-0128d04dec8a_isrForCrosstalkSources_997027_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/4ecc1f8a-6555-4eae-b73a-0128d04dec8a_isrForCrosstalkSources_997027_15.stderr parsl.dataflow.dflow DEBUG: Task 3851 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3851 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3851 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3851 has memoization hash 96dfbd55addab7e2fc7858ac37bf3569 parsl.dataflow.memoization INFO: Task 3851 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f16c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3851 try 0 launched on executor multi with executor id 3852 parsl.dataflow.dflow INFO: Standard output for task 3851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/fb72a926-3db5-403e-90cf-8c548c30dfea_isrForCrosstalkSources_996994_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/fb72a926-3db5-403e-90cf-8c548c30dfea_isrForCrosstalkSources_996994_38.stderr parsl.dataflow.dflow DEBUG: Task 3852 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3852 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3852 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3852 has memoization hash b5c127028586ad18d39ee5862438802c parsl.dataflow.memoization INFO: Task 3852 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f1620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Parsl task 3852 try 0 launched on executor multi with executor id 3853 parsl.dataflow.dflow INFO: Standard output for task 3852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/b41d886d-809c-41ad-b470-8b3010fa5484_isrForCrosstalkSources_996991_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/b41d886d-809c-41ad-b470-8b3010fa5484_isrForCrosstalkSources_996991_7.stderr parsl.dataflow.dflow DEBUG: Task 3853 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3853 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3853 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3853 has memoization hash 9ead73f0d091576fa91a6eb37f71964d parsl.dataflow.memoization INFO: Task 3853 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3853 try 0 launched on executor multi with executor id 3854 parsl.dataflow.dflow INFO: Standard output for task 3853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/bae2e892-7951-4cf0-8b3a-9c959c04fa6d_isrForCrosstalkSources_997029_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/bae2e892-7951-4cf0-8b3a-9c959c04fa6d_isrForCrosstalkSources_997029_21.stderr parsl.dataflow.dflow DEBUG: Task 3854 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3854 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3854 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3854 has memoization hash 0a7aa896cd43ddc6f353cf0b5462c563 parsl.dataflow.memoization INFO: Task 3854 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77cba0b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3854 try 0 launched on executor multi with executor id 3855 parsl.dataflow.dflow INFO: Standard output for task 3854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/cf07f2ac-e419-4ac3-ae74-8f123fee8be3_isrForCrosstalkSources_997021_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/cf07f2ac-e419-4ac3-ae74-8f123fee8be3_isrForCrosstalkSources_997021_31.stderr parsl.dataflow.dflow DEBUG: Task 3855 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3855 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3855 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3855 has memoization hash b0be91296a5a993834b81f202b407e09 parsl.dataflow.memoization INFO: Task 3855 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3855 try 0 launched on executor multi with executor id 3856 parsl.dataflow.dflow INFO: Standard output for task 3855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9c495159-46d0-4fba-a11e-b8de65bf6443_isrForCrosstalkSources_996960_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9c495159-46d0-4fba-a11e-b8de65bf6443_isrForCrosstalkSources_996960_41.stderr parsl.dataflow.dflow DEBUG: Task 3856 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3856 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3856 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3856 has memoization hash 346d2f47b830e69d941cbdf12c519c8d parsl.dataflow.memoization INFO: Task 3856 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3856 try 0 launched on executor multi with executor id 3857 parsl.dataflow.dflow INFO: Standard output for task 3856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/46dd3084-528c-4fa5-bdbb-32eaabbce501_isrForCrosstalkSources_996989_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/46dd3084-528c-4fa5-bdbb-32eaabbce501_isrForCrosstalkSources_996989_41.stderr parsl.dataflow.dflow DEBUG: Task 3857 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3857 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3857 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3857 has memoization hash fd987e8cfa0d5d99fd53b6a0d2cc24ee parsl.dataflow.memoization INFO: Task 3857 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3857 try 0 launched on executor multi with executor id 3858 parsl.dataflow.dflow INFO: Standard output for task 3857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/75554253-227a-42fb-aca5-ac43121fca45_isrForCrosstalkSources_996956_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/75554253-227a-42fb-aca5-ac43121fca45_isrForCrosstalkSources_996956_2.stderr parsl.dataflow.dflow DEBUG: Task 3858 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3858 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3858 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3858 has memoization hash 8dbfb51fe6af39e85a15e685adbfc16b parsl.dataflow.memoization INFO: Task 3858 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3128e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3858 try 0 launched on executor multi with executor id 3859 parsl.dataflow.dflow INFO: Standard output for task 3858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/db020087-c5f3-4dd7-a76c-b4eb5412f371_isrForCrosstalkSources_997014_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/db020087-c5f3-4dd7-a76c-b4eb5412f371_isrForCrosstalkSources_997014_32.stderr parsl.dataflow.dflow DEBUG: Task 3859 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3859 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3859 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3859 has memoization hash dbedf8fee9f7593c895b81b005d2323f parsl.dataflow.memoization INFO: Task 3859 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3859 try 0 launched on executor multi with executor id 3860 parsl.dataflow.dflow INFO: Standard output for task 3859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5d56cc1b-1e07-46d5-82fe-0b78e9e82ba4_isrForCrosstalkSources_996968_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5d56cc1b-1e07-46d5-82fe-0b78e9e82ba4_isrForCrosstalkSources_996968_45.stderr parsl.dataflow.dflow DEBUG: Task 3860 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3860 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3860 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3860 has memoization hash 33d8590ffd419926fcb186b656dccbda parsl.dataflow.memoization INFO: Task 3860 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3860 try 0 launched on executor multi with executor id 3861 parsl.dataflow.dflow INFO: Standard output for task 3860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/1332ebdb-bfaa-47ec-95dd-9014148e22fe_isrForCrosstalkSources_996963_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/1332ebdb-bfaa-47ec-95dd-9014148e22fe_isrForCrosstalkSources_996963_41.stderr parsl.dataflow.dflow DEBUG: Task 3861 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3861 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3861 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3861 has memoization hash 969cdb1c7087c216c1c9071497399a6c parsl.dataflow.memoization INFO: Task 3861 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3861 try 0 launched on executor multi with executor id 3862 parsl.dataflow.dflow INFO: Standard output for task 3861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9cb1968d-2c9a-419a-8d7a-25f92a65d570_isrForCrosstalkSources_997015_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9cb1968d-2c9a-419a-8d7a-25f92a65d570_isrForCrosstalkSources_997015_24.stderr parsl.dataflow.dflow DEBUG: Task 3862 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3862 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3862 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3862 has memoization hash 73254b1dbe67785f56e0b3c159559407 parsl.dataflow.memoization INFO: Task 3862 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3862 try 0 launched on executor multi with executor id 3863 parsl.dataflow.dflow INFO: Standard output for task 3862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3ad37cc5-2530-4595-8cec-5a8f696835b6_isrForCrosstalkSources_996995_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3ad37cc5-2530-4595-8cec-5a8f696835b6_isrForCrosstalkSources_996995_48.stderr parsl.dataflow.dflow DEBUG: Task 3863 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3863 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3863 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3863 has memoization hash 6be5fe89729efd96d3ec6f8adff3cfa7 parsl.dataflow.memoization INFO: Task 3863 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3863 try 0 launched on executor multi with executor id 3864 parsl.dataflow.dflow INFO: Standard output for task 3863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/dbdcdb79-2da4-406a-a79e-2cc1610b5ef2_isrForCrosstalkSources_997029_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/dbdcdb79-2da4-406a-a79e-2cc1610b5ef2_isrForCrosstalkSources_997029_56.stderr parsl.dataflow.dflow DEBUG: Task 3864 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3864 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3864 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3864 has memoization hash 38fdcc08f7be1fefed5d9ea987ac8339 parsl.dataflow.memoization INFO: Task 3864 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3864 try 0 launched on executor multi with executor id 3865 parsl.dataflow.dflow INFO: Standard output for task 3864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/5bd1e177-4100-41df-880f-0053664cd375_isrForCrosstalkSources_996990_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/5bd1e177-4100-41df-880f-0053664cd375_isrForCrosstalkSources_996990_42.stderr parsl.dataflow.dflow DEBUG: Task 3865 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3865 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3865 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3865 has memoization hash 883d3000462ee18117380d5e5b175176 parsl.dataflow.memoization INFO: Task 3865 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3865 try 0 launched on executor multi with executor id 3866 parsl.dataflow.dflow INFO: Standard output for task 3865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/6d0b7c7e-44ce-4631-bd54-fa1ea2d5b0e4_isrForCrosstalkSources_996962_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/6d0b7c7e-44ce-4631-bd54-fa1ea2d5b0e4_isrForCrosstalkSources_996962_39.stderr parsl.dataflow.dflow DEBUG: Task 3866 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3866 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3866 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3866 has memoization hash cb8397f85823b66bcbb46b4fbd7df7fc parsl.dataflow.memoization INFO: Task 3866 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3866 try 0 launched on executor multi with executor id 3867 parsl.dataflow.dflow INFO: Standard output for task 3866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d67d03c7-e642-4690-b9af-b8eec5452dd4_isrForCrosstalkSources_997017_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d67d03c7-e642-4690-b9af-b8eec5452dd4_isrForCrosstalkSources_997017_20.stderr parsl.dataflow.dflow DEBUG: Task 3867 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3867 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3867 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3867 has memoization hash fbe075357b740e1803d84cebf970099b parsl.dataflow.memoization INFO: Task 3867 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3867 try 0 launched on executor multi with executor id 3868 parsl.dataflow.dflow INFO: Standard output for task 3867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/9133372a-6fbb-43f5-9957-876ab93142c9_isrForCrosstalkSources_996974_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/9133372a-6fbb-43f5-9957-876ab93142c9_isrForCrosstalkSources_996974_46.stderr parsl.dataflow.dflow DEBUG: Task 3868 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3868 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3868 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3868 has memoization hash 7743f1b8c69089808a072015616d9d43 parsl.dataflow.memoization INFO: Task 3868 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3868 try 0 launched on executor multi with executor id 3869 parsl.dataflow.dflow INFO: Standard output for task 3868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/248ec971-67ef-414f-9444-5115a078d816_isrForCrosstalkSources_996956_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/248ec971-67ef-414f-9444-5115a078d816_isrForCrosstalkSources_996956_47.stderr parsl.dataflow.dflow DEBUG: Task 3869 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3869 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3869 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3869 has memoization hash 97e818ce4532e4f3c73d7d437432aae2 parsl.dataflow.memoization INFO: Task 3869 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3869 try 0 launched on executor multi with executor id 3870 parsl.dataflow.dflow INFO: Standard output for task 3869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/12b5e499-abed-4874-9fd6-948feed71efc_isrForCrosstalkSources_997033_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/12b5e499-abed-4874-9fd6-948feed71efc_isrForCrosstalkSources_997033_17.stderr parsl.dataflow.dflow DEBUG: Task 3870 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3870 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3870 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3870 has memoization hash 879686fe9f59d083d80ab4e7727bb9d0 parsl.dataflow.memoization INFO: Task 3870 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3870 try 0 launched on executor multi with executor id 3871 parsl.dataflow.dflow INFO: Standard output for task 3870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/37232444-cca8-4809-8053-248d4cf2224d_isrForCrosstalkSources_996952_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/37232444-cca8-4809-8053-248d4cf2224d_isrForCrosstalkSources_996952_44.stderr parsl.dataflow.dflow DEBUG: Task 3871 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3871 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3871 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3871 has memoization hash cd0410e04d4919607a397b9ce8bdbbb6 parsl.dataflow.memoization INFO: Task 3871 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3871 try 0 launched on executor multi with executor id 3872 parsl.dataflow.dflow INFO: Standard output for task 3871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/799ef417-48ef-47c6-afd3-7aef6dc5a45f_isrForCrosstalkSources_996989_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/799ef417-48ef-47c6-afd3-7aef6dc5a45f_isrForCrosstalkSources_996989_31.stderr parsl.dataflow.dflow DEBUG: Task 3872 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3872 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3872 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3872 has memoization hash 5b589d8f48d9a715631dcaddddc03152 parsl.dataflow.memoization INFO: Task 3872 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3872 try 0 launched on executor multi with executor id 3873 parsl.dataflow.dflow INFO: Standard output for task 3872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/36b5df50-2d6c-49b1-8e20-00ce8de5151a_isrForCrosstalkSources_997024_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/36b5df50-2d6c-49b1-8e20-00ce8de5151a_isrForCrosstalkSources_997024_49.stderr parsl.dataflow.dflow DEBUG: Task 3873 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3873 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3873 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3873 has memoization hash 02d17c3c7e6a8bfb7ca86289f03473b4 parsl.dataflow.memoization INFO: Task 3873 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3127a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3873 try 0 launched on executor multi with executor id 3874 parsl.dataflow.dflow INFO: Standard output for task 3873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3e921ba9-1e13-4e08-adaa-b728d880f261_isrForCrosstalkSources_996963_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3e921ba9-1e13-4e08-adaa-b728d880f261_isrForCrosstalkSources_996963_4.stderr parsl.dataflow.dflow DEBUG: Task 3874 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3874 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3874 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3874 has memoization hash 3fa010d02e8ae52b873657d6450ccb65 parsl.dataflow.memoization INFO: Task 3874 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3874 try 0 launched on executor multi with executor id 3875 parsl.dataflow.dflow INFO: Standard output for task 3874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/147d0c21-fad3-4d19-8fb6-7a4ab018f9e6_isrForCrosstalkSources_996980_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/147d0c21-fad3-4d19-8fb6-7a4ab018f9e6_isrForCrosstalkSources_996980_50.stderr parsl.dataflow.dflow DEBUG: Task 3875 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3875 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3875 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3875 has memoization hash deaf354953b493fe554e78ae30469059 parsl.dataflow.memoization INFO: Task 3875 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3875 try 0 launched on executor multi with executor id 3876 parsl.dataflow.dflow INFO: Standard output for task 3875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/ec5d445a-05e8-4725-8e74-d67c92652957_isrForCrosstalkSources_996953_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/ec5d445a-05e8-4725-8e74-d67c92652957_isrForCrosstalkSources_996953_4.stderr parsl.dataflow.dflow DEBUG: Task 3876 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3876 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3876 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3876 has memoization hash f5891b0bb8f784c2c98b86e31ed3b2b0 parsl.dataflow.memoization INFO: Task 3876 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3876 try 0 launched on executor multi with executor id 3877 parsl.dataflow.dflow INFO: Standard output for task 3876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/a0eb375e-e6d5-4fe4-8e2d-0a141376637b_isrForCrosstalkSources_997031_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/a0eb375e-e6d5-4fe4-8e2d-0a141376637b_isrForCrosstalkSources_997031_21.stderr parsl.dataflow.dflow DEBUG: Task 3877 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3877 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3877 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3877 has memoization hash d1e4d807708deba27d543c844c504844 parsl.dataflow.memoization INFO: Task 3877 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3102c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3877 try 0 launched on executor multi with executor id 3878 parsl.dataflow.dflow INFO: Standard output for task 3877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0893aa6c-eac4-4209-9f2a-e2720e37527f_isrForCrosstalkSources_996975_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0893aa6c-eac4-4209-9f2a-e2720e37527f_isrForCrosstalkSources_996975_35.stderr parsl.dataflow.dflow DEBUG: Task 3878 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3878 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3878 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3878 has memoization hash 5d5e43e3b01ecff1c02f762fd0edd423 parsl.dataflow.memoization INFO: Task 3878 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3100e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3878 try 0 launched on executor multi with executor id 3879 parsl.dataflow.dflow INFO: Standard output for task 3878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e3169082-d695-42ea-8924-a3380596cb46_isrForCrosstalkSources_997022_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e3169082-d695-42ea-8924-a3380596cb46_isrForCrosstalkSources_997022_59.stderr parsl.dataflow.dflow DEBUG: Task 3879 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3879 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3879 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3879 has memoization hash ce365fec4ca9162eec03c053aba8d484 parsl.dataflow.memoization INFO: Task 3879 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3104a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3879 try 0 launched on executor multi with executor id 3880 parsl.dataflow.dflow INFO: Standard output for task 3879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e024921c-a29a-463c-a7dc-79bacc7c30ba_isrForCrosstalkSources_997033_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e024921c-a29a-463c-a7dc-79bacc7c30ba_isrForCrosstalkSources_997033_52.stderr parsl.dataflow.dflow DEBUG: Task 3880 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3880 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3880 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3880 has memoization hash 259fd98529f524f73f56ff0a381d4388 parsl.dataflow.memoization INFO: Task 3880 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3880 try 0 launched on executor multi with executor id 3881 parsl.dataflow.dflow INFO: Standard output for task 3880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9b757870-ea23-422c-9a49-458ff07a514d_isrForCrosstalkSources_996959_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9b757870-ea23-422c-9a49-458ff07a514d_isrForCrosstalkSources_996959_51.stderr parsl.dataflow.dflow DEBUG: Task 3881 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3881 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3881 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3881 has memoization hash b41beefc39852e651fc5cb07ec443704 parsl.dataflow.memoization INFO: Task 3881 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3881 try 0 launched on executor multi with executor id 3882 parsl.dataflow.dflow INFO: Standard output for task 3881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4e095491-0579-40b6-b454-1a547285dd0e_isrForCrosstalkSources_997023_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4e095491-0579-40b6-b454-1a547285dd0e_isrForCrosstalkSources_997023_8.stderr parsl.dataflow.dflow DEBUG: Task 3882 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3882 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3882 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3882 has memoization hash 146318dceafb8154af10ccfe6d0d7f68 parsl.dataflow.memoization INFO: Task 3882 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3105e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3882 try 0 launched on executor multi with executor id 3883 parsl.dataflow.dflow INFO: Standard output for task 3882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ab602216-b333-404b-8f21-1b7fe7b8e05e_isrForCrosstalkSources_996975_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ab602216-b333-404b-8f21-1b7fe7b8e05e_isrForCrosstalkSources_996975_55.stderr parsl.dataflow.dflow DEBUG: Task 3883 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3883 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3883 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3883 has memoization hash 9edd78024147f0a37c4d22f4ebbba00d parsl.dataflow.memoization INFO: Task 3883 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3883 try 0 launched on executor multi with executor id 3884 parsl.dataflow.dflow INFO: Standard output for task 3883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/46cce164-2b51-4a88-933b-b5cdff6b0e9a_isrForCrosstalkSources_996999_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/46cce164-2b51-4a88-933b-b5cdff6b0e9a_isrForCrosstalkSources_996999_51.stderr parsl.dataflow.dflow DEBUG: Task 3884 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3884 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3884 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3884 has memoization hash aa4c447dc3188f7b93033f0ebe833072 parsl.dataflow.memoization INFO: Task 3884 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3884 try 0 launched on executor multi with executor id 3885 parsl.dataflow.dflow INFO: Standard output for task 3884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4fcfc62d-b1e0-4d91-99c2-e3722e77f650_isrForCrosstalkSources_996998_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4fcfc62d-b1e0-4d91-99c2-e3722e77f650_isrForCrosstalkSources_996998_12.stderr parsl.dataflow.dflow DEBUG: Task 3885 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3885 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3885 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3885 has memoization hash 58352cb0f62a5e7cdf98e8913c156e1d parsl.dataflow.memoization INFO: Task 3885 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3885 try 0 launched on executor multi with executor id 3886 parsl.dataflow.dflow INFO: Standard output for task 3885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9d0dd8da-e77d-4d87-89f2-8aa1b999f694_isrForCrosstalkSources_996964_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9d0dd8da-e77d-4d87-89f2-8aa1b999f694_isrForCrosstalkSources_996964_20.stderr parsl.dataflow.dflow DEBUG: Task 3886 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3886 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3886 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3886 has memoization hash ebc156439046a149da7113234062a362 parsl.dataflow.memoization INFO: Task 3886 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3107c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3886 try 0 launched on executor multi with executor id 3887 parsl.dataflow.dflow INFO: Standard output for task 3886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2410b7eb-c714-42a6-b534-5b2493714dba_isrForCrosstalkSources_997002_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2410b7eb-c714-42a6-b534-5b2493714dba_isrForCrosstalkSources_997002_9.stderr parsl.dataflow.dflow DEBUG: Task 3887 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3887 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3887 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3887 has memoization hash 76f8312240b4f29266660cff97028351 parsl.dataflow.memoization INFO: Task 3887 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3887 try 0 launched on executor multi with executor id 3888 parsl.dataflow.dflow INFO: Standard output for task 3887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c68ec87e-77b7-4b97-97df-17f29b650610_isrForCrosstalkSources_997004_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c68ec87e-77b7-4b97-97df-17f29b650610_isrForCrosstalkSources_997004_19.stderr parsl.dataflow.dflow DEBUG: Task 3888 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3888 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3888 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3888 has memoization hash 11236c94c5465c6a0adc73f9cb913331 parsl.dataflow.memoization INFO: Task 3888 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3888 try 0 launched on executor multi with executor id 3889 parsl.dataflow.dflow INFO: Standard output for task 3888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/01708eac-5640-41aa-a7ea-4d95a3f1d5ac_isrForCrosstalkSources_996960_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/01708eac-5640-41aa-a7ea-4d95a3f1d5ac_isrForCrosstalkSources_996960_4.stderr parsl.dataflow.dflow DEBUG: Task 3889 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3889 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3889 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3889 has memoization hash 2b27ed0ef4a2f47d0d02c8211f87c568 parsl.dataflow.memoization INFO: Task 3889 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3109a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3889 try 0 launched on executor multi with executor id 3890 parsl.dataflow.dflow INFO: Standard output for task 3889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4b38b924-e7ce-45df-b9a4-5941328b33ff_isrForCrosstalkSources_997023_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4b38b924-e7ce-45df-b9a4-5941328b33ff_isrForCrosstalkSources_997023_48.stderr parsl.dataflow.dflow DEBUG: Task 3890 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3890 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3890 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3890 has memoization hash a43a6400ff17a08230dfbd3d006f6de5 parsl.dataflow.memoization INFO: Task 3890 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3890 try 0 launched on executor multi with executor id 3891 parsl.dataflow.dflow INFO: Standard output for task 3890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1fb33086-b129-48c2-ab8e-c56ccc3e9cf7_isrForCrosstalkSources_997000_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1fb33086-b129-48c2-ab8e-c56ccc3e9cf7_isrForCrosstalkSources_997000_53.stderr parsl.dataflow.dflow DEBUG: Task 3891 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3891 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3891 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3891 has memoization hash 4a2b156276f20812f8c8fe8f2bd099e5 parsl.dataflow.memoization INFO: Task 3891 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3891 try 0 launched on executor multi with executor id 3892 parsl.dataflow.dflow INFO: Standard output for task 3891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/953f238d-18d3-4f82-8947-44cf8853e6e1_isrForCrosstalkSources_996964_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/953f238d-18d3-4f82-8947-44cf8853e6e1_isrForCrosstalkSources_996964_4.stderr parsl.dataflow.dflow DEBUG: Task 3892 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3892 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3892 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3892 has memoization hash d05a63d179adfef1a1a27e430df62e1d parsl.dataflow.memoization INFO: Task 3892 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3892 try 0 launched on executor multi with executor id 3893 parsl.dataflow.dflow INFO: Standard output for task 3892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4b4c13b1-a5e5-450a-aef1-4f243392bab6_isrForCrosstalkSources_996997_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4b4c13b1-a5e5-450a-aef1-4f243392bab6_isrForCrosstalkSources_996997_31.stderr parsl.dataflow.dflow DEBUG: Task 3893 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3893 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3893 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3893 has memoization hash 6b129b05aa99b5a0ae825692901cfcac parsl.dataflow.memoization INFO: Task 3893 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3893 try 0 launched on executor multi with executor id 3894 parsl.dataflow.dflow INFO: Standard output for task 3893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/f781a00d-c5a9-4dd5-accf-c08854ca7448_isrForCrosstalkSources_997002_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/f781a00d-c5a9-4dd5-accf-c08854ca7448_isrForCrosstalkSources_997002_61.stderr parsl.dataflow.dflow DEBUG: Task 3894 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3894 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3894 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3894 has memoization hash 26c8635acb0bcd95e8c19f71119d8e4a parsl.dataflow.memoization INFO: Task 3894 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3894 try 0 launched on executor multi with executor id 3895 parsl.dataflow.dflow INFO: Standard output for task 3894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b0928436-285d-4daa-a6b4-77dddb3a75b5_isrForCrosstalkSources_997006_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b0928436-285d-4daa-a6b4-77dddb3a75b5_isrForCrosstalkSources_997006_8.stderr parsl.dataflow.dflow DEBUG: Task 3895 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3895 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3895 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3895 has memoization hash 99e60e7a7c550372eed71def9014455e parsl.dataflow.memoization INFO: Task 3895 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3895 try 0 launched on executor multi with executor id 3896 parsl.dataflow.dflow INFO: Standard output for task 3895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/236b5fe0-4c79-4a06-8241-301a730f12fe_isrForCrosstalkSources_997002_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/236b5fe0-4c79-4a06-8241-301a730f12fe_isrForCrosstalkSources_997002_23.stderr parsl.dataflow.dflow DEBUG: Task 3896 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3896 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3896 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3896 has memoization hash 5da0defd4c1b90ffe20488ed1cdbd660 parsl.dataflow.memoization INFO: Task 3896 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3896 try 0 launched on executor multi with executor id 3897 parsl.dataflow.dflow INFO: Standard output for task 3896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/49f2faf5-6db1-4d5f-9d6b-ef0c95ce782e_isrForCrosstalkSources_997030_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/49f2faf5-6db1-4d5f-9d6b-ef0c95ce782e_isrForCrosstalkSources_997030_29.stderr parsl.dataflow.dflow DEBUG: Task 3897 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3897 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3897 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3897 has memoization hash d2b8dbfcf3f6d507b800e0d5341e069e parsl.dataflow.memoization INFO: Task 3897 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3897 try 0 launched on executor multi with executor id 3898 parsl.dataflow.dflow INFO: Standard output for task 3897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/50d8d36a-316a-419b-a1de-067e24a55655_isrForCrosstalkSources_996963_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/50d8d36a-316a-419b-a1de-067e24a55655_isrForCrosstalkSources_996963_17.stderr parsl.dataflow.dflow DEBUG: Task 3898 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3898 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3898 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3898 has memoization hash 799df569a3a56ddb923b69b8278fa38f parsl.dataflow.memoization INFO: Task 3898 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3898 try 0 launched on executor multi with executor id 3899 parsl.dataflow.dflow INFO: Standard output for task 3898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/98156d87-8325-4b8c-9c0d-f7f13bfa86c3_isrForCrosstalkSources_996977_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/98156d87-8325-4b8c-9c0d-f7f13bfa86c3_isrForCrosstalkSources_996977_51.stderr parsl.dataflow.dflow DEBUG: Task 3899 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3899 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3899 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3899 has memoization hash 5ea2569cde6707ad465818ce5e1bfc15 parsl.dataflow.memoization INFO: Task 3899 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3899 try 0 launched on executor multi with executor id 3900 parsl.dataflow.dflow INFO: Standard output for task 3899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/d7896fb8-cfc7-41a8-ab35-a03ebd2b55d5_isrForCrosstalkSources_997011_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/d7896fb8-cfc7-41a8-ab35-a03ebd2b55d5_isrForCrosstalkSources_997011_50.stderr parsl.dataflow.dflow DEBUG: Task 3900 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3900 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3900 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3900 has memoization hash 1d634611a5d82fc31e7a73dce05fa9f4 parsl.dataflow.memoization INFO: Task 3900 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3900 try 0 launched on executor multi with executor id 3901 parsl.dataflow.dflow INFO: Standard output for task 3900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4ecde8fa-3fcc-44b9-8038-0f3c37b73d77_isrForCrosstalkSources_997021_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4ecde8fa-3fcc-44b9-8038-0f3c37b73d77_isrForCrosstalkSources_997021_16.stderr parsl.dataflow.dflow DEBUG: Task 3901 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3901 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3901 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3901 has memoization hash 143a6e303d517ba60352037a88a6c67a parsl.dataflow.memoization INFO: Task 3901 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3111c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3901 try 0 launched on executor multi with executor id 3902 parsl.dataflow.dflow INFO: Standard output for task 3901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/c6be46a3-925e-42db-8810-692808c04c60_isrForCrosstalkSources_996974_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/c6be46a3-925e-42db-8810-692808c04c60_isrForCrosstalkSources_996974_11.stderr parsl.dataflow.dflow DEBUG: Task 3902 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3902 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3902 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3902 has memoization hash 94983bd1f4e98face99ab0a15a6bd5d8 parsl.dataflow.memoization INFO: Task 3902 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c310fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3902 try 0 launched on executor multi with executor id 3903 parsl.dataflow.dflow INFO: Standard output for task 3902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2025360-472f-4900-8272-7d1577929b30_isrForCrosstalkSources_996996_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2025360-472f-4900-8272-7d1577929b30_isrForCrosstalkSources_996996_15.stderr parsl.dataflow.dflow DEBUG: Task 3903 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3903 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3903 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3903 has memoization hash d1ea9c153db5a6ee5e9f61a9067d5cb2 parsl.dataflow.memoization INFO: Task 3903 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3903 try 0 launched on executor multi with executor id 3904 parsl.dataflow.dflow INFO: Standard output for task 3903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/c83b3a15-62f6-45b7-9cb7-923a70cdb582_isrForCrosstalkSources_996958_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/c83b3a15-62f6-45b7-9cb7-923a70cdb582_isrForCrosstalkSources_996958_33.stderr parsl.dataflow.dflow DEBUG: Task 3904 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3904 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3904 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3904 has memoization hash 9a0d0f1bf5263d75f9a50eeba2740bfb parsl.dataflow.memoization INFO: Task 3904 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3904 try 0 launched on executor multi with executor id 3905 parsl.dataflow.dflow INFO: Standard output for task 3904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/5f5b470d-d004-43fd-90fb-5613a963436e_isrForCrosstalkSources_996994_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/5f5b470d-d004-43fd-90fb-5613a963436e_isrForCrosstalkSources_996994_22.stderr parsl.dataflow.dflow DEBUG: Task 3905 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3905 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3905 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3905 has memoization hash d2c7a86c5ac5e265c5f0cb57a9a5d95b parsl.dataflow.memoization INFO: Task 3905 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3113a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3905 try 0 launched on executor multi with executor id 3906 parsl.dataflow.dflow INFO: Standard output for task 3905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/01378bc6-d3bc-4fd3-af9e-91c563d3e4d9_isrForCrosstalkSources_996992_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/01378bc6-d3bc-4fd3-af9e-91c563d3e4d9_isrForCrosstalkSources_996992_49.stderr parsl.dataflow.dflow DEBUG: Task 3906 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3906 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3906 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3906 has memoization hash 713db0491cf98eb90999d7d6046738e3 parsl.dataflow.memoization INFO: Task 3906 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3906 try 0 launched on executor multi with executor id 3907 parsl.dataflow.dflow INFO: Standard output for task 3906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/891956f9-676c-4f08-bc34-776ee1106b18_isrForCrosstalkSources_996973_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/891956f9-676c-4f08-bc34-776ee1106b18_isrForCrosstalkSources_996973_27.stderr parsl.dataflow.dflow DEBUG: Task 3907 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3907 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3907 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3907 has memoization hash 867dda5b56dedc71d64a09f571f4f89e parsl.dataflow.memoization INFO: Task 3907 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3114e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3907 try 0 launched on executor multi with executor id 3908 parsl.dataflow.dflow INFO: Standard output for task 3907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/90fd8756-c39d-4029-ad25-54bd9a11e214_isrForCrosstalkSources_996970_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/90fd8756-c39d-4029-ad25-54bd9a11e214_isrForCrosstalkSources_996970_53.stderr parsl.dataflow.dflow DEBUG: Task 3908 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3908 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3908 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3908 has memoization hash 2b0f0c3ca419f6f609617cd36d0d4a35 parsl.dataflow.memoization INFO: Task 3908 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3908 try 0 launched on executor multi with executor id 3909 parsl.dataflow.dflow INFO: Standard output for task 3908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/986b5c43-72c5-4b0e-badc-b4a1ae9f0aec_isrForCrosstalkSources_996983_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/986b5c43-72c5-4b0e-badc-b4a1ae9f0aec_isrForCrosstalkSources_996983_14.stderr parsl.dataflow.dflow DEBUG: Task 3909 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3909 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3909 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3909 has memoization hash cd0b7f31690bbb0325197cdba974bcab parsl.dataflow.memoization INFO: Task 3909 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3909 try 0 launched on executor multi with executor id 3910 parsl.dataflow.dflow INFO: Standard output for task 3909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/4909aaa2-5853-42be-87b0-fc796e55cd88_isrForCrosstalkSources_996979_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/4909aaa2-5853-42be-87b0-fc796e55cd88_isrForCrosstalkSources_996979_24.stderr parsl.dataflow.dflow DEBUG: Task 3910 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3910 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3910 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3910 has memoization hash a3660c49e0b82fb1769c406ab6bad638 parsl.dataflow.memoization INFO: Task 3910 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3116c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3910 try 0 launched on executor multi with executor id 3911 parsl.dataflow.dflow INFO: Standard output for task 3910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2e574040-a26f-4efc-a428-3a313ae16ba5_isrForCrosstalkSources_996980_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2e574040-a26f-4efc-a428-3a313ae16ba5_isrForCrosstalkSources_996980_43.stderr parsl.dataflow.dflow DEBUG: Task 3911 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3911 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3911 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3911 has memoization hash 6b2e8b6c9ce99f9dfd9059be2e49682d parsl.dataflow.memoization INFO: Task 3911 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3911 try 0 launched on executor multi with executor id 3912 parsl.dataflow.dflow INFO: Standard output for task 3911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/6af6e820-fd6f-4564-83f9-10fb6926b859_isrForCrosstalkSources_996981_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/6af6e820-fd6f-4564-83f9-10fb6926b859_isrForCrosstalkSources_996981_12.stderr parsl.dataflow.dflow DEBUG: Task 3912 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3912 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3912 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3912 has memoization hash 51ff5b0a013ae158c4f1be5427c8be23 parsl.dataflow.memoization INFO: Task 3912 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3912 try 0 launched on executor multi with executor id 3913 parsl.dataflow.dflow INFO: Standard output for task 3912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/64bc5f7f-4344-406b-97ad-659b75f9602e_isrForCrosstalkSources_997012_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/64bc5f7f-4344-406b-97ad-659b75f9602e_isrForCrosstalkSources_997012_55.stderr parsl.dataflow.dflow DEBUG: Task 3913 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3913 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3913 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3913 has memoization hash 0897f4dffa42c11cf4f45eac6687ffa7 parsl.dataflow.memoization INFO: Task 3913 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3125c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3913 try 0 launched on executor multi with executor id 3914 parsl.dataflow.dflow INFO: Standard output for task 3913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/dd1e1b40-2879-43d3-809d-44a81ac444d5_isrForCrosstalkSources_996961_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/dd1e1b40-2879-43d3-809d-44a81ac444d5_isrForCrosstalkSources_996961_11.stderr parsl.dataflow.dflow DEBUG: Task 3914 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3914 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3914 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3914 has memoization hash aba1e8339356e3f4a0a6998f61635901 parsl.dataflow.memoization INFO: Task 3914 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3914 try 0 launched on executor multi with executor id 3915 parsl.dataflow.dflow INFO: Standard output for task 3914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/263820f2-1a14-4bf3-8935-6babc2070403_isrForCrosstalkSources_996962_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/263820f2-1a14-4bf3-8935-6babc2070403_isrForCrosstalkSources_996962_3.stderr parsl.dataflow.dflow DEBUG: Task 3915 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3915 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3915 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3915 has memoization hash ba38af361cb36fb05d31f0556288e14f parsl.dataflow.memoization INFO: Task 3915 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3915 try 0 launched on executor multi with executor id 3916 parsl.dataflow.dflow INFO: Standard output for task 3915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3afdbe06-5083-4b2d-a96f-f6ad85c58a2f_isrForCrosstalkSources_997033_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3afdbe06-5083-4b2d-a96f-f6ad85c58a2f_isrForCrosstalkSources_997033_3.stderr parsl.dataflow.dflow DEBUG: Task 3916 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3916 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3916 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3916 has memoization hash 033004d633d1956817ad928c4cb23e57 parsl.dataflow.memoization INFO: Task 3916 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3123e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3916 try 0 launched on executor multi with executor id 3917 parsl.dataflow.dflow INFO: Standard output for task 3916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6cfc74e1-8bdd-4f11-aa87-25caf81beb07_isrForCrosstalkSources_996982_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6cfc74e1-8bdd-4f11-aa87-25caf81beb07_isrForCrosstalkSources_996982_33.stderr parsl.dataflow.dflow DEBUG: Task 3917 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3917 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3917 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3917 has memoization hash 9fb952dbbab71c1aff6735793fced821 parsl.dataflow.memoization INFO: Task 3917 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3917 try 0 launched on executor multi with executor id 3918 parsl.dataflow.dflow INFO: Standard output for task 3917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/82a8eef1-4d24-473d-b01c-c5e74befa9a3_isrForCrosstalkSources_997030_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/82a8eef1-4d24-473d-b01c-c5e74befa9a3_isrForCrosstalkSources_997030_13.stderr parsl.dataflow.dflow DEBUG: Task 3918 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3918 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3918 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3918 has memoization hash a5762a8361dd693ac1b4ec410b74e604 parsl.dataflow.memoization INFO: Task 3918 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3122a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3918 try 0 launched on executor multi with executor id 3919 parsl.dataflow.dflow INFO: Standard output for task 3918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/70ceb363-bf38-4a42-b19a-e9ff781b1184_isrForCrosstalkSources_997023_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/70ceb363-bf38-4a42-b19a-e9ff781b1184_isrForCrosstalkSources_997023_2.stderr parsl.dataflow.dflow DEBUG: Task 3919 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3919 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3919 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3919 has memoization hash aae9172ba3bba187497dfe767f5b47d0 parsl.dataflow.memoization INFO: Task 3919 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3919 try 0 launched on executor multi with executor id 3920 parsl.dataflow.dflow INFO: Standard output for task 3919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d303f3d6-012e-428e-83f9-6287eb90cf0e_isrForCrosstalkSources_997008_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d303f3d6-012e-428e-83f9-6287eb90cf0e_isrForCrosstalkSources_997008_61.stderr parsl.dataflow.dflow DEBUG: Task 3920 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3920 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3920 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3920 has memoization hash 6790e3825fe095893b11d12f36bfd1b9 parsl.dataflow.memoization INFO: Task 3920 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3920 try 0 launched on executor multi with executor id 3921 parsl.dataflow.dflow INFO: Standard output for task 3920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/bd357125-f005-421a-a501-95b15255d2bc_isrForCrosstalkSources_997002_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/bd357125-f005-421a-a501-95b15255d2bc_isrForCrosstalkSources_997002_10.stderr parsl.dataflow.dflow DEBUG: Task 3921 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3921 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3921 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3921 has memoization hash dd6c9adbe309cdee8f58260d9b97493a parsl.dataflow.memoization INFO: Task 3921 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3120c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3921 try 0 launched on executor multi with executor id 3922 parsl.dataflow.dflow INFO: Standard output for task 3921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6627218a-313b-4596-adb8-b2eb5a1c36f5_isrForCrosstalkSources_997005_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6627218a-313b-4596-adb8-b2eb5a1c36f5_isrForCrosstalkSources_997005_31.stderr parsl.dataflow.dflow DEBUG: Task 3922 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3922 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3922 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3922 has memoization hash d5e6e0b22303e165f0bbcf31a5ee8954 parsl.dataflow.memoization INFO: Task 3922 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3922 try 0 launched on executor multi with executor id 3923 parsl.dataflow.dflow INFO: Standard output for task 3922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e697d78e-0a7f-4fed-86c4-bc41fb9ae429_isrForCrosstalkSources_996963_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e697d78e-0a7f-4fed-86c4-bc41fb9ae429_isrForCrosstalkSources_996963_35.stderr parsl.dataflow.dflow DEBUG: Task 3923 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3923 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3923 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3923 has memoization hash 5b8ec39f3a92faa9e6e8d5912c178a3e parsl.dataflow.memoization INFO: Task 3923 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3923 try 0 launched on executor multi with executor id 3924 parsl.dataflow.dflow INFO: Standard output for task 3923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9d7406d8-75f2-416d-8229-85b381fa584a_isrForCrosstalkSources_997030_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9d7406d8-75f2-416d-8229-85b381fa584a_isrForCrosstalkSources_997030_46.stderr parsl.dataflow.dflow DEBUG: Task 3924 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3924 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3924 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3924 has memoization hash 0d21a22e790f1a5dc4475540a796d278 parsl.dataflow.memoization INFO: Task 3924 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3924 try 0 launched on executor multi with executor id 3925 parsl.dataflow.dflow INFO: Standard output for task 3924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/efc2559f-e137-46c9-880c-36cbf3272d63_isrForCrosstalkSources_996997_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/efc2559f-e137-46c9-880c-36cbf3272d63_isrForCrosstalkSources_996997_56.stderr parsl.dataflow.dflow DEBUG: Task 3925 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3925 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3925 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3925 has memoization hash cc3d84d1d5ed52d6af624e40c0d8065c parsl.dataflow.memoization INFO: Task 3925 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3925 try 0 launched on executor multi with executor id 3926 parsl.dataflow.dflow INFO: Standard output for task 3925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/734a7b6e-59f4-4e8e-9fbf-ef5d5b60ef5a_isrForCrosstalkSources_997028_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/734a7b6e-59f4-4e8e-9fbf-ef5d5b60ef5a_isrForCrosstalkSources_997028_38.stderr parsl.dataflow.dflow DEBUG: Task 3926 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3926 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3926 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3926 has memoization hash 7e90885e2131d2f0b555f75323d53c58 parsl.dataflow.memoization INFO: Task 3926 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3926 try 0 launched on executor multi with executor id 3927 parsl.dataflow.dflow INFO: Standard output for task 3926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/00d2fb39-c654-4db2-9266-3e6143c07670_isrForCrosstalkSources_997012_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/00d2fb39-c654-4db2-9266-3e6143c07670_isrForCrosstalkSources_997012_58.stderr parsl.dataflow.dflow DEBUG: Task 3927 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3927 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3927 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3927 has memoization hash c043a23792ad2c2a4da0f767db920779 parsl.dataflow.memoization INFO: Task 3927 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3927 try 0 launched on executor multi with executor id 3928 parsl.dataflow.dflow INFO: Standard output for task 3927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a09296f2-6b8c-4a3b-bdab-80cdb37c9201_isrForCrosstalkSources_996978_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a09296f2-6b8c-4a3b-bdab-80cdb37c9201_isrForCrosstalkSources_996978_54.stderr parsl.dataflow.dflow DEBUG: Task 3928 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3928 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3928 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3928 has memoization hash fc7ccba6e9c817047c187bbf09f50eea parsl.dataflow.memoization INFO: Task 3928 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3928 try 0 launched on executor multi with executor id 3929 parsl.dataflow.dflow INFO: Standard output for task 3928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0fa028b7-0098-4218-87f2-3990e1c95346_isrForCrosstalkSources_996982_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0fa028b7-0098-4218-87f2-3990e1c95346_isrForCrosstalkSources_996982_42.stderr parsl.dataflow.dflow DEBUG: Task 3929 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3929 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3929 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3929 has memoization hash c2f18110ff7de029be88d633d3c04fb7 parsl.dataflow.memoization INFO: Task 3929 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3929 try 0 launched on executor multi with executor id 3930 parsl.dataflow.dflow INFO: Standard output for task 3929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/26eb14b9-4ccf-4ee7-b6c7-ade3d207ecc7_isrForCrosstalkSources_997028_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/26eb14b9-4ccf-4ee7-b6c7-ade3d207ecc7_isrForCrosstalkSources_997028_42.stderr parsl.dataflow.dflow DEBUG: Task 3930 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3930 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3930 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3930 has memoization hash 0435f941f3dc4fcc453c10f3bc552979 parsl.dataflow.memoization INFO: Task 3930 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3930 try 0 launched on executor multi with executor id 3931 parsl.dataflow.dflow INFO: Standard output for task 3930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/724f7fb1-5948-49b3-9b3d-0a691427966a_isrForCrosstalkSources_996973_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/724f7fb1-5948-49b3-9b3d-0a691427966a_isrForCrosstalkSources_996973_42.stderr parsl.dataflow.dflow DEBUG: Task 3931 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3931 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3931 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3931 has memoization hash b05cfbc29dc429564de9ff6257d2fd2d parsl.dataflow.memoization INFO: Task 3931 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3931 try 0 launched on executor multi with executor id 3932 parsl.dataflow.dflow INFO: Standard output for task 3931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/59c43751-9d6f-4e36-b133-2b998463f261_isrForCrosstalkSources_996984_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/59c43751-9d6f-4e36-b133-2b998463f261_isrForCrosstalkSources_996984_57.stderr parsl.dataflow.dflow DEBUG: Task 3932 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3932 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3932 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3932 has memoization hash 78cae430ae687eeb40bfee34ca755e67 parsl.dataflow.memoization INFO: Task 3932 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3119e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3932 try 0 launched on executor multi with executor id 3933 parsl.dataflow.dflow INFO: Standard output for task 3932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/71018524-c372-4267-98ae-d980d8c550d3_isrForCrosstalkSources_996973_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/71018524-c372-4267-98ae-d980d8c550d3_isrForCrosstalkSources_996973_48.stderr parsl.dataflow.dflow DEBUG: Task 3933 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3933 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3933 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3933 has memoization hash de5cb0a728a360d7a35b8e5eb76ed9ed parsl.dataflow.memoization INFO: Task 3933 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3933 try 0 launched on executor multi with executor id 3934 parsl.dataflow.dflow INFO: Standard output for task 3933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/49a0cff5-98b0-46f0-a814-4dd2a02aac96_isrForCrosstalkSources_996998_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/49a0cff5-98b0-46f0-a814-4dd2a02aac96_isrForCrosstalkSources_996998_32.stderr parsl.dataflow.dflow DEBUG: Task 3934 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3934 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3934 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3934 has memoization hash fde1c5e1fdfbf1e1b5a7f89e2df67a62 parsl.dataflow.memoization INFO: Task 3934 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3934 try 0 launched on executor multi with executor id 3935 parsl.dataflow.dflow INFO: Standard output for task 3934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/8b6bfe4c-5a5f-4b0a-8a96-370aa3180bbc_isrForCrosstalkSources_996982_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/8b6bfe4c-5a5f-4b0a-8a96-370aa3180bbc_isrForCrosstalkSources_996982_17.stderr parsl.dataflow.dflow DEBUG: Task 3935 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3935 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3935 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3935 has memoization hash 50a2a839c4832e8eefda5f56079205bd parsl.dataflow.memoization INFO: Task 3935 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3118a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3935 try 0 launched on executor multi with executor id 3936 parsl.dataflow.dflow INFO: Standard output for task 3935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/251e940d-d9c0-45e3-bb3a-9e4c428cb50d_isrForCrosstalkSources_996955_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/251e940d-d9c0-45e3-bb3a-9e4c428cb50d_isrForCrosstalkSources_996955_4.stderr parsl.dataflow.dflow DEBUG: Task 3936 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3936 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3936 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3936 has memoization hash 8c3681f5f026aa00dd71ea7e0c1dc918 parsl.dataflow.memoization INFO: Task 3936 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3936 try 0 launched on executor multi with executor id 3937 parsl.dataflow.dflow INFO: Standard output for task 3936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/bd9eca6d-1191-4b37-afab-966aa6220a59_isrForCrosstalkSources_996972_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/bd9eca6d-1191-4b37-afab-966aa6220a59_isrForCrosstalkSources_996972_20.stderr parsl.dataflow.dflow DEBUG: Task 3937 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3937 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3937 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3937 has memoization hash f6b99a7f2cd09185ae24c01fdbf30621 parsl.dataflow.memoization INFO: Task 3937 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c311620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3937 try 0 launched on executor multi with executor id 3938 parsl.dataflow.dflow INFO: Standard output for task 3937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e86cb61f-9ea5-441d-9eff-7bef0211c326_isrForCrosstalkSources_997022_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e86cb61f-9ea5-441d-9eff-7bef0211c326_isrForCrosstalkSources_997022_60.stderr parsl.dataflow.dflow DEBUG: Task 3938 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3938 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3938 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3938 has memoization hash 749d7f5bfd59b4006cd851ac0f3ca61d parsl.dataflow.memoization INFO: Task 3938 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3938 try 0 launched on executor multi with executor id 3939 parsl.dataflow.dflow INFO: Standard output for task 3938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/975c472a-9194-4dbe-a6e7-23f9895af637_isrForCrosstalkSources_996961_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/975c472a-9194-4dbe-a6e7-23f9895af637_isrForCrosstalkSources_996961_38.stderr parsl.dataflow.dflow DEBUG: Task 3939 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3939 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3939 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3939 has memoization hash eb4a4936b2c14e0bf5970b5ac1bd77c0 parsl.dataflow.memoization INFO: Task 3939 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3939 try 0 launched on executor multi with executor id 3940 parsl.dataflow.dflow INFO: Standard output for task 3939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/8d590be7-8d88-4c6b-bc72-516a7ea800d3_isrForCrosstalkSources_997002_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/8d590be7-8d88-4c6b-bc72-516a7ea800d3_isrForCrosstalkSources_997002_16.stderr parsl.dataflow.dflow DEBUG: Task 3940 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3940 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3940 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3940 has memoization hash 29d3474a0825a1a4b53b7ee95e9b983f parsl.dataflow.memoization INFO: Task 3940 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3940 try 0 launched on executor multi with executor id 3941 parsl.dataflow.dflow INFO: Standard output for task 3940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8cfe6893-5172-41b7-84c8-b59e10c1d676_isrForCrosstalkSources_997032_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8cfe6893-5172-41b7-84c8-b59e10c1d676_isrForCrosstalkSources_997032_47.stderr parsl.dataflow.dflow DEBUG: Task 3941 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3941 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3941 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3941 has memoization hash 1ee212a61b1918520dcdd3fbf7bb69b3 parsl.dataflow.memoization INFO: Task 3941 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3941 try 0 launched on executor multi with executor id 3942 parsl.dataflow.dflow INFO: Standard output for task 3941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1d4c218b-69db-4ca8-b383-7485fe4b7f0d_isrForCrosstalkSources_996967_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1d4c218b-69db-4ca8-b383-7485fe4b7f0d_isrForCrosstalkSources_996967_38.stderr parsl.dataflow.dflow DEBUG: Task 3942 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3942 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3942 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3942 has memoization hash 17a5011689c7b25447ca84ee3fb46ac9 parsl.dataflow.memoization INFO: Task 3942 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3942 try 0 launched on executor multi with executor id 3943 parsl.dataflow.dflow INFO: Standard output for task 3942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/65c5f76b-f7cb-456e-ba08-1abfde7938ce_isrForCrosstalkSources_996952_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/65c5f76b-f7cb-456e-ba08-1abfde7938ce_isrForCrosstalkSources_996952_36.stderr parsl.dataflow.dflow DEBUG: Task 3943 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3943 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3943 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3943 has memoization hash c8b83b2194350090fd4a47bd08eb76a8 parsl.dataflow.memoization INFO: Task 3943 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3943 try 0 launched on executor multi with executor id 3944 parsl.dataflow.dflow INFO: Standard output for task 3943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f0fcf9e9-1105-44c6-878a-ee6bcc62bdf5_isrForCrosstalkSources_996993_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f0fcf9e9-1105-44c6-878a-ee6bcc62bdf5_isrForCrosstalkSources_996993_59.stderr parsl.dataflow.dflow DEBUG: Task 3944 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3944 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3944 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3944 has memoization hash 326ac28962f109e85e061afc141f2800 parsl.dataflow.memoization INFO: Task 3944 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3944 try 0 launched on executor multi with executor id 3945 parsl.dataflow.dflow INFO: Standard output for task 3944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f536e41f-6223-4f84-b04f-71a0909a6822_isrForCrosstalkSources_997019_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f536e41f-6223-4f84-b04f-71a0909a6822_isrForCrosstalkSources_997019_38.stderr parsl.dataflow.dflow DEBUG: Task 3945 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3945 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3945 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3945 has memoization hash 7060687ccf021dbf0c72424d1ac81bcd parsl.dataflow.memoization INFO: Task 3945 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3945 try 0 launched on executor multi with executor id 3946 parsl.dataflow.dflow INFO: Standard output for task 3945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b1d3c471-1328-4dba-97c7-e92d653ff687_isrForCrosstalkSources_997004_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b1d3c471-1328-4dba-97c7-e92d653ff687_isrForCrosstalkSources_997004_50.stderr parsl.dataflow.dflow DEBUG: Task 3946 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3946 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3946 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3946 has memoization hash 099f98d1b9713f77f43a5f4852c1d528 parsl.dataflow.memoization INFO: Task 3946 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3946 try 0 launched on executor multi with executor id 3947 parsl.dataflow.dflow INFO: Standard output for task 3946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/91082b54-897c-4c66-a06b-af9df9aa8138_isrForCrosstalkSources_997000_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/91082b54-897c-4c66-a06b-af9df9aa8138_isrForCrosstalkSources_997000_22.stderr parsl.dataflow.dflow DEBUG: Task 3947 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3947 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3947 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3947 has memoization hash 3f4955a43c415971fa65d011a5030c21 parsl.dataflow.memoization INFO: Task 3947 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3947 try 0 launched on executor multi with executor id 3948 parsl.dataflow.dflow INFO: Standard output for task 3947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/160665b5-2cff-4b4e-8210-2c2706c80a19_isrForCrosstalkSources_996994_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/160665b5-2cff-4b4e-8210-2c2706c80a19_isrForCrosstalkSources_996994_47.stderr parsl.dataflow.dflow DEBUG: Task 3948 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3948 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3948 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3948 has memoization hash a2f1d291a76f6132302e2d1ced86318b parsl.dataflow.memoization INFO: Task 3948 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3131a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3948 try 0 launched on executor multi with executor id 3949 parsl.dataflow.dflow INFO: Standard output for task 3948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f452c756-d342-4adc-b36d-d56570e98d70_isrForCrosstalkSources_996990_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f452c756-d342-4adc-b36d-d56570e98d70_isrForCrosstalkSources_996990_3.stderr parsl.dataflow.dflow DEBUG: Task 3949 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3949 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3949 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3949 has memoization hash 51d4e25a9b5987ebee7fa7989d1521ad parsl.dataflow.memoization INFO: Task 3949 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3132e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3949 try 0 launched on executor multi with executor id 3950 parsl.dataflow.dflow INFO: Standard output for task 3949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5119db02-c28a-40d0-8194-81b2d4d65e00_isrForCrosstalkSources_997017_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5119db02-c28a-40d0-8194-81b2d4d65e00_isrForCrosstalkSources_997017_21.stderr parsl.dataflow.dflow DEBUG: Task 3950 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3950 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3950 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3950 has memoization hash 57b9d1d7ce4c2eedfca91753e3072f51 parsl.dataflow.memoization INFO: Task 3950 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3950 try 0 launched on executor multi with executor id 3951 parsl.dataflow.dflow INFO: Standard output for task 3950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7058a74d-746f-4b8d-bfa8-6f25ffa10810_isrForCrosstalkSources_996990_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7058a74d-746f-4b8d-bfa8-6f25ffa10810_isrForCrosstalkSources_996990_53.stderr parsl.dataflow.dflow DEBUG: Task 3951 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3951 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3951 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3951 has memoization hash b17efd9a3ee6a8b88eb33b18fbab0d71 parsl.dataflow.memoization INFO: Task 3951 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3951 try 0 launched on executor multi with executor id 3952 parsl.dataflow.dflow INFO: Standard output for task 3951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/0da5816e-e956-4a9a-9eb3-aec237c20fc4_isrForCrosstalkSources_996962_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/0da5816e-e956-4a9a-9eb3-aec237c20fc4_isrForCrosstalkSources_996962_26.stderr parsl.dataflow.dflow DEBUG: Task 3952 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3952 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3952 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3952 has memoization hash edd7ab652a98cd2328561db70f22cc9c parsl.dataflow.memoization INFO: Task 3952 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3952 try 0 launched on executor multi with executor id 3953 parsl.dataflow.dflow INFO: Standard output for task 3952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/4e38ca91-a5f0-4276-9571-6dcbf918d0ee_isrForCrosstalkSources_996989_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/4e38ca91-a5f0-4276-9571-6dcbf918d0ee_isrForCrosstalkSources_996989_62.stderr parsl.dataflow.dflow DEBUG: Task 3953 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3953 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3953 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3953 has memoization hash 0391875a935be92c63ac18ac3d94f6fb parsl.dataflow.memoization INFO: Task 3953 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3134c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3953 try 0 launched on executor multi with executor id 3954 parsl.dataflow.dflow INFO: Standard output for task 3953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0eaba4eb-d87e-4cc5-b54c-6805f50ffe9d_isrForCrosstalkSources_996970_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0eaba4eb-d87e-4cc5-b54c-6805f50ffe9d_isrForCrosstalkSources_996970_40.stderr parsl.dataflow.dflow DEBUG: Task 3954 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3954 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3954 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3954 has memoization hash deef4d204c82c6efd6cefbb98978e05f parsl.dataflow.memoization INFO: Task 3954 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3954 try 0 launched on executor multi with executor id 3955 parsl.dataflow.dflow INFO: Standard output for task 3954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0b9bcfe9-78ca-45d5-bde8-c6a96e053622_isrForCrosstalkSources_996983_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0b9bcfe9-78ca-45d5-bde8-c6a96e053622_isrForCrosstalkSources_996983_11.stderr parsl.dataflow.dflow DEBUG: Task 3955 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3955 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3955 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3955 has memoization hash 56c8bbfb66a108a703e3fae8c4d0f8e4 parsl.dataflow.memoization INFO: Task 3955 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3955 try 0 launched on executor multi with executor id 3956 parsl.dataflow.dflow INFO: Standard output for task 3955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/af087199-3599-477a-b2ec-d60f8cc528d4_isrForCrosstalkSources_996972_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/af087199-3599-477a-b2ec-d60f8cc528d4_isrForCrosstalkSources_996972_56.stderr parsl.dataflow.dflow DEBUG: Task 3956 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3956 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3956 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3956 has memoization hash fdf0e74466739dde5814a0b1b9a343ce parsl.dataflow.memoization INFO: Task 3956 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3136a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3956 try 0 launched on executor multi with executor id 3957 parsl.dataflow.dflow INFO: Standard output for task 3956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b4505b9c-7e78-467d-ac33-4cbdc241b0bc_isrForCrosstalkSources_996957_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b4505b9c-7e78-467d-ac33-4cbdc241b0bc_isrForCrosstalkSources_996957_61.stderr parsl.dataflow.dflow DEBUG: Task 3957 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3957 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3957 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3957 has memoization hash eff353b0c125e1610a2eb67dab5c24b1 parsl.dataflow.memoization INFO: Task 3957 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3957 try 0 launched on executor multi with executor id 3958 parsl.dataflow.dflow INFO: Standard output for task 3957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/5a3e8b94-747a-4d82-9dcb-98fc68aa78b1_isrForCrosstalkSources_997001_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/5a3e8b94-747a-4d82-9dcb-98fc68aa78b1_isrForCrosstalkSources_997001_36.stderr parsl.dataflow.dflow DEBUG: Task 3958 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3958 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3958 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3958 has memoization hash 0db9950bd69192adb81ae207eab993bc parsl.dataflow.memoization INFO: Task 3958 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3137e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3958 try 0 launched on executor multi with executor id 3959 parsl.dataflow.dflow INFO: Standard output for task 3958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/ab0a924d-d713-42c8-bdea-bea0d95cce76_isrForCrosstalkSources_996977_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/ab0a924d-d713-42c8-bdea-bea0d95cce76_isrForCrosstalkSources_996977_46.stderr parsl.dataflow.dflow DEBUG: Task 3959 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3959 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3959 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3959 has memoization hash 214022bd0462ee1e4c0860ac6f7a00c2 parsl.dataflow.memoization INFO: Task 3959 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3959 try 0 launched on executor multi with executor id 3960 parsl.dataflow.dflow INFO: Standard output for task 3959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/d1c0344d-6040-4345-beec-44fca68aca87_isrForCrosstalkSources_996996_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/d1c0344d-6040-4345-beec-44fca68aca87_isrForCrosstalkSources_996996_21.stderr parsl.dataflow.dflow DEBUG: Task 3960 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3960 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3960 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3960 has memoization hash 8065184f8039860af66587a71e4de1ec parsl.dataflow.memoization INFO: Task 3960 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3960 try 0 launched on executor multi with executor id 3961 parsl.dataflow.dflow INFO: Standard output for task 3960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/7cf31f18-2403-4653-8d46-b02f4af3322f_isrForCrosstalkSources_997031_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/7cf31f18-2403-4653-8d46-b02f4af3322f_isrForCrosstalkSources_997031_20.stderr parsl.dataflow.dflow DEBUG: Task 3961 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3961 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3961 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3961 has memoization hash 263b0ede77ee98883e83b6cb8ecc136b parsl.dataflow.memoization INFO: Task 3961 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3139c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3961 try 0 launched on executor multi with executor id 3962 parsl.dataflow.dflow INFO: Standard output for task 3961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/30519971-2f04-4776-a2c8-15d7d400a935_isrForCrosstalkSources_996967_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/30519971-2f04-4776-a2c8-15d7d400a935_isrForCrosstalkSources_996967_28.stderr parsl.dataflow.dflow DEBUG: Task 3962 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3962 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3962 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3962 has memoization hash 0524eb22c6a3ab550e400dbc053d1fd9 parsl.dataflow.memoization INFO: Task 3962 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3962 try 0 launched on executor multi with executor id 3963 parsl.dataflow.dflow INFO: Standard output for task 3962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/c6c6eaf9-1455-4734-a552-9a7822a0ab20_isrForCrosstalkSources_996971_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/c6c6eaf9-1455-4734-a552-9a7822a0ab20_isrForCrosstalkSources_996971_42.stderr parsl.dataflow.dflow DEBUG: Task 3963 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3963 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3963 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3963 has memoization hash 18d92ca7734db8190c4deaf13748ab0f parsl.dataflow.memoization INFO: Task 3963 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3963 try 0 launched on executor multi with executor id 3964 parsl.dataflow.dflow INFO: Standard output for task 3963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/fca94434-a89a-4d09-92f7-e51aee01fc73_isrForCrosstalkSources_997026_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/fca94434-a89a-4d09-92f7-e51aee01fc73_isrForCrosstalkSources_997026_4.stderr parsl.dataflow.dflow DEBUG: Task 3964 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3964 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3964 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3964 has memoization hash 7d66200386d5cfe038111ec732da5958 parsl.dataflow.memoization INFO: Task 3964 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3964 try 0 launched on executor multi with executor id 3965 parsl.dataflow.dflow INFO: Standard output for task 3964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/cb38b111-ce66-4dfa-a058-8ff0bfb274fa_isrForCrosstalkSources_996989_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/cb38b111-ce66-4dfa-a058-8ff0bfb274fa_isrForCrosstalkSources_996989_30.stderr parsl.dataflow.dflow DEBUG: Task 3965 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3965 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3965 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3965 has memoization hash 6b473701f4b0df4b9aa476eefaf1cdd4 parsl.dataflow.memoization INFO: Task 3965 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3965 try 0 launched on executor multi with executor id 3966 parsl.dataflow.dflow INFO: Standard output for task 3965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f9b05847-94fd-4af7-8b4d-a3fbab3f0d09_isrForCrosstalkSources_996978_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f9b05847-94fd-4af7-8b4d-a3fbab3f0d09_isrForCrosstalkSources_996978_57.stderr parsl.dataflow.dflow DEBUG: Task 3966 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3966 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3966 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3966 has memoization hash 36c19b0f86f5e05d3c3b311ff54204b5 parsl.dataflow.memoization INFO: Task 3966 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3966 try 0 launched on executor multi with executor id 3967 parsl.dataflow.dflow INFO: Standard output for task 3966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c5977cf7-41da-4bfd-931a-9cdc4f4404e7_isrForCrosstalkSources_997014_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c5977cf7-41da-4bfd-931a-9cdc4f4404e7_isrForCrosstalkSources_997014_8.stderr parsl.dataflow.dflow DEBUG: Task 3967 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3967 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3967 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3967 has memoization hash fef271f92c7b22ffe6306b9196c90944 parsl.dataflow.memoization INFO: Task 3967 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3967 try 0 launched on executor multi with executor id 3968 parsl.dataflow.dflow INFO: Standard output for task 3967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/582ddccc-2939-4394-a2f0-3e1c0629484f_isrForCrosstalkSources_996975_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/582ddccc-2939-4394-a2f0-3e1c0629484f_isrForCrosstalkSources_996975_56.stderr parsl.dataflow.dflow DEBUG: Task 3968 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3968 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3968 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3968 has memoization hash c33e2ae27b6f2a5565f5231adab906cc parsl.dataflow.memoization INFO: Task 3968 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3968 try 0 launched on executor multi with executor id 3969 parsl.dataflow.dflow INFO: Standard output for task 3968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8e81859a-e401-4ca0-bda8-ee7fa61e7fe4_isrForCrosstalkSources_996963_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8e81859a-e401-4ca0-bda8-ee7fa61e7fe4_isrForCrosstalkSources_996963_55.stderr parsl.dataflow.dflow DEBUG: Task 3969 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3969 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3969 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3969 has memoization hash 35aab918d30db6b32b87fa48f0b79267 parsl.dataflow.memoization INFO: Task 3969 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3969 try 0 launched on executor multi with executor id 3970 parsl.dataflow.dflow INFO: Standard output for task 3969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/771755d8-c768-47e5-9c8f-6acfc850a5cf_isrForCrosstalkSources_996999_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/771755d8-c768-47e5-9c8f-6acfc850a5cf_isrForCrosstalkSources_996999_58.stderr parsl.dataflow.dflow DEBUG: Task 3970 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3970 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3970 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3970 has memoization hash d9e9f63567d7d3203938002e02753554 parsl.dataflow.memoization INFO: Task 3970 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c313ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3970 try 0 launched on executor multi with executor id 3971 parsl.dataflow.dflow INFO: Standard output for task 3970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/44e8c673-4ac8-4e04-9a93-63fc5fb8f4f1_isrForCrosstalkSources_996975_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/44e8c673-4ac8-4e04-9a93-63fc5fb8f4f1_isrForCrosstalkSources_996975_24.stderr parsl.dataflow.dflow DEBUG: Task 3971 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3971 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3971 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3971 has memoization hash a163c5e8436e4d40ad88cd5fdcec7f83 parsl.dataflow.memoization INFO: Task 3971 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c312700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3971 try 0 launched on executor multi with executor id 3972 parsl.dataflow.dflow INFO: Standard output for task 3971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4be43b07-0edf-4bd1-8477-166fc0e90694_isrForCrosstalkSources_997030_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4be43b07-0edf-4bd1-8477-166fc0e90694_isrForCrosstalkSources_997030_28.stderr parsl.dataflow.dflow DEBUG: Task 3972 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3972 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3972 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3972 has memoization hash 99b3f835b517e641c89d164ee3bfebd4 parsl.dataflow.memoization INFO: Task 3972 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b80e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3972 try 0 launched on executor multi with executor id 3973 parsl.dataflow.dflow INFO: Standard output for task 3972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ce36c364-9258-4472-a7b9-b679ba63416f_isrForCrosstalkSources_996988_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ce36c364-9258-4472-a7b9-b679ba63416f_isrForCrosstalkSources_996988_47.stderr parsl.dataflow.dflow DEBUG: Task 3973 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3973 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3973 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3973 has memoization hash 5b460570d1a61d968efd7a1ef5faf220 parsl.dataflow.memoization INFO: Task 3973 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3973 try 0 launched on executor multi with executor id 3974 parsl.dataflow.dflow INFO: Standard output for task 3973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/eea46093-9b38-4039-932b-a8cd5a69e33d_isrForCrosstalkSources_996967_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/eea46093-9b38-4039-932b-a8cd5a69e33d_isrForCrosstalkSources_996967_13.stderr parsl.dataflow.dflow DEBUG: Task 3974 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3974 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3974 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3974 has memoization hash d69f34f9d019775b823a4c2f8bb68327 parsl.dataflow.memoization INFO: Task 3974 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b82c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3974 try 0 launched on executor multi with executor id 3975 parsl.dataflow.dflow INFO: Standard output for task 3974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/12342e66-40a8-4bc6-9bd5-9d26242b871b_isrForCrosstalkSources_996995_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/12342e66-40a8-4bc6-9bd5-9d26242b871b_isrForCrosstalkSources_996995_35.stderr parsl.dataflow.dflow DEBUG: Task 3975 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3975 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3975 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3975 has memoization hash 3d9cd0700bc647cc33a71cb6ae821d57 parsl.dataflow.memoization INFO: Task 3975 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3975 try 0 launched on executor multi with executor id 3976 parsl.dataflow.dflow INFO: Standard output for task 3975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c3b80d89-b704-481a-a3b2-1466ff7b2455_isrForCrosstalkSources_996967_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c3b80d89-b704-481a-a3b2-1466ff7b2455_isrForCrosstalkSources_996967_20.stderr parsl.dataflow.dflow DEBUG: Task 3976 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3976 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3976 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3976 has memoization hash ee1420d5583c16e7f7779fb808e0400b parsl.dataflow.memoization INFO: Task 3976 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3976 try 0 launched on executor multi with executor id 3977 parsl.dataflow.dflow INFO: Standard output for task 3976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/e7554664-d4eb-4443-937e-0705743d4306_isrForCrosstalkSources_997012_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/e7554664-d4eb-4443-937e-0705743d4306_isrForCrosstalkSources_997012_17.stderr parsl.dataflow.dflow DEBUG: Task 3977 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3977 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3977 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3977 has memoization hash 1c9c51c301f2cac5a3be7c74271bf774 parsl.dataflow.memoization INFO: Task 3977 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3977 try 0 launched on executor multi with executor id 3978 parsl.dataflow.dflow INFO: Standard output for task 3977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ce5fc6f5-f13a-44b8-9df3-97e43804bf71_isrForCrosstalkSources_996988_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ce5fc6f5-f13a-44b8-9df3-97e43804bf71_isrForCrosstalkSources_996988_57.stderr parsl.dataflow.dflow DEBUG: Task 3978 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3978 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3978 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3978 has memoization hash 696974ca25ad81ce9ec0c47d85fd2dbe parsl.dataflow.memoization INFO: Task 3978 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b84a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3978 try 0 launched on executor multi with executor id 3979 parsl.dataflow.dflow INFO: Standard output for task 3978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/533aa9dc-aa2c-4779-bef3-5185a9b22988_isrForCrosstalkSources_997028_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/533aa9dc-aa2c-4779-bef3-5185a9b22988_isrForCrosstalkSources_997028_61.stderr parsl.dataflow.dflow DEBUG: Task 3979 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3979 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3979 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3979 has memoization hash cab786325e8dd91066f522a52b3f64d2 parsl.dataflow.memoization INFO: Task 3979 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b85e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3979 try 0 launched on executor multi with executor id 3980 parsl.dataflow.dflow INFO: Standard output for task 3979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/4716cb7c-62cc-4581-be97-5538dedb3961_isrForCrosstalkSources_996971_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/4716cb7c-62cc-4581-be97-5538dedb3961_isrForCrosstalkSources_996971_37.stderr parsl.dataflow.dflow DEBUG: Task 3980 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3980 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3980 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3980 has memoization hash 507cdcdfa5ad49565f00dedf30e9c925 parsl.dataflow.memoization INFO: Task 3980 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3980 try 0 launched on executor multi with executor id 3981 parsl.dataflow.dflow INFO: Standard output for task 3980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/b41c9c56-96d7-412d-b781-3a48b9e55be7_isrForCrosstalkSources_996952_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/b41c9c56-96d7-412d-b781-3a48b9e55be7_isrForCrosstalkSources_996952_2.stderr parsl.dataflow.dflow DEBUG: Task 3981 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3981 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3981 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3981 has memoization hash 0178a4f8bff25b0c01cf9ed8feb6d068 parsl.dataflow.memoization INFO: Task 3981 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3981 try 0 launched on executor multi with executor id 3982 parsl.dataflow.dflow INFO: Standard output for task 3981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ede650df-bb3e-46cb-9b5c-4e003cea2eea_isrForCrosstalkSources_997006_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ede650df-bb3e-46cb-9b5c-4e003cea2eea_isrForCrosstalkSources_997006_20.stderr parsl.dataflow.dflow DEBUG: Task 3982 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3982 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3982 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3982 has memoization hash 0c23ccc065d24e541d4874959f51b456 parsl.dataflow.memoization INFO: Task 3982 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3982 try 0 launched on executor multi with executor id 3983 parsl.dataflow.dflow INFO: Standard output for task 3982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b7a4677c-784e-4ee4-99ec-acb5001bee37_isrForCrosstalkSources_996966_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b7a4677c-784e-4ee4-99ec-acb5001bee37_isrForCrosstalkSources_996966_56.stderr parsl.dataflow.dflow DEBUG: Task 3983 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3983 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3983 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3983 has memoization hash bcaa556577bb32402b84cfb037bf04f3 parsl.dataflow.memoization INFO: Task 3983 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b87c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3983 try 0 launched on executor multi with executor id 3984 parsl.dataflow.dflow INFO: Standard output for task 3983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c556eb16-1681-44d6-a61e-7dc753193ea8_isrForCrosstalkSources_997019_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c556eb16-1681-44d6-a61e-7dc753193ea8_isrForCrosstalkSources_997019_18.stderr parsl.dataflow.dflow DEBUG: Task 3984 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3984 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3984 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3984 has memoization hash a8364365d14bf81fbfd729a072b55ba7 parsl.dataflow.memoization INFO: Task 3984 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3984 try 0 launched on executor multi with executor id 3985 parsl.dataflow.dflow INFO: Standard output for task 3984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/ee601cfd-b28d-4239-b4ea-258068245388_isrForCrosstalkSources_997008_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/ee601cfd-b28d-4239-b4ea-258068245388_isrForCrosstalkSources_997008_22.stderr parsl.dataflow.dflow DEBUG: Task 3985 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3985 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3985 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3985 has memoization hash 07567b3d8781c4f60a58f3632dd4884f parsl.dataflow.memoization INFO: Task 3985 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3985 try 0 launched on executor multi with executor id 3986 parsl.dataflow.dflow INFO: Standard output for task 3985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b53ef759-8bd5-4dd7-8c33-6331e7f676ac_isrForCrosstalkSources_996998_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b53ef759-8bd5-4dd7-8c33-6331e7f676ac_isrForCrosstalkSources_996998_28.stderr parsl.dataflow.dflow DEBUG: Task 3986 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3986 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3986 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3986 has memoization hash 292fd1ba0af6da61ddb0db3a573d6f99 parsl.dataflow.memoization INFO: Task 3986 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b89a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3986 try 0 launched on executor multi with executor id 3987 parsl.dataflow.dflow INFO: Standard output for task 3986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/c3547417-c095-48a3-8f89-a1c56cf5204b_isrForCrosstalkSources_996956_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/c3547417-c095-48a3-8f89-a1c56cf5204b_isrForCrosstalkSources_996956_48.stderr parsl.dataflow.dflow DEBUG: Task 3987 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3987 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3987 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3987 has memoization hash f607b2492618fda27e54e01ada5a7bcf parsl.dataflow.memoization INFO: Task 3987 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3987 try 0 launched on executor multi with executor id 3988 parsl.dataflow.dflow INFO: Standard output for task 3987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9842d687-1e35-45dd-b5e6-11a9fdeb3687_isrForCrosstalkSources_996988_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9842d687-1e35-45dd-b5e6-11a9fdeb3687_isrForCrosstalkSources_996988_41.stderr parsl.dataflow.dflow DEBUG: Task 3988 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3988 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3988 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3988 has memoization hash 64a7bcf3c0a68ae0713a820cae45adb7 parsl.dataflow.memoization INFO: Task 3988 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3988 try 0 launched on executor multi with executor id 3989 parsl.dataflow.dflow INFO: Standard output for task 3988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8f379c12-8114-4f2b-97e7-7b95f121ec2d_isrForCrosstalkSources_996960_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8f379c12-8114-4f2b-97e7-7b95f121ec2d_isrForCrosstalkSources_996960_8.stderr parsl.dataflow.dflow DEBUG: Task 3989 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3989 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3989 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3989 has memoization hash 69626b92ae2409f783b77a920f3ac920 parsl.dataflow.memoization INFO: Task 3989 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3989 try 0 launched on executor multi with executor id 3990 parsl.dataflow.dflow INFO: Standard output for task 3989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c04671fa-05c1-4192-97c5-67c549abd8e9_isrForCrosstalkSources_997006_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c04671fa-05c1-4192-97c5-67c549abd8e9_isrForCrosstalkSources_997006_35.stderr parsl.dataflow.dflow DEBUG: Task 3990 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3990 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3990 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3990 has memoization hash ade6e57ed0f1a8ab641d55743007408e parsl.dataflow.memoization INFO: Task 3990 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3990 try 0 launched on executor multi with executor id 3991 parsl.dataflow.dflow INFO: Standard output for task 3990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0cc70593-6513-491b-94df-840b5b801cd1_isrForCrosstalkSources_997013_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0cc70593-6513-491b-94df-840b5b801cd1_isrForCrosstalkSources_997013_15.stderr parsl.dataflow.dflow DEBUG: Task 3991 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3991 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3991 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3991 has memoization hash 055d10abdbe19dccf942a7e244e2bef8 parsl.dataflow.memoization INFO: Task 3991 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3991 try 0 launched on executor multi with executor id 3992 parsl.dataflow.dflow INFO: Standard output for task 3991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/6258eed0-8aa1-4d0b-bc4e-1a4d08f27d3c_isrForCrosstalkSources_996987_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/6258eed0-8aa1-4d0b-bc4e-1a4d08f27d3c_isrForCrosstalkSources_996987_7.stderr parsl.dataflow.dflow DEBUG: Task 3992 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3992 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3992 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3992 has memoization hash 90d05cad723f533bc8fa294e8e5cdc21 parsl.dataflow.memoization INFO: Task 3992 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3992 try 0 launched on executor multi with executor id 3993 parsl.dataflow.dflow INFO: Standard output for task 3992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e985fcb1-c51d-4523-814a-577800433c61_isrForCrosstalkSources_996970_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e985fcb1-c51d-4523-814a-577800433c61_isrForCrosstalkSources_996970_4.stderr parsl.dataflow.dflow DEBUG: Task 3993 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3993 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3993 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3993 has memoization hash 1093f09c430f8e3db0d8cda6566bb216 parsl.dataflow.memoization INFO: Task 3993 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3993 try 0 launched on executor multi with executor id 3994 parsl.dataflow.dflow INFO: Standard output for task 3993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/78e9deaf-aa46-4995-9026-9b64690298f1_isrForCrosstalkSources_997031_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/78e9deaf-aa46-4995-9026-9b64690298f1_isrForCrosstalkSources_997031_62.stderr parsl.dataflow.dflow DEBUG: Task 3994 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3994 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3994 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3994 has memoization hash a6e67e00c196e8109260e90a266c55c5 parsl.dataflow.memoization INFO: Task 3994 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3994 try 0 launched on executor multi with executor id 3995 parsl.dataflow.dflow INFO: Standard output for task 3994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0da4e8e0-386f-4be4-ba4c-199d52620697_isrForCrosstalkSources_997028_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0da4e8e0-386f-4be4-ba4c-199d52620697_isrForCrosstalkSources_997028_7.stderr parsl.dataflow.dflow DEBUG: Task 3995 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3995 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3995 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3995 has memoization hash 834d9ece1fc3d9ac5cb880eb6c2f2726 parsl.dataflow.memoization INFO: Task 3995 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3995 try 0 launched on executor multi with executor id 3996 parsl.dataflow.dflow INFO: Standard output for task 3995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/d0c95ecc-c21e-48b5-9ba1-9a1625311fe6_isrForCrosstalkSources_996976_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/d0c95ecc-c21e-48b5-9ba1-9a1625311fe6_isrForCrosstalkSources_996976_62.stderr parsl.dataflow.dflow DEBUG: Task 3996 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3996 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3996 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3996 has memoization hash 2de3ca65b8fbfe4c9335f71a74909e64 parsl.dataflow.memoization INFO: Task 3996 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3996 try 0 launched on executor multi with executor id 3997 parsl.dataflow.dflow INFO: Standard output for task 3996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/8388030f-bd9c-438a-a710-b4936bfe4952_isrForCrosstalkSources_997025_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/8388030f-bd9c-438a-a710-b4936bfe4952_isrForCrosstalkSources_997025_46.stderr parsl.dataflow.dflow DEBUG: Task 3997 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3997 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3997 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3997 has memoization hash 7fbf590f4cb378bfd300c6fdaeea775d parsl.dataflow.memoization INFO: Task 3997 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3997 try 0 launched on executor multi with executor id 3998 parsl.dataflow.dflow INFO: Standard output for task 3997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c94adbc0-37c1-420a-b610-7b1e19bb5542_isrForCrosstalkSources_996994_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c94adbc0-37c1-420a-b610-7b1e19bb5542_isrForCrosstalkSources_996994_59.stderr parsl.dataflow.dflow DEBUG: Task 3998 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3998 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3998 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3998 has memoization hash 670f8c6fe7fccfcf487fa6d2708efb0a parsl.dataflow.memoization INFO: Task 3998 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3998 try 0 launched on executor multi with executor id 3999 parsl.dataflow.dflow INFO: Standard output for task 3998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/17141304-cfc4-40cf-a072-3a824b2fdf4b_isrForCrosstalkSources_996968_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/17141304-cfc4-40cf-a072-3a824b2fdf4b_isrForCrosstalkSources_996968_37.stderr parsl.dataflow.dflow DEBUG: Task 3999 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 3999 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 3999 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3999 has memoization hash 7d92029fd74e019edc4085b02ec43acc parsl.dataflow.memoization INFO: Task 3999 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3999 try 0 launched on executor multi with executor id 4000 parsl.dataflow.dflow INFO: Standard output for task 3999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/72666815-174e-4608-a618-a362e2bdb576_isrForCrosstalkSources_997012_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/72666815-174e-4608-a618-a362e2bdb576_isrForCrosstalkSources_997012_39.stderr parsl.dataflow.dflow DEBUG: Task 4000 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4000 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4000 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4000 has memoization hash b7645d8de582da5ca2afd817defce2ba parsl.dataflow.memoization INFO: Task 4000 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b91c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4000 try 0 launched on executor multi with executor id 4001 parsl.dataflow.dflow INFO: Standard output for task 4000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d34017b4-770b-48f4-bbd8-c5757dc41e06_isrForCrosstalkSources_997025_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d34017b4-770b-48f4-bbd8-c5757dc41e06_isrForCrosstalkSources_997025_8.stderr parsl.dataflow.dflow DEBUG: Task 4001 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4001 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4001 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4001 has memoization hash f09405b0453f19f77eb7b6687323f868 parsl.dataflow.memoization INFO: Task 4001 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4001 try 0 launched on executor multi with executor id 4002 parsl.dataflow.dflow INFO: Standard output for task 4001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/64fda880-f143-4025-85be-6cdf594be4a7_isrForCrosstalkSources_996974_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/64fda880-f143-4025-85be-6cdf594be4a7_isrForCrosstalkSources_996974_27.stderr parsl.dataflow.dflow DEBUG: Task 4002 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4002 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4002 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4002 has memoization hash 24dc5b58be966fa5463fe9cf547c623e parsl.dataflow.memoization INFO: Task 4002 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b93a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4002 try 0 launched on executor multi with executor id 4003 parsl.dataflow.dflow INFO: Standard output for task 4002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc5b58f7-1cda-43c1-a255-7544822d47b6_isrForCrosstalkSources_997025_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc5b58f7-1cda-43c1-a255-7544822d47b6_isrForCrosstalkSources_997025_17.stderr parsl.dataflow.dflow DEBUG: Task 4003 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4003 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4003 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4003 has memoization hash 0de5bb89de6799e43f9ec230258e26a6 parsl.dataflow.memoization INFO: Task 4003 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4003 try 0 launched on executor multi with executor id 4004 parsl.dataflow.dflow INFO: Standard output for task 4003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/e23a6c09-55fe-49a2-8177-d6b32d7898a7_isrForCrosstalkSources_996991_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/e23a6c09-55fe-49a2-8177-d6b32d7898a7_isrForCrosstalkSources_996991_17.stderr parsl.dataflow.dflow DEBUG: Task 4004 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4004 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4004 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4004 has memoization hash 838707e695aaa84573402b3e7b0f6da3 parsl.dataflow.memoization INFO: Task 4004 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4004 try 0 launched on executor multi with executor id 4005 parsl.dataflow.dflow INFO: Standard output for task 4004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0f13cc09-985d-4fec-821a-591f2996e68e_isrForCrosstalkSources_997023_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0f13cc09-985d-4fec-821a-591f2996e68e_isrForCrosstalkSources_997023_52.stderr parsl.dataflow.dflow DEBUG: Task 4005 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4005 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4005 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4005 has memoization hash 2e83499a06aa287232fdd41281b6e8c4 parsl.dataflow.memoization INFO: Task 4005 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b94e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4005 try 0 launched on executor multi with executor id 4006 parsl.dataflow.dflow INFO: Standard output for task 4005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5d42ff7e-cd65-4451-bc78-aea3294d4361_isrForCrosstalkSources_996978_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5d42ff7e-cd65-4451-bc78-aea3294d4361_isrForCrosstalkSources_996978_56.stderr parsl.dataflow.dflow DEBUG: Task 4006 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4006 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4006 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4006 has memoization hash 93773d002c459ad07ee38e57a7a8406d parsl.dataflow.memoization INFO: Task 4006 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4006 try 0 launched on executor multi with executor id 4007 parsl.dataflow.dflow INFO: Standard output for task 4006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/700823b4-21ab-44cb-b37b-e34a0614bf8e_isrForCrosstalkSources_996976_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/700823b4-21ab-44cb-b37b-e34a0614bf8e_isrForCrosstalkSources_996976_38.stderr parsl.dataflow.dflow DEBUG: Task 4007 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4007 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4007 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4007 has memoization hash 94eaf39462f4eeec15db8354669e7089 parsl.dataflow.memoization INFO: Task 4007 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b96c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4007 try 0 launched on executor multi with executor id 4008 parsl.dataflow.dflow INFO: Standard output for task 4007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e3140b86-f5c6-4244-97ff-2877c37553b5_isrForCrosstalkSources_996983_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e3140b86-f5c6-4244-97ff-2877c37553b5_isrForCrosstalkSources_996983_40.stderr parsl.dataflow.dflow DEBUG: Task 4008 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4008 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4008 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4008 has memoization hash e8947e40a44157747733fcd700f9e0e6 parsl.dataflow.memoization INFO: Task 4008 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4008 try 0 launched on executor multi with executor id 4009 parsl.dataflow.dflow INFO: Standard output for task 4008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/60b48a7b-f112-429d-b47d-ff79de0dd112_isrForCrosstalkSources_996970_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/60b48a7b-f112-429d-b47d-ff79de0dd112_isrForCrosstalkSources_996970_31.stderr parsl.dataflow.dflow DEBUG: Task 4009 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4009 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4009 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4009 has memoization hash 09fe73f266a88f19b5f1e59b3955f648 parsl.dataflow.memoization INFO: Task 4009 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b98a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4009 try 0 launched on executor multi with executor id 4010 parsl.dataflow.dflow INFO: Standard output for task 4009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/05fcd1f4-08ed-4d0f-ba6b-1bad6c59fda3_isrForCrosstalkSources_996953_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/05fcd1f4-08ed-4d0f-ba6b-1bad6c59fda3_isrForCrosstalkSources_996953_36.stderr parsl.dataflow.dflow DEBUG: Task 4010 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4010 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4010 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4010 has memoization hash 23950d90a0be4d2bd0f471d9cc21e626 parsl.dataflow.memoization INFO: Task 4010 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4010 try 0 launched on executor multi with executor id 4011 parsl.dataflow.dflow INFO: Standard output for task 4010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6ea2297e-348e-4972-bcfc-09bbd8201a74_isrForCrosstalkSources_996954_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6ea2297e-348e-4972-bcfc-09bbd8201a74_isrForCrosstalkSources_996954_33.stderr parsl.dataflow.dflow DEBUG: Task 4011 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4011 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4011 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4011 has memoization hash 27a8d8659691be87c167e53f94a9b043 parsl.dataflow.memoization INFO: Task 4011 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4011 try 0 launched on executor multi with executor id 4012 parsl.dataflow.dflow INFO: Standard output for task 4011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/4ad0bfa6-f471-4a56-ace0-c4712f6e936b_isrForCrosstalkSources_997002_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/4ad0bfa6-f471-4a56-ace0-c4712f6e936b_isrForCrosstalkSources_997002_3.stderr parsl.dataflow.dflow DEBUG: Task 4012 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4012 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4012 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4012 has memoization hash 56a5ca0661d6ced38566bac339a00a2d parsl.dataflow.memoization INFO: Task 4012 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b99e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4012 try 0 launched on executor multi with executor id 4013 parsl.dataflow.dflow INFO: Standard output for task 4012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0a3c32fc-4e37-4bd5-ba84-977675118796_isrForCrosstalkSources_997020_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0a3c32fc-4e37-4bd5-ba84-977675118796_isrForCrosstalkSources_997020_25.stderr parsl.dataflow.dflow DEBUG: Task 4013 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4013 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4013 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4013 has memoization hash 784e9cd7b60a0fcce0572faa8b3cdceb parsl.dataflow.memoization INFO: Task 4013 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4013 try 0 launched on executor multi with executor id 4014 parsl.dataflow.dflow INFO: Standard output for task 4013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/dcc8270d-7943-4d9b-b00b-a0cb52b0f937_isrForCrosstalkSources_997032_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/dcc8270d-7943-4d9b-b00b-a0cb52b0f937_isrForCrosstalkSources_997032_46.stderr parsl.dataflow.dflow DEBUG: Task 4014 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4014 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4014 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4014 has memoization hash 8c126e6edaf97386a832f3a0a31d0d05 parsl.dataflow.memoization INFO: Task 4014 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4014 try 0 launched on executor multi with executor id 4015 parsl.dataflow.dflow INFO: Standard output for task 4014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a8ae4991-e8ef-48e7-8cd7-100137451ee5_isrForCrosstalkSources_997000_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a8ae4991-e8ef-48e7-8cd7-100137451ee5_isrForCrosstalkSources_997000_24.stderr parsl.dataflow.dflow DEBUG: Task 4015 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4015 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4015 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4015 has memoization hash 9071a990338ee7a21efbc6ffce013128 parsl.dataflow.memoization INFO: Task 4015 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4015 try 0 launched on executor multi with executor id 4016 parsl.dataflow.dflow INFO: Standard output for task 4015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/8e781840-b022-4524-8ad2-923877223d04_isrForCrosstalkSources_996972_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/8e781840-b022-4524-8ad2-923877223d04_isrForCrosstalkSources_996972_21.stderr parsl.dataflow.dflow DEBUG: Task 4016 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4016 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4016 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4016 has memoization hash 2b01122924d65f64a337a679af117b27 parsl.dataflow.memoization INFO: Task 4016 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4016 try 0 launched on executor multi with executor id 4017 parsl.dataflow.dflow INFO: Standard output for task 4016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5a7da13c-d0a4-42f6-8b0b-f83c4ee24e82_isrForCrosstalkSources_997018_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5a7da13c-d0a4-42f6-8b0b-f83c4ee24e82_isrForCrosstalkSources_997018_42.stderr parsl.dataflow.dflow DEBUG: Task 4017 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4017 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4017 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4017 has memoization hash 9d3dd48b3777a6960d1f49d352f53aec parsl.dataflow.memoization INFO: Task 4017 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4017 try 0 launched on executor multi with executor id 4018 parsl.dataflow.dflow INFO: Standard output for task 4017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/308ea9fe-9b71-4b07-af6c-eca8b4190e02_isrForCrosstalkSources_997009_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/308ea9fe-9b71-4b07-af6c-eca8b4190e02_isrForCrosstalkSources_997009_4.stderr parsl.dataflow.dflow DEBUG: Task 4018 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4018 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4018 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4018 has memoization hash aebb3f8ec67bd9c97a46d2650407ab50 parsl.dataflow.memoization INFO: Task 4018 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4018 try 0 launched on executor multi with executor id 4019 parsl.dataflow.dflow INFO: Standard output for task 4018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/fd257609-0333-418b-b8e5-55902fffd9b8_isrForCrosstalkSources_997023_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/fd257609-0333-418b-b8e5-55902fffd9b8_isrForCrosstalkSources_997023_54.stderr parsl.dataflow.dflow DEBUG: Task 4019 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4019 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4019 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4019 has memoization hash 65208478258176adc323d6c62901424f parsl.dataflow.memoization INFO: Task 4019 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4019 try 0 launched on executor multi with executor id 4020 parsl.dataflow.dflow INFO: Standard output for task 4019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/86afdad4-53aa-4d67-bec1-2038025f7afa_isrForCrosstalkSources_996978_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/86afdad4-53aa-4d67-bec1-2038025f7afa_isrForCrosstalkSources_996978_46.stderr parsl.dataflow.dflow DEBUG: Task 4020 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4020 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4020 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4020 has memoization hash e50d1c54e747309cfe64f2ffc7c7fba5 parsl.dataflow.memoization INFO: Task 4020 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4020 try 0 launched on executor multi with executor id 4021 parsl.dataflow.dflow INFO: Standard output for task 4020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/83cd4117-1b9d-4054-81f0-947b13556397_isrForCrosstalkSources_996964_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/83cd4117-1b9d-4054-81f0-947b13556397_isrForCrosstalkSources_996964_3.stderr parsl.dataflow.dflow DEBUG: Task 4021 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4021 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4021 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4021 has memoization hash 36fd21edce92648fa01bf4244409ed74 parsl.dataflow.memoization INFO: Task 4021 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b9f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4021 try 0 launched on executor multi with executor id 4022 parsl.dataflow.dflow INFO: Standard output for task 4021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/34d207bd-f400-4aef-8eda-28c9cd380ed3_isrForCrosstalkSources_996996_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/34d207bd-f400-4aef-8eda-28c9cd380ed3_isrForCrosstalkSources_996996_46.stderr parsl.dataflow.dflow DEBUG: Task 4022 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4022 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4022 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4022 has memoization hash 1083539ea131d08e53507cc33524a254 parsl.dataflow.memoization INFO: Task 4022 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4022 try 0 launched on executor multi with executor id 4023 parsl.dataflow.dflow INFO: Standard output for task 4022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/21d0d287-1e2d-4c3a-850c-210009a18482_isrForCrosstalkSources_996959_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/21d0d287-1e2d-4c3a-850c-210009a18482_isrForCrosstalkSources_996959_14.stderr parsl.dataflow.dflow DEBUG: Task 4023 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4023 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4023 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4023 has memoization hash 05ea44a20fadfb2316fcbc355b73e903 parsl.dataflow.memoization INFO: Task 4023 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4023 try 0 launched on executor multi with executor id 4024 parsl.dataflow.dflow INFO: Standard output for task 4023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a06daf9-ec8a-4473-986e-731bdd7a42a7_isrForCrosstalkSources_996994_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a06daf9-ec8a-4473-986e-731bdd7a42a7_isrForCrosstalkSources_996994_3.stderr parsl.dataflow.dflow DEBUG: Task 4024 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4024 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4024 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4024 has memoization hash 22f06e5dc0eac090022abf6d934816fe parsl.dataflow.memoization INFO: Task 4024 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4024 try 0 launched on executor multi with executor id 4025 parsl.dataflow.dflow INFO: Standard output for task 4024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/bb523ac4-df53-44d5-8326-dd5221423faa_isrForCrosstalkSources_996983_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/bb523ac4-df53-44d5-8326-dd5221423faa_isrForCrosstalkSources_996983_54.stderr parsl.dataflow.dflow DEBUG: Task 4025 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4025 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4025 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4025 has memoization hash c79cadfa3d8edf842d466215fbf33809 parsl.dataflow.memoization INFO: Task 4025 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 36 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12b4c1a9aefebdc13dbfdf6dac3212af with result from task 36 parsl.dataflow.dflow DEBUG: Task 36 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 36 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7e888150-9a69-411c-8ba4-e88da8177835_isrForCrosstalkSources_997033_33.stdout parsl.dataflow.dflow INFO: Standard error for task 36 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7e888150-9a69-411c-8ba4-e88da8177835_isrForCrosstalkSources_997033_33.stderr parsl.dataflow.dflow INFO: Parsl task 4025 try 0 launched on executor multi with executor id 4026 parsl.dataflow.dflow INFO: Standard output for task 4025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/b18a10fc-d4a4-45d7-aa9a-0c1f4b1b379d_isrForCrosstalkSources_997029_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/b18a10fc-d4a4-45d7-aa9a-0c1f4b1b379d_isrForCrosstalkSources_997029_26.stderr parsl.dataflow.dflow DEBUG: Task 4026 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4026 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4026 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4026 has memoization hash 30c5b789f5234ea6ed2499bf69fa030d parsl.dataflow.memoization INFO: Task 4026 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4026 try 0 launched on executor multi with executor id 4027 parsl.dataflow.dflow INFO: Standard output for task 4026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/3616d9dd-937b-4070-8f86-396a119ee2f5_isrForCrosstalkSources_997022_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/3616d9dd-937b-4070-8f86-396a119ee2f5_isrForCrosstalkSources_997022_50.stderr parsl.dataflow.dflow DEBUG: Task 4027 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4027 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4027 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4027 has memoization hash b2b72a1b17d20af5e5ab38097d7484a4 parsl.dataflow.memoization INFO: Task 4027 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4027 try 0 launched on executor multi with executor id 4028 parsl.dataflow.dflow INFO: Standard output for task 4027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/bd1952a7-7e37-4eb0-8bb7-f31d21a5f0d4_isrForCrosstalkSources_996975_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/bd1952a7-7e37-4eb0-8bb7-f31d21a5f0d4_isrForCrosstalkSources_996975_9.stderr parsl.dataflow.dflow DEBUG: Task 4028 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4028 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4028 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4028 has memoization hash 21f829b4f4048877f5e2319ff8015559 parsl.dataflow.memoization INFO: Task 4028 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4028 try 0 launched on executor multi with executor id 4029 parsl.dataflow.dflow INFO: Standard output for task 4028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a2592c3d-80f8-4610-9b17-0416632635da_isrForCrosstalkSources_997032_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a2592c3d-80f8-4610-9b17-0416632635da_isrForCrosstalkSources_997032_60.stderr parsl.dataflow.dflow DEBUG: Task 4029 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4029 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4029 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4029 has memoization hash cc85f9a4c5228589cae1d631ec443125 parsl.dataflow.memoization INFO: Task 4029 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4029 try 0 launched on executor multi with executor id 4030 parsl.dataflow.dflow INFO: Standard output for task 4029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b7f7e401-1ad7-4856-bd9d-6cc6bdec23ac_isrForCrosstalkSources_996959_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b7f7e401-1ad7-4856-bd9d-6cc6bdec23ac_isrForCrosstalkSources_996959_15.stderr parsl.dataflow.dflow DEBUG: Task 4030 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4030 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4030 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4030 has memoization hash d70fc5f929ba178f88e6ab90b9f933e5 parsl.dataflow.memoization INFO: Task 4030 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4030 try 0 launched on executor multi with executor id 4031 parsl.dataflow.dflow INFO: Standard output for task 4030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/2ac33dc2-a453-4263-bb1a-3790ff71d36a_isrForCrosstalkSources_996972_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/2ac33dc2-a453-4263-bb1a-3790ff71d36a_isrForCrosstalkSources_996972_27.stderr parsl.dataflow.dflow DEBUG: Task 4031 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4031 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4031 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4031 has memoization hash c626a4628d474226823572538a73a923 parsl.dataflow.memoization INFO: Task 4031 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4031 try 0 launched on executor multi with executor id 4032 parsl.dataflow.dflow INFO: Standard output for task 4031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/34e01d0c-37be-455a-bcd4-e16d7ba53baf_isrForCrosstalkSources_996988_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/34e01d0c-37be-455a-bcd4-e16d7ba53baf_isrForCrosstalkSources_996988_14.stderr parsl.dataflow.dflow DEBUG: Task 4032 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4032 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4032 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4032 has memoization hash 9dd10217c3e2c71db3e173d6c6d1bd6b parsl.dataflow.memoization INFO: Task 4032 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4032 try 0 launched on executor multi with executor id 4033 parsl.dataflow.dflow INFO: Standard output for task 4032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/1b481612-8b66-44b1-9b9f-020824dfc101_isrForCrosstalkSources_997012_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/1b481612-8b66-44b1-9b9f-020824dfc101_isrForCrosstalkSources_997012_23.stderr parsl.dataflow.dflow DEBUG: Task 4033 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4033 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4033 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4033 has memoization hash 85784e9203d9cfaf0cae257f7b1fbc04 parsl.dataflow.memoization INFO: Task 4033 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4033 try 0 launched on executor multi with executor id 4034 parsl.dataflow.dflow INFO: Standard output for task 4033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2549b6c1-f1f9-436a-b8bd-5b9be9c64b5b_isrForCrosstalkSources_997003_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2549b6c1-f1f9-436a-b8bd-5b9be9c64b5b_isrForCrosstalkSources_997003_31.stderr parsl.dataflow.dflow DEBUG: Task 4034 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4034 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4034 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4034 has memoization hash a585474cc07ba9fa2c707767e5d920ad parsl.dataflow.memoization INFO: Task 4034 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4034 try 0 launched on executor multi with executor id 4035 parsl.dataflow.dflow INFO: Standard output for task 4034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/900ffbcf-aaf6-4781-9b75-47ec2a882a93_isrForCrosstalkSources_996980_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/900ffbcf-aaf6-4781-9b75-47ec2a882a93_isrForCrosstalkSources_996980_44.stderr parsl.dataflow.dflow DEBUG: Task 4035 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4035 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4035 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4035 has memoization hash 2aacd5467a2733bfdb57c88df7ac87f8 parsl.dataflow.memoization INFO: Task 4035 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4035 try 0 launched on executor multi with executor id 4036 parsl.dataflow.dflow INFO: Standard output for task 4035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2ee63fc2-13f2-4f18-b0cc-f8bfa3fcedff_isrForCrosstalkSources_997006_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2ee63fc2-13f2-4f18-b0cc-f8bfa3fcedff_isrForCrosstalkSources_997006_12.stderr parsl.dataflow.dflow DEBUG: Task 4036 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4036 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4036 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4036 has memoization hash f63fa2b6dc3f2c6c3f1b49eabe47d97c parsl.dataflow.memoization INFO: Task 4036 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4036 try 0 launched on executor multi with executor id 4037 parsl.dataflow.dflow INFO: Standard output for task 4036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/87f89b21-cfe5-46af-8c77-7f9a5129b14b_isrForCrosstalkSources_996994_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/87f89b21-cfe5-46af-8c77-7f9a5129b14b_isrForCrosstalkSources_996994_54.stderr parsl.dataflow.dflow DEBUG: Task 4037 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4037 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4037 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4037 has memoization hash e27eb4489510b6c52087bdcaa5da9da4 parsl.dataflow.memoization INFO: Task 4037 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1ba980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4037 try 0 launched on executor multi with executor id 4038 parsl.dataflow.dflow INFO: Standard output for task 4037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/bbbb6adf-6fd4-4980-8459-40fafaad9aba_isrForCrosstalkSources_996967_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/bbbb6adf-6fd4-4980-8459-40fafaad9aba_isrForCrosstalkSources_996967_4.stderr parsl.dataflow.dflow DEBUG: Task 4038 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4038 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4038 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4038 has memoization hash 5b315b5b24e560644b09e0d9b5334d89 parsl.dataflow.memoization INFO: Task 4038 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1baa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4038 try 0 launched on executor multi with executor id 4039 parsl.dataflow.dflow INFO: Standard output for task 4038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/fb893e15-5025-4c9f-bb30-d28c2c0d81b9_isrForCrosstalkSources_997029_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/fb893e15-5025-4c9f-bb30-d28c2c0d81b9_isrForCrosstalkSources_997029_59.stderr parsl.dataflow.dflow DEBUG: Task 4039 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4039 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4039 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4039 has memoization hash 1a0386efd37f1d2963e859c3ee65014e parsl.dataflow.memoization INFO: Task 4039 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4039 try 0 launched on executor multi with executor id 4040 parsl.dataflow.dflow INFO: Standard output for task 4039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d438732d-de89-4707-b813-759f97022e85_isrForCrosstalkSources_997024_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d438732d-de89-4707-b813-759f97022e85_isrForCrosstalkSources_997024_54.stderr parsl.dataflow.dflow DEBUG: Task 4040 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4040 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4040 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4040 has memoization hash deb7b936d36330f093495532120eb977 parsl.dataflow.memoization INFO: Task 4040 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1baac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4040 try 0 launched on executor multi with executor id 4041 parsl.dataflow.dflow INFO: Standard output for task 4040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6bd79186-92e3-455d-aa8a-08750f46bbf6_isrForCrosstalkSources_996970_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6bd79186-92e3-455d-aa8a-08750f46bbf6_isrForCrosstalkSources_996970_14.stderr parsl.dataflow.dflow DEBUG: Task 4041 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4041 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4041 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4041 has memoization hash 7b01aa8a113265cc0124410b9f42dd74 parsl.dataflow.memoization INFO: Task 4041 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4041 try 0 launched on executor multi with executor id 4042 parsl.dataflow.dflow INFO: Standard output for task 4041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b8104e4d-161c-46d9-be4f-4d610e2bb012_isrForCrosstalkSources_996983_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b8104e4d-161c-46d9-be4f-4d610e2bb012_isrForCrosstalkSources_996983_59.stderr parsl.dataflow.dflow DEBUG: Task 4042 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4042 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4042 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4042 has memoization hash 10c88a9faa9c9f384cbf50e13c015453 parsl.dataflow.memoization INFO: Task 4042 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1baca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4042 try 0 launched on executor multi with executor id 4043 parsl.dataflow.dflow INFO: Standard output for task 4042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/6d1550b6-214c-44a3-8b3c-a83515aac1e9_isrForCrosstalkSources_996985_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/6d1550b6-214c-44a3-8b3c-a83515aac1e9_isrForCrosstalkSources_996985_13.stderr parsl.dataflow.dflow DEBUG: Task 4043 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4043 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4043 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4043 has memoization hash 1143f4fa636d82bab88fa19c6d32d99b parsl.dataflow.memoization INFO: Task 4043 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4043 try 0 launched on executor multi with executor id 4044 parsl.dataflow.dflow INFO: Standard output for task 4043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/5ca1d33b-36b2-4c40-a98f-bed0499b0505_isrForCrosstalkSources_996994_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/5ca1d33b-36b2-4c40-a98f-bed0499b0505_isrForCrosstalkSources_996994_5.stderr parsl.dataflow.dflow DEBUG: Task 4044 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4044 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4044 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4044 has memoization hash 84d7c44e56ccb9465d03e969cde334e3 parsl.dataflow.memoization INFO: Task 4044 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4044 try 0 launched on executor multi with executor id 4045 parsl.dataflow.dflow INFO: Standard output for task 4044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/984e59a4-3b51-42b1-a091-cf3028f65170_isrForCrosstalkSources_996998_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/984e59a4-3b51-42b1-a091-cf3028f65170_isrForCrosstalkSources_996998_45.stderr parsl.dataflow.dflow DEBUG: Task 4045 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4045 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4045 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4045 has memoization hash a151312c4f0e94baa58044e9bc4ee634 parsl.dataflow.memoization INFO: Task 4045 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4045 try 0 launched on executor multi with executor id 4046 parsl.dataflow.dflow INFO: Standard output for task 4045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a646866e-1f60-4c69-97cd-e892a4f203af_isrForCrosstalkSources_996994_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a646866e-1f60-4c69-97cd-e892a4f203af_isrForCrosstalkSources_996994_44.stderr parsl.dataflow.dflow DEBUG: Task 4046 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4046 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4046 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4046 has memoization hash 26ab9dfad4a51900c8d6edc22c194253 parsl.dataflow.memoization INFO: Task 4046 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1baf20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4046 try 0 launched on executor multi with executor id 4047 parsl.dataflow.dflow INFO: Standard output for task 4046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/de7d3066-6ac1-4a4b-9bfe-1f7d5c29949c_isrForCrosstalkSources_997013_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/de7d3066-6ac1-4a4b-9bfe-1f7d5c29949c_isrForCrosstalkSources_997013_23.stderr parsl.dataflow.dflow DEBUG: Task 4047 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4047 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4047 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4047 has memoization hash 0ca80818b6a5530cee5fac820a07a4dc parsl.dataflow.memoization INFO: Task 4047 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bafc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4047 try 0 launched on executor multi with executor id 4048 parsl.dataflow.dflow INFO: Standard output for task 4047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/9c3b144f-372c-4205-a765-1d7c43cfd6e2_isrForCrosstalkSources_996970_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/9c3b144f-372c-4205-a765-1d7c43cfd6e2_isrForCrosstalkSources_996970_52.stderr parsl.dataflow.dflow DEBUG: Task 4048 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4048 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4048 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4048 has memoization hash 97a05cd04cebf5b3f6ea5ff77b25b2f1 parsl.dataflow.memoization INFO: Task 4048 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4048 try 0 launched on executor multi with executor id 4049 parsl.dataflow.dflow INFO: Standard output for task 4048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/66f8c903-4626-49f8-b9a6-7449a12e975e_isrForCrosstalkSources_996999_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/66f8c903-4626-49f8-b9a6-7449a12e975e_isrForCrosstalkSources_996999_27.stderr parsl.dataflow.dflow DEBUG: Task 4049 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4049 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4049 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4049 has memoization hash b7a6480c57992b5759841b1a9dbb74e6 parsl.dataflow.memoization INFO: Task 4049 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4049 try 0 launched on executor multi with executor id 4050 parsl.dataflow.dflow INFO: Standard output for task 4049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/73c0a4c0-1fce-4de8-94dd-82a2589af7e1_isrForCrosstalkSources_997021_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/73c0a4c0-1fce-4de8-94dd-82a2589af7e1_isrForCrosstalkSources_997021_20.stderr parsl.dataflow.dflow DEBUG: Task 4050 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4050 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4050 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4050 has memoization hash 662dfaf5c187a6a84d9bd57f4defaffe parsl.dataflow.memoization INFO: Task 4050 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 12 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c26be93964f79c76d6e1daf03d9e51e with result from task 12 parsl.dataflow.dflow INFO: Parsl task 4050 try 0 launched on executor multi with executor id 4051 parsl.dataflow.dflow INFO: Standard output for task 4050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/dc8290e3-74f3-48ad-a197-df467499a1a7_isrForCrosstalkSources_996965_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/dc8290e3-74f3-48ad-a197-df467499a1a7_isrForCrosstalkSources_996965_25.stderr parsl.dataflow.dflow DEBUG: Task 4051 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4051 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4051 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4051 has memoization hash 764f1b3a311159b125a9ba981e77fbcb parsl.dataflow.memoization INFO: Task 4051 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1b8040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4051 try 0 launched on executor multi with executor id 4052 parsl.dataflow.dflow INFO: Standard output for task 4051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c819094f-a72d-4583-9e24-791b5eaabb04_isrForCrosstalkSources_997025_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c819094f-a72d-4583-9e24-791b5eaabb04_isrForCrosstalkSources_997025_25.stderr parsl.dataflow.dflow DEBUG: Task 4052 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4052 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4052 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4052 has memoization hash 44c3f7c8a800e57760e850cf3ca24945 parsl.dataflow.memoization INFO: Task 4052 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4052 try 0 launched on executor multi with executor id 4053 parsl.dataflow.dflow INFO: Standard output for task 4052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a81bb35b-adc9-42c3-a57d-e62abab5d110_isrForCrosstalkSources_997015_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a81bb35b-adc9-42c3-a57d-e62abab5d110_isrForCrosstalkSources_997015_61.stderr parsl.dataflow.dflow DEBUG: Task 4053 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4053 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4053 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4053 has memoization hash 2d622f7c99132d7ef7b4adf59b883570 parsl.dataflow.memoization INFO: Task 4053 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 12 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 12 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/111f589a-79d9-4da6-9f23-946436de9347_isrForCrosstalkSources_996956_62.stdout parsl.dataflow.dflow INFO: Standard error for task 12 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/111f589a-79d9-4da6-9f23-946436de9347_isrForCrosstalkSources_996956_62.stderr parsl.dataflow.dflow INFO: Parsl task 4053 try 0 launched on executor multi with executor id 4054 parsl.dataflow.dflow INFO: Standard output for task 4053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1d69ddf2-6992-4680-8b23-bbd3a728b34e_isrForCrosstalkSources_996998_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1d69ddf2-6992-4680-8b23-bbd3a728b34e_isrForCrosstalkSources_996998_31.stderr parsl.dataflow.dflow DEBUG: Task 4054 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4054 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4054 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4054 has memoization hash c49f057d302260f2b1254a24c8f78169 parsl.dataflow.memoization INFO: Task 4054 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4054 try 0 launched on executor multi with executor id 4055 parsl.dataflow.dflow INFO: Standard output for task 4054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b07814cb-7dfe-4453-ba4e-cec96dafc499_isrForCrosstalkSources_997021_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b07814cb-7dfe-4453-ba4e-cec96dafc499_isrForCrosstalkSources_997021_5.stderr parsl.dataflow.dflow DEBUG: Task 4055 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4055 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4055 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4055 has memoization hash 07a7195377904b9b49668bc7ddc2cabe parsl.dataflow.memoization INFO: Task 4055 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4055 try 0 launched on executor multi with executor id 4056 parsl.dataflow.dflow INFO: Standard output for task 4055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cc260ada-d804-4aed-962a-248eae8f0fc5_isrForCrosstalkSources_996988_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cc260ada-d804-4aed-962a-248eae8f0fc5_isrForCrosstalkSources_996988_38.stderr parsl.dataflow.dflow DEBUG: Task 4056 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4056 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4056 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4056 has memoization hash b4c71382b5291dab118f96dad61283dd parsl.dataflow.memoization INFO: Task 4056 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4056 try 0 launched on executor multi with executor id 4057 parsl.dataflow.dflow INFO: Standard output for task 4056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/75a39256-c5fc-4174-ae39-6b7c37b9dba8_isrForCrosstalkSources_996958_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/75a39256-c5fc-4174-ae39-6b7c37b9dba8_isrForCrosstalkSources_996958_26.stderr parsl.dataflow.dflow DEBUG: Task 4057 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4057 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4057 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4057 has memoization hash 496db61a9c3782ffbb7c8a1b77938943 parsl.dataflow.memoization INFO: Task 4057 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4057 try 0 launched on executor multi with executor id 4058 parsl.dataflow.dflow INFO: Standard output for task 4057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/bef53721-baf1-4323-9d64-86f1232a4e22_isrForCrosstalkSources_996997_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/bef53721-baf1-4323-9d64-86f1232a4e22_isrForCrosstalkSources_996997_33.stderr parsl.dataflow.dflow DEBUG: Task 4058 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4058 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4058 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4058 has memoization hash c5a4f3e2c42722f59fe8314a017d5c2c parsl.dataflow.memoization INFO: Task 4058 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4058 try 0 launched on executor multi with executor id 4059 parsl.dataflow.dflow INFO: Standard output for task 4058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/761825ae-6127-489e-8488-7bce5e51a25f_isrForCrosstalkSources_997027_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/761825ae-6127-489e-8488-7bce5e51a25f_isrForCrosstalkSources_997027_10.stderr parsl.dataflow.dflow DEBUG: Task 4059 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4059 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4059 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4059 has memoization hash e1a4e0e0c5781aff89ee7cc426a2703d parsl.dataflow.memoization INFO: Task 4059 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4059 try 0 launched on executor multi with executor id 4060 parsl.dataflow.dflow INFO: Standard output for task 4059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/34b12def-fa76-42fd-88b5-a59953be4f7d_isrForCrosstalkSources_997021_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/34b12def-fa76-42fd-88b5-a59953be4f7d_isrForCrosstalkSources_997021_15.stderr parsl.dataflow.dflow DEBUG: Task 4060 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4060 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4060 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4060 has memoization hash 2b7ba401a07cf91027a026e346d99bb5 parsl.dataflow.memoization INFO: Task 4060 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4060 try 0 launched on executor multi with executor id 4061 parsl.dataflow.dflow INFO: Standard output for task 4060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/23b8fbac-ef02-417b-95fb-6a8a9853db92_isrForCrosstalkSources_996959_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/23b8fbac-ef02-417b-95fb-6a8a9853db92_isrForCrosstalkSources_996959_26.stderr parsl.dataflow.dflow DEBUG: Task 4061 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4061 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4061 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4061 has memoization hash 3523f45429f850ec37cae5faacf3d14f parsl.dataflow.memoization INFO: Task 4061 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4061 try 0 launched on executor multi with executor id 4062 parsl.dataflow.dflow INFO: Standard output for task 4061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/0051d814-0ff5-4e34-bdc1-8f551a67f5d3_isrForCrosstalkSources_996960_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/0051d814-0ff5-4e34-bdc1-8f551a67f5d3_isrForCrosstalkSources_996960_37.stderr parsl.dataflow.dflow DEBUG: Task 4062 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4062 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4062 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4062 has memoization hash 5fc5255ef6a52076c091277d96b9dc84 parsl.dataflow.memoization INFO: Task 4062 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4062 try 0 launched on executor multi with executor id 4063 parsl.dataflow.dflow INFO: Standard output for task 4062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/f4fc5900-9d04-4448-9e32-cad9b4a9c4f8_isrForCrosstalkSources_996963_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/f4fc5900-9d04-4448-9e32-cad9b4a9c4f8_isrForCrosstalkSources_996963_6.stderr parsl.dataflow.dflow DEBUG: Task 4063 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4063 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4063 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4063 has memoization hash eb29d7bd0c29c2137f2cce2a0497bbcf parsl.dataflow.memoization INFO: Task 4063 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4063 try 0 launched on executor multi with executor id 4064 parsl.dataflow.dflow INFO: Standard output for task 4063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/391d17eb-1377-408a-abdf-c32a531b631b_isrForCrosstalkSources_997029_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/391d17eb-1377-408a-abdf-c32a531b631b_isrForCrosstalkSources_997029_15.stderr parsl.dataflow.dflow DEBUG: Task 4064 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4064 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4064 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4064 has memoization hash 8a4a810a9424ed422bafd773eb32643a parsl.dataflow.memoization INFO: Task 4064 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bbb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4064 try 0 launched on executor multi with executor id 4065 parsl.dataflow.dflow INFO: Standard output for task 4064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/39477b9e-c0ac-4d9a-8060-b5a54a0c7aa0_isrForCrosstalkSources_996969_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/39477b9e-c0ac-4d9a-8060-b5a54a0c7aa0_isrForCrosstalkSources_996969_14.stderr parsl.dataflow.dflow DEBUG: Task 4065 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4065 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4065 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4065 has memoization hash 46222b58e46d56846d8e04434366eb7e parsl.dataflow.memoization INFO: Task 4065 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4065 try 0 launched on executor multi with executor id 4066 parsl.dataflow.dflow INFO: Standard output for task 4065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/db947eb1-caf1-4f86-939a-082c850b6b27_isrForCrosstalkSources_996953_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/db947eb1-caf1-4f86-939a-082c850b6b27_isrForCrosstalkSources_996953_57.stderr parsl.dataflow.dflow DEBUG: Task 4066 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4066 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4066 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4066 has memoization hash 0c7dda9bbe69dd52bbc4e7561a24d2c5 parsl.dataflow.memoization INFO: Task 4066 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bbba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4066 try 0 launched on executor multi with executor id 4067 parsl.dataflow.dflow INFO: Standard output for task 4066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/9fb3e7ef-bcf0-46d1-94a2-47b7955d30eb_isrForCrosstalkSources_996987_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/9fb3e7ef-bcf0-46d1-94a2-47b7955d30eb_isrForCrosstalkSources_996987_30.stderr parsl.dataflow.dflow DEBUG: Task 4067 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4067 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4067 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4067 has memoization hash ef87a944d6fe1dc64e7299d7854de843 parsl.dataflow.memoization INFO: Task 4067 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bbc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4067 try 0 launched on executor multi with executor id 4068 parsl.dataflow.dflow INFO: Standard output for task 4067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/6125ebd7-f2fb-4a55-b9e4-a252051294db_isrForCrosstalkSources_997033_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/6125ebd7-f2fb-4a55-b9e4-a252051294db_isrForCrosstalkSources_997033_50.stderr parsl.dataflow.dflow DEBUG: Task 4068 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4068 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4068 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4068 has memoization hash 8523e2605dea846aa53ed946fb539c43 parsl.dataflow.memoization INFO: Task 4068 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bbce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4068 try 0 launched on executor multi with executor id 4069 parsl.dataflow.dflow INFO: Standard output for task 4068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ad7915a6-7cdc-4266-a387-6c044c195442_isrForCrosstalkSources_996962_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ad7915a6-7cdc-4266-a387-6c044c195442_isrForCrosstalkSources_996962_53.stderr parsl.dataflow.dflow DEBUG: Task 4069 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4069 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4069 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4069 has memoization hash c6ababd825e6bee829decac7658952bd parsl.dataflow.memoization INFO: Task 4069 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bbe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4069 try 0 launched on executor multi with executor id 4070 parsl.dataflow.dflow INFO: Standard output for task 4069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fba0d047-a55a-4ed0-8451-6c8e07ac70e9_isrForCrosstalkSources_996968_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fba0d047-a55a-4ed0-8451-6c8e07ac70e9_isrForCrosstalkSources_996968_49.stderr parsl.dataflow.dflow DEBUG: Task 4070 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4070 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4070 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4070 has memoization hash 545ecefc098f95a14fd0613e9886e6d5 parsl.dataflow.memoization INFO: Task 4070 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bbd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4070 try 0 launched on executor multi with executor id 4071 parsl.dataflow.dflow INFO: Standard output for task 4070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/1292fbcc-caca-40c5-ad74-7f2624bee17b_isrForCrosstalkSources_997011_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/1292fbcc-caca-40c5-ad74-7f2624bee17b_isrForCrosstalkSources_997011_46.stderr parsl.dataflow.dflow DEBUG: Task 4071 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4071 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4071 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4071 has memoization hash 3310cfca9a90f4a14e10649a56920874 parsl.dataflow.memoization INFO: Task 4071 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bbec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4071 try 0 launched on executor multi with executor id 4072 parsl.dataflow.dflow INFO: Standard output for task 4071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bad13cbb-0c2c-4a50-b274-ffc7188b14fa_isrForCrosstalkSources_996960_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bad13cbb-0c2c-4a50-b274-ffc7188b14fa_isrForCrosstalkSources_996960_36.stderr parsl.dataflow.dflow DEBUG: Task 4072 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4072 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4072 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4072 has memoization hash 57a9d735b3b9a96ffb4a06455fba53d3 parsl.dataflow.memoization INFO: Task 4072 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bbf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4072 try 0 launched on executor multi with executor id 4073 parsl.dataflow.dflow INFO: Standard output for task 4072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/654e64fc-4af7-419a-9b34-2b78b6dfdabf_isrForCrosstalkSources_996992_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/654e64fc-4af7-419a-9b34-2b78b6dfdabf_isrForCrosstalkSources_996992_4.stderr parsl.dataflow.dflow DEBUG: Task 4073 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4073 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4073 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4073 has memoization hash 8478393cbe8f06a02dc1b3ce7aa97789 parsl.dataflow.memoization INFO: Task 4073 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1bb420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4073 try 0 launched on executor multi with executor id 4074 parsl.dataflow.dflow INFO: Standard output for task 4073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/035bd836-996a-4dfd-a4dc-f140933377d4_isrForCrosstalkSources_996998_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/035bd836-996a-4dfd-a4dc-f140933377d4_isrForCrosstalkSources_996998_6.stderr parsl.dataflow.dflow DEBUG: Task 4074 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4074 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4074 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4074 has memoization hash a05a0e9ab4a82f459fc47617a500038e parsl.dataflow.memoization INFO: Task 4074 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4074 try 0 launched on executor multi with executor id 4075 parsl.dataflow.dflow INFO: Standard output for task 4074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/b09c66ae-f054-4f5b-a780-07c991e96cc1_isrForCrosstalkSources_996956_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/b09c66ae-f054-4f5b-a780-07c991e96cc1_isrForCrosstalkSources_996956_11.stderr parsl.dataflow.dflow DEBUG: Task 4075 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4075 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4075 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4075 has memoization hash b9ef7a1f03c1e6d4c16826a36c8af84e parsl.dataflow.memoization INFO: Task 4075 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2540e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4075 try 0 launched on executor multi with executor id 4076 parsl.dataflow.dflow INFO: Standard output for task 4075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/aae1b624-2ac6-46de-ae86-26a7f0d130af_isrForCrosstalkSources_996980_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/aae1b624-2ac6-46de-ae86-26a7f0d130af_isrForCrosstalkSources_996980_55.stderr parsl.dataflow.dflow DEBUG: Task 4076 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4076 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4076 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4076 has memoization hash ebadfa49cecf1ba6a0aa1502a88dc294 parsl.dataflow.memoization INFO: Task 4076 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4076 try 0 launched on executor multi with executor id 4077 parsl.dataflow.dflow INFO: Standard output for task 4076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1acadc24-de64-43be-9160-fa266740c9d8_isrForCrosstalkSources_996977_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1acadc24-de64-43be-9160-fa266740c9d8_isrForCrosstalkSources_996977_5.stderr parsl.dataflow.dflow DEBUG: Task 4077 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4077 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4077 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4077 has memoization hash 7edcacd2e21baadc3382b9d0948bc611 parsl.dataflow.memoization INFO: Task 4077 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2542c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4077 try 0 launched on executor multi with executor id 4078 parsl.dataflow.dflow INFO: Standard output for task 4077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/651e9a2c-0b47-459d-ae50-e3c2ba0fbad1_isrForCrosstalkSources_997029_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/651e9a2c-0b47-459d-ae50-e3c2ba0fbad1_isrForCrosstalkSources_997029_11.stderr parsl.dataflow.dflow DEBUG: Task 4078 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4078 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4078 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4078 has memoization hash ccd60bad19533c4080e391e076457141 parsl.dataflow.memoization INFO: Task 4078 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4078 try 0 launched on executor multi with executor id 4079 parsl.dataflow.dflow INFO: Standard output for task 4078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0888f7e8-6157-48af-8777-861ef9d3a261_isrForCrosstalkSources_996973_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0888f7e8-6157-48af-8777-861ef9d3a261_isrForCrosstalkSources_996973_24.stderr parsl.dataflow.dflow DEBUG: Task 4079 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4079 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4079 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4079 has memoization hash 08b66b2865ac2d840a7f3ec8f4fbde29 parsl.dataflow.memoization INFO: Task 4079 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2544a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4079 try 0 launched on executor multi with executor id 4080 parsl.dataflow.dflow INFO: Standard output for task 4079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/7ddf6eff-1278-4a7c-9ac6-ef7f2b2013e6_isrForCrosstalkSources_996970_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/7ddf6eff-1278-4a7c-9ac6-ef7f2b2013e6_isrForCrosstalkSources_996970_41.stderr parsl.dataflow.dflow DEBUG: Task 4080 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4080 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4080 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4080 has memoization hash b4912986cb23e609e9ffd55022652a0e parsl.dataflow.memoization INFO: Task 4080 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4080 try 0 launched on executor multi with executor id 4081 parsl.dataflow.dflow INFO: Standard output for task 4080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/57b0d630-bbc0-4672-897b-cff5da1fb44d_isrForCrosstalkSources_997019_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/57b0d630-bbc0-4672-897b-cff5da1fb44d_isrForCrosstalkSources_997019_35.stderr parsl.dataflow.dflow DEBUG: Task 4081 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4081 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4081 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4081 has memoization hash 4aeab56b2e1ed904b93f4f7fd879e567 parsl.dataflow.memoization INFO: Task 4081 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4081 try 0 launched on executor multi with executor id 4082 parsl.dataflow.dflow INFO: Standard output for task 4081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/9e99e92c-05d1-47ed-baad-7ac666d42868_isrForCrosstalkSources_997024_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/9e99e92c-05d1-47ed-baad-7ac666d42868_isrForCrosstalkSources_997024_35.stderr parsl.dataflow.dflow DEBUG: Task 4082 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4082 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4082 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4082 has memoization hash c98392077edbcb6295b3767c67e0c6b8 parsl.dataflow.memoization INFO: Task 4082 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2545e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4082 try 0 launched on executor multi with executor id 4083 parsl.dataflow.dflow INFO: Standard output for task 4082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2b554c3b-2571-418f-ad52-70a986195b82_isrForCrosstalkSources_996989_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2b554c3b-2571-418f-ad52-70a986195b82_isrForCrosstalkSources_996989_6.stderr parsl.dataflow.dflow DEBUG: Task 4083 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4083 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4083 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4083 has memoization hash 91ba152af64b4eecb666d05cb20afd17 parsl.dataflow.memoization INFO: Task 4083 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4083 try 0 launched on executor multi with executor id 4084 parsl.dataflow.dflow INFO: Standard output for task 4083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/fecdcd53-c143-40c5-b17f-2ecf942161ae_isrForCrosstalkSources_996981_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/fecdcd53-c143-40c5-b17f-2ecf942161ae_isrForCrosstalkSources_996981_3.stderr parsl.dataflow.dflow DEBUG: Task 4084 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4084 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4084 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4084 has memoization hash 7270dbd39cc5c5d53b860cdd41fad603 parsl.dataflow.memoization INFO: Task 4084 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2547c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4084 try 0 launched on executor multi with executor id 4085 parsl.dataflow.dflow INFO: Standard output for task 4084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7d44aca6-3d81-4d9b-9d1e-b1e5d9cefb28_isrForCrosstalkSources_996994_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7d44aca6-3d81-4d9b-9d1e-b1e5d9cefb28_isrForCrosstalkSources_996994_32.stderr parsl.dataflow.dflow DEBUG: Task 4085 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4085 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4085 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4085 has memoization hash 2abeb310bcbc1f072f0f46fde9fc4a44 parsl.dataflow.memoization INFO: Task 4085 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4085 try 0 launched on executor multi with executor id 4086 parsl.dataflow.dflow INFO: Standard output for task 4085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/0778fe5a-d566-4c5c-99cd-d7d777cb054a_isrForCrosstalkSources_996964_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/0778fe5a-d566-4c5c-99cd-d7d777cb054a_isrForCrosstalkSources_996964_34.stderr parsl.dataflow.dflow DEBUG: Task 4086 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4086 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4086 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4086 has memoization hash 15ef7fabf5bbfb837dd652685fa5ac70 parsl.dataflow.memoization INFO: Task 4086 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4086 try 0 launched on executor multi with executor id 4087 parsl.dataflow.dflow INFO: Standard output for task 4086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/d910ca4d-a629-4d38-adb8-0aa9fbd9fe3a_isrForCrosstalkSources_997015_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/d910ca4d-a629-4d38-adb8-0aa9fbd9fe3a_isrForCrosstalkSources_997015_53.stderr parsl.dataflow.dflow DEBUG: Task 4087 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4087 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4087 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4087 has memoization hash 1604a236c3fe8bfefe84ae077985d343 parsl.dataflow.memoization INFO: Task 4087 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4087 try 0 launched on executor multi with executor id 4088 parsl.dataflow.dflow INFO: Standard output for task 4087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/1dbcd8c9-8270-4d96-a9eb-bd3fcbe5fb7a_isrForCrosstalkSources_997016_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/1dbcd8c9-8270-4d96-a9eb-bd3fcbe5fb7a_isrForCrosstalkSources_997016_7.stderr parsl.dataflow.dflow DEBUG: Task 4088 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4088 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4088 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4088 has memoization hash a420eb001da1cc43434e80966d6f1fac parsl.dataflow.memoization INFO: Task 4088 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2549a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4088 try 0 launched on executor multi with executor id 4089 parsl.dataflow.dflow INFO: Standard output for task 4088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f1004d21-25f8-4ce6-bffa-84d0b19380c3_isrForCrosstalkSources_996989_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f1004d21-25f8-4ce6-bffa-84d0b19380c3_isrForCrosstalkSources_996989_27.stderr parsl.dataflow.dflow DEBUG: Task 4089 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4089 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4089 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4089 has memoization hash c25acac3c462f7aebc12b6e6078c81de parsl.dataflow.memoization INFO: Task 4089 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4089 try 0 launched on executor multi with executor id 4090 parsl.dataflow.dflow INFO: Standard output for task 4089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/2242a595-3027-40da-af6e-530cef73615f_isrForCrosstalkSources_997021_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/2242a595-3027-40da-af6e-530cef73615f_isrForCrosstalkSources_997021_40.stderr parsl.dataflow.dflow DEBUG: Task 4090 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4090 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4090 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4090 has memoization hash 5f8e99b7da06ef4c69797008dce8d418 parsl.dataflow.memoization INFO: Task 4090 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4090 try 0 launched on executor multi with executor id 4091 parsl.dataflow.dflow INFO: Standard output for task 4090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/f689a925-506a-4c65-8595-11a82e9284d5_isrForCrosstalkSources_997021_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/f689a925-506a-4c65-8595-11a82e9284d5_isrForCrosstalkSources_997021_24.stderr parsl.dataflow.dflow DEBUG: Task 4091 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4091 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4091 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4091 has memoization hash bba9150d519b6686024b51b462d1ccf4 parsl.dataflow.memoization INFO: Task 4091 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4091 try 0 launched on executor multi with executor id 4092 parsl.dataflow.dflow INFO: Standard output for task 4091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c27fb42f-0e0a-404b-af81-01987ce4e5b0_isrForCrosstalkSources_996992_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c27fb42f-0e0a-404b-af81-01987ce4e5b0_isrForCrosstalkSources_996992_55.stderr parsl.dataflow.dflow DEBUG: Task 4092 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4092 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4092 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4092 has memoization hash 3d4b59cb921a8fc341af54186fa510a0 parsl.dataflow.memoization INFO: Task 4092 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4092 try 0 launched on executor multi with executor id 4093 parsl.dataflow.dflow INFO: Standard output for task 4092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/99f9e8b4-78f9-4554-8d0e-131520033d80_isrForCrosstalkSources_996965_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/99f9e8b4-78f9-4554-8d0e-131520033d80_isrForCrosstalkSources_996965_48.stderr parsl.dataflow.dflow DEBUG: Task 4093 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4093 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4093 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4093 has memoization hash 3680740c3af364d7337bdbff9bb78106 parsl.dataflow.memoization INFO: Task 4093 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4093 try 0 launched on executor multi with executor id 4094 parsl.dataflow.dflow INFO: Standard output for task 4093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/2d8579b6-5036-41ec-9cfc-4e94bb0a58dd_isrForCrosstalkSources_996957_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/2d8579b6-5036-41ec-9cfc-4e94bb0a58dd_isrForCrosstalkSources_996957_58.stderr parsl.dataflow.dflow DEBUG: Task 4094 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4094 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4094 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4094 has memoization hash 3d0e24e6acc45020298efd8668db5b0e parsl.dataflow.memoization INFO: Task 4094 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4094 try 0 launched on executor multi with executor id 4095 parsl.dataflow.dflow INFO: Standard output for task 4094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/052094d5-d50b-4cfb-b9a3-ef2fcfd4807a_isrForCrosstalkSources_996959_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/052094d5-d50b-4cfb-b9a3-ef2fcfd4807a_isrForCrosstalkSources_996959_8.stderr parsl.dataflow.dflow DEBUG: Task 4095 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4095 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4095 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4095 has memoization hash 4f2adefbdff40e2b73a9d06921db3dac parsl.dataflow.memoization INFO: Task 4095 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4095 try 0 launched on executor multi with executor id 4096 parsl.dataflow.dflow INFO: Standard output for task 4095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/0fb50ef0-beba-4fc4-87c4-257a520d6a39_isrForCrosstalkSources_996967_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/0fb50ef0-beba-4fc4-87c4-257a520d6a39_isrForCrosstalkSources_996967_10.stderr parsl.dataflow.dflow DEBUG: Task 4096 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4096 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4096 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4096 has memoization hash 8889b563ad2808d64a0dbfecb2e15b70 parsl.dataflow.memoization INFO: Task 4096 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4096 try 0 launched on executor multi with executor id 4097 parsl.dataflow.dflow INFO: Standard output for task 4096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4f9103da-4728-4480-b5cf-def4ae47444a_isrForCrosstalkSources_996954_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4f9103da-4728-4480-b5cf-def4ae47444a_isrForCrosstalkSources_996954_44.stderr parsl.dataflow.dflow DEBUG: Task 4097 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4097 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4097 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4097 has memoization hash c44a0a8ee057fc37893d98487bbf60b7 parsl.dataflow.memoization INFO: Task 4097 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4097 try 0 launched on executor multi with executor id 4098 parsl.dataflow.dflow INFO: Standard output for task 4097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0f41d76f-7fe9-498b-a1f1-0682bcf99c03_isrForCrosstalkSources_997013_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0f41d76f-7fe9-498b-a1f1-0682bcf99c03_isrForCrosstalkSources_997013_17.stderr parsl.dataflow.dflow DEBUG: Task 4098 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4098 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4098 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4098 has memoization hash c7e1ec8ff65ee5d218ac9f387c93e0e2 parsl.dataflow.memoization INFO: Task 4098 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 20 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0fef4e256784c12e0df787d0a3063f9d with result from task 20 parsl.dataflow.dflow INFO: Parsl task 4098 try 0 launched on executor multi with executor id 4099 parsl.dataflow.dflow INFO: Standard output for task 4098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/97993fa0-9760-4ecc-831c-2916da5d71da_isrForCrosstalkSources_997001_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/97993fa0-9760-4ecc-831c-2916da5d71da_isrForCrosstalkSources_997001_59.stderr parsl.dataflow.dflow DEBUG: Task 4099 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4099 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4099 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4099 has memoization hash 582182aaf1739491f885800ba3171295 parsl.dataflow.memoization INFO: Task 4099 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 20 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 20 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/13e7e8e5-8559-44ed-bf58-8d913741d954_isrForCrosstalkSources_997016_39.stdout parsl.dataflow.dflow INFO: Standard error for task 20 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/13e7e8e5-8559-44ed-bf58-8d913741d954_isrForCrosstalkSources_997016_39.stderr parsl.dataflow.dflow INFO: Parsl task 4099 try 0 launched on executor multi with executor id 4100 parsl.dataflow.dflow INFO: Standard output for task 4099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/c9c249fc-fec1-415b-be3e-fc9d01de5d57_isrForCrosstalkSources_996963_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/c9c249fc-fec1-415b-be3e-fc9d01de5d57_isrForCrosstalkSources_996963_54.stderr parsl.dataflow.dflow DEBUG: Task 4100 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4100 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4100 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4100 has memoization hash 3a2c396244b9fa04b099884a147e1dcc parsl.dataflow.memoization INFO: Task 4100 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4100 try 0 launched on executor multi with executor id 4101 parsl.dataflow.dflow INFO: Standard output for task 4100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d99e00bf-109f-47ae-9d74-6b12c1632abf_isrForCrosstalkSources_997004_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d99e00bf-109f-47ae-9d74-6b12c1632abf_isrForCrosstalkSources_997004_37.stderr parsl.dataflow.dflow DEBUG: Task 4101 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4101 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4101 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4101 has memoization hash 6bb51c057d1ecfdc3d78b9e005f51372 parsl.dataflow.memoization INFO: Task 4101 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4101 try 0 launched on executor multi with executor id 4102 parsl.dataflow.dflow INFO: Standard output for task 4101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/fd11f259-48e9-41ea-b00c-3aae4708d8e0_isrForCrosstalkSources_996979_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/fd11f259-48e9-41ea-b00c-3aae4708d8e0_isrForCrosstalkSources_996979_15.stderr parsl.dataflow.dflow DEBUG: Task 4102 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4102 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4102 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4102 has memoization hash cc9121eb30a0fe163a631e59ebd56690 parsl.dataflow.memoization INFO: Task 4102 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4102 try 0 launched on executor multi with executor id 4103 parsl.dataflow.dflow INFO: Standard output for task 4102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/454b18b1-6eef-456f-a963-2ea477843cea_isrForCrosstalkSources_997004_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/454b18b1-6eef-456f-a963-2ea477843cea_isrForCrosstalkSources_997004_51.stderr parsl.dataflow.dflow DEBUG: Task 4103 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4103 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4103 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4103 has memoization hash 17c897aa7e43c82575d9fae97e5274c9 parsl.dataflow.memoization INFO: Task 4103 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2551c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4103 try 0 launched on executor multi with executor id 4104 parsl.dataflow.dflow INFO: Standard output for task 4103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/be020954-1366-4b89-9bc7-24bca9e90bfc_isrForCrosstalkSources_997015_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/be020954-1366-4b89-9bc7-24bca9e90bfc_isrForCrosstalkSources_997015_19.stderr parsl.dataflow.dflow DEBUG: Task 4104 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4104 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4104 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4104 has memoization hash 454a96ebb935118411045a885e7b9a48 parsl.dataflow.memoization INFO: Task 4104 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4104 try 0 launched on executor multi with executor id 4105 parsl.dataflow.dflow INFO: Standard output for task 4104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/5c6c9dd1-118a-4207-9a1a-c1a57f14d82b_isrForCrosstalkSources_997028_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/5c6c9dd1-118a-4207-9a1a-c1a57f14d82b_isrForCrosstalkSources_997028_16.stderr parsl.dataflow.dflow DEBUG: Task 4105 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4105 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4105 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4105 has memoization hash 8240755dc47065f3e21c63f9b398bb5c parsl.dataflow.memoization INFO: Task 4105 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2553a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4105 try 0 launched on executor multi with executor id 4106 parsl.dataflow.dflow INFO: Standard output for task 4105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b1a9f1e4-1c40-4bbb-95ba-0072aa1a8a7c_isrForCrosstalkSources_997030_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b1a9f1e4-1c40-4bbb-95ba-0072aa1a8a7c_isrForCrosstalkSources_997030_25.stderr parsl.dataflow.dflow DEBUG: Task 4106 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4106 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4106 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4106 has memoization hash 88acee77c4e5d3447afc97b9563e3ee9 parsl.dataflow.memoization INFO: Task 4106 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2554e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4106 try 0 launched on executor multi with executor id 4107 parsl.dataflow.dflow INFO: Standard output for task 4106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b016b1cc-8225-4855-8d60-de3a872625af_isrForCrosstalkSources_997015_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b016b1cc-8225-4855-8d60-de3a872625af_isrForCrosstalkSources_997015_47.stderr parsl.dataflow.dflow DEBUG: Task 4107 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4107 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4107 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4107 has memoization hash 7d4e9d0aba76b99905df4e269063ae38 parsl.dataflow.memoization INFO: Task 4107 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4107 try 0 launched on executor multi with executor id 4108 parsl.dataflow.dflow INFO: Standard output for task 4107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7f335333-a87a-4a8a-aab7-e197ee221418_isrForCrosstalkSources_997006_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7f335333-a87a-4a8a-aab7-e197ee221418_isrForCrosstalkSources_997006_32.stderr parsl.dataflow.dflow DEBUG: Task 4108 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4108 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4108 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4108 has memoization hash 0cc456fb810ef1db86b67190a9895a88 parsl.dataflow.memoization INFO: Task 4108 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4108 try 0 launched on executor multi with executor id 4109 parsl.dataflow.dflow INFO: Standard output for task 4108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e1be73b5-213d-4744-b30c-4a582aa42e3d_isrForCrosstalkSources_997026_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e1be73b5-213d-4744-b30c-4a582aa42e3d_isrForCrosstalkSources_997026_8.stderr parsl.dataflow.dflow DEBUG: Task 4109 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4109 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4109 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4109 has memoization hash 70e46b4728c2812d409481215fd77369 parsl.dataflow.memoization INFO: Task 4109 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4109 try 0 launched on executor multi with executor id 4110 parsl.dataflow.dflow INFO: Standard output for task 4109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a49cf293-ca5e-4a7a-9244-23710d1414a5_isrForCrosstalkSources_996984_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a49cf293-ca5e-4a7a-9244-23710d1414a5_isrForCrosstalkSources_996984_42.stderr parsl.dataflow.dflow DEBUG: Task 4110 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4110 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4110 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4110 has memoization hash 534daafa5ad567e45262bba0f464acce parsl.dataflow.memoization INFO: Task 4110 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2556c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4110 try 0 launched on executor multi with executor id 4111 parsl.dataflow.dflow INFO: Standard output for task 4110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f3eefc6c-b517-4d73-aab9-a54af69cfcdc_isrForCrosstalkSources_997018_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f3eefc6c-b517-4d73-aab9-a54af69cfcdc_isrForCrosstalkSources_997018_48.stderr parsl.dataflow.dflow DEBUG: Task 4111 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4111 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4111 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4111 has memoization hash deb421054e4c5d38b02005af715e463a parsl.dataflow.memoization INFO: Task 4111 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4111 try 0 launched on executor multi with executor id 4112 parsl.dataflow.dflow INFO: Standard output for task 4111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b578b5e5-57fb-457e-9f2f-fa25bd163c9d_isrForCrosstalkSources_997021_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b578b5e5-57fb-457e-9f2f-fa25bd163c9d_isrForCrosstalkSources_997021_1.stderr parsl.dataflow.dflow DEBUG: Task 4112 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4112 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4112 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4112 has memoization hash 7afe6279b87e2ba493654ac83c21f554 parsl.dataflow.memoization INFO: Task 4112 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2558a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4112 try 0 launched on executor multi with executor id 4113 parsl.dataflow.dflow INFO: Standard output for task 4112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/49c6eac4-2169-4fd6-a226-da8c832169f9_isrForCrosstalkSources_997024_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/49c6eac4-2169-4fd6-a226-da8c832169f9_isrForCrosstalkSources_997024_39.stderr parsl.dataflow.dflow DEBUG: Task 4113 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4113 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4113 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4113 has memoization hash a8eb727d700be1e2d7f6178ad620a95d parsl.dataflow.memoization INFO: Task 4113 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4113 try 0 launched on executor multi with executor id 4114 parsl.dataflow.dflow INFO: Standard output for task 4113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/d2d3a68e-52aa-4995-b5bd-974a627a9be2_isrForCrosstalkSources_997021_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/d2d3a68e-52aa-4995-b5bd-974a627a9be2_isrForCrosstalkSources_997021_33.stderr parsl.dataflow.dflow DEBUG: Task 4114 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4114 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4114 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4114 has memoization hash 9233cad2bcae30bce63764fe43d0d4e8 parsl.dataflow.memoization INFO: Task 4114 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) 2024-03-15 13:50:04 proc_lsst.multi:170 [INFO] [multi] job_id 17074898 parsl.dataflow.dflow INFO: Parsl task 4114 try 0 launched on executor multi with executor id 4115 parsl.dataflow.dflow INFO: Standard output for task 4114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/983d17e5-908c-4e97-aa04-14f8e105f3e1_isrForCrosstalkSources_996952_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/983d17e5-908c-4e97-aa04-14f8e105f3e1_isrForCrosstalkSources_996952_30.stderr parsl.dataflow.dflow DEBUG: Task 4115 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4115 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4115 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4115 has memoization hash 4e5d28b7a2a25d115863996e9e003023 parsl.dataflow.memoization INFO: Task 4115 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) proc_lsst.multi INFO: [multi] job_id 17074898 2024-03-15 13:50:04 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 4 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 4 2024-03-15 13:50:04 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:04 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074898 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074898 parsl.executors.status_handling DEBUG: Launched block 39 on executor multi with job ID 17074898 parsl.process_loggers DEBUG: Normal ending for _general_strategy on thread JobStatusPoller-Timer-Thread-22709913200592 2024-03-15 13:50:04 proc_lsst.multi:146 [INFO] found job 91028 in provider local proc_lsst.multi INFO: found job 91028 in provider local parsl.dataflow.dflow INFO: Parsl task 4115 try 0 launched on executor multi with executor id 4116 parsl.dataflow.dflow INFO: Standard output for task 4115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/2cddb7f2-36d7-406c-bf19-b6562f46655b_isrForCrosstalkSources_997019_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/2cddb7f2-36d7-406c-bf19-b6562f46655b_isrForCrosstalkSources_997019_9.stderr parsl.dataflow.dflow DEBUG: Task 4116 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4116 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4116 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4116 has memoization hash 3d5529cf4ad38b212ffc253dc380ef87 parsl.dataflow.memoization INFO: Task 4116 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c3f0ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4116 try 0 launched on executor multi with executor id 4117 parsl.dataflow.dflow INFO: Standard output for task 4116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/6a54ff00-6b18-41c8-abd7-8c0028d2da73_isrForCrosstalkSources_996986_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/6a54ff00-6b18-41c8-abd7-8c0028d2da73_isrForCrosstalkSources_996986_59.stderr parsl.dataflow.dflow DEBUG: Task 4117 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4117 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4117 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4117 has memoization hash f611fee5cfc429f51f47a45eb86ba3b3 parsl.dataflow.memoization INFO: Task 4117 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4117 try 0 launched on executor multi with executor id 4118 parsl.dataflow.dflow INFO: Standard output for task 4117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/ef581af9-ac04-4a51-95ad-40735c28960d_isrForCrosstalkSources_996973_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/ef581af9-ac04-4a51-95ad-40735c28960d_isrForCrosstalkSources_996973_23.stderr parsl.dataflow.dflow DEBUG: Task 4118 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4118 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4118 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4118 has memoization hash b2020b95be520557e6d9d09c5be015cb parsl.dataflow.memoization INFO: Task 4118 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4118 try 0 launched on executor multi with executor id 4119 parsl.dataflow.dflow INFO: Standard output for task 4118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1f2a995c-ddc4-4d95-b38c-273b65887c27_isrForCrosstalkSources_997032_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1f2a995c-ddc4-4d95-b38c-273b65887c27_isrForCrosstalkSources_997032_14.stderr parsl.dataflow.dflow DEBUG: Task 4119 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4119 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4119 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4119 has memoization hash f94191a669cad51251c5d5ed63f78f78 parsl.dataflow.memoization INFO: Task 4119 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4119 try 0 launched on executor multi with executor id 4120 parsl.dataflow.dflow INFO: Standard output for task 4119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a1a44dce-1acc-4cf5-bb86-1b8f750a8123_isrForCrosstalkSources_996977_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a1a44dce-1acc-4cf5-bb86-1b8f750a8123_isrForCrosstalkSources_996977_21.stderr parsl.dataflow.dflow DEBUG: Task 4120 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4120 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4120 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4120 has memoization hash aac9a54b80768beb8927b61897105e2b parsl.dataflow.memoization INFO: Task 4120 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c254040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4120 try 0 launched on executor multi with executor id 4121 parsl.dataflow.dflow INFO: Standard output for task 4120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/649d483a-824c-4f29-ba8f-f4774380ef97_isrForCrosstalkSources_997000_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/649d483a-824c-4f29-ba8f-f4774380ef97_isrForCrosstalkSources_997000_52.stderr parsl.dataflow.dflow DEBUG: Task 4121 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4121 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4121 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4121 has memoization hash cff1bf60b1e013c232844c30573f2852 parsl.dataflow.memoization INFO: Task 4121 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4121 try 0 launched on executor multi with executor id 4122 parsl.dataflow.dflow INFO: Standard output for task 4121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/73217616-2029-4563-be0f-d6e91c1d3dca_isrForCrosstalkSources_997011_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/73217616-2029-4563-be0f-d6e91c1d3dca_isrForCrosstalkSources_997011_49.stderr parsl.dataflow.dflow DEBUG: Task 4122 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4122 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4122 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4122 has memoization hash 3baedfc3a73d29ace10d9df36bf363fe parsl.dataflow.memoization INFO: Task 4122 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4122 try 0 launched on executor multi with executor id 4123 parsl.dataflow.dflow INFO: Standard output for task 4122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d1df3b93-9477-48e9-867c-ab05e801a187_isrForCrosstalkSources_996987_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d1df3b93-9477-48e9-867c-ab05e801a187_isrForCrosstalkSources_996987_49.stderr parsl.dataflow.dflow DEBUG: Task 4123 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4123 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4123 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4123 has memoization hash bbd7c121e0246682d6b9ace4f1961649 parsl.dataflow.memoization INFO: Task 4123 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4123 try 0 launched on executor multi with executor id 4124 parsl.dataflow.dflow INFO: Standard output for task 4123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/61e7d61c-9385-4259-8082-82c0dc22d634_isrForCrosstalkSources_996987_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/61e7d61c-9385-4259-8082-82c0dc22d634_isrForCrosstalkSources_996987_11.stderr parsl.dataflow.dflow DEBUG: Task 4124 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4124 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4124 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4124 has memoization hash 77fd25016a0329d8b5598a280ae5fb02 parsl.dataflow.memoization INFO: Task 4124 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 21 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d442684d45ccb25cc306cdcedcc16877 with result from task 21 parsl.dataflow.dflow DEBUG: Task 21 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 21 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/bf2640a4-a504-4aa6-b5ec-0f0fd2d42b63_isrForCrosstalkSources_997011_21.stdout parsl.dataflow.dflow INFO: Standard error for task 21 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/bf2640a4-a504-4aa6-b5ec-0f0fd2d42b63_isrForCrosstalkSources_997011_21.stderr parsl.dataflow.dflow INFO: Task 24 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 501c93a680b428fa1d9357c6e35abb03 with result from task 24 parsl.dataflow.dflow DEBUG: Task 24 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 24 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9bd3c513-b556-4e37-affd-38a7beef8ef3_isrForCrosstalkSources_997023_56.stdout parsl.dataflow.dflow INFO: Standard error for task 24 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9bd3c513-b556-4e37-affd-38a7beef8ef3_isrForCrosstalkSources_997023_56.stderr 2024-03-15 13:50:04 proc_lsst.multi:146 [INFO] found job 17074833 in provider astro proc_lsst.multi INFO: found job 17074833 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4124 try 0 launched on executor multi with executor id 4125 parsl.dataflow.dflow INFO: Standard output for task 4124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/28b7e20c-65e0-41bf-bd23-32e707725b14_isrForCrosstalkSources_996953_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/28b7e20c-65e0-41bf-bd23-32e707725b14_isrForCrosstalkSources_996953_19.stderr parsl.dataflow.dflow DEBUG: Task 4125 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4125 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4125 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4125 has memoization hash 6124831ab0565daa98a5edb3d5051dcc parsl.dataflow.memoization INFO: Task 4125 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4125 try 0 launched on executor multi with executor id 4126 parsl.dataflow.dflow INFO: Standard output for task 4125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c99ed4ec-fe96-4f2c-a90f-9566728cef85_isrForCrosstalkSources_996972_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c99ed4ec-fe96-4f2c-a90f-9566728cef85_isrForCrosstalkSources_996972_7.stderr parsl.dataflow.dflow DEBUG: Task 4126 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4126 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4126 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4126 has memoization hash 0bedc8bb88a3b518cf5a6de07e1abb54 parsl.dataflow.memoization INFO: Task 4126 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4126 try 0 launched on executor multi with executor id 4127 parsl.dataflow.dflow INFO: Standard output for task 4126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8cb7c2d0-9edf-404b-91fc-48a4b7e266b2_isrForCrosstalkSources_997027_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8cb7c2d0-9edf-404b-91fc-48a4b7e266b2_isrForCrosstalkSources_997027_62.stderr parsl.dataflow.dflow DEBUG: Task 4127 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4127 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4127 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4127 has memoization hash cd5cb134507183e1040e2cc310960847 parsl.dataflow.memoization INFO: Task 4127 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4127 try 0 launched on executor multi with executor id 4128 parsl.dataflow.dflow INFO: Standard output for task 4127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/05b1c6ad-3799-4dda-919e-e4612f55fa5d_isrForCrosstalkSources_997006_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/05b1c6ad-3799-4dda-919e-e4612f55fa5d_isrForCrosstalkSources_997006_18.stderr parsl.dataflow.dflow DEBUG: Task 4128 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4128 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4128 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4128 has memoization hash 4acaf04d63048f70c944d432806ab47d parsl.dataflow.memoization INFO: Task 4128 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4128 try 0 launched on executor multi with executor id 4129 parsl.dataflow.dflow INFO: Standard output for task 4128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/3ecbd8a5-15d8-4c0f-9cca-6cc2d59108a8_isrForCrosstalkSources_997002_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/3ecbd8a5-15d8-4c0f-9cca-6cc2d59108a8_isrForCrosstalkSources_997002_37.stderr parsl.dataflow.dflow DEBUG: Task 4129 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4129 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4129 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4129 has memoization hash b5b512c2ef5fc7fa46f5d612d27a5cd5 parsl.dataflow.memoization INFO: Task 4129 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2560c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4129 try 0 launched on executor multi with executor id 4130 parsl.dataflow.dflow INFO: Standard output for task 4129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/cd22cccb-b455-404a-92ac-3234361266ae_isrForCrosstalkSources_996959_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/cd22cccb-b455-404a-92ac-3234361266ae_isrForCrosstalkSources_996959_24.stderr parsl.dataflow.dflow DEBUG: Task 4130 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4130 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4130 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4130 has memoization hash 46c485b9d67d18f30f43a38868fc187b parsl.dataflow.memoization INFO: Task 4130 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4130 try 0 launched on executor multi with executor id 4131 parsl.dataflow.dflow INFO: Standard output for task 4130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/85720f88-e3ca-4be5-a8c6-91b4c47889e9_isrForCrosstalkSources_997010_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/85720f88-e3ca-4be5-a8c6-91b4c47889e9_isrForCrosstalkSources_997010_14.stderr parsl.dataflow.dflow DEBUG: Task 4131 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4131 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4131 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4131 has memoization hash e07ee9c5e793d69969bd30f3ecd8f702 parsl.dataflow.memoization INFO: Task 4131 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2563e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4131 try 0 launched on executor multi with executor id 4132 parsl.dataflow.dflow INFO: Standard output for task 4131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/630a00ea-cd76-42f0-92e1-c313d10fd19b_isrForCrosstalkSources_997028_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/630a00ea-cd76-42f0-92e1-c313d10fd19b_isrForCrosstalkSources_997028_20.stderr parsl.dataflow.dflow DEBUG: Task 4132 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4132 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4132 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4132 has memoization hash 440b5fbe3ae3eebb4e941c3b6d892064 parsl.dataflow.memoization INFO: Task 4132 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2565c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4132 try 0 launched on executor multi with executor id 4133 parsl.dataflow.dflow INFO: Standard output for task 4132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2e628b3c-7ae3-410c-a64d-c33b99f460cd_isrForCrosstalkSources_996970_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2e628b3c-7ae3-410c-a64d-c33b99f460cd_isrForCrosstalkSources_996970_28.stderr parsl.dataflow.dflow DEBUG: Task 4133 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4133 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4133 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4133 has memoization hash c37e400a10626d0ff211dcec2ddda103 parsl.dataflow.memoization INFO: Task 4133 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4133 try 0 launched on executor multi with executor id 4134 parsl.dataflow.dflow INFO: Standard output for task 4133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/48bcfc13-f82b-4bb2-b3b6-898712cd8b98_isrForCrosstalkSources_997031_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/48bcfc13-f82b-4bb2-b3b6-898712cd8b98_isrForCrosstalkSources_997031_57.stderr parsl.dataflow.dflow DEBUG: Task 4134 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4134 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4134 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4134 has memoization hash 211fc6c285af33faed08edba655a3aab parsl.dataflow.memoization INFO: Task 4134 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4134 try 0 launched on executor multi with executor id 4135 parsl.dataflow.dflow INFO: Standard output for task 4134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ce1f7a10-772e-423a-b431-9de22abad73b_isrForCrosstalkSources_996975_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ce1f7a10-772e-423a-b431-9de22abad73b_isrForCrosstalkSources_996975_50.stderr parsl.dataflow.dflow DEBUG: Task 4135 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4135 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4135 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4135 has memoization hash 996b353404ec61630a4b9a156f6d2e04 parsl.dataflow.memoization INFO: Task 4135 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4135 try 0 launched on executor multi with executor id 4136 parsl.dataflow.dflow INFO: Standard output for task 4135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6bbd4c91-bcf9-4abe-ac27-b8515dc2933c_isrForCrosstalkSources_996952_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6bbd4c91-bcf9-4abe-ac27-b8515dc2933c_isrForCrosstalkSources_996952_46.stderr parsl.dataflow.dflow DEBUG: Task 4136 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4136 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4136 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4136 has memoization hash e98c8410fa94e019df273f3fa0c16407 parsl.dataflow.memoization INFO: Task 4136 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4136 try 0 launched on executor multi with executor id 4137 parsl.dataflow.dflow INFO: Standard output for task 4136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/229f64af-8b93-4e8b-acd4-8a81e10e74e9_isrForCrosstalkSources_996987_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/229f64af-8b93-4e8b-acd4-8a81e10e74e9_isrForCrosstalkSources_996987_21.stderr parsl.dataflow.dflow DEBUG: Task 4137 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4137 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4137 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4137 has memoization hash 82e339374e5ca8221610b4f33a161a6c parsl.dataflow.memoization INFO: Task 4137 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2568e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4137 try 0 launched on executor multi with executor id 4138 parsl.dataflow.dflow INFO: Standard output for task 4137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/46c16a19-1ea9-407d-919f-fca018e0e555_isrForCrosstalkSources_996969_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/46c16a19-1ea9-407d-919f-fca018e0e555_isrForCrosstalkSources_996969_31.stderr parsl.dataflow.dflow DEBUG: Task 4138 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4138 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4138 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4138 has memoization hash ef5bdd0041ae955fde9a9998839af27a parsl.dataflow.memoization INFO: Task 4138 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4138 try 0 launched on executor multi with executor id 4139 parsl.dataflow.dflow INFO: Standard output for task 4138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/caddeb77-0558-40b9-abf7-6312963ae369_isrForCrosstalkSources_996961_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/caddeb77-0558-40b9-abf7-6312963ae369_isrForCrosstalkSources_996961_31.stderr parsl.dataflow.dflow DEBUG: Task 4139 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4139 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4139 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4139 has memoization hash c14722bf0140f47741bd6ae0e5241f65 parsl.dataflow.memoization INFO: Task 4139 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2567a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4139 try 0 launched on executor multi with executor id 4140 parsl.dataflow.dflow INFO: Standard output for task 4139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ab4b06a6-4388-46d4-b036-41fb8903b29a_isrForCrosstalkSources_997000_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ab4b06a6-4388-46d4-b036-41fb8903b29a_isrForCrosstalkSources_997000_18.stderr parsl.dataflow.dflow DEBUG: Task 4140 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4140 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4140 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4140 has memoization hash eda9bfb1c7f14e1dcc0a64ceeb526f5d parsl.dataflow.memoization INFO: Task 4140 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4140 try 0 launched on executor multi with executor id 4141 parsl.dataflow.dflow INFO: Standard output for task 4140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b969646b-a8d7-4bd4-8be8-ad29ddd80a97_isrForCrosstalkSources_996979_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b969646b-a8d7-4bd4-8be8-ad29ddd80a97_isrForCrosstalkSources_996979_54.stderr parsl.dataflow.dflow DEBUG: Task 4141 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4141 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4141 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4141 has memoization hash 359d125692573a226cb07cccc638f125 parsl.dataflow.memoization INFO: Task 4141 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4141 try 0 launched on executor multi with executor id 4142 parsl.dataflow.dflow INFO: Standard output for task 4141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/73b9acd3-2dd9-4a98-8f6b-79dc68be948a_isrForCrosstalkSources_996985_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/73b9acd3-2dd9-4a98-8f6b-79dc68be948a_isrForCrosstalkSources_996985_56.stderr parsl.dataflow.dflow DEBUG: Task 4142 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4142 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4142 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4142 has memoization hash e080372eda0519595855048b5f606fed parsl.dataflow.memoization INFO: Task 4142 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4142 try 0 launched on executor multi with executor id 4143 parsl.dataflow.dflow INFO: Standard output for task 4142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4ae80b42-1e06-4f43-a4f7-7c8fbb5aef41_isrForCrosstalkSources_997021_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4ae80b42-1e06-4f43-a4f7-7c8fbb5aef41_isrForCrosstalkSources_997021_18.stderr parsl.dataflow.dflow DEBUG: Task 4143 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4143 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4143 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4143 has memoization hash 677cd5421f0ed8e0b74dbe0e8f74bed1 parsl.dataflow.memoization INFO: Task 4143 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4143 try 0 launched on executor multi with executor id 4144 parsl.dataflow.dflow INFO: Standard output for task 4143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ce5ec4db-1076-41ec-92ad-c61630bcd9da_isrForCrosstalkSources_996969_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ce5ec4db-1076-41ec-92ad-c61630bcd9da_isrForCrosstalkSources_996969_18.stderr parsl.dataflow.dflow DEBUG: Task 4144 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4144 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4144 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4144 has memoization hash 12bd9523caceb1c51fbb2aa0a741d9ca parsl.dataflow.memoization INFO: Task 4144 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4144 try 0 launched on executor multi with executor id 4145 parsl.dataflow.dflow INFO: Standard output for task 4144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/9acb5344-5013-44f2-95be-8288833bc1be_isrForCrosstalkSources_997026_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/9acb5344-5013-44f2-95be-8288833bc1be_isrForCrosstalkSources_997026_26.stderr parsl.dataflow.dflow DEBUG: Task 4145 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4145 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4145 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4145 has memoization hash 42e5a43d7df4822a2d3b7f32a819c68b parsl.dataflow.memoization INFO: Task 4145 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4145 try 0 launched on executor multi with executor id 4146 parsl.dataflow.dflow INFO: Standard output for task 4145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8eb7f91c-4277-4ef9-937d-6569aaf9718c_isrForCrosstalkSources_996973_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8eb7f91c-4277-4ef9-937d-6569aaf9718c_isrForCrosstalkSources_996973_15.stderr parsl.dataflow.dflow DEBUG: Task 4146 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4146 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4146 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4146 has memoization hash 972c54c424ac3703e6e0da38a5ee73c7 parsl.dataflow.memoization INFO: Task 4146 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4146 try 0 launched on executor multi with executor id 4147 parsl.dataflow.dflow INFO: Standard output for task 4146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c6bef2d1-9842-4722-8d55-7b984e78303d_isrForCrosstalkSources_997007_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c6bef2d1-9842-4722-8d55-7b984e78303d_isrForCrosstalkSources_997007_4.stderr parsl.dataflow.dflow DEBUG: Task 4147 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4147 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4147 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4147 has memoization hash fd01c33084e7d9b654998debd03c21e8 parsl.dataflow.memoization INFO: Task 4147 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4147 try 0 launched on executor multi with executor id 4148 parsl.dataflow.dflow INFO: Standard output for task 4147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f98650ef-5f71-49c5-af02-246b42ce9b1a_isrForCrosstalkSources_997032_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f98650ef-5f71-49c5-af02-246b42ce9b1a_isrForCrosstalkSources_997032_57.stderr parsl.dataflow.dflow DEBUG: Task 4148 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4148 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4148 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4148 has memoization hash 4b0640846d3183372fff7343d4d9100f parsl.dataflow.memoization INFO: Task 4148 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4148 try 0 launched on executor multi with executor id 4149 parsl.dataflow.dflow INFO: Standard output for task 4148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/78ddabd4-c910-4c02-b97b-47af64ab8951_isrForCrosstalkSources_996981_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/78ddabd4-c910-4c02-b97b-47af64ab8951_isrForCrosstalkSources_996981_40.stderr parsl.dataflow.dflow DEBUG: Task 4149 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4149 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4149 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4149 has memoization hash dd4eb4cc0bfe19ded56a0e10a322064b parsl.dataflow.memoization INFO: Task 4149 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4149 try 0 launched on executor multi with executor id 4150 parsl.dataflow.dflow INFO: Standard output for task 4149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/88892a16-3df8-4143-8608-6f43ec305f53_isrForCrosstalkSources_996984_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/88892a16-3df8-4143-8608-6f43ec305f53_isrForCrosstalkSources_996984_44.stderr parsl.dataflow.dflow DEBUG: Task 4150 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4150 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4150 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4150 has memoization hash 83b495377604e4517aab428608c66e58 parsl.dataflow.memoization INFO: Task 4150 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4150 try 0 launched on executor multi with executor id 4151 parsl.dataflow.dflow INFO: Standard output for task 4150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/47dc76cc-36fc-4df9-92a7-6f0dca8f79be_isrForCrosstalkSources_997031_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/47dc76cc-36fc-4df9-92a7-6f0dca8f79be_isrForCrosstalkSources_997031_3.stderr parsl.dataflow.dflow DEBUG: Task 4151 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4151 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4151 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4151 has memoization hash 3be5e972f659fad40b897b4e58ad4ce1 parsl.dataflow.memoization INFO: Task 4151 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4151 try 0 launched on executor multi with executor id 4152 parsl.dataflow.dflow INFO: Standard output for task 4151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7188aa70-d8a5-4e45-8b8b-f4c89d4ef759_isrForCrosstalkSources_997021_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7188aa70-d8a5-4e45-8b8b-f4c89d4ef759_isrForCrosstalkSources_997021_2.stderr parsl.dataflow.dflow DEBUG: Task 4152 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4152 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4152 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4152 has memoization hash 97fc08ad3310eea7d1a8519c3f1390f4 parsl.dataflow.memoization INFO: Task 4152 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2572e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4152 try 0 launched on executor multi with executor id 4153 parsl.dataflow.dflow INFO: Standard output for task 4152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/1480630c-0858-4e6f-bba6-967096e7f632_isrForCrosstalkSources_997002_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/1480630c-0858-4e6f-bba6-967096e7f632_isrForCrosstalkSources_997002_60.stderr parsl.dataflow.dflow DEBUG: Task 4153 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4153 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4153 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4153 has memoization hash 5ce872ec3301e4cddbbd69cf3087b2bd parsl.dataflow.memoization INFO: Task 4153 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4153 try 0 launched on executor multi with executor id 4154 parsl.dataflow.dflow INFO: Standard output for task 4153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8c2d38ca-17cd-4f58-be55-bacdfa856ed1_isrForCrosstalkSources_996956_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8c2d38ca-17cd-4f58-be55-bacdfa856ed1_isrForCrosstalkSources_996956_54.stderr parsl.dataflow.dflow DEBUG: Task 4154 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4154 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4154 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4154 has memoization hash d9f46fef031e37772c075aafc0cc1fd9 parsl.dataflow.memoization INFO: Task 4154 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4154 try 0 launched on executor multi with executor id 4155 parsl.dataflow.dflow INFO: Standard output for task 4154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/93d35b35-83ff-4e99-a97c-c62a8097f4c8_isrForCrosstalkSources_996998_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/93d35b35-83ff-4e99-a97c-c62a8097f4c8_isrForCrosstalkSources_996998_22.stderr parsl.dataflow.dflow DEBUG: Task 4155 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4155 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4155 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4155 has memoization hash 1f371cbda97e9f9913896cca75240b82 parsl.dataflow.memoization INFO: Task 4155 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2574c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4155 try 0 launched on executor multi with executor id 4156 parsl.dataflow.dflow INFO: Standard output for task 4155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/493338f7-d670-4798-863a-3ee3df092cef_isrForCrosstalkSources_996974_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/493338f7-d670-4798-863a-3ee3df092cef_isrForCrosstalkSources_996974_25.stderr parsl.dataflow.dflow DEBUG: Task 4156 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4156 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4156 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4156 has memoization hash 46252afcd820f20a9b38bc4a7ddc71a5 parsl.dataflow.memoization INFO: Task 4156 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2571a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4156 try 0 launched on executor multi with executor id 4157 parsl.dataflow.dflow INFO: Standard output for task 4156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/e18e5beb-78b6-4cbf-beaf-d3f169570390_isrForCrosstalkSources_997000_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/e18e5beb-78b6-4cbf-beaf-d3f169570390_isrForCrosstalkSources_997000_47.stderr parsl.dataflow.dflow DEBUG: Task 4157 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4157 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4157 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4157 has memoization hash cb2aa342f64bf1559e8730b3b2730361 parsl.dataflow.memoization INFO: Task 4157 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4157 try 0 launched on executor multi with executor id 4158 parsl.dataflow.dflow INFO: Standard output for task 4157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/1c89dd15-e892-4e87-8635-c3a097cef58b_isrForCrosstalkSources_997008_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/1c89dd15-e892-4e87-8635-c3a097cef58b_isrForCrosstalkSources_997008_53.stderr parsl.dataflow.dflow DEBUG: Task 4158 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4158 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4158 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4158 has memoization hash 00bbe70ef127b45d75d25b7f80bea7c8 parsl.dataflow.memoization INFO: Task 4158 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4158 try 0 launched on executor multi with executor id 4159 parsl.dataflow.dflow INFO: Standard output for task 4158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/eddbdf41-916e-4ec2-87dd-6ace3035f384_isrForCrosstalkSources_996977_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/eddbdf41-916e-4ec2-87dd-6ace3035f384_isrForCrosstalkSources_996977_54.stderr parsl.dataflow.dflow DEBUG: Task 4159 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4159 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4159 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4159 has memoization hash c6801a8f65a41ed336edfb60e3434b3f parsl.dataflow.memoization INFO: Task 4159 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4159 try 0 launched on executor multi with executor id 4160 parsl.dataflow.dflow INFO: Standard output for task 4159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/1a3364ac-eb5d-4f27-9a84-2764a32ff099_isrForCrosstalkSources_996990_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/1a3364ac-eb5d-4f27-9a84-2764a32ff099_isrForCrosstalkSources_996990_5.stderr parsl.dataflow.dflow DEBUG: Task 4160 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4160 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4160 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4160 has memoization hash 636d624dde0a992e932a4ac66af4c7f4 parsl.dataflow.memoization INFO: Task 4160 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4160 try 0 launched on executor multi with executor id 4161 parsl.dataflow.dflow INFO: Standard output for task 4160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/950b1801-fa98-4751-9de6-748af10f2f2d_isrForCrosstalkSources_997021_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/950b1801-fa98-4751-9de6-748af10f2f2d_isrForCrosstalkSources_997021_47.stderr parsl.dataflow.dflow DEBUG: Task 4161 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4161 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4161 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4161 has memoization hash 80876451aeb98a2f3464bc6da205b37d parsl.dataflow.memoization INFO: Task 4161 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2576a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4161 try 0 launched on executor multi with executor id 4162 parsl.dataflow.dflow INFO: Standard output for task 4161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/e01626b5-494c-4e36-b550-bfec6c22f7f4_isrForCrosstalkSources_996974_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/e01626b5-494c-4e36-b550-bfec6c22f7f4_isrForCrosstalkSources_996974_33.stderr parsl.dataflow.dflow DEBUG: Task 4162 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4162 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4162 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4162 has memoization hash e5c01f224003a911056870216bd0d934 parsl.dataflow.memoization INFO: Task 4162 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4162 try 0 launched on executor multi with executor id 4163 parsl.dataflow.dflow INFO: Standard output for task 4162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/aadee31f-5e25-4038-bffc-65e18cb5e7ff_isrForCrosstalkSources_996957_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/aadee31f-5e25-4038-bffc-65e18cb5e7ff_isrForCrosstalkSources_996957_32.stderr parsl.dataflow.dflow DEBUG: Task 4163 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4163 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4163 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4163 has memoization hash bcdaecea48c4efeb630a38e097d098f2 parsl.dataflow.memoization INFO: Task 4163 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2577e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4163 try 0 launched on executor multi with executor id 4164 parsl.dataflow.dflow INFO: Standard output for task 4163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1ebc5081-d79d-448c-a81a-afdba4b6c588_isrForCrosstalkSources_996964_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1ebc5081-d79d-448c-a81a-afdba4b6c588_isrForCrosstalkSources_996964_10.stderr parsl.dataflow.dflow DEBUG: Task 4164 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4164 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4164 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4164 has memoization hash 5b6332273a73719c46707d136a30469e parsl.dataflow.memoization INFO: Task 4164 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4164 try 0 launched on executor multi with executor id 4165 parsl.dataflow.dflow INFO: Standard output for task 4164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/65c89758-1653-41db-9358-35fd9c955afe_isrForCrosstalkSources_996972_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/65c89758-1653-41db-9358-35fd9c955afe_isrForCrosstalkSources_996972_44.stderr parsl.dataflow.dflow DEBUG: Task 4165 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4165 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4165 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4165 has memoization hash 1f48173614fc058ab1b1cc7122077cc4 parsl.dataflow.memoization INFO: Task 4165 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4165 try 0 launched on executor multi with executor id 4166 parsl.dataflow.dflow INFO: Standard output for task 4165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a42f1078-aac2-4d79-b0d6-0c119db67e00_isrForCrosstalkSources_996954_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a42f1078-aac2-4d79-b0d6-0c119db67e00_isrForCrosstalkSources_996954_43.stderr parsl.dataflow.dflow DEBUG: Task 4166 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4166 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4166 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4166 has memoization hash 701fe1bf71f647d9d73925f2dd3d2f9e parsl.dataflow.memoization INFO: Task 4166 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4166 try 0 launched on executor multi with executor id 4167 parsl.dataflow.dflow INFO: Standard output for task 4166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/15fa2398-ac49-4c1f-beee-9bbd15dc0900_isrForCrosstalkSources_997009_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/15fa2398-ac49-4c1f-beee-9bbd15dc0900_isrForCrosstalkSources_997009_21.stderr parsl.dataflow.dflow DEBUG: Task 4167 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4167 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4167 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4167 has memoization hash 448887ef484a7930fb621e192ac35f56 parsl.dataflow.memoization INFO: Task 4167 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4167 try 0 launched on executor multi with executor id 4168 parsl.dataflow.dflow INFO: Standard output for task 4167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/400487b6-78f1-4033-80b8-5d32f010685a_isrForCrosstalkSources_996955_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/400487b6-78f1-4033-80b8-5d32f010685a_isrForCrosstalkSources_996955_10.stderr parsl.dataflow.dflow DEBUG: Task 4168 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4168 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4168 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4168 has memoization hash 0a33b8738952d490ae96f1a0859288b7 parsl.dataflow.memoization INFO: Task 4168 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4168 try 0 launched on executor multi with executor id 4169 parsl.dataflow.dflow INFO: Standard output for task 4168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/16ffb8f2-5ad0-4de6-b929-cf29139e7acd_isrForCrosstalkSources_997024_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/16ffb8f2-5ad0-4de6-b929-cf29139e7acd_isrForCrosstalkSources_997024_32.stderr parsl.dataflow.dflow DEBUG: Task 4169 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4169 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4169 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4169 has memoization hash 6bab71556478705e84c4d338a27cd070 parsl.dataflow.memoization INFO: Task 4169 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4169 try 0 launched on executor multi with executor id 4170 parsl.dataflow.dflow INFO: Standard output for task 4169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/52f7e4e0-4381-4f4c-acae-28286a816f92_isrForCrosstalkSources_997014_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/52f7e4e0-4381-4f4c-acae-28286a816f92_isrForCrosstalkSources_997014_35.stderr parsl.dataflow.dflow DEBUG: Task 4170 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4170 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4170 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4170 has memoization hash 757f96cafda15301c436d56fe0b3272f parsl.dataflow.memoization INFO: Task 4170 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4170 try 0 launched on executor multi with executor id 4171 parsl.dataflow.dflow INFO: Standard output for task 4170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f5af03d1-535c-427f-8ec2-9f05cc83d5d7_isrForCrosstalkSources_996953_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f5af03d1-535c-427f-8ec2-9f05cc83d5d7_isrForCrosstalkSources_996953_12.stderr parsl.dataflow.dflow DEBUG: Task 4171 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4171 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4171 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4171 has memoization hash c80df2a6b06b0cf4bdcb0155bf485d3e parsl.dataflow.memoization INFO: Task 4171 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c257b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4171 try 0 launched on executor multi with executor id 4172 parsl.dataflow.dflow INFO: Standard output for task 4171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/1900199a-1006-4fc7-b09a-c9fdfd3febd0_isrForCrosstalkSources_996989_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/1900199a-1006-4fc7-b09a-c9fdfd3febd0_isrForCrosstalkSources_996989_15.stderr parsl.dataflow.dflow DEBUG: Task 4172 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4172 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4172 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4172 has memoization hash c412d6dc9402c7951fbfa21ddc405094 parsl.dataflow.memoization INFO: Task 4172 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2579c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4172 try 0 launched on executor multi with executor id 4173 parsl.dataflow.dflow INFO: Standard output for task 4172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/fc58541b-8063-4872-b290-94edd79ca642_isrForCrosstalkSources_997009_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/fc58541b-8063-4872-b290-94edd79ca642_isrForCrosstalkSources_997009_15.stderr parsl.dataflow.dflow DEBUG: Task 4173 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4173 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4173 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4173 has memoization hash 458df32076ef2c1331df08be4d4e5f63 parsl.dataflow.memoization INFO: Task 4173 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4173 try 0 launched on executor multi with executor id 4174 parsl.dataflow.dflow INFO: Standard output for task 4173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/11eb4d01-d653-460c-b9ad-a16526e6b9ad_isrForCrosstalkSources_996997_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/11eb4d01-d653-460c-b9ad-a16526e6b9ad_isrForCrosstalkSources_996997_32.stderr parsl.dataflow.dflow DEBUG: Task 4174 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4174 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4174 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4174 has memoization hash b9141211f8fce8fad1b1da80f47254b9 parsl.dataflow.memoization INFO: Task 4174 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4174 try 0 launched on executor multi with executor id 4175 parsl.dataflow.dflow INFO: Standard output for task 4174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/697a0adc-f57b-4405-9507-4569a572c9e9_isrForCrosstalkSources_997001_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/697a0adc-f57b-4405-9507-4569a572c9e9_isrForCrosstalkSources_997001_61.stderr parsl.dataflow.dflow DEBUG: Task 4175 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4175 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4175 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4175 has memoization hash 7032366b7359504ac634f34dd8a76614 parsl.dataflow.memoization INFO: Task 4175 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c256020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4175 try 0 launched on executor multi with executor id 4176 parsl.dataflow.dflow INFO: Standard output for task 4175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/cd319707-7a2b-495c-8736-4f528ab81f1a_isrForCrosstalkSources_996990_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/cd319707-7a2b-495c-8736-4f528ab81f1a_isrForCrosstalkSources_996990_20.stderr parsl.dataflow.dflow DEBUG: Task 4176 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4176 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4176 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4176 has memoization hash ef9e4b5d11739e095a63f8a7c4214b8e parsl.dataflow.memoization INFO: Task 4176 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2562a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4176 try 0 launched on executor multi with executor id 4177 parsl.dataflow.dflow INFO: Standard output for task 4176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/cf8db234-4486-4a7e-8b43-e0f4f3531613_isrForCrosstalkSources_996962_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/cf8db234-4486-4a7e-8b43-e0f4f3531613_isrForCrosstalkSources_996962_62.stderr parsl.dataflow.dflow DEBUG: Task 4177 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4177 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4177 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4177 has memoization hash 1e771711cd4043482b3efe54d961b923 parsl.dataflow.memoization INFO: Task 4177 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f00e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4177 try 0 launched on executor multi with executor id 4178 parsl.dataflow.dflow INFO: Standard output for task 4177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4a1c33eb-f2f4-4210-9677-6edbe1b0d9ac_isrForCrosstalkSources_996959_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4a1c33eb-f2f4-4210-9677-6edbe1b0d9ac_isrForCrosstalkSources_996959_46.stderr parsl.dataflow.dflow DEBUG: Task 4178 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4178 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4178 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4178 has memoization hash 9d9fc446f93604534f0a47df4ae54de7 parsl.dataflow.memoization INFO: Task 4178 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4178 try 0 launched on executor multi with executor id 4179 parsl.dataflow.dflow INFO: Standard output for task 4178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/8d359b4b-ede0-4c3c-94f0-1347ef92883d_isrForCrosstalkSources_997003_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/8d359b4b-ede0-4c3c-94f0-1347ef92883d_isrForCrosstalkSources_997003_25.stderr parsl.dataflow.dflow DEBUG: Task 4179 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4179 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4179 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4179 has memoization hash 75f1bd820d756adf61149194b99ead44 parsl.dataflow.memoization INFO: Task 4179 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4179 try 0 launched on executor multi with executor id 4180 parsl.dataflow.dflow INFO: Standard output for task 4179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/47d5b8dd-115e-4fa9-8d3c-5d56169fecef_isrForCrosstalkSources_997009_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/47d5b8dd-115e-4fa9-8d3c-5d56169fecef_isrForCrosstalkSources_997009_34.stderr parsl.dataflow.dflow DEBUG: Task 4180 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4180 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4180 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4180 has memoization hash e133a9636b23dc094456f5da02741d12 parsl.dataflow.memoization INFO: Task 4180 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4180 try 0 launched on executor multi with executor id 4181 parsl.dataflow.dflow INFO: Standard output for task 4180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/5f4d5d13-7d12-48d9-ba25-c4631233a368_isrForCrosstalkSources_996993_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/5f4d5d13-7d12-48d9-ba25-c4631233a368_isrForCrosstalkSources_996993_30.stderr parsl.dataflow.dflow DEBUG: Task 4181 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4181 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4181 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4181 has memoization hash 4c0a504bb63cbe7503135aa0973d6140 parsl.dataflow.memoization INFO: Task 4181 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4181 try 0 launched on executor multi with executor id 4182 parsl.dataflow.dflow INFO: Standard output for task 4181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/2fe2a9b5-ce0f-4225-b371-2bdee8c6591c_isrForCrosstalkSources_996982_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/2fe2a9b5-ce0f-4225-b371-2bdee8c6591c_isrForCrosstalkSources_996982_19.stderr parsl.dataflow.dflow DEBUG: Task 4182 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4182 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4182 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4182 has memoization hash 07cdbc1bfdd3d534243b7e08e20b4373 parsl.dataflow.memoization INFO: Task 4182 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 18 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 474ac1c7bb5b44420c1c17851d9a5add with result from task 18 parsl.dataflow.dflow INFO: Parsl task 4182 try 0 launched on executor multi with executor id 4183 parsl.dataflow.dflow INFO: Standard output for task 4182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/8ae844d7-37db-49c8-8b59-e1d230021bee_isrForCrosstalkSources_997023_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/8ae844d7-37db-49c8-8b59-e1d230021bee_isrForCrosstalkSources_997023_22.stderr parsl.dataflow.dflow DEBUG: Task 4183 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4183 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4183 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4183 has memoization hash e70c8aa219539853a7cf026a2048290b parsl.dataflow.memoization INFO: Task 4183 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f04a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 18 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 18 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/08b2b231-2f60-4566-b2ff-0373f2ffd571_isrForCrosstalkSources_997032_15.stdout parsl.dataflow.dflow INFO: Standard error for task 18 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/08b2b231-2f60-4566-b2ff-0373f2ffd571_isrForCrosstalkSources_997032_15.stderr parsl.dataflow.dflow INFO: Parsl task 4183 try 0 launched on executor multi with executor id 4184 parsl.dataflow.dflow INFO: Standard output for task 4183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/6b2b9caa-5bcd-4e61-9255-7957686f7732_isrForCrosstalkSources_997021_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/6b2b9caa-5bcd-4e61-9255-7957686f7732_isrForCrosstalkSources_997021_51.stderr parsl.dataflow.dflow DEBUG: Task 4184 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4184 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4184 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4184 has memoization hash 31b285c1d55ff6b29a2d1b2481b3d93a parsl.dataflow.memoization INFO: Task 4184 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f05e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4184 try 0 launched on executor multi with executor id 4185 parsl.dataflow.dflow INFO: Standard output for task 4184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/10fb6ade-fdd5-415d-a827-c097385e5f80_isrForCrosstalkSources_996960_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/10fb6ade-fdd5-415d-a827-c097385e5f80_isrForCrosstalkSources_996960_10.stderr parsl.dataflow.dflow DEBUG: Task 4185 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4185 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4185 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4185 has memoization hash 5404062efe545d900c16a94d6125d378 parsl.dataflow.memoization INFO: Task 4185 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4185 try 0 launched on executor multi with executor id 4186 parsl.dataflow.dflow INFO: Standard output for task 4185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4c02647f-3e2d-459c-9f61-4ccbd8943d81_isrForCrosstalkSources_996995_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4c02647f-3e2d-459c-9f61-4ccbd8943d81_isrForCrosstalkSources_996995_31.stderr parsl.dataflow.dflow DEBUG: Task 4186 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4186 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4186 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4186 has memoization hash 181509bd0c5355637232c832dc3f1223 parsl.dataflow.memoization INFO: Task 4186 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4186 try 0 launched on executor multi with executor id 4187 parsl.dataflow.dflow INFO: Standard output for task 4186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/0977445e-9929-432d-a449-d9f22793d10b_isrForCrosstalkSources_996954_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/0977445e-9929-432d-a449-d9f22793d10b_isrForCrosstalkSources_996954_18.stderr parsl.dataflow.dflow DEBUG: Task 4187 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4187 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4187 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4187 has memoization hash e13d6be369dd7016660ebf491e211772 parsl.dataflow.memoization INFO: Task 4187 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4187 try 0 launched on executor multi with executor id 4188 parsl.dataflow.dflow INFO: Standard output for task 4187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ef5a9c98-786d-4a90-90ea-5cd804f795be_isrForCrosstalkSources_996954_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ef5a9c98-786d-4a90-90ea-5cd804f795be_isrForCrosstalkSources_996954_12.stderr parsl.dataflow.dflow DEBUG: Task 4188 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4188 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4188 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4188 has memoization hash 4e7871db5d09a59a0124e97371955900 parsl.dataflow.memoization INFO: Task 4188 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f07c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4188 try 0 launched on executor multi with executor id 4189 parsl.dataflow.dflow INFO: Standard output for task 4188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/ffeef2ca-8554-464c-93b5-a431112337dc_isrForCrosstalkSources_996965_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/ffeef2ca-8554-464c-93b5-a431112337dc_isrForCrosstalkSources_996965_35.stderr parsl.dataflow.dflow DEBUG: Task 4189 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4189 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4189 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4189 has memoization hash a4e988f86033d9c6bdabb1c22361e299 parsl.dataflow.memoization INFO: Task 4189 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4189 try 0 launched on executor multi with executor id 4190 parsl.dataflow.dflow INFO: Standard output for task 4189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/5705dbd0-dc0f-45e8-b60c-4ccc33adbb2f_isrForCrosstalkSources_996961_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/5705dbd0-dc0f-45e8-b60c-4ccc33adbb2f_isrForCrosstalkSources_996961_29.stderr parsl.dataflow.dflow DEBUG: Task 4190 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4190 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4190 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4190 has memoization hash 87aeb187ed7197a99cc8f6663da67cf9 parsl.dataflow.memoization INFO: Task 4190 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4190 try 0 launched on executor multi with executor id 4191 parsl.dataflow.dflow INFO: Standard output for task 4190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1f180f24-235c-4764-8fe1-0a41feab2f5f_isrForCrosstalkSources_997032_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1f180f24-235c-4764-8fe1-0a41feab2f5f_isrForCrosstalkSources_997032_49.stderr parsl.dataflow.dflow DEBUG: Task 4191 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4191 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4191 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4191 has memoization hash 6de01ae3c5455a1a21a759e4a0c7a4f8 parsl.dataflow.memoization INFO: Task 4191 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f09a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4191 try 0 launched on executor multi with executor id 4192 parsl.dataflow.dflow INFO: Standard output for task 4191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/958fc4b5-bc1e-4154-81a9-32214722dfc2_isrForCrosstalkSources_997016_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/958fc4b5-bc1e-4154-81a9-32214722dfc2_isrForCrosstalkSources_997016_60.stderr parsl.dataflow.dflow DEBUG: Task 4192 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4192 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4192 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4192 has memoization hash 2a49ecc37e00be8824baeb095fd75dd2 parsl.dataflow.memoization INFO: Task 4192 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4192 try 0 launched on executor multi with executor id 4193 parsl.dataflow.dflow INFO: Standard output for task 4192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7483437d-a0ce-4e2d-8b58-5774831bf904_isrForCrosstalkSources_997005_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7483437d-a0ce-4e2d-8b58-5774831bf904_isrForCrosstalkSources_997005_35.stderr parsl.dataflow.dflow DEBUG: Task 4193 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4193 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4193 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4193 has memoization hash 0abc7cb9f118d6a144af3418ddcb0e6b parsl.dataflow.memoization INFO: Task 4193 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4193 try 0 launched on executor multi with executor id 4194 parsl.dataflow.dflow INFO: Standard output for task 4193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/94f6be8e-2c65-4c42-861b-69808fcc8856_isrForCrosstalkSources_996980_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/94f6be8e-2c65-4c42-861b-69808fcc8856_isrForCrosstalkSources_996980_27.stderr parsl.dataflow.dflow DEBUG: Task 4194 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4194 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4194 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4194 has memoization hash 04f690b83da5c977ca75ea103f44c32c parsl.dataflow.memoization INFO: Task 4194 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4194 try 0 launched on executor multi with executor id 4195 parsl.dataflow.dflow INFO: Standard output for task 4194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b16f4e70-4090-4b03-8162-1d681f041250_isrForCrosstalkSources_997031_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b16f4e70-4090-4b03-8162-1d681f041250_isrForCrosstalkSources_997031_17.stderr parsl.dataflow.dflow DEBUG: Task 4195 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4195 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4195 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4195 has memoization hash ba9f750195624b941e7b6eb722d464b8 parsl.dataflow.memoization INFO: Task 4195 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4195 try 0 launched on executor multi with executor id 4196 parsl.dataflow.dflow INFO: Standard output for task 4195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d9824884-dfec-40aa-b495-77253f5ee51c_isrForCrosstalkSources_996985_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d9824884-dfec-40aa-b495-77253f5ee51c_isrForCrosstalkSources_996985_32.stderr parsl.dataflow.dflow DEBUG: Task 4196 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4196 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4196 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4196 has memoization hash 4fc30c8dfb06da806024ca98221b85c6 parsl.dataflow.memoization INFO: Task 4196 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4196 try 0 launched on executor multi with executor id 4197 parsl.dataflow.dflow INFO: Standard output for task 4196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/fb9ffdb7-fd2d-4d6e-a729-8b73f87dfef5_isrForCrosstalkSources_996956_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/fb9ffdb7-fd2d-4d6e-a729-8b73f87dfef5_isrForCrosstalkSources_996956_58.stderr parsl.dataflow.dflow DEBUG: Task 4197 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4197 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4197 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4197 has memoization hash af8c889aac2f113b9fac7366838256ef parsl.dataflow.memoization INFO: Task 4197 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4197 try 0 launched on executor multi with executor id 4198 parsl.dataflow.dflow INFO: Standard output for task 4197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/233d8bf0-da34-4623-a7ec-501f4e0db985_isrForCrosstalkSources_996976_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/233d8bf0-da34-4623-a7ec-501f4e0db985_isrForCrosstalkSources_996976_17.stderr parsl.dataflow.dflow DEBUG: Task 4198 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4198 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4198 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4198 has memoization hash 2f7fc79fd09515fc4d60b26c38d67686 parsl.dataflow.memoization INFO: Task 4198 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4198 try 0 launched on executor multi with executor id 4199 parsl.dataflow.dflow INFO: Standard output for task 4198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1ed29ae9-055a-4db2-8878-d9302182fd17_isrForCrosstalkSources_996967_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1ed29ae9-055a-4db2-8878-d9302182fd17_isrForCrosstalkSources_996967_15.stderr parsl.dataflow.dflow DEBUG: Task 4199 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4199 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4199 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4199 has memoization hash 9808f9fb31b10872bed2a58cb0457950 parsl.dataflow.memoization INFO: Task 4199 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4199 try 0 launched on executor multi with executor id 4200 parsl.dataflow.dflow INFO: Standard output for task 4199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/678cbcd2-4d30-4d52-b03c-147019160006_isrForCrosstalkSources_996988_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/678cbcd2-4d30-4d52-b03c-147019160006_isrForCrosstalkSources_996988_4.stderr parsl.dataflow.dflow DEBUG: Task 4200 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4200 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4200 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4200 has memoization hash c3125b00dd2fe58cd55536eeeba3d70c parsl.dataflow.memoization INFO: Task 4200 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4200 try 0 launched on executor multi with executor id 4201 parsl.dataflow.dflow INFO: Standard output for task 4200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/75804330-87f1-4b53-bd30-73d301948ffb_isrForCrosstalkSources_996981_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/75804330-87f1-4b53-bd30-73d301948ffb_isrForCrosstalkSources_996981_29.stderr parsl.dataflow.dflow DEBUG: Task 4201 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4201 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4201 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4201 has memoization hash d25c6dac8d35d36f896554e249f0caf3 parsl.dataflow.memoization INFO: Task 4201 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4201 try 0 launched on executor multi with executor id 4202 parsl.dataflow.dflow INFO: Standard output for task 4201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/adea6b4f-f785-4b03-9ded-cf415524126f_isrForCrosstalkSources_997007_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/adea6b4f-f785-4b03-9ded-cf415524126f_isrForCrosstalkSources_997007_13.stderr parsl.dataflow.dflow DEBUG: Task 4202 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4202 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4202 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4202 has memoization hash e33d906e1296f9634ee107000141dc97 parsl.dataflow.memoization INFO: Task 4202 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4202 try 0 launched on executor multi with executor id 4203 parsl.dataflow.dflow INFO: Standard output for task 4202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/19fa080b-d55e-4074-8186-e4f524ffe70f_isrForCrosstalkSources_997010_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/19fa080b-d55e-4074-8186-e4f524ffe70f_isrForCrosstalkSources_997010_19.stderr parsl.dataflow.dflow DEBUG: Task 4203 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4203 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4203 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4203 has memoization hash 652dabd4e8d4fe43135bda91b7fd75d1 parsl.dataflow.memoization INFO: Task 4203 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4203 try 0 launched on executor multi with executor id 4204 parsl.dataflow.dflow INFO: Standard output for task 4203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/cacb8d7c-6750-4b3e-bd61-b49aac149092_isrForCrosstalkSources_996982_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/cacb8d7c-6750-4b3e-bd61-b49aac149092_isrForCrosstalkSources_996982_48.stderr parsl.dataflow.dflow DEBUG: Task 4204 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4204 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4204 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4204 has memoization hash 7c81b8e127e4873fa3cf5ab2b704972c parsl.dataflow.memoization INFO: Task 4204 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4204 try 0 launched on executor multi with executor id 4205 parsl.dataflow.dflow INFO: Standard output for task 4204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ade87412-3a01-4716-95a4-34b2468eb1db_isrForCrosstalkSources_997002_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ade87412-3a01-4716-95a4-34b2468eb1db_isrForCrosstalkSources_997002_22.stderr parsl.dataflow.dflow DEBUG: Task 4205 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4205 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4205 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4205 has memoization hash 8170337b154278dc10ad252eb881a114 parsl.dataflow.memoization INFO: Task 4205 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f11c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4205 try 0 launched on executor multi with executor id 4206 parsl.dataflow.dflow INFO: Standard output for task 4205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/eab6b0f3-31d3-4b9b-b5d9-696a20c38db3_isrForCrosstalkSources_996976_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/eab6b0f3-31d3-4b9b-b5d9-696a20c38db3_isrForCrosstalkSources_996976_41.stderr parsl.dataflow.dflow DEBUG: Task 4206 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4206 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4206 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4206 has memoization hash e6c9cf0c67983dc095b9275462890d98 parsl.dataflow.memoization INFO: Task 4206 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4206 try 0 launched on executor multi with executor id 4207 parsl.dataflow.dflow INFO: Standard output for task 4206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0ffa1115-317e-4ae7-87b4-e352e125938e_isrForCrosstalkSources_997021_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0ffa1115-317e-4ae7-87b4-e352e125938e_isrForCrosstalkSources_997021_30.stderr parsl.dataflow.dflow DEBUG: Task 4207 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4207 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4207 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4207 has memoization hash 063c14af82c731875ec6e4752676279b parsl.dataflow.memoization INFO: Task 4207 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f13a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4207 try 0 launched on executor multi with executor id 4208 parsl.dataflow.dflow INFO: Standard output for task 4207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e84c3c63-c558-4217-8914-0533b95ad9e2_isrForCrosstalkSources_996970_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e84c3c63-c558-4217-8914-0533b95ad9e2_isrForCrosstalkSources_996970_15.stderr parsl.dataflow.dflow DEBUG: Task 4208 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4208 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4208 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4208 has memoization hash abc2e9bb9fca8e084d4bdf9d83fb735c parsl.dataflow.memoization INFO: Task 4208 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4208 try 0 launched on executor multi with executor id 4209 parsl.dataflow.dflow INFO: Standard output for task 4208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/92f3bf55-ab8b-4572-a732-aa1a6666bf24_isrForCrosstalkSources_997016_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/92f3bf55-ab8b-4572-a732-aa1a6666bf24_isrForCrosstalkSources_997016_48.stderr parsl.dataflow.dflow DEBUG: Task 4209 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4209 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4209 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4209 has memoization hash 1d853b94a94362011ed94c77651c7694 parsl.dataflow.memoization INFO: Task 4209 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4209 try 0 launched on executor multi with executor id 4210 parsl.dataflow.dflow INFO: Standard output for task 4209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/344cb7ff-6064-4722-bc6d-7f1c935431bd_isrForCrosstalkSources_996955_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/344cb7ff-6064-4722-bc6d-7f1c935431bd_isrForCrosstalkSources_996955_34.stderr parsl.dataflow.dflow DEBUG: Task 4210 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4210 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4210 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4210 has memoization hash 7ed3227749662040cd6fdbacb68abb81 parsl.dataflow.memoization INFO: Task 4210 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f14e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4210 try 0 launched on executor multi with executor id 4211 parsl.dataflow.dflow INFO: Standard output for task 4210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/2c047c19-37f3-4294-8be5-2f7a54525633_isrForCrosstalkSources_996988_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/2c047c19-37f3-4294-8be5-2f7a54525633_isrForCrosstalkSources_996988_29.stderr parsl.dataflow.dflow DEBUG: Task 4211 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4211 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4211 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4211 has memoization hash bfc1f43e72f07a5a95f3a95d008b927a parsl.dataflow.memoization INFO: Task 4211 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4211 try 0 launched on executor multi with executor id 4212 parsl.dataflow.dflow INFO: Standard output for task 4211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/90ea80fe-bc2b-4121-8467-df28e28861b3_isrForCrosstalkSources_997024_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/90ea80fe-bc2b-4121-8467-df28e28861b3_isrForCrosstalkSources_997024_28.stderr parsl.dataflow.dflow DEBUG: Task 4212 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4212 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4212 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4212 has memoization hash cb4fcaab49e893a007a48c285fa9cd6b parsl.dataflow.memoization INFO: Task 4212 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f16c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4212 try 0 launched on executor multi with executor id 4213 parsl.dataflow.dflow INFO: Standard output for task 4212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/25c9ccb1-c506-49f3-aa40-7a215c601b3f_isrForCrosstalkSources_997032_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/25c9ccb1-c506-49f3-aa40-7a215c601b3f_isrForCrosstalkSources_997032_40.stderr parsl.dataflow.dflow DEBUG: Task 4213 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4213 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4213 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4213 has memoization hash 793f354206b62e114ceb07b3e84b3a21 parsl.dataflow.memoization INFO: Task 4213 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4213 try 0 launched on executor multi with executor id 4214 parsl.dataflow.dflow INFO: Standard output for task 4213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/8376a5a4-4364-406f-9af6-02000f22e757_isrForCrosstalkSources_997011_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/8376a5a4-4364-406f-9af6-02000f22e757_isrForCrosstalkSources_997011_10.stderr parsl.dataflow.dflow DEBUG: Task 4214 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4214 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4214 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4214 has memoization hash 44e28a95604f76de255d5931a6dd32cc parsl.dataflow.memoization INFO: Task 4214 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f18a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4214 try 0 launched on executor multi with executor id 4215 parsl.dataflow.dflow INFO: Standard output for task 4214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/fc650015-32a3-4bee-9989-671c27ed0b55_isrForCrosstalkSources_997028_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/fc650015-32a3-4bee-9989-671c27ed0b55_isrForCrosstalkSources_997028_40.stderr parsl.dataflow.dflow DEBUG: Task 4215 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4215 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4215 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4215 has memoization hash 8d77d9062a6bc9cc830d6cadb315fc8c parsl.dataflow.memoization INFO: Task 4215 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4215 try 0 launched on executor multi with executor id 4216 parsl.dataflow.dflow INFO: Standard output for task 4215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/ccabb9bc-2720-4f8b-a1d2-05ebb8966e7a_isrForCrosstalkSources_997023_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/ccabb9bc-2720-4f8b-a1d2-05ebb8966e7a_isrForCrosstalkSources_997023_60.stderr parsl.dataflow.dflow DEBUG: Task 4216 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4216 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4216 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4216 has memoization hash 6c9476d09b918b1ac9edc411c79e9fe6 parsl.dataflow.memoization INFO: Task 4216 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4216 try 0 launched on executor multi with executor id 4217 parsl.dataflow.dflow INFO: Standard output for task 4216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e426e024-7f79-48e2-9e5b-3c06344323c0_isrForCrosstalkSources_997011_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e426e024-7f79-48e2-9e5b-3c06344323c0_isrForCrosstalkSources_997011_14.stderr parsl.dataflow.dflow DEBUG: Task 4217 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4217 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4217 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4217 has memoization hash ea6bb4a181e2bb4253fa4c120295540f parsl.dataflow.memoization INFO: Task 4217 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f19e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4217 try 0 launched on executor multi with executor id 4218 parsl.dataflow.dflow INFO: Standard output for task 4217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9d254a22-e5f6-46aa-8ecc-614e04b8b594_isrForCrosstalkSources_997004_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9d254a22-e5f6-46aa-8ecc-614e04b8b594_isrForCrosstalkSources_997004_38.stderr parsl.dataflow.dflow DEBUG: Task 4218 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4218 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4218 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4218 has memoization hash d4d56df95c520662773317725450bcea parsl.dataflow.memoization INFO: Task 4218 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4218 try 0 launched on executor multi with executor id 4219 parsl.dataflow.dflow INFO: Standard output for task 4218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/02bfc06a-d221-404c-af07-00641b74e8ab_isrForCrosstalkSources_996990_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/02bfc06a-d221-404c-af07-00641b74e8ab_isrForCrosstalkSources_996990_45.stderr parsl.dataflow.dflow DEBUG: Task 4219 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4219 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4219 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4219 has memoization hash 1dc4bb633a6e409eb4d4acef1a56558b parsl.dataflow.memoization INFO: Task 4219 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4219 try 0 launched on executor multi with executor id 4220 parsl.dataflow.dflow INFO: Standard output for task 4219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/b39560b2-c54a-4aa6-a5c9-a1a552e915b6_isrForCrosstalkSources_996993_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/b39560b2-c54a-4aa6-a5c9-a1a552e915b6_isrForCrosstalkSources_996993_37.stderr parsl.dataflow.dflow DEBUG: Task 4220 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4220 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4220 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4220 has memoization hash 00e92c2efa666c6fa8a9719d9af9bd04 parsl.dataflow.memoization INFO: Task 4220 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4220 try 0 launched on executor multi with executor id 4221 parsl.dataflow.dflow INFO: Standard output for task 4220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9bdfdb3b-68ed-49af-9c93-fec3829d9d76_isrForCrosstalkSources_996984_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9bdfdb3b-68ed-49af-9c93-fec3829d9d76_isrForCrosstalkSources_996984_18.stderr parsl.dataflow.dflow DEBUG: Task 4221 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4221 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4221 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4221 has memoization hash 950ead4c6ef69226c439abb848da24cd parsl.dataflow.memoization INFO: Task 4221 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4221 try 0 launched on executor multi with executor id 4222 parsl.dataflow.dflow INFO: Standard output for task 4221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/082178dc-332c-4ce3-b17b-738713d50400_isrForCrosstalkSources_996966_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/082178dc-332c-4ce3-b17b-738713d50400_isrForCrosstalkSources_996966_36.stderr parsl.dataflow.dflow DEBUG: Task 4222 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4222 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4222 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4222 has memoization hash 680783e25fd63149b9ceeccd68c85109 parsl.dataflow.memoization INFO: Task 4222 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4222 try 0 launched on executor multi with executor id 4223 parsl.dataflow.dflow INFO: Standard output for task 4222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/6c75c8bc-26b9-4ecc-a996-eb8da8faf4a4_isrForCrosstalkSources_997001_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/6c75c8bc-26b9-4ecc-a996-eb8da8faf4a4_isrForCrosstalkSources_997001_45.stderr parsl.dataflow.dflow DEBUG: Task 4223 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4223 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4223 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4223 has memoization hash 2c9d814d3b90f148568137c859a97165 parsl.dataflow.memoization INFO: Task 4223 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4223 try 0 launched on executor multi with executor id 4224 parsl.dataflow.dflow INFO: Standard output for task 4223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7bb5177a-7e81-40bc-8945-3bc3fe4b6539_isrForCrosstalkSources_997015_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7bb5177a-7e81-40bc-8945-3bc3fe4b6539_isrForCrosstalkSources_997015_38.stderr parsl.dataflow.dflow DEBUG: Task 4224 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4224 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4224 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4224 has memoization hash 163feeffdb5a2e1070cf4a94aa0a9973 parsl.dataflow.memoization INFO: Task 4224 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4224 try 0 launched on executor multi with executor id 4225 parsl.dataflow.dflow INFO: Standard output for task 4224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/201cd0a5-32e9-453b-8ca1-568613a74163_isrForCrosstalkSources_997004_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/201cd0a5-32e9-453b-8ca1-568613a74163_isrForCrosstalkSources_997004_60.stderr parsl.dataflow.dflow DEBUG: Task 4225 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4225 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4225 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4225 has memoization hash 31b44a132b84ef66fcf4c60f994da32f parsl.dataflow.memoization INFO: Task 4225 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4225 try 0 launched on executor multi with executor id 4226 parsl.dataflow.dflow INFO: Standard output for task 4225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0a8eff75-dde9-4ec9-8c97-1c171015f500_isrForCrosstalkSources_997023_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0a8eff75-dde9-4ec9-8c97-1c171015f500_isrForCrosstalkSources_997023_31.stderr parsl.dataflow.dflow DEBUG: Task 4226 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4226 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4226 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4226 has memoization hash 4dcf7c4ac5b7792b07756c92c96762a8 parsl.dataflow.memoization INFO: Task 4226 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f1f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4226 try 0 launched on executor multi with executor id 4227 parsl.dataflow.dflow INFO: Standard output for task 4226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/438c10d4-1b17-47b9-9114-2bf0de49ad79_isrForCrosstalkSources_996974_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/438c10d4-1b17-47b9-9114-2bf0de49ad79_isrForCrosstalkSources_996974_1.stderr parsl.dataflow.dflow DEBUG: Task 4227 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4227 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4227 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4227 has memoization hash ec11028ef8887e5514d4c54d2d00ec0f parsl.dataflow.memoization INFO: Task 4227 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4227 try 0 launched on executor multi with executor id 4228 parsl.dataflow.dflow INFO: Standard output for task 4227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/7e86c72b-e108-41ad-8328-7bfc4b095dd4_isrForCrosstalkSources_997018_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/7e86c72b-e108-41ad-8328-7bfc4b095dd4_isrForCrosstalkSources_997018_25.stderr parsl.dataflow.dflow DEBUG: Task 4228 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4228 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4228 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4228 has memoization hash fa0574ef2b1b79400fcbffd6978ed578 parsl.dataflow.memoization INFO: Task 4228 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f20c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4228 try 0 launched on executor multi with executor id 4229 parsl.dataflow.dflow INFO: Standard output for task 4228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ed79a39d-6b44-4198-88ed-5e3b0094de61_isrForCrosstalkSources_996998_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ed79a39d-6b44-4198-88ed-5e3b0094de61_isrForCrosstalkSources_996998_25.stderr parsl.dataflow.dflow DEBUG: Task 4229 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4229 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4229 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4229 has memoization hash 46165493ff0de1786ba670d4be0bbcce parsl.dataflow.memoization INFO: Task 4229 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4229 try 0 launched on executor multi with executor id 4230 parsl.dataflow.dflow INFO: Standard output for task 4229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/48d8e5e2-0b1d-4e17-b856-cb339e9dfaa2_isrForCrosstalkSources_996974_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/48d8e5e2-0b1d-4e17-b856-cb339e9dfaa2_isrForCrosstalkSources_996974_51.stderr parsl.dataflow.dflow DEBUG: Task 4230 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4230 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4230 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4230 has memoization hash bfc5a9f9bda956afc0da553b9aa80082 parsl.dataflow.memoization INFO: Task 4230 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4230 try 0 launched on executor multi with executor id 4231 parsl.dataflow.dflow INFO: Standard output for task 4230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/512673a7-4ae8-4bae-a3d7-863bc70e8deb_isrForCrosstalkSources_997019_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/512673a7-4ae8-4bae-a3d7-863bc70e8deb_isrForCrosstalkSources_997019_15.stderr parsl.dataflow.dflow DEBUG: Task 4231 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4231 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4231 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4231 has memoization hash a8dcbc0b8d5180fea10836ee75730292 parsl.dataflow.memoization INFO: Task 4231 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f22a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4231 try 0 launched on executor multi with executor id 4232 parsl.dataflow.dflow INFO: Standard output for task 4231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c9feeab0-3cd6-4b59-8779-83bdf26b9077_isrForCrosstalkSources_996969_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c9feeab0-3cd6-4b59-8779-83bdf26b9077_isrForCrosstalkSources_996969_57.stderr parsl.dataflow.dflow DEBUG: Task 4232 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4232 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4232 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4232 has memoization hash 7834fda28d75359b1bec7b40584c1830 parsl.dataflow.memoization INFO: Task 4232 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4232 try 0 launched on executor multi with executor id 4233 parsl.dataflow.dflow INFO: Standard output for task 4232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/8c73f837-f92c-4bbe-99ca-2c35be2a8d5f_isrForCrosstalkSources_996985_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/8c73f837-f92c-4bbe-99ca-2c35be2a8d5f_isrForCrosstalkSources_996985_25.stderr parsl.dataflow.dflow DEBUG: Task 4233 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4233 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4233 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4233 has memoization hash 177c7bb29a25d6821300f712edef49b7 parsl.dataflow.memoization INFO: Task 4233 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f23e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4233 try 0 launched on executor multi with executor id 4234 parsl.dataflow.dflow INFO: Standard output for task 4233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/1993d646-0316-4509-b624-bd2f3063c322_isrForCrosstalkSources_996970_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/1993d646-0316-4509-b624-bd2f3063c322_isrForCrosstalkSources_996970_22.stderr parsl.dataflow.dflow DEBUG: Task 4234 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4234 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4234 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4234 has memoization hash d9c2a80ad8cd3b49855746cbcc413b1e parsl.dataflow.memoization INFO: Task 4234 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4234 try 0 launched on executor multi with executor id 4235 parsl.dataflow.dflow INFO: Standard output for task 4234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/ea015206-a2b1-496f-95d6-a8e229501fc1_isrForCrosstalkSources_996966_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/ea015206-a2b1-496f-95d6-a8e229501fc1_isrForCrosstalkSources_996966_49.stderr parsl.dataflow.dflow DEBUG: Task 4235 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4235 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4235 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4235 has memoization hash 73fb0771be2690b891ff5cb0fbd47057 parsl.dataflow.memoization INFO: Task 4235 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4235 try 0 launched on executor multi with executor id 4236 parsl.dataflow.dflow INFO: Standard output for task 4235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c8315153-9cb1-4914-940d-8d306e92a2a3_isrForCrosstalkSources_996968_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c8315153-9cb1-4914-940d-8d306e92a2a3_isrForCrosstalkSources_996968_14.stderr parsl.dataflow.dflow DEBUG: Task 4236 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4236 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4236 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4236 has memoization hash 56a513eb6619a56b419f7fde85db2ed9 parsl.dataflow.memoization INFO: Task 4236 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f25c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4236 try 0 launched on executor multi with executor id 4237 parsl.dataflow.dflow INFO: Standard output for task 4236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/28217a73-f482-4f94-b5d6-7c160031eed3_isrForCrosstalkSources_997033_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/28217a73-f482-4f94-b5d6-7c160031eed3_isrForCrosstalkSources_997033_23.stderr parsl.dataflow.dflow DEBUG: Task 4237 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4237 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4237 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4237 has memoization hash 9fc05727e97c769d3f20f092cab8f93c parsl.dataflow.memoization INFO: Task 4237 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4237 try 0 launched on executor multi with executor id 4238 parsl.dataflow.dflow INFO: Standard output for task 4237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/81af871e-2e6c-488b-937d-a9e4a237a70c_isrForCrosstalkSources_996975_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/81af871e-2e6c-488b-937d-a9e4a237a70c_isrForCrosstalkSources_996975_20.stderr parsl.dataflow.dflow DEBUG: Task 4238 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4238 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4238 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4238 has memoization hash 972d445bb482a9222c60bb9fabb51993 parsl.dataflow.memoization INFO: Task 4238 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4238 try 0 launched on executor multi with executor id 4239 parsl.dataflow.dflow INFO: Standard output for task 4238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/26e7e675-8aad-4dd3-8018-6c282b657ce7_isrForCrosstalkSources_996984_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/26e7e675-8aad-4dd3-8018-6c282b657ce7_isrForCrosstalkSources_996984_46.stderr parsl.dataflow.dflow DEBUG: Task 4239 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4239 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4239 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4239 has memoization hash 98d8dffc25be4bd219b936851f6739ef parsl.dataflow.memoization INFO: Task 4239 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4239 try 0 launched on executor multi with executor id 4240 parsl.dataflow.dflow INFO: Standard output for task 4239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/403f446a-af80-46cb-bbce-d1af4713b520_isrForCrosstalkSources_996998_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/403f446a-af80-46cb-bbce-d1af4713b520_isrForCrosstalkSources_996998_13.stderr parsl.dataflow.dflow DEBUG: Task 4240 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4240 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4240 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4240 has memoization hash 50d86dabceecdb3ca08d5897bc3f6156 parsl.dataflow.memoization INFO: Task 4240 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f27a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4240 try 0 launched on executor multi with executor id 4241 parsl.dataflow.dflow INFO: Standard output for task 4240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3409e5ae-da2c-43b3-ac0f-078da9174de3_isrForCrosstalkSources_996955_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3409e5ae-da2c-43b3-ac0f-078da9174de3_isrForCrosstalkSources_996955_45.stderr parsl.dataflow.dflow DEBUG: Task 4241 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4241 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4241 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4241 has memoization hash e79bfe92b24d4944c8876b026a7b026f parsl.dataflow.memoization INFO: Task 4241 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f28e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4241 try 0 launched on executor multi with executor id 4242 parsl.dataflow.dflow INFO: Standard output for task 4241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7dd66e7e-c356-4cc3-9ace-db2a940c8555_isrForCrosstalkSources_996989_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7dd66e7e-c356-4cc3-9ace-db2a940c8555_isrForCrosstalkSources_996989_34.stderr parsl.dataflow.dflow DEBUG: Task 4242 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4242 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4242 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4242 has memoization hash 437e0e7e587503bbd94760724ff756f6 parsl.dataflow.memoization INFO: Task 4242 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4242 try 0 launched on executor multi with executor id 4243 parsl.dataflow.dflow INFO: Standard output for task 4242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/44012aad-b28f-4a26-91cb-628f10999a4c_isrForCrosstalkSources_996980_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/44012aad-b28f-4a26-91cb-628f10999a4c_isrForCrosstalkSources_996980_2.stderr parsl.dataflow.dflow DEBUG: Task 4243 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4243 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4243 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4243 has memoization hash 15b4c8e94669ae26dd13ecd9c2d6c1fe parsl.dataflow.memoization INFO: Task 4243 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4243 try 0 launched on executor multi with executor id 4244 parsl.dataflow.dflow INFO: Standard output for task 4243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/48bb62a2-6d2f-4c1f-a6e2-83ab31958a8a_isrForCrosstalkSources_997010_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/48bb62a2-6d2f-4c1f-a6e2-83ab31958a8a_isrForCrosstalkSources_997010_38.stderr parsl.dataflow.dflow DEBUG: Task 4244 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4244 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4244 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4244 has memoization hash fe8dd61343f3f6f52508fb8d5549c653 parsl.dataflow.memoization INFO: Task 4244 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:05 proc_lsst.multi:146 [INFO] found job 17074834 in provider astro proc_lsst.multi INFO: found job 17074834 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4244 try 0 launched on executor multi with executor id 4245 parsl.dataflow.dflow INFO: Standard output for task 4244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/c09fb3e9-f663-4c0c-b577-ffffb1dc8b11_isrForCrosstalkSources_996988_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/c09fb3e9-f663-4c0c-b577-ffffb1dc8b11_isrForCrosstalkSources_996988_21.stderr parsl.dataflow.dflow DEBUG: Task 4245 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4245 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4245 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4245 has memoization hash 53557ac54657271588f8f55af6648937 parsl.dataflow.memoization INFO: Task 4245 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4245 try 0 launched on executor multi with executor id 4246 parsl.dataflow.dflow INFO: Standard output for task 4245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1264c6fe-2eef-4ee0-a7c5-dde25ea5f4a7_isrForCrosstalkSources_996957_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1264c6fe-2eef-4ee0-a7c5-dde25ea5f4a7_isrForCrosstalkSources_996957_52.stderr parsl.dataflow.dflow DEBUG: Task 4246 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4246 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4246 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4246 has memoization hash 0d847fd6ecffaa9dd9238ef69e8aee83 parsl.dataflow.memoization INFO: Task 4246 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4246 try 0 launched on executor multi with executor id 4247 parsl.dataflow.dflow INFO: Standard output for task 4246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f307728e-2925-41fc-aed9-c02fc0e4db8f_isrForCrosstalkSources_996966_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f307728e-2925-41fc-aed9-c02fc0e4db8f_isrForCrosstalkSources_996966_35.stderr parsl.dataflow.dflow DEBUG: Task 4247 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4247 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4247 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4247 has memoization hash 9ed3363e609a62d0432750b1daf38b3b parsl.dataflow.memoization INFO: Task 4247 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4247 try 0 launched on executor multi with executor id 4248 parsl.dataflow.dflow INFO: Standard output for task 4247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/74d985e0-25d1-451a-9e9c-e097e69542e1_isrForCrosstalkSources_996990_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/74d985e0-25d1-451a-9e9c-e097e69542e1_isrForCrosstalkSources_996990_52.stderr parsl.dataflow.dflow DEBUG: Task 4248 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4248 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4248 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4248 has memoization hash bcc5473212ccdf75c764021b7545045b parsl.dataflow.memoization INFO: Task 4248 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c2559e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4248 try 0 launched on executor multi with executor id 4249 parsl.dataflow.dflow INFO: Standard output for task 4248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5a519c36-4ec3-4c28-ae2f-9642f9cc8a7c_isrForCrosstalkSources_996987_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5a519c36-4ec3-4c28-ae2f-9642f9cc8a7c_isrForCrosstalkSources_996987_29.stderr parsl.dataflow.dflow DEBUG: Task 4249 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4249 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4249 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4249 has memoization hash f13c4dc550aae900df88e494bfec5c71 parsl.dataflow.memoization INFO: Task 4249 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c255b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4249 try 0 launched on executor multi with executor id 4250 parsl.dataflow.dflow INFO: Standard output for task 4249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/349c138b-222d-4978-a36c-2a13e0642006_isrForCrosstalkSources_997012_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/349c138b-222d-4978-a36c-2a13e0642006_isrForCrosstalkSources_997012_11.stderr parsl.dataflow.dflow DEBUG: Task 4250 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4250 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4250 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4250 has memoization hash 558b0180805e634092f1b7896ce13e1e parsl.dataflow.memoization INFO: Task 4250 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4250 try 0 launched on executor multi with executor id 4251 parsl.dataflow.dflow INFO: Standard output for task 4250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/467c8c24-d776-4595-b3b3-66c3a5cbbf24_isrForCrosstalkSources_996969_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/467c8c24-d776-4595-b3b3-66c3a5cbbf24_isrForCrosstalkSources_996969_21.stderr parsl.dataflow.dflow DEBUG: Task 4251 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4251 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4251 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4251 has memoization hash 549a42e1fd9edb8d3bf105d06a4fb33a parsl.dataflow.memoization INFO: Task 4251 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4251 try 0 launched on executor multi with executor id 4252 parsl.dataflow.dflow INFO: Standard output for task 4251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/2ebb57ee-4271-441e-be56-2f84ef944ae7_isrForCrosstalkSources_997009_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/2ebb57ee-4271-441e-be56-2f84ef944ae7_isrForCrosstalkSources_997009_11.stderr parsl.dataflow.dflow DEBUG: Task 4252 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4252 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4252 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4252 has memoization hash 4682730096ab90eb5fd34da01e427ff9 parsl.dataflow.memoization INFO: Task 4252 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4252 try 0 launched on executor multi with executor id 4253 parsl.dataflow.dflow INFO: Standard output for task 4252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/8627908c-d843-474d-8e8e-8884d86d3176_isrForCrosstalkSources_997030_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/8627908c-d843-474d-8e8e-8884d86d3176_isrForCrosstalkSources_997030_21.stderr parsl.dataflow.dflow DEBUG: Task 4253 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4253 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4253 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4253 has memoization hash c1e5c4e957283ab93782ba756b26483f parsl.dataflow.memoization INFO: Task 4253 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4253 try 0 launched on executor multi with executor id 4254 parsl.dataflow.dflow INFO: Standard output for task 4253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1cf65b55-2d5d-45c0-9444-6d5c73193e08_isrForCrosstalkSources_997003_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1cf65b55-2d5d-45c0-9444-6d5c73193e08_isrForCrosstalkSources_997003_18.stderr parsl.dataflow.dflow DEBUG: Task 4254 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4254 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4254 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4254 has memoization hash fe54e1f94f684581b6ea5e4b2407a444 parsl.dataflow.memoization INFO: Task 4254 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4254 try 0 launched on executor multi with executor id 4255 parsl.dataflow.dflow INFO: Standard output for task 4254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/27788f04-3a6f-47e4-896f-f1f821422ced_isrForCrosstalkSources_996975_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/27788f04-3a6f-47e4-896f-f1f821422ced_isrForCrosstalkSources_996975_60.stderr parsl.dataflow.dflow DEBUG: Task 4255 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4255 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4255 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4255 has memoization hash 5963d2470df43c7f6bd24858a3d15116 parsl.dataflow.memoization INFO: Task 4255 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4255 try 0 launched on executor multi with executor id 4256 parsl.dataflow.dflow INFO: Standard output for task 4255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/33506b5c-d824-4b39-99d4-7c623a9f124f_isrForCrosstalkSources_997016_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/33506b5c-d824-4b39-99d4-7c623a9f124f_isrForCrosstalkSources_997016_31.stderr parsl.dataflow.dflow DEBUG: Task 4256 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4256 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4256 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4256 has memoization hash ff483c4c0dfe10927a63048dc52afa5f parsl.dataflow.memoization INFO: Task 4256 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4256 try 0 launched on executor multi with executor id 4257 parsl.dataflow.dflow INFO: Standard output for task 4256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/99688dd0-5794-452b-8e0c-4671d60842a6_isrForCrosstalkSources_997013_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/99688dd0-5794-452b-8e0c-4671d60842a6_isrForCrosstalkSources_997013_39.stderr parsl.dataflow.dflow DEBUG: Task 4257 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4257 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4257 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4257 has memoization hash 57db0fc08cab6bc0ffea26adffc131be parsl.dataflow.memoization INFO: Task 4257 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f32e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4257 try 0 launched on executor multi with executor id 4258 parsl.dataflow.dflow INFO: Standard output for task 4257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/2bfa982a-2895-482f-b4d8-2e126e2caaaf_isrForCrosstalkSources_996966_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/2bfa982a-2895-482f-b4d8-2e126e2caaaf_isrForCrosstalkSources_996966_53.stderr parsl.dataflow.dflow DEBUG: Task 4258 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4258 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4258 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4258 has memoization hash 0590655d8824f771212ec7db471fb494 parsl.dataflow.memoization INFO: Task 4258 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f31a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4258 try 0 launched on executor multi with executor id 4259 parsl.dataflow.dflow INFO: Standard output for task 4258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/5a1b9e49-1591-4ddd-8b57-0c19704a7d36_isrForCrosstalkSources_996986_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/5a1b9e49-1591-4ddd-8b57-0c19704a7d36_isrForCrosstalkSources_996986_32.stderr parsl.dataflow.dflow DEBUG: Task 4259 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4259 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4259 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4259 has memoization hash 23322d401768747160e625518e0ae925 parsl.dataflow.memoization INFO: Task 4259 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4259 try 0 launched on executor multi with executor id 4260 parsl.dataflow.dflow INFO: Standard output for task 4259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/0537a6a6-9722-4f61-9968-c5589413f131_isrForCrosstalkSources_997005_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/0537a6a6-9722-4f61-9968-c5589413f131_isrForCrosstalkSources_997005_49.stderr parsl.dataflow.dflow DEBUG: Task 4260 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4260 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4260 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4260 has memoization hash 000c138b9679c125df1ca781eef8587c parsl.dataflow.memoization INFO: Task 4260 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4260 try 0 launched on executor multi with executor id 4261 parsl.dataflow.dflow INFO: Standard output for task 4260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/37132177-4bc8-49c1-a1e1-7c8f7fa31e26_isrForCrosstalkSources_996991_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/37132177-4bc8-49c1-a1e1-7c8f7fa31e26_isrForCrosstalkSources_996991_53.stderr parsl.dataflow.dflow DEBUG: Task 4261 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4261 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4261 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4261 has memoization hash d31f094ffb2864b736856dc3d73fbf81 parsl.dataflow.memoization INFO: Task 4261 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f34c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4261 try 0 launched on executor multi with executor id 4262 parsl.dataflow.dflow INFO: Standard output for task 4261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/cff37941-d50d-49c4-bf0f-023177e91dea_isrForCrosstalkSources_997011_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/cff37941-d50d-49c4-bf0f-023177e91dea_isrForCrosstalkSources_997011_24.stderr parsl.dataflow.dflow DEBUG: Task 4262 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4262 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4262 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4262 has memoization hash 3011fff44afaab9bd49e627de67eec00 parsl.dataflow.memoization INFO: Task 4262 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4262 try 0 launched on executor multi with executor id 4263 parsl.dataflow.dflow INFO: Standard output for task 4262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a98710d7-d1d6-41b0-bccf-535dc0fe5e95_isrForCrosstalkSources_997026_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a98710d7-d1d6-41b0-bccf-535dc0fe5e95_isrForCrosstalkSources_997026_9.stderr parsl.dataflow.dflow DEBUG: Task 4263 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4263 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4263 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4263 has memoization hash 9911e1efd231b6b05bb3cc2f56ac8e15 parsl.dataflow.memoization INFO: Task 4263 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f36a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4263 try 0 launched on executor multi with executor id 4264 parsl.dataflow.dflow INFO: Standard output for task 4263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2bac6407-c81b-4161-94aa-5679e4756140_isrForCrosstalkSources_996977_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2bac6407-c81b-4161-94aa-5679e4756140_isrForCrosstalkSources_996977_29.stderr parsl.dataflow.dflow DEBUG: Task 4264 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4264 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4264 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4264 has memoization hash 39a1e6cdf8966313bf1e82971a3a5bfb parsl.dataflow.memoization INFO: Task 4264 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4264 try 0 launched on executor multi with executor id 4265 parsl.dataflow.dflow INFO: Standard output for task 4264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/405a5e43-3502-4d7e-8672-cbc063248059_isrForCrosstalkSources_997026_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/405a5e43-3502-4d7e-8672-cbc063248059_isrForCrosstalkSources_997026_33.stderr parsl.dataflow.dflow DEBUG: Task 4265 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4265 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4265 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4265 has memoization hash e0a7972a28f2a469bb9ab5ec86aaea78 parsl.dataflow.memoization INFO: Task 4265 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4265 try 0 launched on executor multi with executor id 4266 parsl.dataflow.dflow INFO: Standard output for task 4265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/eb38b678-1907-4a12-bd9a-0a5d1ecf1b90_isrForCrosstalkSources_997027_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/eb38b678-1907-4a12-bd9a-0a5d1ecf1b90_isrForCrosstalkSources_997027_43.stderr parsl.dataflow.dflow DEBUG: Task 4266 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4266 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4266 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4266 has memoization hash fda24defd9e9f55e4ba04a3321a323f5 parsl.dataflow.memoization INFO: Task 4266 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f37e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4266 try 0 launched on executor multi with executor id 4267 parsl.dataflow.dflow INFO: Standard output for task 4266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/41f95b41-bbe9-412e-bc8d-a4c0360ce2df_isrForCrosstalkSources_997033_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/41f95b41-bbe9-412e-bc8d-a4c0360ce2df_isrForCrosstalkSources_997033_53.stderr parsl.dataflow.dflow DEBUG: Task 4267 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4267 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4267 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4267 has memoization hash d5b089b7e67c5bd58945dcf81ace5fc0 parsl.dataflow.memoization INFO: Task 4267 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4267 try 0 launched on executor multi with executor id 4268 parsl.dataflow.dflow INFO: Standard output for task 4267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a261f3fe-33cb-43e3-979f-e09db6d7cd27_isrForCrosstalkSources_996964_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a261f3fe-33cb-43e3-979f-e09db6d7cd27_isrForCrosstalkSources_996964_12.stderr parsl.dataflow.dflow DEBUG: Task 4268 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4268 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4268 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4268 has memoization hash 40fe4287a28e7915b12ce2a47810f3a7 parsl.dataflow.memoization INFO: Task 4268 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f39c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4268 try 0 launched on executor multi with executor id 4269 parsl.dataflow.dflow INFO: Standard output for task 4268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a703a434-2797-4794-ac9c-7f1872af2f9e_isrForCrosstalkSources_997021_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a703a434-2797-4794-ac9c-7f1872af2f9e_isrForCrosstalkSources_997021_19.stderr parsl.dataflow.dflow DEBUG: Task 4269 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4269 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4269 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4269 has memoization hash ae7fa99b748092269d337d3411d8bc13 parsl.dataflow.memoization INFO: Task 4269 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4269 try 0 launched on executor multi with executor id 4270 parsl.dataflow.dflow INFO: Standard output for task 4269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/96e834d1-1e7d-4ddf-9970-236eed987d4f_isrForCrosstalkSources_996970_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/96e834d1-1e7d-4ddf-9970-236eed987d4f_isrForCrosstalkSources_996970_32.stderr parsl.dataflow.dflow DEBUG: Task 4270 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4270 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4270 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4270 has memoization hash 5a168c785cadff7515cdc4bab343d3f5 parsl.dataflow.memoization INFO: Task 4270 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4270 try 0 launched on executor multi with executor id 4271 parsl.dataflow.dflow INFO: Standard output for task 4270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/974c50ca-6c36-4764-9943-daac8a0f312c_isrForCrosstalkSources_997011_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/974c50ca-6c36-4764-9943-daac8a0f312c_isrForCrosstalkSources_997011_23.stderr parsl.dataflow.dflow DEBUG: Task 4271 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4271 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4271 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4271 has memoization hash 3b6c005683595b7aaa6d2b4805ed4f30 parsl.dataflow.memoization INFO: Task 4271 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4271 try 0 launched on executor multi with executor id 4272 parsl.dataflow.dflow INFO: Standard output for task 4271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/367761cd-8464-4cdc-b993-5da0987f0583_isrForCrosstalkSources_996967_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/367761cd-8464-4cdc-b993-5da0987f0583_isrForCrosstalkSources_996967_52.stderr parsl.dataflow.dflow DEBUG: Task 4272 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4272 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4272 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4272 has memoization hash 1f71834ee8526fac7e0b731ce25b01ad parsl.dataflow.memoization INFO: Task 4272 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4272 try 0 launched on executor multi with executor id 4273 parsl.dataflow.dflow INFO: Standard output for task 4272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1e4d37c6-f9fc-43e4-8f23-801482c14d4f_isrForCrosstalkSources_996967_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1e4d37c6-f9fc-43e4-8f23-801482c14d4f_isrForCrosstalkSources_996967_21.stderr parsl.dataflow.dflow DEBUG: Task 4273 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4273 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4273 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4273 has memoization hash 8f60457ad69b6c214fad25fe49a134bf parsl.dataflow.memoization INFO: Task 4273 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4273 try 0 launched on executor multi with executor id 4274 parsl.dataflow.dflow INFO: Standard output for task 4273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/799e61bf-8a1c-4405-9070-fb3a5ea3cc33_isrForCrosstalkSources_997007_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/799e61bf-8a1c-4405-9070-fb3a5ea3cc33_isrForCrosstalkSources_997007_20.stderr parsl.dataflow.dflow DEBUG: Task 4274 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4274 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4274 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4274 has memoization hash a140711d5f1010b3230858c39de2f1c4 parsl.dataflow.memoization INFO: Task 4274 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4274 try 0 launched on executor multi with executor id 4275 parsl.dataflow.dflow INFO: Standard output for task 4274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/52a7b3c4-ea41-405d-84e2-34bfc7a8697e_isrForCrosstalkSources_997019_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/52a7b3c4-ea41-405d-84e2-34bfc7a8697e_isrForCrosstalkSources_997019_21.stderr parsl.dataflow.dflow DEBUG: Task 4275 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4275 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4275 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4275 has memoization hash 01f453a99603e60af078b07af409e148 parsl.dataflow.memoization INFO: Task 4275 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4275 try 0 launched on executor multi with executor id 4276 parsl.dataflow.dflow INFO: Standard output for task 4275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/496493d3-6a96-4ad9-926e-1baf2f7f95ab_isrForCrosstalkSources_996960_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/496493d3-6a96-4ad9-926e-1baf2f7f95ab_isrForCrosstalkSources_996960_56.stderr parsl.dataflow.dflow DEBUG: Task 4276 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4276 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4276 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4276 has memoization hash 273d552bbb6832342886b081c53f2e2d parsl.dataflow.memoization INFO: Task 4276 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4276 try 0 launched on executor multi with executor id 4277 parsl.dataflow.dflow INFO: Standard output for task 4276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/52925fdb-99f0-4ea6-8ddf-821bbbac7741_isrForCrosstalkSources_996997_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/52925fdb-99f0-4ea6-8ddf-821bbbac7741_isrForCrosstalkSources_996997_30.stderr parsl.dataflow.dflow DEBUG: Task 4277 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4277 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4277 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4277 has memoization hash b197c2d0389a4372338c7d8b1f13adac parsl.dataflow.memoization INFO: Task 4277 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4277 try 0 launched on executor multi with executor id 4278 parsl.dataflow.dflow INFO: Standard output for task 4277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/bf85befa-bf41-4e7e-9475-3c0c66f1786a_isrForCrosstalkSources_996971_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/bf85befa-bf41-4e7e-9475-3c0c66f1786a_isrForCrosstalkSources_996971_29.stderr parsl.dataflow.dflow DEBUG: Task 4278 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4278 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4278 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4278 has memoization hash d4de27e1de8c50c253dad2f631837b3a parsl.dataflow.memoization INFO: Task 4278 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f3f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4278 try 0 launched on executor multi with executor id 4279 parsl.dataflow.dflow INFO: Standard output for task 4278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/c239d8bf-f7b1-466a-9cac-9613f68f3909_isrForCrosstalkSources_997008_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/c239d8bf-f7b1-466a-9cac-9613f68f3909_isrForCrosstalkSources_997008_11.stderr parsl.dataflow.dflow DEBUG: Task 4279 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4279 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4279 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4279 has memoization hash b6c1312adeca1548b1d714f1fa9d32c9 parsl.dataflow.memoization INFO: Task 4279 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f2de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4279 try 0 launched on executor multi with executor id 4280 parsl.dataflow.dflow INFO: Standard output for task 4279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/74a9e3d4-d0ad-449d-bd70-6efad244c794_isrForCrosstalkSources_996985_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/74a9e3d4-d0ad-449d-bd70-6efad244c794_isrForCrosstalkSources_996985_31.stderr parsl.dataflow.dflow DEBUG: Task 4280 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4280 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4280 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4280 has memoization hash 3a6c892875b4ca6819500059a20117dd parsl.dataflow.memoization INFO: Task 4280 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c0f0040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4280 try 0 launched on executor multi with executor id 4281 parsl.dataflow.dflow INFO: Standard output for task 4280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/122d889f-28b4-4b8d-95bc-818775159030_isrForCrosstalkSources_996955_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/122d889f-28b4-4b8d-95bc-818775159030_isrForCrosstalkSources_996955_20.stderr parsl.dataflow.dflow DEBUG: Task 4281 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4281 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4281 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4281 has memoization hash e941d61b3a314260c828cd600ae106ee parsl.dataflow.memoization INFO: Task 4281 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1840e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4281 try 0 launched on executor multi with executor id 4282 parsl.dataflow.dflow INFO: Standard output for task 4281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/939ea313-3f4a-4603-b8aa-1f69979abc66_isrForCrosstalkSources_997029_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/939ea313-3f4a-4603-b8aa-1f69979abc66_isrForCrosstalkSources_997029_17.stderr parsl.dataflow.dflow DEBUG: Task 4282 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4282 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4282 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4282 has memoization hash a5dc5a7f2c87ed72938fb0aa4f0b288d parsl.dataflow.memoization INFO: Task 4282 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4282 try 0 launched on executor multi with executor id 4283 parsl.dataflow.dflow INFO: Standard output for task 4282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/d06a860c-e816-4778-a4c3-0b7e43f030b7_isrForCrosstalkSources_996955_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/d06a860c-e816-4778-a4c3-0b7e43f030b7_isrForCrosstalkSources_996955_8.stderr parsl.dataflow.dflow DEBUG: Task 4283 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4283 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4283 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4283 has memoization hash 2409fd723b34cb842fbd79bd0897b3a7 parsl.dataflow.memoization INFO: Task 4283 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4283 try 0 launched on executor multi with executor id 4284 parsl.dataflow.dflow INFO: Standard output for task 4283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f0e5fe4f-5d6b-4efe-b903-ba83eeae7484_isrForCrosstalkSources_996975_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f0e5fe4f-5d6b-4efe-b903-ba83eeae7484_isrForCrosstalkSources_996975_36.stderr parsl.dataflow.dflow DEBUG: Task 4284 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4284 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4284 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4284 has memoization hash ce3bd09cd975f354adf6ecb8fa86004c parsl.dataflow.memoization INFO: Task 4284 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4284 try 0 launched on executor multi with executor id 4285 parsl.dataflow.dflow INFO: Standard output for task 4284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/486ac73b-8f24-4a9f-aaa8-4fa8888897aa_isrForCrosstalkSources_996995_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/486ac73b-8f24-4a9f-aaa8-4fa8888897aa_isrForCrosstalkSources_996995_42.stderr parsl.dataflow.dflow DEBUG: Task 4285 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4285 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4285 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4285 has memoization hash dec0628b4a6adbdea6ba64d76ef053f0 parsl.dataflow.memoization INFO: Task 4285 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1842c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4285 try 0 launched on executor multi with executor id 4286 parsl.dataflow.dflow INFO: Standard output for task 4285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/102a8930-afa3-4714-beea-5245e916340f_isrForCrosstalkSources_996995_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/102a8930-afa3-4714-beea-5245e916340f_isrForCrosstalkSources_996995_58.stderr parsl.dataflow.dflow DEBUG: Task 4286 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4286 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4286 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4286 has memoization hash c7fd37c5a2f59b779ccffa8ce96894e2 parsl.dataflow.memoization INFO: Task 4286 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 19 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db74ae9f0a876970c23c390c07b9ee54 with result from task 19 parsl.dataflow.dflow DEBUG: Task 19 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 19 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/889f548b-d883-45a7-80dc-af175f69d35c_isrForCrosstalkSources_996997_16.stdout parsl.dataflow.dflow INFO: Standard error for task 19 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/889f548b-d883-45a7-80dc-af175f69d35c_isrForCrosstalkSources_996997_16.stderr parsl.dataflow.dflow INFO: Task 23 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 903b8693c585c7f592f9b0b286961b85 with result from task 23 parsl.dataflow.dflow INFO: Parsl task 4286 try 0 launched on executor multi with executor id 4287 parsl.dataflow.dflow INFO: Standard output for task 4286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4fbb04a1-598b-4134-b005-09996bcee303_isrForCrosstalkSources_997014_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4fbb04a1-598b-4134-b005-09996bcee303_isrForCrosstalkSources_997014_31.stderr parsl.dataflow.dflow DEBUG: Task 4287 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4287 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4287 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4287 has memoization hash 3fe3918dbe4b973ecb3e59b2e3047c3b parsl.dataflow.memoization INFO: Task 4287 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1844a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4287 try 0 launched on executor multi with executor id 4288 parsl.dataflow.dflow INFO: Standard output for task 4287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ebd81182-cc9c-4ab6-a4a0-fb1265a116cd_isrForCrosstalkSources_997021_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ebd81182-cc9c-4ab6-a4a0-fb1265a116cd_isrForCrosstalkSources_997021_42.stderr parsl.dataflow.dflow DEBUG: Task 4288 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4288 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4288 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4288 has memoization hash db3925ac404ba1c8166074368fd96401 parsl.dataflow.memoization INFO: Task 4288 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4288 try 0 launched on executor multi with executor id 4289 parsl.dataflow.dflow INFO: Standard output for task 4288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/6bde540a-5896-40ef-bbe9-4ee70a2ce528_isrForCrosstalkSources_996992_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/6bde540a-5896-40ef-bbe9-4ee70a2ce528_isrForCrosstalkSources_996992_21.stderr parsl.dataflow.dflow DEBUG: Task 4289 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4289 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4289 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4289 has memoization hash 81ab25946243c566bfff2a3ea56f3931 parsl.dataflow.memoization INFO: Task 4289 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4289 try 0 launched on executor multi with executor id 4290 parsl.dataflow.dflow INFO: Standard output for task 4289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fc002d14-77bd-4ef7-a948-624c45cff4c7_isrForCrosstalkSources_997002_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fc002d14-77bd-4ef7-a948-624c45cff4c7_isrForCrosstalkSources_997002_45.stderr parsl.dataflow.dflow DEBUG: Task 4290 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4290 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4290 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4290 has memoization hash 90efc874228356e68179b57407cead8f parsl.dataflow.memoization INFO: Task 4290 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1845e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4290 try 0 launched on executor multi with executor id 4291 parsl.dataflow.dflow INFO: Standard output for task 4290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/7c682adf-b85d-4e87-a011-0015a248f869_isrForCrosstalkSources_996960_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/7c682adf-b85d-4e87-a011-0015a248f869_isrForCrosstalkSources_996960_44.stderr parsl.dataflow.dflow DEBUG: Task 4291 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4291 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4291 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4291 has memoization hash ea0acd866ec8ad02e8d00abd6a990d83 parsl.dataflow.memoization INFO: Task 4291 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 23 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 23 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/92f9b283-80d2-47d0-ad23-0680444f4226_isrForCrosstalkSources_996959_54.stdout parsl.dataflow.dflow INFO: Standard error for task 23 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/92f9b283-80d2-47d0-ad23-0680444f4226_isrForCrosstalkSources_996959_54.stderr parsl.dataflow.dflow INFO: Parsl task 4291 try 0 launched on executor multi with executor id 4292 parsl.dataflow.dflow INFO: Standard output for task 4291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6f497fc6-ebaf-4f6a-9f0c-ec7c4e47f161_isrForCrosstalkSources_997015_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6f497fc6-ebaf-4f6a-9f0c-ec7c4e47f161_isrForCrosstalkSources_997015_54.stderr parsl.dataflow.dflow DEBUG: Task 4292 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4292 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4292 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4292 has memoization hash d78a599382a1177787e592ce6f505dde parsl.dataflow.memoization INFO: Task 4292 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1847c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 13 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12f23acd5743b465efce4ba908d7a22c with result from task 13 parsl.dataflow.dflow INFO: Parsl task 4292 try 0 launched on executor multi with executor id 4293 parsl.dataflow.dflow INFO: Standard output for task 4292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7fdc5c7a-a8fb-47fb-9cfb-54d83afc2cdc_isrForCrosstalkSources_996988_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7fdc5c7a-a8fb-47fb-9cfb-54d83afc2cdc_isrForCrosstalkSources_996988_26.stderr parsl.dataflow.dflow DEBUG: Task 4293 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4293 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4293 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4293 has memoization hash 33b0c288e8dbf619e232b259837103e0 parsl.dataflow.memoization INFO: Task 4293 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4293 try 0 launched on executor multi with executor id 4294 parsl.dataflow.dflow INFO: Standard output for task 4293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/58daa97e-70c9-4910-980a-f9ddead62443_isrForCrosstalkSources_997012_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/58daa97e-70c9-4910-980a-f9ddead62443_isrForCrosstalkSources_997012_45.stderr parsl.dataflow.dflow DEBUG: Task 4294 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4294 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4294 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4294 has memoization hash 29344119ece6c036c7d2bcd095d0b415 parsl.dataflow.memoization INFO: Task 4294 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1849a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4294 try 0 launched on executor multi with executor id 4295 parsl.dataflow.dflow INFO: Standard output for task 4294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/90659ea9-a39e-489c-95cb-3cfcf92227da_isrForCrosstalkSources_997012_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/90659ea9-a39e-489c-95cb-3cfcf92227da_isrForCrosstalkSources_997012_60.stderr parsl.dataflow.dflow DEBUG: Task 4295 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4295 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4295 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4295 has memoization hash 2fbff6e844b844064171ee427b89464d parsl.dataflow.memoization INFO: Task 4295 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 13 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 13 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/32afa545-8f66-4faa-a15e-c67b94a00320_isrForCrosstalkSources_996975_1.stdout parsl.dataflow.dflow INFO: Standard error for task 13 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/32afa545-8f66-4faa-a15e-c67b94a00320_isrForCrosstalkSources_996975_1.stderr parsl.dataflow.dflow INFO: Parsl task 4295 try 0 launched on executor multi with executor id 4296 parsl.dataflow.dflow INFO: Standard output for task 4295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1099ac38-0317-41d4-a200-63d2d9845e3c_isrForCrosstalkSources_996958_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1099ac38-0317-41d4-a200-63d2d9845e3c_isrForCrosstalkSources_996958_51.stderr parsl.dataflow.dflow DEBUG: Task 4296 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4296 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4296 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4296 has memoization hash 899e5400af82135e4aeef7c99c3e42d2 parsl.dataflow.memoization INFO: Task 4296 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4296 try 0 launched on executor multi with executor id 4297 parsl.dataflow.dflow INFO: Standard output for task 4296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/31adb687-30d6-48f0-bfd8-d18a506ea554_isrForCrosstalkSources_997024_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/31adb687-30d6-48f0-bfd8-d18a506ea554_isrForCrosstalkSources_997024_26.stderr parsl.dataflow.dflow DEBUG: Task 4297 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4297 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4297 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4297 has memoization hash 321be3b1c9eadfd6aa254d637c4e4987 parsl.dataflow.memoization INFO: Task 4297 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4297 try 0 launched on executor multi with executor id 4298 parsl.dataflow.dflow INFO: Standard output for task 4297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/f896ce3e-79b4-45b3-a41e-25705e053f0c_isrForCrosstalkSources_996952_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/f896ce3e-79b4-45b3-a41e-25705e053f0c_isrForCrosstalkSources_996952_10.stderr parsl.dataflow.dflow DEBUG: Task 4298 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4298 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4298 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4298 has memoization hash 0452f80d6e39ab38f595970f1f43a3dc parsl.dataflow.memoization INFO: Task 4298 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4298 try 0 launched on executor multi with executor id 4299 parsl.dataflow.dflow INFO: Standard output for task 4298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/74f68542-35be-4ba6-930b-289c65d08f9c_isrForCrosstalkSources_996966_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/74f68542-35be-4ba6-930b-289c65d08f9c_isrForCrosstalkSources_996966_10.stderr parsl.dataflow.dflow DEBUG: Task 4299 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4299 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4299 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4299 has memoization hash d6d7ed399fc8b2c6180d204aad3f912a parsl.dataflow.memoization INFO: Task 4299 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4299 try 0 launched on executor multi with executor id 4300 parsl.dataflow.dflow INFO: Standard output for task 4299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1dc5f539-8158-4a90-8a7d-7675c7191867_isrForCrosstalkSources_997032_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1dc5f539-8158-4a90-8a7d-7675c7191867_isrForCrosstalkSources_997032_4.stderr parsl.dataflow.dflow DEBUG: Task 4300 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4300 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4300 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4300 has memoization hash 9930c98ff16b355acea28f44ebdd24da parsl.dataflow.memoization INFO: Task 4300 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4300 try 0 launched on executor multi with executor id 4301 parsl.dataflow.dflow INFO: Standard output for task 4300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/4541df1e-75a2-4d2f-8d69-143f20620366_isrForCrosstalkSources_996952_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/4541df1e-75a2-4d2f-8d69-143f20620366_isrForCrosstalkSources_996952_9.stderr parsl.dataflow.dflow DEBUG: Task 4301 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4301 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4301 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4301 has memoization hash 3787bf645a3da4f1ebcd4632bf25a866 parsl.dataflow.memoization INFO: Task 4301 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4301 try 0 launched on executor multi with executor id 4302 parsl.dataflow.dflow INFO: Standard output for task 4301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/839f1c1b-4ef3-4c06-845b-0962c7e38ca3_isrForCrosstalkSources_997027_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/839f1c1b-4ef3-4c06-845b-0962c7e38ca3_isrForCrosstalkSources_997027_6.stderr parsl.dataflow.dflow DEBUG: Task 4302 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4302 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4302 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4302 has memoization hash 28531514f8c3341e0b9c102abc926568 parsl.dataflow.memoization INFO: Task 4302 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4302 try 0 launched on executor multi with executor id 4303 parsl.dataflow.dflow INFO: Standard output for task 4302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/dd4ce919-4530-4d70-9173-d70ffe830143_isrForCrosstalkSources_996993_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/dd4ce919-4530-4d70-9173-d70ffe830143_isrForCrosstalkSources_996993_51.stderr parsl.dataflow.dflow DEBUG: Task 4303 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4303 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4303 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4303 has memoization hash bc7904b25a32215e5d52869019b9509a parsl.dataflow.memoization INFO: Task 4303 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4303 try 0 launched on executor multi with executor id 4304 parsl.dataflow.dflow INFO: Standard output for task 4303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/abf9385a-e125-46c2-bdaf-8bb2c01936bf_isrForCrosstalkSources_996972_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/abf9385a-e125-46c2-bdaf-8bb2c01936bf_isrForCrosstalkSources_996972_18.stderr parsl.dataflow.dflow DEBUG: Task 4304 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4304 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4304 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4304 has memoization hash 624f2aff452cd3ec47ad2a848f48f94c parsl.dataflow.memoization INFO: Task 4304 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4304 try 0 launched on executor multi with executor id 4305 parsl.dataflow.dflow INFO: Standard output for task 4304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/499d1a4f-1bd5-4874-a303-f3960c1da952_isrForCrosstalkSources_997012_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/499d1a4f-1bd5-4874-a303-f3960c1da952_isrForCrosstalkSources_997012_6.stderr parsl.dataflow.dflow DEBUG: Task 4305 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4305 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4305 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4305 has memoization hash db9d94e9ee3dec490b3a50be6158a41a parsl.dataflow.memoization INFO: Task 4305 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 17 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb424b1bb4808962c6e309202b9d6ee3 with result from task 17 parsl.dataflow.dflow INFO: Parsl task 4305 try 0 launched on executor multi with executor id 4306 parsl.dataflow.dflow INFO: Standard output for task 4305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/47e303e2-ba8b-4593-8319-384246584060_isrForCrosstalkSources_997003_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/47e303e2-ba8b-4593-8319-384246584060_isrForCrosstalkSources_997003_41.stderr parsl.dataflow.dflow DEBUG: Task 4306 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4306 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4306 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4306 has memoization hash cf033904b0e18221f7eb004fd2636e65 parsl.dataflow.memoization INFO: Task 4306 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4306 try 0 launched on executor multi with executor id 4307 parsl.dataflow.dflow INFO: Standard output for task 4306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/8725a853-6c5a-4a74-8e17-60daad879948_isrForCrosstalkSources_996999_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/8725a853-6c5a-4a74-8e17-60daad879948_isrForCrosstalkSources_996999_57.stderr parsl.dataflow.dflow DEBUG: Task 4307 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4307 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4307 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4307 has memoization hash 3a1066f877c224c4ad06dbc5e472a380 parsl.dataflow.memoization INFO: Task 4307 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 17 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 17 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/3b03cf9c-14a6-4098-87e5-9d2cbacb7bbc_isrForCrosstalkSources_996981_39.stdout parsl.dataflow.dflow INFO: Standard error for task 17 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/3b03cf9c-14a6-4098-87e5-9d2cbacb7bbc_isrForCrosstalkSources_996981_39.stderr parsl.dataflow.dflow INFO: Parsl task 4307 try 0 launched on executor multi with executor id 4308 parsl.dataflow.dflow INFO: Standard output for task 4307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/bdc02af8-5d9c-4911-ba8d-db410328a452_isrForCrosstalkSources_997022_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/bdc02af8-5d9c-4911-ba8d-db410328a452_isrForCrosstalkSources_997022_41.stderr parsl.dataflow.dflow DEBUG: Task 4308 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4308 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4308 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4308 has memoization hash 8aed4a7fb5b0b74eeb9e7a387215ab82 parsl.dataflow.memoization INFO: Task 4308 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4308 try 0 launched on executor multi with executor id 4309 parsl.dataflow.dflow INFO: Standard output for task 4308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/16aac535-a7e9-43db-976b-6ec2bb72ce37_isrForCrosstalkSources_997019_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/16aac535-a7e9-43db-976b-6ec2bb72ce37_isrForCrosstalkSources_997019_52.stderr parsl.dataflow.dflow DEBUG: Task 4309 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4309 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4309 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4309 has memoization hash 503f850a7ec4ad0dc2f3097ee5346609 parsl.dataflow.memoization INFO: Task 4309 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1851c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4309 try 0 launched on executor multi with executor id 4310 parsl.dataflow.dflow INFO: Standard output for task 4309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/bdb9ec38-9df2-42cc-b879-b8c8e1265685_isrForCrosstalkSources_996965_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/bdb9ec38-9df2-42cc-b879-b8c8e1265685_isrForCrosstalkSources_996965_62.stderr parsl.dataflow.dflow DEBUG: Task 4310 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4310 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4310 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4310 has memoization hash b2c6db41bc61dc8b7e943c6a904c46b5 parsl.dataflow.memoization INFO: Task 4310 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4310 try 0 launched on executor multi with executor id 4311 parsl.dataflow.dflow INFO: Standard output for task 4310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/8a9883a7-2c12-42ad-988a-dc17cda4578d_isrForCrosstalkSources_996989_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/8a9883a7-2c12-42ad-988a-dc17cda4578d_isrForCrosstalkSources_996989_7.stderr parsl.dataflow.dflow DEBUG: Task 4311 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4311 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4311 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4311 has memoization hash 9e1aab78def83b25fac751e88dadb504 parsl.dataflow.memoization INFO: Task 4311 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1853a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 14 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81f903f0c937ca949caabf359162dbca with result from task 14 parsl.dataflow.dflow INFO: Parsl task 4311 try 0 launched on executor multi with executor id 4312 parsl.dataflow.dflow INFO: Standard output for task 4311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/36b5bfd6-9406-45cc-9daa-455326fb3eac_isrForCrosstalkSources_996965_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/36b5bfd6-9406-45cc-9daa-455326fb3eac_isrForCrosstalkSources_996965_31.stderr parsl.dataflow.dflow DEBUG: Task 4312 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4312 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4312 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4312 has memoization hash 268d1639a388ff90af787dee8bc8eb20 parsl.dataflow.memoization INFO: Task 4312 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4312 try 0 launched on executor multi with executor id 4313 parsl.dataflow.dflow INFO: Standard output for task 4312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/129db217-7d1e-43ba-949a-707409ae4f4a_isrForCrosstalkSources_997019_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/129db217-7d1e-43ba-949a-707409ae4f4a_isrForCrosstalkSources_997019_46.stderr parsl.dataflow.dflow DEBUG: Task 4313 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4313 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4313 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4313 has memoization hash 6a1370edf3dfaabc8f42e7c1bb7d529d parsl.dataflow.memoization INFO: Task 4313 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c184040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4313 try 0 launched on executor multi with executor id 4314 parsl.dataflow.dflow INFO: Standard output for task 4313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f9735b49-f07e-4fa2-9684-df0239ddefe0_isrForCrosstalkSources_997018_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f9735b49-f07e-4fa2-9684-df0239ddefe0_isrForCrosstalkSources_997018_22.stderr parsl.dataflow.dflow DEBUG: Task 4314 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4314 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4314 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4314 has memoization hash 29c2940ef0c6195ccfb3637d895d6c5a parsl.dataflow.memoization INFO: Task 4314 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1854e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 14 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 14 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/5c4d2d03-e29c-4199-baea-e35f0f3ef3a2_isrForCrosstalkSources_996967_23.stdout parsl.dataflow.dflow INFO: Standard error for task 14 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/5c4d2d03-e29c-4199-baea-e35f0f3ef3a2_isrForCrosstalkSources_996967_23.stderr parsl.dataflow.dflow INFO: Parsl task 4314 try 0 launched on executor multi with executor id 4315 parsl.dataflow.dflow INFO: Standard output for task 4314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c3be24fc-abb2-4d0d-9af6-893fb27b84a6_isrForCrosstalkSources_997009_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c3be24fc-abb2-4d0d-9af6-893fb27b84a6_isrForCrosstalkSources_997009_61.stderr parsl.dataflow.dflow DEBUG: Task 4315 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4315 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4315 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4315 has memoization hash 0c343e6b970cfc9c5443a1ad23d2b3ae parsl.dataflow.memoization INFO: Task 4315 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 15 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2e523bf3981ebc95272d5ce140c88bb with result from task 15 parsl.dataflow.dflow INFO: Parsl task 4315 try 0 launched on executor multi with executor id 4316 parsl.dataflow.dflow INFO: Standard output for task 4315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/678518ed-aa2f-4c76-8b9b-a502205d8979_isrForCrosstalkSources_997025_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/678518ed-aa2f-4c76-8b9b-a502205d8979_isrForCrosstalkSources_997025_51.stderr parsl.dataflow.dflow DEBUG: Task 4316 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4316 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4316 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4316 has memoization hash e6128804be3295e35df5eca1752ea7a8 parsl.dataflow.memoization INFO: Task 4316 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1856c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4316 try 0 launched on executor multi with executor id 4317 parsl.dataflow.dflow INFO: Standard output for task 4316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3d04cd7b-f3a4-492b-8954-4f38142923de_isrForCrosstalkSources_996959_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3d04cd7b-f3a4-492b-8954-4f38142923de_isrForCrosstalkSources_996959_6.stderr parsl.dataflow.dflow DEBUG: Task 4317 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4317 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4317 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4317 has memoization hash cd1944ff0871283512674a38089c8716 parsl.dataflow.memoization INFO: Task 4317 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1858a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4317 try 0 launched on executor multi with executor id 4318 parsl.dataflow.dflow INFO: Standard output for task 4317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/4e93dc14-a76d-4405-ba35-5979e1eccd0c_isrForCrosstalkSources_997010_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/4e93dc14-a76d-4405-ba35-5979e1eccd0c_isrForCrosstalkSources_997010_10.stderr parsl.dataflow.dflow DEBUG: Task 4318 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4318 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4318 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4318 has memoization hash 63e5f6c04308ee940c577591806c3a18 parsl.dataflow.memoization INFO: Task 4318 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 15 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 15 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/447f0501-20b3-40a3-9d03-5d4d5e5bfb59_isrForCrosstalkSources_997006_9.stdout parsl.dataflow.dflow INFO: Standard error for task 15 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/447f0501-20b3-40a3-9d03-5d4d5e5bfb59_isrForCrosstalkSources_997006_9.stderr parsl.dataflow.dflow INFO: Parsl task 4318 try 0 launched on executor multi with executor id 4319 parsl.dataflow.dflow INFO: Standard output for task 4318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/80bd226e-13de-420a-b9da-f99a4c35dce1_isrForCrosstalkSources_996952_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/80bd226e-13de-420a-b9da-f99a4c35dce1_isrForCrosstalkSources_996952_17.stderr parsl.dataflow.dflow DEBUG: Task 4319 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4319 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4319 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4319 has memoization hash 9530657eb9b576ca21fddc598db9e81e parsl.dataflow.memoization INFO: Task 4319 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1859e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4319 try 0 launched on executor multi with executor id 4320 parsl.dataflow.dflow INFO: Standard output for task 4319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d4433bb5-b5ff-4e93-8c1f-39a3f569788f_isrForCrosstalkSources_997000_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d4433bb5-b5ff-4e93-8c1f-39a3f569788f_isrForCrosstalkSources_997000_14.stderr parsl.dataflow.dflow DEBUG: Task 4320 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4320 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4320 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4320 has memoization hash 02d3f17f71cd0b3d7896c163dd159cbc parsl.dataflow.memoization INFO: Task 4320 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 16 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f14e8840306656adc97ff5dfbe81f780 with result from task 16 parsl.dataflow.dflow INFO: Parsl task 4320 try 0 launched on executor multi with executor id 4321 parsl.dataflow.dflow INFO: Standard output for task 4320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/53120f4b-b538-452d-8191-997a1cdc43e9_isrForCrosstalkSources_996981_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/53120f4b-b538-452d-8191-997a1cdc43e9_isrForCrosstalkSources_996981_5.stderr parsl.dataflow.dflow DEBUG: Task 4321 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4321 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4321 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4321 has memoization hash 404979965dc54a1d053f6d008646431f parsl.dataflow.memoization INFO: Task 4321 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 16 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 16 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9f4c1d8b-91e9-40e0-a979-d4a1359a795f_isrForCrosstalkSources_996995_41.stdout parsl.dataflow.dflow INFO: Standard error for task 16 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9f4c1d8b-91e9-40e0-a979-d4a1359a795f_isrForCrosstalkSources_996995_41.stderr parsl.dataflow.dflow INFO: Parsl task 4321 try 0 launched on executor multi with executor id 4322 parsl.dataflow.dflow INFO: Standard output for task 4321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/fec6715d-a714-4e1c-8782-3f7ee4d6fc4a_isrForCrosstalkSources_997021_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/fec6715d-a714-4e1c-8782-3f7ee4d6fc4a_isrForCrosstalkSources_997021_46.stderr parsl.dataflow.dflow DEBUG: Task 4322 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4322 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4322 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4322 has memoization hash 21e6bf4bc846faf9ccda327968b2ebdb parsl.dataflow.memoization INFO: Task 4322 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4322 try 0 launched on executor multi with executor id 4323 parsl.dataflow.dflow INFO: Standard output for task 4322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/586ef330-5159-4a5e-8b0a-37e245331a68_isrForCrosstalkSources_996966_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/586ef330-5159-4a5e-8b0a-37e245331a68_isrForCrosstalkSources_996966_51.stderr parsl.dataflow.dflow DEBUG: Task 4323 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4323 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4323 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4323 has memoization hash 4f324c2d7c47540691d0c27ca5f93715 parsl.dataflow.memoization INFO: Task 4323 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4323 try 0 launched on executor multi with executor id 4324 parsl.dataflow.dflow INFO: Standard output for task 4323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/aa8319c4-d2c2-4aed-afbe-b6a474b195db_isrForCrosstalkSources_996983_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/aa8319c4-d2c2-4aed-afbe-b6a474b195db_isrForCrosstalkSources_996983_33.stderr parsl.dataflow.dflow DEBUG: Task 4324 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4324 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4324 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4324 has memoization hash f822a7afa193d7f32059cdb5ac645e3d parsl.dataflow.memoization INFO: Task 4324 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4324 try 0 launched on executor multi with executor id 4325 parsl.dataflow.dflow INFO: Standard output for task 4324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/586dd823-efaa-4f84-8147-257eb15c713c_isrForCrosstalkSources_996962_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/586dd823-efaa-4f84-8147-257eb15c713c_isrForCrosstalkSources_996962_41.stderr parsl.dataflow.dflow DEBUG: Task 4325 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4325 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4325 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4325 has memoization hash 93b9497f2cd3d2696635fb13a17fc7f2 parsl.dataflow.memoization INFO: Task 4325 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4325 try 0 launched on executor multi with executor id 4326 parsl.dataflow.dflow INFO: Standard output for task 4325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1f5a6f1e-879c-44f2-ae22-3b9a562f45e0_isrForCrosstalkSources_996977_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1f5a6f1e-879c-44f2-ae22-3b9a562f45e0_isrForCrosstalkSources_996977_9.stderr parsl.dataflow.dflow DEBUG: Task 4326 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4326 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4326 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4326 has memoization hash ea8566fbce0408e78fadb0e2e5baf4e0 parsl.dataflow.memoization INFO: Task 4326 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4326 try 0 launched on executor multi with executor id 4327 parsl.dataflow.dflow INFO: Standard output for task 4326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/41b752bd-81ba-48b1-902f-83acda1b27e2_isrForCrosstalkSources_997031_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/41b752bd-81ba-48b1-902f-83acda1b27e2_isrForCrosstalkSources_997031_16.stderr parsl.dataflow.dflow DEBUG: Task 4327 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4327 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4327 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4327 has memoization hash 26dcbbb7964f9f2e961edfc36795fcf4 parsl.dataflow.memoization INFO: Task 4327 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:06 proc_lsst.multi:146 [INFO] found job 17074835 in provider astro proc_lsst.multi INFO: found job 17074835 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4327 try 0 launched on executor multi with executor id 4328 parsl.dataflow.dflow INFO: Standard output for task 4327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e8f03c71-8714-41c8-9079-302b53dedf0f_isrForCrosstalkSources_996964_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e8f03c71-8714-41c8-9079-302b53dedf0f_isrForCrosstalkSources_996964_22.stderr parsl.dataflow.dflow DEBUG: Task 4328 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4328 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4328 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4328 has memoization hash 8250afb0826531393188095e4ff50623 parsl.dataflow.memoization INFO: Task 4328 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4328 try 0 launched on executor multi with executor id 4329 parsl.dataflow.dflow INFO: Standard output for task 4328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/fffccc60-2aa5-4d76-8e34-f23e78ef819c_isrForCrosstalkSources_996986_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/fffccc60-2aa5-4d76-8e34-f23e78ef819c_isrForCrosstalkSources_996986_48.stderr parsl.dataflow.dflow DEBUG: Task 4329 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4329 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4329 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4329 has memoization hash 753debb6963060f04b11da3edfdf31bc parsl.dataflow.memoization INFO: Task 4329 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4329 try 0 launched on executor multi with executor id 4330 parsl.dataflow.dflow INFO: Standard output for task 4329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/e425102b-c957-449e-b083-adf24bd4c86d_isrForCrosstalkSources_996975_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/e425102b-c957-449e-b083-adf24bd4c86d_isrForCrosstalkSources_996975_32.stderr parsl.dataflow.dflow DEBUG: Task 4330 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4330 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4330 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4330 has memoization hash f3700385e277b545859ef03c8b2639a2 parsl.dataflow.memoization INFO: Task 4330 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4330 try 0 launched on executor multi with executor id 4331 parsl.dataflow.dflow INFO: Standard output for task 4330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/12af1763-235e-41e6-96dc-b0ff15195e76_isrForCrosstalkSources_997032_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/12af1763-235e-41e6-96dc-b0ff15195e76_isrForCrosstalkSources_997032_48.stderr parsl.dataflow.dflow DEBUG: Task 4331 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4331 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4331 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4331 has memoization hash fdf7cd606567462a8208493d3b9b770f parsl.dataflow.memoization INFO: Task 4331 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 22 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de687cddcffba15a565a147fe345b35e with result from task 22 parsl.dataflow.dflow DEBUG: Task 22 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 22 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/990e58e9-4394-4a84-b6ce-b7960cc72c83_isrForCrosstalkSources_997023_49.stdout parsl.dataflow.dflow INFO: Standard error for task 22 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/990e58e9-4394-4a84-b6ce-b7960cc72c83_isrForCrosstalkSources_997023_49.stderr parsl.dataflow.dflow INFO: Parsl task 4331 try 0 launched on executor multi with executor id 4332 parsl.dataflow.dflow INFO: Standard output for task 4331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/eb923953-063b-47ef-b8ac-5cd9fc482df7_isrForCrosstalkSources_996967_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/eb923953-063b-47ef-b8ac-5cd9fc482df7_isrForCrosstalkSources_996967_35.stderr parsl.dataflow.dflow DEBUG: Task 4332 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4332 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4332 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4332 has memoization hash cf60e9d12ce7a5d5fb0f3ef74eee12a5 parsl.dataflow.memoization INFO: Task 4332 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4332 try 0 launched on executor multi with executor id 4333 parsl.dataflow.dflow INFO: Standard output for task 4332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/fac44a26-6066-412e-b046-7b6db17e63fb_isrForCrosstalkSources_997031_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/fac44a26-6066-412e-b046-7b6db17e63fb_isrForCrosstalkSources_997031_58.stderr parsl.dataflow.dflow DEBUG: Task 4333 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4333 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4333 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4333 has memoization hash 1a282609833c679cf6a12f832bc2f68a parsl.dataflow.memoization INFO: Task 4333 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4333 try 0 launched on executor multi with executor id 4334 parsl.dataflow.dflow INFO: Standard output for task 4333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/f120c409-aed0-4c53-9d2e-73ccd1370adb_isrForCrosstalkSources_997000_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/f120c409-aed0-4c53-9d2e-73ccd1370adb_isrForCrosstalkSources_997000_55.stderr parsl.dataflow.dflow DEBUG: Task 4334 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4334 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4334 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4334 has memoization hash 72b7a77d0591350c75ac51c3e4af4641 parsl.dataflow.memoization INFO: Task 4334 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4334 try 0 launched on executor multi with executor id 4335 parsl.dataflow.dflow INFO: Standard output for task 4334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/65f9cb3e-b55e-45ee-9533-44cf52c9be2d_isrForCrosstalkSources_996993_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/65f9cb3e-b55e-45ee-9533-44cf52c9be2d_isrForCrosstalkSources_996993_45.stderr parsl.dataflow.dflow DEBUG: Task 4335 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4335 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4335 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4335 has memoization hash 10bad021730db682e65a1f9c3b34e04a parsl.dataflow.memoization INFO: Task 4335 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1860c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4335 try 0 launched on executor multi with executor id 4336 parsl.dataflow.dflow INFO: Standard output for task 4335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/f499209e-42d1-4c83-8634-a20640918968_isrForCrosstalkSources_997023_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/f499209e-42d1-4c83-8634-a20640918968_isrForCrosstalkSources_997023_9.stderr parsl.dataflow.dflow DEBUG: Task 4336 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4336 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4336 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4336 has memoization hash c8a9753418774c8ba5d8dfcf700f3983 parsl.dataflow.memoization INFO: Task 4336 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4336 try 0 launched on executor multi with executor id 4337 parsl.dataflow.dflow INFO: Standard output for task 4336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/c6f20bf7-683e-429f-b694-63e0f1a3d89c_isrForCrosstalkSources_997027_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/c6f20bf7-683e-429f-b694-63e0f1a3d89c_isrForCrosstalkSources_997027_58.stderr parsl.dataflow.dflow DEBUG: Task 4337 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4337 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4337 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4337 has memoization hash ce8d49c2c0441b5726e5e6e61c31a5f5 parsl.dataflow.memoization INFO: Task 4337 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1862a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4337 try 0 launched on executor multi with executor id 4338 parsl.dataflow.dflow INFO: Standard output for task 4337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d55519a6-0f53-463e-930e-83ab65d3bba5_isrForCrosstalkSources_997026_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d55519a6-0f53-463e-930e-83ab65d3bba5_isrForCrosstalkSources_997026_32.stderr parsl.dataflow.dflow DEBUG: Task 4338 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4338 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4338 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4338 has memoization hash 8f931a4e1c7a6b1bc3c32a090fa30c60 parsl.dataflow.memoization INFO: Task 4338 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4338 try 0 launched on executor multi with executor id 4339 parsl.dataflow.dflow INFO: Standard output for task 4338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/584add02-456f-4486-9efa-2c99804b4987_isrForCrosstalkSources_997010_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/584add02-456f-4486-9efa-2c99804b4987_isrForCrosstalkSources_997010_8.stderr parsl.dataflow.dflow DEBUG: Task 4339 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4339 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4339 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4339 has memoization hash bce6726d809752b241e6408418015a70 parsl.dataflow.memoization INFO: Task 4339 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4339 try 0 launched on executor multi with executor id 4340 parsl.dataflow.dflow INFO: Standard output for task 4339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/4562e5f6-4601-405a-b3d4-7b4367116aee_isrForCrosstalkSources_997003_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/4562e5f6-4601-405a-b3d4-7b4367116aee_isrForCrosstalkSources_997003_9.stderr parsl.dataflow.dflow DEBUG: Task 4340 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4340 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4340 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4340 has memoization hash b7f945b9fcee0b62dfde5f6a929eb5bb parsl.dataflow.memoization INFO: Task 4340 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1863e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4340 try 0 launched on executor multi with executor id 4341 parsl.dataflow.dflow INFO: Standard output for task 4340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/55d3e3bc-6c63-460b-b64f-d181b0723d62_isrForCrosstalkSources_997023_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/55d3e3bc-6c63-460b-b64f-d181b0723d62_isrForCrosstalkSources_997023_21.stderr parsl.dataflow.dflow DEBUG: Task 4341 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4341 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4341 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4341 has memoization hash 025af1e120f4a6c462b770347b031698 parsl.dataflow.memoization INFO: Task 4341 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1865c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4341 try 0 launched on executor multi with executor id 4342 parsl.dataflow.dflow INFO: Standard output for task 4341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/060cb5b1-9717-4eb8-8872-45412d697828_isrForCrosstalkSources_996960_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/060cb5b1-9717-4eb8-8872-45412d697828_isrForCrosstalkSources_996960_39.stderr parsl.dataflow.dflow DEBUG: Task 4342 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4342 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4342 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4342 has memoization hash 747abb568d5faf8c729eb2c2624d394e parsl.dataflow.memoization INFO: Task 4342 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Task 38 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad81832faa76c990fe606945513d8fed with result from task 38 parsl.dataflow.dflow INFO: Parsl task 4342 try 0 launched on executor multi with executor id 4343 parsl.dataflow.dflow INFO: Standard output for task 4342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bcc2f85e-79f4-4038-9f5d-133121a56bf9_isrForCrosstalkSources_997027_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bcc2f85e-79f4-4038-9f5d-133121a56bf9_isrForCrosstalkSources_997027_25.stderr parsl.dataflow.dflow DEBUG: Task 4343 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4343 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4343 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4343 has memoization hash b43969451aad1568a6129383aeeee79a parsl.dataflow.memoization INFO: Task 4343 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4343 try 0 launched on executor multi with executor id 4344 parsl.dataflow.dflow INFO: Standard output for task 4343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/2d801789-67c2-48c1-a73f-3ea496aa27d5_isrForCrosstalkSources_997028_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/2d801789-67c2-48c1-a73f-3ea496aa27d5_isrForCrosstalkSources_997028_50.stderr parsl.dataflow.dflow DEBUG: Task 4344 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4344 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4344 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4344 has memoization hash da2c66875458ea4245e96f703ebd7a6f parsl.dataflow.memoization INFO: Task 4344 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1868e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4344 try 0 launched on executor multi with executor id 4345 parsl.dataflow.dflow INFO: Standard output for task 4344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/a0a470ad-08f1-4011-bdc9-abcfcc066d3e_isrForCrosstalkSources_996966_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/a0a470ad-08f1-4011-bdc9-abcfcc066d3e_isrForCrosstalkSources_996966_26.stderr parsl.dataflow.dflow DEBUG: Task 4345 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4345 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4345 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4345 has memoization hash 16299ec51a64605dc185aa84ec45bcc0 parsl.dataflow.memoization INFO: Task 4345 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c185f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow DEBUG: Task 38 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 38 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/df3895d8-03a7-45c7-8a90-67639359d2d4_isrForCrosstalkSources_997029_55.stdout parsl.dataflow.dflow INFO: Standard error for task 38 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/df3895d8-03a7-45c7-8a90-67639359d2d4_isrForCrosstalkSources_997029_55.stderr parsl.dataflow.dflow INFO: Parsl task 4345 try 0 launched on executor multi with executor id 4346 parsl.dataflow.dflow INFO: Standard output for task 4345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/460c4e6a-ee33-4a34-9072-80fab7a21c4e_isrForCrosstalkSources_996991_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/460c4e6a-ee33-4a34-9072-80fab7a21c4e_isrForCrosstalkSources_996991_5.stderr parsl.dataflow.dflow DEBUG: Task 4346 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4346 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4346 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4346 has memoization hash 2177f40e45a17a6205c953dd292cadc4 parsl.dataflow.memoization INFO: Task 4346 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4346 try 0 launched on executor multi with executor id 4347 parsl.dataflow.dflow INFO: Standard output for task 4346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/65b19e1e-9476-4439-91f0-6b91f037cc21_isrForCrosstalkSources_996993_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/65b19e1e-9476-4439-91f0-6b91f037cc21_isrForCrosstalkSources_996993_39.stderr parsl.dataflow.dflow DEBUG: Task 4347 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4347 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4347 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4347 has memoization hash 6d922be2025983479d29009218ccf443 parsl.dataflow.memoization INFO: Task 4347 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4347 try 0 launched on executor multi with executor id 4348 parsl.dataflow.dflow INFO: Standard output for task 4347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/a28a0386-87a3-4c1b-8eba-2f5b69fa0d0a_isrForCrosstalkSources_996998_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/a28a0386-87a3-4c1b-8eba-2f5b69fa0d0a_isrForCrosstalkSources_996998_1.stderr parsl.dataflow.dflow DEBUG: Task 4348 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4348 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4348 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4348 has memoization hash 085c044df93105ac27e2c6fb2379b423 parsl.dataflow.memoization INFO: Task 4348 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4348 try 0 launched on executor multi with executor id 4349 parsl.dataflow.dflow INFO: Standard output for task 4348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/50c14ec3-4256-4506-9963-275eeef245cc_isrForCrosstalkSources_997014_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/50c14ec3-4256-4506-9963-275eeef245cc_isrForCrosstalkSources_997014_3.stderr parsl.dataflow.dflow DEBUG: Task 4349 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4349 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4349 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4349 has memoization hash d8e1d77d03bd2e84d2ed7db08811b34b parsl.dataflow.memoization INFO: Task 4349 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4349 try 0 launched on executor multi with executor id 4350 parsl.dataflow.dflow INFO: Standard output for task 4349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d1bf0ece-63f1-4ef7-9145-7ac5d3467628_isrForCrosstalkSources_997001_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d1bf0ece-63f1-4ef7-9145-7ac5d3467628_isrForCrosstalkSources_997001_48.stderr parsl.dataflow.dflow DEBUG: Task 4350 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4350 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4350 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4350 has memoization hash d672c97a70dd86bd77611adae3f8cf44 parsl.dataflow.memoization INFO: Task 4350 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4350 try 0 launched on executor multi with executor id 4351 parsl.dataflow.dflow INFO: Standard output for task 4350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/abdfb6af-abb0-41f3-ac8b-7349e0b75e54_isrForCrosstalkSources_997028_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/abdfb6af-abb0-41f3-ac8b-7349e0b75e54_isrForCrosstalkSources_997028_54.stderr parsl.dataflow.dflow DEBUG: Task 4351 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4351 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4351 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4351 has memoization hash edb8e8b71c40f2efce66b99ab8d63fad parsl.dataflow.memoization INFO: Task 4351 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4351 try 0 launched on executor multi with executor id 4352 parsl.dataflow.dflow INFO: Standard output for task 4351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/97cdb5c9-de15-438d-b5d0-c6245e51d19b_isrForCrosstalkSources_997012_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/97cdb5c9-de15-438d-b5d0-c6245e51d19b_isrForCrosstalkSources_997012_9.stderr parsl.dataflow.dflow DEBUG: Task 4352 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4352 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4352 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4352 has memoization hash be7a4a519abf5e448457bae0a4b9b250 parsl.dataflow.memoization INFO: Task 4352 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4352 try 0 launched on executor multi with executor id 4353 parsl.dataflow.dflow INFO: Standard output for task 4352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c203052d-4f78-411e-8545-bf286e89e515_isrForCrosstalkSources_996965_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c203052d-4f78-411e-8545-bf286e89e515_isrForCrosstalkSources_996965_23.stderr parsl.dataflow.dflow DEBUG: Task 4353 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4353 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4353 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4353 has memoization hash 7f1637a32225b7b904cbe52fc7a1b81e parsl.dataflow.memoization INFO: Task 4353 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4353 try 0 launched on executor multi with executor id 4354 parsl.dataflow.dflow INFO: Standard output for task 4353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/29342b58-221d-4960-916f-64c9ada8232e_isrForCrosstalkSources_997016_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/29342b58-221d-4960-916f-64c9ada8232e_isrForCrosstalkSources_997016_19.stderr parsl.dataflow.dflow DEBUG: Task 4354 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4354 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4354 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4354 has memoization hash 597a86690d7e488889632a944c7f85f1 parsl.dataflow.memoization INFO: Task 4354 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4354 try 0 launched on executor multi with executor id 4355 parsl.dataflow.dflow INFO: Standard output for task 4354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/27b44e8c-5bc0-4ffd-bd26-2bb62031ee46_isrForCrosstalkSources_996980_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/27b44e8c-5bc0-4ffd-bd26-2bb62031ee46_isrForCrosstalkSources_996980_6.stderr parsl.dataflow.dflow DEBUG: Task 4355 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4355 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4355 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4355 has memoization hash 43b56288bb12e6c62a4c5941b504c6fe parsl.dataflow.memoization INFO: Task 4355 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4355 try 0 launched on executor multi with executor id 4356 parsl.dataflow.dflow INFO: Standard output for task 4355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e253e809-139d-4c68-a894-908902788ac7_isrForCrosstalkSources_996978_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e253e809-139d-4c68-a894-908902788ac7_isrForCrosstalkSources_996978_38.stderr parsl.dataflow.dflow DEBUG: Task 4356 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4356 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4356 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4356 has memoization hash 7d0834fca80be07a81fb434ad992b6c7 parsl.dataflow.memoization INFO: Task 4356 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4356 try 0 launched on executor multi with executor id 4357 parsl.dataflow.dflow INFO: Standard output for task 4356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/df17f5d8-8617-4776-a41f-5482a30bb261_isrForCrosstalkSources_996978_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/df17f5d8-8617-4776-a41f-5482a30bb261_isrForCrosstalkSources_996978_49.stderr parsl.dataflow.dflow DEBUG: Task 4357 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4357 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4357 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4357 has memoization hash 2987a9aeed41d31923da644b747174f0 parsl.dataflow.memoization INFO: Task 4357 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4357 try 0 launched on executor multi with executor id 4358 parsl.dataflow.dflow INFO: Standard output for task 4357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/978d8e9e-6c27-4a55-9177-5afc92de1b7c_isrForCrosstalkSources_996967_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/978d8e9e-6c27-4a55-9177-5afc92de1b7c_isrForCrosstalkSources_996967_51.stderr parsl.dataflow.dflow DEBUG: Task 4358 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4358 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4358 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4358 has memoization hash 3694d74bfa62bcfdb97a302a8edc90ab parsl.dataflow.memoization INFO: Task 4358 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4358 try 0 launched on executor multi with executor id 4359 parsl.dataflow.dflow INFO: Standard output for task 4358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/02e962ad-47c1-498b-8eb9-c51859c1a1f1_isrForCrosstalkSources_997025_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/02e962ad-47c1-498b-8eb9-c51859c1a1f1_isrForCrosstalkSources_997025_21.stderr parsl.dataflow.dflow DEBUG: Task 4359 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4359 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4359 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4359 has memoization hash 44f24b8bc03f1b4fbc02e8a3cdb4f78e parsl.dataflow.memoization INFO: Task 4359 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1871a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4359 try 0 launched on executor multi with executor id 4360 parsl.dataflow.dflow INFO: Standard output for task 4359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/59d0c293-3aae-4b87-a740-d9769c5711fd_isrForCrosstalkSources_996970_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/59d0c293-3aae-4b87-a740-d9769c5711fd_isrForCrosstalkSources_996970_18.stderr parsl.dataflow.dflow DEBUG: Task 4360 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4360 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4360 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4360 has memoization hash 2d6d308be608325171355b32b83bbbee parsl.dataflow.memoization INFO: Task 4360 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c186fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4360 try 0 launched on executor multi with executor id 4361 parsl.dataflow.dflow INFO: Standard output for task 4360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fb9777d8-838e-4773-98bb-762efb35b279_isrForCrosstalkSources_996976_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fb9777d8-838e-4773-98bb-762efb35b279_isrForCrosstalkSources_996976_24.stderr parsl.dataflow.dflow DEBUG: Task 4361 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4361 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4361 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4361 has memoization hash c5c20a856479802368de999e18d0999e parsl.dataflow.memoization INFO: Task 4361 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4361 try 0 launched on executor multi with executor id 4362 parsl.dataflow.dflow INFO: Standard output for task 4361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/942f9dc9-4b2d-49f2-a16f-33c3913b1e9b_isrForCrosstalkSources_996955_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/942f9dc9-4b2d-49f2-a16f-33c3913b1e9b_isrForCrosstalkSources_996955_12.stderr parsl.dataflow.dflow DEBUG: Task 4362 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4362 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4362 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4362 has memoization hash 55cf2dd98f359d0f577206be291d6781 parsl.dataflow.memoization INFO: Task 4362 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4362 try 0 launched on executor multi with executor id 4363 parsl.dataflow.dflow INFO: Standard output for task 4362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/f32b5a3b-9e95-4af5-9e11-60ad194f967d_isrForCrosstalkSources_996991_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/f32b5a3b-9e95-4af5-9e11-60ad194f967d_isrForCrosstalkSources_996991_11.stderr parsl.dataflow.dflow DEBUG: Task 4363 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4363 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4363 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4363 has memoization hash e36e6fbad48ba96e8c75b73e0303868c parsl.dataflow.memoization INFO: Task 4363 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4363 try 0 launched on executor multi with executor id 4364 parsl.dataflow.dflow INFO: Standard output for task 4363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8aa5ea30-d15b-41b9-b6b5-116742ef6077_isrForCrosstalkSources_997013_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8aa5ea30-d15b-41b9-b6b5-116742ef6077_isrForCrosstalkSources_997013_52.stderr parsl.dataflow.dflow DEBUG: Task 4364 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4364 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4364 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4364 has memoization hash 97c7989638b5fa1b4c52c98e3cab6f66 parsl.dataflow.memoization INFO: Task 4364 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1874c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4364 try 0 launched on executor multi with executor id 4365 parsl.dataflow.dflow INFO: Standard output for task 4364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ab2c92c9-5338-44cf-82b0-2d354679c7c2_isrForCrosstalkSources_996980_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ab2c92c9-5338-44cf-82b0-2d354679c7c2_isrForCrosstalkSources_996980_37.stderr parsl.dataflow.dflow DEBUG: Task 4365 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4365 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4365 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4365 has memoization hash dcefdd9f87210cca781065cee7117324 parsl.dataflow.memoization INFO: Task 4365 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1872e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4365 try 0 launched on executor multi with executor id 4366 parsl.dataflow.dflow INFO: Standard output for task 4365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/563c1e8a-0042-4d75-b6c8-0b4778b502a7_isrForCrosstalkSources_996966_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/563c1e8a-0042-4d75-b6c8-0b4778b502a7_isrForCrosstalkSources_996966_33.stderr parsl.dataflow.dflow DEBUG: Task 4366 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4366 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4366 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4366 has memoization hash b7e0a30b18364e695ebb809c3c4c08de parsl.dataflow.memoization INFO: Task 4366 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4366 try 0 launched on executor multi with executor id 4367 parsl.dataflow.dflow INFO: Standard output for task 4366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/176a68d9-4f07-4751-b852-8b65b9ad33e5_isrForCrosstalkSources_996993_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/176a68d9-4f07-4751-b852-8b65b9ad33e5_isrForCrosstalkSources_996993_14.stderr parsl.dataflow.dflow DEBUG: Task 4367 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4367 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4367 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4367 has memoization hash df094b900aebd0a3de1cfa01f7978b3c parsl.dataflow.memoization INFO: Task 4367 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4367 try 0 launched on executor multi with executor id 4368 parsl.dataflow.dflow INFO: Standard output for task 4367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/13f92840-d2cd-4aee-bc2f-296af3e79704_isrForCrosstalkSources_997022_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/13f92840-d2cd-4aee-bc2f-296af3e79704_isrForCrosstalkSources_997022_11.stderr parsl.dataflow.dflow DEBUG: Task 4368 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4368 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4368 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4368 has memoization hash 2f3c6fe43b121364144f2634f50d83ed parsl.dataflow.memoization INFO: Task 4368 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1876a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4368 try 0 launched on executor multi with executor id 4369 parsl.dataflow.dflow INFO: Standard output for task 4368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/ea76a04c-35af-42a7-916f-ac32a4435908_isrForCrosstalkSources_996952_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/ea76a04c-35af-42a7-916f-ac32a4435908_isrForCrosstalkSources_996952_37.stderr parsl.dataflow.dflow DEBUG: Task 4369 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4369 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4369 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4369 has memoization hash dea01d6010f563b1168de0de92d3959f parsl.dataflow.memoization INFO: Task 4369 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1877e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4369 try 0 launched on executor multi with executor id 4370 parsl.dataflow.dflow INFO: Standard output for task 4369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/ac2d9dff-ac28-4cfe-ad8d-4ab8976b0668_isrForCrosstalkSources_996961_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/ac2d9dff-ac28-4cfe-ad8d-4ab8976b0668_isrForCrosstalkSources_996961_20.stderr parsl.dataflow.dflow DEBUG: Task 4370 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4370 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4370 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4370 has memoization hash 3cd1b847b669038ad049036646304a9f parsl.dataflow.memoization INFO: Task 4370 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4370 try 0 launched on executor multi with executor id 4371 parsl.dataflow.dflow INFO: Standard output for task 4370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d61da60d-36ee-4dbb-a011-9eb26b748081_isrForCrosstalkSources_996979_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d61da60d-36ee-4dbb-a011-9eb26b748081_isrForCrosstalkSources_996979_19.stderr parsl.dataflow.dflow DEBUG: Task 4371 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4371 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4371 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4371 has memoization hash 98971daccfebc68e14c0e2cfe9acf547 parsl.dataflow.memoization INFO: Task 4371 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4371 try 0 launched on executor multi with executor id 4372 parsl.dataflow.dflow INFO: Standard output for task 4371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/0cfb53af-ebd2-4413-92d5-406f496bcdf4_isrForCrosstalkSources_996972_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/0cfb53af-ebd2-4413-92d5-406f496bcdf4_isrForCrosstalkSources_996972_23.stderr parsl.dataflow.dflow DEBUG: Task 4372 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4372 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4372 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4372 has memoization hash 4814afa1ff5f3319d136839b0f061f0f parsl.dataflow.memoization INFO: Task 4372 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4372 try 0 launched on executor multi with executor id 4373 parsl.dataflow.dflow INFO: Standard output for task 4372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/f1cc18f7-d3da-455d-92b9-2c2747c0436d_isrForCrosstalkSources_997029_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/f1cc18f7-d3da-455d-92b9-2c2747c0436d_isrForCrosstalkSources_997029_57.stderr parsl.dataflow.dflow DEBUG: Task 4373 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4373 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4373 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4373 has memoization hash 2079e756df10c3ed34522b9a99bb2736 parsl.dataflow.memoization INFO: Task 4373 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4373 try 0 launched on executor multi with executor id 4374 parsl.dataflow.dflow INFO: Standard output for task 4373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/acbe27b0-298b-46bb-9258-bb0f09b23894_isrForCrosstalkSources_996972_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/acbe27b0-298b-46bb-9258-bb0f09b23894_isrForCrosstalkSources_996972_37.stderr parsl.dataflow.dflow DEBUG: Task 4374 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4374 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4374 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4374 has memoization hash 67299b687bc2e869c5f90f92e3541b10 parsl.dataflow.memoization INFO: Task 4374 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4374 try 0 launched on executor multi with executor id 4375 parsl.dataflow.dflow INFO: Standard output for task 4374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/857afb47-9db0-4eee-b5b2-3014a85af708_isrForCrosstalkSources_996954_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/857afb47-9db0-4eee-b5b2-3014a85af708_isrForCrosstalkSources_996954_28.stderr parsl.dataflow.dflow DEBUG: Task 4375 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4375 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4375 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4375 has memoization hash 2217e0d18563b23ffe271ba682c3c404 parsl.dataflow.memoization INFO: Task 4375 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4375 try 0 launched on executor multi with executor id 4376 parsl.dataflow.dflow INFO: Standard output for task 4375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/6b09887d-7a43-4435-a4ef-598ef1824cdc_isrForCrosstalkSources_997028_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/6b09887d-7a43-4435-a4ef-598ef1824cdc_isrForCrosstalkSources_997028_4.stderr parsl.dataflow.dflow DEBUG: Task 4376 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4376 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4376 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4376 has memoization hash ab37de05d90188027a3a45f809f45d0d parsl.dataflow.memoization INFO: Task 4376 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4376 try 0 launched on executor multi with executor id 4377 parsl.dataflow.dflow INFO: Standard output for task 4376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/7a12d075-d2f1-4e0b-911c-b09623e97e2f_isrForCrosstalkSources_997004_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/7a12d075-d2f1-4e0b-911c-b09623e97e2f_isrForCrosstalkSources_997004_25.stderr parsl.dataflow.dflow DEBUG: Task 4377 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4377 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4377 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4377 has memoization hash 43af7fcd07347fd1887823859133ba13 parsl.dataflow.memoization INFO: Task 4377 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1879c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4377 try 0 launched on executor multi with executor id 4378 parsl.dataflow.dflow INFO: Standard output for task 4377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/c434cab6-f6b5-4205-8bc6-ec92caf9de4f_isrForCrosstalkSources_996957_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/c434cab6-f6b5-4205-8bc6-ec92caf9de4f_isrForCrosstalkSources_996957_57.stderr parsl.dataflow.dflow DEBUG: Task 4378 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4378 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4378 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4378 has memoization hash a391bb080436efd8ad9219fe8bde4d29 parsl.dataflow.memoization INFO: Task 4378 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4378 try 0 launched on executor multi with executor id 4379 parsl.dataflow.dflow INFO: Standard output for task 4378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f325644-ca42-4ca4-8cb9-37801a9a896b_isrForCrosstalkSources_996994_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f325644-ca42-4ca4-8cb9-37801a9a896b_isrForCrosstalkSources_996994_53.stderr parsl.dataflow.dflow DEBUG: Task 4379 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4379 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4379 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4379 has memoization hash 1ae59fdb1009284bad2448903c62878d parsl.dataflow.memoization INFO: Task 4379 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4379 try 0 launched on executor multi with executor id 4380 parsl.dataflow.dflow INFO: Standard output for task 4379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1129c0a9-760e-4ac9-9177-e35a12329866_isrForCrosstalkSources_996979_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1129c0a9-760e-4ac9-9177-e35a12329866_isrForCrosstalkSources_996979_30.stderr parsl.dataflow.dflow DEBUG: Task 4380 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4380 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4380 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4380 has memoization hash eb1b4173bfbae4bb85ae967bac99e767 parsl.dataflow.memoization INFO: Task 4380 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4380 try 0 launched on executor multi with executor id 4381 parsl.dataflow.dflow INFO: Standard output for task 4380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c8ac9bbb-3f37-456e-84dc-7a388c5ac31e_isrForCrosstalkSources_997014_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c8ac9bbb-3f37-456e-84dc-7a388c5ac31e_isrForCrosstalkSources_997014_7.stderr parsl.dataflow.dflow DEBUG: Task 4381 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4381 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4381 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4381 has memoization hash 90c4d6c39467d647fdd0375e62f8b21c parsl.dataflow.memoization INFO: Task 4381 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c187f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4381 try 0 launched on executor multi with executor id 4382 parsl.dataflow.dflow INFO: Standard output for task 4381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fa8a4834-5113-45f7-b870-8501cba7becd_isrForCrosstalkSources_996976_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fa8a4834-5113-45f7-b870-8501cba7becd_isrForCrosstalkSources_996976_33.stderr parsl.dataflow.dflow DEBUG: Task 4382 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4382 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4382 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4382 has memoization hash ab5730d370bc515db2dd74acaa6b59b6 parsl.dataflow.memoization INFO: Task 4382 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c1867a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4382 try 0 launched on executor multi with executor id 4383 parsl.dataflow.dflow INFO: Standard output for task 4382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/6e285c0d-350c-49e2-81bb-41d5f6cb1fa3_isrForCrosstalkSources_996984_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/6e285c0d-350c-49e2-81bb-41d5f6cb1fa3_isrForCrosstalkSources_996984_54.stderr parsl.dataflow.dflow DEBUG: Task 4383 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4383 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4383 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4383 has memoization hash 85259dd9fb5fc405c0145a05cab4089e parsl.dataflow.memoization INFO: Task 4383 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f840e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4383 try 0 launched on executor multi with executor id 4384 parsl.dataflow.dflow INFO: Standard output for task 4383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/82217c1f-ff80-4f7e-80d6-670629e8f2bf_isrForCrosstalkSources_997025_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/82217c1f-ff80-4f7e-80d6-670629e8f2bf_isrForCrosstalkSources_997025_15.stderr parsl.dataflow.dflow DEBUG: Task 4384 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4384 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4384 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4384 has memoization hash 4ac20fe232f7df0f77891e0551baa35f parsl.dataflow.memoization INFO: Task 4384 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4384 try 0 launched on executor multi with executor id 4385 parsl.dataflow.dflow INFO: Standard output for task 4384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3917908b-cb1f-4b7e-ad99-83f5472012da_isrForCrosstalkSources_997019_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3917908b-cb1f-4b7e-ad99-83f5472012da_isrForCrosstalkSources_997019_5.stderr parsl.dataflow.dflow DEBUG: Task 4385 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4385 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4385 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4385 has memoization hash 5af068a1741ef426f54640456a74bd20 parsl.dataflow.memoization INFO: Task 4385 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4385 try 0 launched on executor multi with executor id 4386 parsl.dataflow.dflow INFO: Standard output for task 4385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/324fa248-8458-46d6-b1a4-44094af0381c_isrForCrosstalkSources_996972_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/324fa248-8458-46d6-b1a4-44094af0381c_isrForCrosstalkSources_996972_15.stderr parsl.dataflow.dflow DEBUG: Task 4386 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4386 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4386 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4386 has memoization hash ae55b70f84c5f1099054ac948f543755 parsl.dataflow.memoization INFO: Task 4386 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f842c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4386 try 0 launched on executor multi with executor id 4387 parsl.dataflow.dflow INFO: Standard output for task 4386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0f38e91a-8779-4786-a307-009c6fafb2ad_isrForCrosstalkSources_997021_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0f38e91a-8779-4786-a307-009c6fafb2ad_isrForCrosstalkSources_997021_39.stderr parsl.dataflow.dflow DEBUG: Task 4387 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4387 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4387 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4387 has memoization hash 877f720ce769fa28bc7baccb18d035c7 parsl.dataflow.memoization INFO: Task 4387 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4387 try 0 launched on executor multi with executor id 4388 parsl.dataflow.dflow INFO: Standard output for task 4387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/251a336f-0326-40c7-8fa8-b3046655e8a7_isrForCrosstalkSources_996971_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/251a336f-0326-40c7-8fa8-b3046655e8a7_isrForCrosstalkSources_996971_53.stderr parsl.dataflow.dflow DEBUG: Task 4388 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4388 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4388 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4388 has memoization hash e0c003d331a1f38b736316a63ce91d7a parsl.dataflow.memoization INFO: Task 4388 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4388 try 0 launched on executor multi with executor id 4389 parsl.dataflow.dflow INFO: Standard output for task 4388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9884be5d-7c5a-4d79-af31-88f2213582fd_isrForCrosstalkSources_996964_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9884be5d-7c5a-4d79-af31-88f2213582fd_isrForCrosstalkSources_996964_2.stderr parsl.dataflow.dflow DEBUG: Task 4389 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4389 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4389 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4389 has memoization hash 682446cfae344845b3b619edf1acdd30 parsl.dataflow.memoization INFO: Task 4389 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4389 try 0 launched on executor multi with executor id 4390 parsl.dataflow.dflow INFO: Standard output for task 4389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e6774f98-9b00-44f4-a86e-275d7777471b_isrForCrosstalkSources_996992_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e6774f98-9b00-44f4-a86e-275d7777471b_isrForCrosstalkSources_996992_29.stderr parsl.dataflow.dflow DEBUG: Task 4390 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4390 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4390 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4390 has memoization hash 8207dd8b27d73bccde86898511b6352c parsl.dataflow.memoization INFO: Task 4390 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f844a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4390 try 0 launched on executor multi with executor id 4391 parsl.dataflow.dflow INFO: Standard output for task 4390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/de3e5d4b-e478-47b1-bc93-21a732ff3134_isrForCrosstalkSources_997030_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/de3e5d4b-e478-47b1-bc93-21a732ff3134_isrForCrosstalkSources_997030_16.stderr parsl.dataflow.dflow DEBUG: Task 4391 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4391 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4391 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4391 has memoization hash 026d8805cabb04bf61e326481b675a1c parsl.dataflow.memoization INFO: Task 4391 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f845e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4391 try 0 launched on executor multi with executor id 4392 parsl.dataflow.dflow INFO: Standard output for task 4391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/31fa0f08-84f7-4ccf-8de0-66b665318879_isrForCrosstalkSources_996973_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/31fa0f08-84f7-4ccf-8de0-66b665318879_isrForCrosstalkSources_996973_28.stderr parsl.dataflow.dflow DEBUG: Task 4392 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4392 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4392 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4392 has memoization hash b156930a8c6cb64d8fea165a1e4d81b3 parsl.dataflow.memoization INFO: Task 4392 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4392 try 0 launched on executor multi with executor id 4393 parsl.dataflow.dflow INFO: Standard output for task 4392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0b9627d5-8c57-4aa0-9b4e-244ee2b699fa_isrForCrosstalkSources_997012_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0b9627d5-8c57-4aa0-9b4e-244ee2b699fa_isrForCrosstalkSources_997012_13.stderr parsl.dataflow.dflow DEBUG: Task 4393 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4393 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4393 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4393 has memoization hash 5dc165743af693850e60586f2d7e8a5b parsl.dataflow.memoization INFO: Task 4393 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4393 try 0 launched on executor multi with executor id 4394 parsl.dataflow.dflow INFO: Standard output for task 4393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a28746a9-c48c-4f51-a7c7-d69458d7f44d_isrForCrosstalkSources_996991_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a28746a9-c48c-4f51-a7c7-d69458d7f44d_isrForCrosstalkSources_996991_18.stderr parsl.dataflow.dflow DEBUG: Task 4394 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4394 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4394 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4394 has memoization hash 2b58169658efbaad77c293a506cb6165 parsl.dataflow.memoization INFO: Task 4394 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4394 try 0 launched on executor multi with executor id 4395 parsl.dataflow.dflow INFO: Standard output for task 4394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a4d5ddf-e6d0-4312-858d-c696c6f9d66a_isrForCrosstalkSources_997004_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a4d5ddf-e6d0-4312-858d-c696c6f9d66a_isrForCrosstalkSources_997004_5.stderr parsl.dataflow.dflow DEBUG: Task 4395 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4395 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4395 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4395 has memoization hash 25b00943589566006f4b672d7080c6e3 parsl.dataflow.memoization INFO: Task 4395 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f847c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4395 try 0 launched on executor multi with executor id 4396 parsl.dataflow.dflow INFO: Standard output for task 4395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/32ceae19-a55f-42d6-939c-10581b49d84e_isrForCrosstalkSources_997007_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/32ceae19-a55f-42d6-939c-10581b49d84e_isrForCrosstalkSources_997007_27.stderr parsl.dataflow.dflow DEBUG: Task 4396 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4396 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4396 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4396 has memoization hash f19f0e9a6ed5bc2a72caa7ad6708943f parsl.dataflow.memoization INFO: Task 4396 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4396 try 0 launched on executor multi with executor id 4397 parsl.dataflow.dflow INFO: Standard output for task 4396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/f5d12716-b830-4c68-9c59-8a322cea0b8e_isrForCrosstalkSources_997002_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/f5d12716-b830-4c68-9c59-8a322cea0b8e_isrForCrosstalkSources_997002_33.stderr parsl.dataflow.dflow DEBUG: Task 4397 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4397 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4397 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4397 has memoization hash 92985a837ab004024fc6d819373a4e58 parsl.dataflow.memoization INFO: Task 4397 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f849a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4397 try 0 launched on executor multi with executor id 4398 parsl.dataflow.dflow INFO: Standard output for task 4397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/89babaae-eff8-4779-afa8-e37636543322_isrForCrosstalkSources_997014_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/89babaae-eff8-4779-afa8-e37636543322_isrForCrosstalkSources_997014_56.stderr parsl.dataflow.dflow DEBUG: Task 4398 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4398 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4398 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4398 has memoization hash 71c3b978fe3a64d9642e142bc6ecca02 parsl.dataflow.memoization INFO: Task 4398 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4398 try 0 launched on executor multi with executor id 4399 parsl.dataflow.dflow INFO: Standard output for task 4398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/914a8c2b-ba8b-4b85-b905-cf42fa3a12c0_isrForCrosstalkSources_997024_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/914a8c2b-ba8b-4b85-b905-cf42fa3a12c0_isrForCrosstalkSources_997024_12.stderr parsl.dataflow.dflow DEBUG: Task 4399 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4399 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4399 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4399 has memoization hash e996717b7526ff70e4234a41f7bb3d02 parsl.dataflow.memoization INFO: Task 4399 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4399 try 0 launched on executor multi with executor id 4400 parsl.dataflow.dflow INFO: Standard output for task 4399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/4885c83d-bce3-4865-bfcc-a61f09ebabd0_isrForCrosstalkSources_996961_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/4885c83d-bce3-4865-bfcc-a61f09ebabd0_isrForCrosstalkSources_996961_41.stderr parsl.dataflow.dflow DEBUG: Task 4400 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4400 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4400 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4400 has memoization hash 19bb641406c384399bbe418d19a5f1e2 parsl.dataflow.memoization INFO: Task 4400 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4400 try 0 launched on executor multi with executor id 4401 parsl.dataflow.dflow INFO: Standard output for task 4400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c8bc8f27-4d94-4245-a01a-0d20c6af5cc1_isrForCrosstalkSources_996980_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c8bc8f27-4d94-4245-a01a-0d20c6af5cc1_isrForCrosstalkSources_996980_15.stderr parsl.dataflow.dflow DEBUG: Task 4401 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4401 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4401 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4401 has memoization hash 78d652759b0ae7ae0d8bb8bf367bbdc0 parsl.dataflow.memoization INFO: Task 4401 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4401 try 0 launched on executor multi with executor id 4402 parsl.dataflow.dflow INFO: Standard output for task 4401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/685279a8-2bd5-41ee-ab01-efe0fe2e5fa4_isrForCrosstalkSources_997020_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/685279a8-2bd5-41ee-ab01-efe0fe2e5fa4_isrForCrosstalkSources_997020_54.stderr parsl.dataflow.dflow DEBUG: Task 4402 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4402 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4402 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4402 has memoization hash 5911ca6a9296b055480a96a3b1e46538 parsl.dataflow.memoization INFO: Task 4402 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4402 try 0 launched on executor multi with executor id 4403 parsl.dataflow.dflow INFO: Standard output for task 4402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a81bc4b8-e845-40ff-8b72-c3d7af70e716_isrForCrosstalkSources_997008_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a81bc4b8-e845-40ff-8b72-c3d7af70e716_isrForCrosstalkSources_997008_36.stderr parsl.dataflow.dflow DEBUG: Task 4403 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4403 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4403 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4403 has memoization hash 03d4ee7fb26bc9124b3a9be6d0fbff5e parsl.dataflow.memoization INFO: Task 4403 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4403 try 0 launched on executor multi with executor id 4404 parsl.dataflow.dflow INFO: Standard output for task 4403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/90f7c3a5-3893-4fb0-b06d-05b695fb7195_isrForCrosstalkSources_997032_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/90f7c3a5-3893-4fb0-b06d-05b695fb7195_isrForCrosstalkSources_997032_26.stderr parsl.dataflow.dflow DEBUG: Task 4404 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4404 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4404 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4404 has memoization hash 007fa2ec7f4fefe185c0c7e472b86aba parsl.dataflow.memoization INFO: Task 4404 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4404 try 0 launched on executor multi with executor id 4405 parsl.dataflow.dflow INFO: Standard output for task 4404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/d7bf6875-e2c8-4d87-9fda-ab91fdcb82ce_isrForCrosstalkSources_997020_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/d7bf6875-e2c8-4d87-9fda-ab91fdcb82ce_isrForCrosstalkSources_997020_21.stderr parsl.dataflow.dflow DEBUG: Task 4405 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4405 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4405 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4405 has memoization hash 26922c1ad5ec88551577f9f523db06d2 parsl.dataflow.memoization INFO: Task 4405 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4405 try 0 launched on executor multi with executor id 4406 parsl.dataflow.dflow INFO: Standard output for task 4405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/ec32f988-0a5c-4dda-9278-1f02e3990036_isrForCrosstalkSources_996955_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/ec32f988-0a5c-4dda-9278-1f02e3990036_isrForCrosstalkSources_996955_2.stderr parsl.dataflow.dflow DEBUG: Task 4406 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4406 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4406 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4406 has memoization hash 1edcfdaa6c1e4e01c34c7356cc1e8fd5 parsl.dataflow.memoization INFO: Task 4406 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4406 try 0 launched on executor multi with executor id 4407 parsl.dataflow.dflow INFO: Standard output for task 4406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1cca67a3-c358-4eec-a559-48b58987f7e2_isrForCrosstalkSources_996983_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1cca67a3-c358-4eec-a559-48b58987f7e2_isrForCrosstalkSources_996983_49.stderr parsl.dataflow.dflow DEBUG: Task 4407 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4407 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4407 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4407 has memoization hash fd25b9e46d978394b5cc0e2f07f22176 parsl.dataflow.memoization INFO: Task 4407 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4407 try 0 launched on executor multi with executor id 4408 parsl.dataflow.dflow INFO: Standard output for task 4407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/49681749-34fd-4cc8-9939-09afbf5d88fa_isrForCrosstalkSources_996992_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/49681749-34fd-4cc8-9939-09afbf5d88fa_isrForCrosstalkSources_996992_14.stderr parsl.dataflow.dflow DEBUG: Task 4408 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4408 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4408 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4408 has memoization hash 9e181d3bdc9051194cf9dba8cd1f78aa parsl.dataflow.memoization INFO: Task 4408 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4408 try 0 launched on executor multi with executor id 4409 parsl.dataflow.dflow INFO: Standard output for task 4408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/2fa59036-3c1e-4f4a-a9fd-80c2a1647502_isrForCrosstalkSources_997011_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/2fa59036-3c1e-4f4a-a9fd-80c2a1647502_isrForCrosstalkSources_997011_16.stderr parsl.dataflow.dflow DEBUG: Task 4409 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4409 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4409 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4409 has memoization hash 1a0006fbb7bb529a184e461e88a199fb parsl.dataflow.memoization INFO: Task 4409 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f851c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4409 try 0 launched on executor multi with executor id 4410 parsl.dataflow.dflow INFO: Standard output for task 4409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b53a540e-3b5d-4125-9734-4ca7e0fc660b_isrForCrosstalkSources_997022_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b53a540e-3b5d-4125-9734-4ca7e0fc660b_isrForCrosstalkSources_997022_27.stderr parsl.dataflow.dflow DEBUG: Task 4410 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4410 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4410 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4410 has memoization hash 00ea050434de9cf19fd10edc76e28ccb parsl.dataflow.memoization INFO: Task 4410 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4410 try 0 launched on executor multi with executor id 4411 parsl.dataflow.dflow INFO: Standard output for task 4410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/b694d2ad-1f6b-41e0-b6f1-855c352ee038_isrForCrosstalkSources_997018_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/b694d2ad-1f6b-41e0-b6f1-855c352ee038_isrForCrosstalkSources_997018_49.stderr parsl.dataflow.dflow DEBUG: Task 4411 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4411 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4411 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4411 has memoization hash 19b051a7439f9bfce07b62a9efd76c7e parsl.dataflow.memoization INFO: Task 4411 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4411 try 0 launched on executor multi with executor id 4412 parsl.dataflow.dflow INFO: Standard output for task 4411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8ed4aa16-7cec-4c1b-995b-3d00cb280e8a_isrForCrosstalkSources_996967_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8ed4aa16-7cec-4c1b-995b-3d00cb280e8a_isrForCrosstalkSources_996967_31.stderr parsl.dataflow.dflow DEBUG: Task 4412 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4412 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4412 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4412 has memoization hash 7b30e4bf322c423062ed30665ce93889 parsl.dataflow.memoization INFO: Task 4412 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4412 try 0 launched on executor multi with executor id 4413 parsl.dataflow.dflow INFO: Standard output for task 4412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4b14cfb7-ba81-4887-abe9-3b6ccf652b40_isrForCrosstalkSources_996976_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4b14cfb7-ba81-4887-abe9-3b6ccf652b40_isrForCrosstalkSources_996976_40.stderr parsl.dataflow.dflow DEBUG: Task 4413 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4413 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4413 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4413 has memoization hash 5dc17fc8b7c708aed96cb4c418499c6c parsl.dataflow.memoization INFO: Task 4413 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f853a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4413 try 0 launched on executor multi with executor id 4414 parsl.dataflow.dflow INFO: Standard output for task 4413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a53f999a-76e9-4851-8245-b26653382cdc_isrForCrosstalkSources_996993_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a53f999a-76e9-4851-8245-b26653382cdc_isrForCrosstalkSources_996993_1.stderr parsl.dataflow.dflow DEBUG: Task 4414 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4414 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4414 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4414 has memoization hash ac695b621857e0ea4ae1099a9b2ffe82 parsl.dataflow.memoization INFO: Task 4414 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f854e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4414 try 0 launched on executor multi with executor id 4415 parsl.dataflow.dflow INFO: Standard output for task 4414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/350af285-d959-4e3a-8c27-54ca70da7dba_isrForCrosstalkSources_996970_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/350af285-d959-4e3a-8c27-54ca70da7dba_isrForCrosstalkSources_996970_12.stderr parsl.dataflow.dflow DEBUG: Task 4415 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4415 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4415 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4415 has memoization hash 58c4014c5f80d97ffd3044477378d016 parsl.dataflow.memoization INFO: Task 4415 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4415 try 0 launched on executor multi with executor id 4416 parsl.dataflow.dflow INFO: Standard output for task 4415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/5b6233d9-1043-456e-94df-24c6d6e84427_isrForCrosstalkSources_996984_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/5b6233d9-1043-456e-94df-24c6d6e84427_isrForCrosstalkSources_996984_17.stderr parsl.dataflow.dflow DEBUG: Task 4416 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4416 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4416 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4416 has memoization hash fb418a6bb58be2f743d214ecd8b0755f parsl.dataflow.memoization INFO: Task 4416 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4416 try 0 launched on executor multi with executor id 4417 parsl.dataflow.dflow INFO: Standard output for task 4416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/aa137d9b-d586-4d80-82ca-9c990f540385_isrForCrosstalkSources_996991_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/aa137d9b-d586-4d80-82ca-9c990f540385_isrForCrosstalkSources_996991_52.stderr parsl.dataflow.dflow DEBUG: Task 4417 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4417 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4417 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4417 has memoization hash a2441b83fd7c4a5fa414c64763245052 parsl.dataflow.memoization INFO: Task 4417 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4417 try 0 launched on executor multi with executor id 4418 parsl.dataflow.dflow INFO: Standard output for task 4417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/febcfde9-e100-417a-9b21-2c5ba47163a1_isrForCrosstalkSources_996996_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/febcfde9-e100-417a-9b21-2c5ba47163a1_isrForCrosstalkSources_996996_58.stderr parsl.dataflow.dflow DEBUG: Task 4418 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4418 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4418 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4418 has memoization hash 43442687367c887f49ee6d1ec4d9dec2 parsl.dataflow.memoization INFO: Task 4418 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f856c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4418 try 0 launched on executor multi with executor id 4419 parsl.dataflow.dflow INFO: Standard output for task 4418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/8020b4ad-1f46-4b32-ae51-a7216e339602_isrForCrosstalkSources_997025_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/8020b4ad-1f46-4b32-ae51-a7216e339602_isrForCrosstalkSources_997025_37.stderr parsl.dataflow.dflow DEBUG: Task 4419 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4419 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4419 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4419 has memoization hash f9ec08f96b265f68c0444b22d5920690 parsl.dataflow.memoization INFO: Task 4419 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4419 try 0 launched on executor multi with executor id 4420 parsl.dataflow.dflow INFO: Standard output for task 4419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2770f4fa-f399-4757-9687-a317ded134f1_isrForCrosstalkSources_996990_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2770f4fa-f399-4757-9687-a317ded134f1_isrForCrosstalkSources_996990_21.stderr parsl.dataflow.dflow DEBUG: Task 4420 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4420 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4420 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4420 has memoization hash 05f18963eade4cc1aa19b7225897fbac parsl.dataflow.memoization INFO: Task 4420 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4420 try 0 launched on executor multi with executor id 4421 parsl.dataflow.dflow INFO: Standard output for task 4420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/16bf48c2-9821-4c36-a82d-81d39db88e00_isrForCrosstalkSources_996974_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/16bf48c2-9821-4c36-a82d-81d39db88e00_isrForCrosstalkSources_996974_16.stderr parsl.dataflow.dflow DEBUG: Task 4421 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4421 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4421 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4421 has memoization hash b07a1122858d4fffcbcca1b602a83c05 parsl.dataflow.memoization INFO: Task 4421 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f858a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4421 try 0 launched on executor multi with executor id 4422 parsl.dataflow.dflow INFO: Standard output for task 4421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/dd16daf2-5b2a-4d7c-be04-e398d359839f_isrForCrosstalkSources_997022_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/dd16daf2-5b2a-4d7c-be04-e398d359839f_isrForCrosstalkSources_997022_30.stderr parsl.dataflow.dflow DEBUG: Task 4422 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4422 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4422 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4422 has memoization hash 5c9c9ac91ed7b339e7adc3cc579e3ed4 parsl.dataflow.memoization INFO: Task 4422 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4422 try 0 launched on executor multi with executor id 4423 parsl.dataflow.dflow INFO: Standard output for task 4422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b3978908-c77d-4544-9861-5ce3f5c24575_isrForCrosstalkSources_996986_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b3978908-c77d-4544-9861-5ce3f5c24575_isrForCrosstalkSources_996986_4.stderr parsl.dataflow.dflow DEBUG: Task 4423 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4423 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4423 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4423 has memoization hash 3edf1fdde22d9982babdb24899a7eebe parsl.dataflow.memoization INFO: Task 4423 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f859e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4423 try 0 launched on executor multi with executor id 4424 parsl.dataflow.dflow INFO: Standard output for task 4423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/82705e31-463b-489b-8b28-6efbb2662acb_isrForCrosstalkSources_996998_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/82705e31-463b-489b-8b28-6efbb2662acb_isrForCrosstalkSources_996998_17.stderr parsl.dataflow.dflow DEBUG: Task 4424 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4424 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4424 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4424 has memoization hash 74530eebc695a7e297237b60b09b2a1c parsl.dataflow.memoization INFO: Task 4424 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4424 try 0 launched on executor multi with executor id 4425 parsl.dataflow.dflow INFO: Standard output for task 4424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/dd51df0e-303d-44d3-bc8e-8ed700ec48f7_isrForCrosstalkSources_996987_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/dd51df0e-303d-44d3-bc8e-8ed700ec48f7_isrForCrosstalkSources_996987_22.stderr parsl.dataflow.dflow DEBUG: Task 4425 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4425 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4425 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4425 has memoization hash 91378317d989ebcdb7cf658ee4720dfe parsl.dataflow.memoization INFO: Task 4425 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4425 try 0 launched on executor multi with executor id 4426 parsl.dataflow.dflow INFO: Standard output for task 4425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/36ae0023-58e7-4e11-bac7-f799099fff33_isrForCrosstalkSources_997006_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/36ae0023-58e7-4e11-bac7-f799099fff33_isrForCrosstalkSources_997006_27.stderr parsl.dataflow.dflow DEBUG: Task 4426 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4426 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4426 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4426 has memoization hash 48cca45f81d22db0efe4935b46605b3f parsl.dataflow.memoization INFO: Task 4426 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4426 try 0 launched on executor multi with executor id 4427 parsl.dataflow.dflow INFO: Standard output for task 4426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/28475fdc-93e5-4b9a-8aea-38609c8492f4_isrForCrosstalkSources_996972_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/28475fdc-93e5-4b9a-8aea-38609c8492f4_isrForCrosstalkSources_996972_10.stderr parsl.dataflow.dflow DEBUG: Task 4427 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4427 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4427 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4427 has memoization hash bb560916d1c46113609c99fb8aa49d20 parsl.dataflow.memoization INFO: Task 4427 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4427 try 0 launched on executor multi with executor id 4428 parsl.dataflow.dflow INFO: Standard output for task 4427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/35b49ebb-78d7-46d2-95b9-d8ccb297b939_isrForCrosstalkSources_996979_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/35b49ebb-78d7-46d2-95b9-d8ccb297b939_isrForCrosstalkSources_996979_18.stderr parsl.dataflow.dflow DEBUG: Task 4428 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4428 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4428 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4428 has memoization hash d06ff40e882c48de448b314031f3b44a parsl.dataflow.memoization INFO: Task 4428 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4428 try 0 launched on executor multi with executor id 4429 parsl.dataflow.dflow INFO: Standard output for task 4428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/50850fbf-9573-4427-9dbe-917d4e97ce8e_isrForCrosstalkSources_997005_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/50850fbf-9573-4427-9dbe-917d4e97ce8e_isrForCrosstalkSources_997005_9.stderr parsl.dataflow.dflow DEBUG: Task 4429 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4429 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4429 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4429 has memoization hash 8d18a40384922919e07c7cab5a05bfdf parsl.dataflow.memoization INFO: Task 4429 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4429 try 0 launched on executor multi with executor id 4430 parsl.dataflow.dflow INFO: Standard output for task 4429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/17331fa3-baeb-4a62-8cc5-da0a842c1ccb_isrForCrosstalkSources_997021_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/17331fa3-baeb-4a62-8cc5-da0a842c1ccb_isrForCrosstalkSources_997021_48.stderr parsl.dataflow.dflow DEBUG: Task 4430 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4430 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4430 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4430 has memoization hash abaac825711c7ae8909a77b57e649d98 parsl.dataflow.memoization INFO: Task 4430 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4430 try 0 launched on executor multi with executor id 4431 parsl.dataflow.dflow INFO: Standard output for task 4430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/8913404d-4554-4a58-9802-be5cad119d00_isrForCrosstalkSources_997010_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/8913404d-4554-4a58-9802-be5cad119d00_isrForCrosstalkSources_997010_1.stderr parsl.dataflow.dflow DEBUG: Task 4431 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4431 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4431 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4431 has memoization hash 9cdf84130fc17e8fd8b92907b42a27e0 parsl.dataflow.memoization INFO: Task 4431 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4431 try 0 launched on executor multi with executor id 4432 parsl.dataflow.dflow INFO: Standard output for task 4431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/a5fb33c7-92ea-4c8b-85f2-71be42018b67_isrForCrosstalkSources_996979_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/a5fb33c7-92ea-4c8b-85f2-71be42018b67_isrForCrosstalkSources_996979_46.stderr parsl.dataflow.dflow DEBUG: Task 4432 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4432 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4432 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4432 has memoization hash 4ac9e559633e72240c0d6279b321e2a6 parsl.dataflow.memoization INFO: Task 4432 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f85f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4432 try 0 launched on executor multi with executor id 4433 parsl.dataflow.dflow INFO: Standard output for task 4432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b2523a06-99ee-422e-a172-10e144c3de4f_isrForCrosstalkSources_996986_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b2523a06-99ee-422e-a172-10e144c3de4f_isrForCrosstalkSources_996986_12.stderr parsl.dataflow.dflow DEBUG: Task 4433 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4433 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4433 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4433 has memoization hash f7a47a9689b2c3bd510af9a060e207d3 parsl.dataflow.memoization INFO: Task 4433 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4433 try 0 launched on executor multi with executor id 4434 parsl.dataflow.dflow INFO: Standard output for task 4433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/94567f2a-0d67-47d9-bb30-55620136b102_isrForCrosstalkSources_997020_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/94567f2a-0d67-47d9-bb30-55620136b102_isrForCrosstalkSources_997020_24.stderr parsl.dataflow.dflow DEBUG: Task 4434 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4434 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4434 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4434 has memoization hash c80d149dced7219666ac114f8fa73282 parsl.dataflow.memoization INFO: Task 4434 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4434 try 0 launched on executor multi with executor id 4435 parsl.dataflow.dflow INFO: Standard output for task 4434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/ddede68c-cc46-46ce-a430-9abe9e40b956_isrForCrosstalkSources_996985_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/ddede68c-cc46-46ce-a430-9abe9e40b956_isrForCrosstalkSources_996985_20.stderr parsl.dataflow.dflow DEBUG: Task 4435 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4435 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4435 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4435 has memoization hash a10dc3f5d28f028f0db63930f0b509c8 parsl.dataflow.memoization INFO: Task 4435 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f860c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4435 try 0 launched on executor multi with executor id 4436 parsl.dataflow.dflow INFO: Standard output for task 4435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7080564a-8094-4048-b5f9-c682d01b6da8_isrForCrosstalkSources_996973_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7080564a-8094-4048-b5f9-c682d01b6da8_isrForCrosstalkSources_996973_59.stderr parsl.dataflow.dflow DEBUG: Task 4436 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4436 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4436 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4436 has memoization hash d1dea7de5dc562d6338bc8443d55945c parsl.dataflow.memoization INFO: Task 4436 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4436 try 0 launched on executor multi with executor id 4437 parsl.dataflow.dflow INFO: Standard output for task 4436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/db38b9e4-e7e1-4a47-b4f1-6e0dca2b1d95_isrForCrosstalkSources_997017_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/db38b9e4-e7e1-4a47-b4f1-6e0dca2b1d95_isrForCrosstalkSources_997017_61.stderr parsl.dataflow.dflow DEBUG: Task 4437 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4437 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4437 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4437 has memoization hash a3885d64143b708c6ff181ac625fd5d7 parsl.dataflow.memoization INFO: Task 4437 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f862a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4437 try 0 launched on executor multi with executor id 4438 parsl.dataflow.dflow INFO: Standard output for task 4437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/66b3e468-8ae2-4876-8569-ca47fa82ac3d_isrForCrosstalkSources_997026_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/66b3e468-8ae2-4876-8569-ca47fa82ac3d_isrForCrosstalkSources_997026_18.stderr parsl.dataflow.dflow DEBUG: Task 4438 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4438 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4438 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4438 has memoization hash e3cad89840dc2368686917db70ded505 parsl.dataflow.memoization INFO: Task 4438 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4438 try 0 launched on executor multi with executor id 4439 parsl.dataflow.dflow INFO: Standard output for task 4438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8c9b0c96-8ec1-42e7-9b3c-4bb2aafb58af_isrForCrosstalkSources_996952_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8c9b0c96-8ec1-42e7-9b3c-4bb2aafb58af_isrForCrosstalkSources_996952_43.stderr parsl.dataflow.dflow DEBUG: Task 4439 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4439 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4439 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4439 has memoization hash dd478dda5998caa7732830b26702d013 parsl.dataflow.memoization INFO: Task 4439 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4439 try 0 launched on executor multi with executor id 4440 parsl.dataflow.dflow INFO: Standard output for task 4439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/be44508b-1d91-479f-bc54-702ad721b2fb_isrForCrosstalkSources_996990_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/be44508b-1d91-479f-bc54-702ad721b2fb_isrForCrosstalkSources_996990_32.stderr parsl.dataflow.dflow DEBUG: Task 4440 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4440 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4440 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4440 has memoization hash 805703a669ba23c3f6c229301e63e9a8 parsl.dataflow.memoization INFO: Task 4440 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f863e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4440 try 0 launched on executor multi with executor id 4441 parsl.dataflow.dflow INFO: Standard output for task 4440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2a51d949-211d-4350-b3ea-0f65df25d599_isrForCrosstalkSources_996977_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2a51d949-211d-4350-b3ea-0f65df25d599_isrForCrosstalkSources_996977_60.stderr parsl.dataflow.dflow DEBUG: Task 4441 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4441 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4441 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4441 has memoization hash 4d2dbe2d50c072eefd198b0ea5dd35e9 parsl.dataflow.memoization INFO: Task 4441 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4441 try 0 launched on executor multi with executor id 4442 parsl.dataflow.dflow INFO: Standard output for task 4441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/54e86e31-e60f-438d-9aea-c677ebc853b1_isrForCrosstalkSources_997007_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/54e86e31-e60f-438d-9aea-c677ebc853b1_isrForCrosstalkSources_997007_34.stderr parsl.dataflow.dflow DEBUG: Task 4442 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4442 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4442 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4442 has memoization hash e784af375d6328cba2ce752e6ad4d587 parsl.dataflow.memoization INFO: Task 4442 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f865c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4442 try 0 launched on executor multi with executor id 4443 parsl.dataflow.dflow INFO: Standard output for task 4442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d46eccf9-35a6-4137-aa9b-27297f8be649_isrForCrosstalkSources_997030_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d46eccf9-35a6-4137-aa9b-27297f8be649_isrForCrosstalkSources_997030_62.stderr parsl.dataflow.dflow DEBUG: Task 4443 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4443 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4443 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4443 has memoization hash c614ee477bbaa1b82732a14fb9883422 parsl.dataflow.memoization INFO: Task 4443 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4443 try 0 launched on executor multi with executor id 4444 parsl.dataflow.dflow INFO: Standard output for task 4443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/7e4c6609-8eb1-443a-b676-ac47c6b74e5b_isrForCrosstalkSources_997032_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/7e4c6609-8eb1-443a-b676-ac47c6b74e5b_isrForCrosstalkSources_997032_62.stderr parsl.dataflow.dflow DEBUG: Task 4444 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4444 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4444 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4444 has memoization hash fd34e7eaac5ed9e3ce3e0b787c6c7cdb parsl.dataflow.memoization INFO: Task 4444 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f867a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4444 try 0 launched on executor multi with executor id 4445 parsl.dataflow.dflow INFO: Standard output for task 4444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/16d08bc4-5592-48e2-a92e-d66d3a312943_isrForCrosstalkSources_996954_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/16d08bc4-5592-48e2-a92e-d66d3a312943_isrForCrosstalkSources_996954_38.stderr parsl.dataflow.dflow DEBUG: Task 4445 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4445 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4445 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4445 has memoization hash 8e0760272f4d0cfaa66736ef79866b34 parsl.dataflow.memoization INFO: Task 4445 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4445 try 0 launched on executor multi with executor id 4446 parsl.dataflow.dflow INFO: Standard output for task 4445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/670dbd10-ce23-4a12-8d72-7fcca8f31fb9_isrForCrosstalkSources_996991_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/670dbd10-ce23-4a12-8d72-7fcca8f31fb9_isrForCrosstalkSources_996991_57.stderr parsl.dataflow.dflow DEBUG: Task 4446 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4446 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4446 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4446 has memoization hash a21e4bc21d0975b3aa398b2560f11680 parsl.dataflow.memoization INFO: Task 4446 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4446 try 0 launched on executor multi with executor id 4447 parsl.dataflow.dflow INFO: Standard output for task 4446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/f5b300ef-288e-4858-a702-f98c337b2d79_isrForCrosstalkSources_996954_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/f5b300ef-288e-4858-a702-f98c337b2d79_isrForCrosstalkSources_996954_30.stderr parsl.dataflow.dflow DEBUG: Task 4447 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4447 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4447 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4447 has memoization hash 5d0c7441d9341b6b12115bdaf0018ff7 parsl.dataflow.memoization INFO: Task 4447 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f868e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4447 try 0 launched on executor multi with executor id 4448 parsl.dataflow.dflow INFO: Standard output for task 4447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/37add00c-bc0c-42c5-b257-ef8264e95a41_isrForCrosstalkSources_997010_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/37add00c-bc0c-42c5-b257-ef8264e95a41_isrForCrosstalkSources_997010_35.stderr parsl.dataflow.dflow DEBUG: Task 4448 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4448 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4448 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4448 has memoization hash 53931c7020a1100e49c9d7a408b1977c parsl.dataflow.memoization INFO: Task 4448 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4448 try 0 launched on executor multi with executor id 4449 parsl.dataflow.dflow INFO: Standard output for task 4448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4d4066da-d051-431e-adf6-e27755ae5785_isrForCrosstalkSources_996987_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4d4066da-d051-431e-adf6-e27755ae5785_isrForCrosstalkSources_996987_26.stderr parsl.dataflow.dflow DEBUG: Task 4449 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4449 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4449 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4449 has memoization hash 385837a53009ef04ffa706e63bb1b340 parsl.dataflow.memoization INFO: Task 4449 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4449 try 0 launched on executor multi with executor id 4450 parsl.dataflow.dflow INFO: Standard output for task 4449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5b44acfb-57a7-4f65-a491-15928938f554_isrForCrosstalkSources_996955_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5b44acfb-57a7-4f65-a491-15928938f554_isrForCrosstalkSources_996955_23.stderr parsl.dataflow.dflow DEBUG: Task 4450 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4450 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4450 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4450 has memoization hash 7f467e55002c44111256f32ba2e605dd parsl.dataflow.memoization INFO: Task 4450 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4450 try 0 launched on executor multi with executor id 4451 parsl.dataflow.dflow INFO: Standard output for task 4450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6af55b86-f833-4927-98e2-2ab2951aa74a_isrForCrosstalkSources_996954_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6af55b86-f833-4927-98e2-2ab2951aa74a_isrForCrosstalkSources_996954_51.stderr parsl.dataflow.dflow DEBUG: Task 4451 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4451 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4451 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4451 has memoization hash 9960b1cbe30c8a72f148b9a3dac4f024 parsl.dataflow.memoization INFO: Task 4451 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4451 try 0 launched on executor multi with executor id 4452 parsl.dataflow.dflow INFO: Standard output for task 4451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/c27c8358-a018-4109-a200-f038cf976825_isrForCrosstalkSources_996983_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/c27c8358-a018-4109-a200-f038cf976825_isrForCrosstalkSources_996983_31.stderr parsl.dataflow.dflow DEBUG: Task 4452 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4452 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4452 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4452 has memoization hash 5fcb2f0a5fa64e5b76e6c288307031e1 parsl.dataflow.memoization INFO: Task 4452 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4452 try 0 launched on executor multi with executor id 4453 parsl.dataflow.dflow INFO: Standard output for task 4452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ba485dc9-07a9-4ad3-ac79-96a2d39e99e7_isrForCrosstalkSources_996981_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ba485dc9-07a9-4ad3-ac79-96a2d39e99e7_isrForCrosstalkSources_996981_43.stderr parsl.dataflow.dflow DEBUG: Task 4453 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4453 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4453 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4453 has memoization hash 711938829902876d3ff6f7ad2e331b37 parsl.dataflow.memoization INFO: Task 4453 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4453 try 0 launched on executor multi with executor id 4454 parsl.dataflow.dflow INFO: Standard output for task 4453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9665953b-2216-490b-8d12-15d0f242a834_isrForCrosstalkSources_996984_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9665953b-2216-490b-8d12-15d0f242a834_isrForCrosstalkSources_996984_19.stderr parsl.dataflow.dflow DEBUG: Task 4454 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4454 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4454 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4454 has memoization hash 4ab0837481677506691408438277c02c parsl.dataflow.memoization INFO: Task 4454 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4454 try 0 launched on executor multi with executor id 4455 parsl.dataflow.dflow INFO: Standard output for task 4454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/c7aec28c-003a-4b63-a6b8-a05c47381574_isrForCrosstalkSources_996990_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/c7aec28c-003a-4b63-a6b8-a05c47381574_isrForCrosstalkSources_996990_26.stderr parsl.dataflow.dflow DEBUG: Task 4455 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4455 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4455 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4455 has memoization hash 90559f099dba75799e353476885dac91 parsl.dataflow.memoization INFO: Task 4455 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4455 try 0 launched on executor multi with executor id 4456 parsl.dataflow.dflow INFO: Standard output for task 4455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0b90a53c-6013-46e8-a2d8-6a0d2c86b983_isrForCrosstalkSources_996997_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0b90a53c-6013-46e8-a2d8-6a0d2c86b983_isrForCrosstalkSources_996997_46.stderr parsl.dataflow.dflow DEBUG: Task 4456 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4456 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4456 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4456 has memoization hash 9f3ae94fedfe5aad0e468fd95f3bbd78 parsl.dataflow.memoization INFO: Task 4456 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4456 try 0 launched on executor multi with executor id 4457 parsl.dataflow.dflow INFO: Standard output for task 4456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/1b7f0786-7e29-4fc4-9b5c-d66d7c2772a4_isrForCrosstalkSources_996995_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/1b7f0786-7e29-4fc4-9b5c-d66d7c2772a4_isrForCrosstalkSources_996995_61.stderr parsl.dataflow.dflow DEBUG: Task 4457 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4457 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4457 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4457 has memoization hash c4f99d65e682651b67f75b242db06b42 parsl.dataflow.memoization INFO: Task 4457 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:08 proc_lsst.multi:146 [INFO] found job 17074836 in provider astro proc_lsst.multi INFO: found job 17074836 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4457 try 0 launched on executor multi with executor id 4458 parsl.dataflow.dflow INFO: Standard output for task 4457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/fbe8dfb2-da21-48e8-9e45-98e9c62a2955_isrForCrosstalkSources_997018_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/fbe8dfb2-da21-48e8-9e45-98e9c62a2955_isrForCrosstalkSources_997018_20.stderr parsl.dataflow.dflow DEBUG: Task 4458 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4458 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4458 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4458 has memoization hash 8c1380aca5311a9d94033b482affc0b5 parsl.dataflow.memoization INFO: Task 4458 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4458 try 0 launched on executor multi with executor id 4459 parsl.dataflow.dflow INFO: Standard output for task 4458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/f74a3fe6-fb76-44f2-996d-5b09dbf6c90b_isrForCrosstalkSources_996959_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/f74a3fe6-fb76-44f2-996d-5b09dbf6c90b_isrForCrosstalkSources_996959_1.stderr parsl.dataflow.dflow DEBUG: Task 4459 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4459 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4459 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4459 has memoization hash 5ed7b890a4876489b2455cabf382df68 parsl.dataflow.memoization INFO: Task 4459 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4459 try 0 launched on executor multi with executor id 4460 parsl.dataflow.dflow INFO: Standard output for task 4459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/016725d3-bad2-47f9-a8bf-7be53077b111_isrForCrosstalkSources_996954_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/016725d3-bad2-47f9-a8bf-7be53077b111_isrForCrosstalkSources_996954_4.stderr parsl.dataflow.dflow DEBUG: Task 4460 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4460 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4460 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4460 has memoization hash 3b66b35b5cef51b1b82e2d36f453cc9b parsl.dataflow.memoization INFO: Task 4460 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4460 try 0 launched on executor multi with executor id 4461 parsl.dataflow.dflow INFO: Standard output for task 4460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/2aaf28e8-1624-4f06-b3b7-9fe02d81f89f_isrForCrosstalkSources_996996_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/2aaf28e8-1624-4f06-b3b7-9fe02d81f89f_isrForCrosstalkSources_996996_13.stderr parsl.dataflow.dflow DEBUG: Task 4461 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4461 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4461 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4461 has memoization hash b97b284b559546ea0797221dea32b87d parsl.dataflow.memoization INFO: Task 4461 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4461 try 0 launched on executor multi with executor id 4462 parsl.dataflow.dflow INFO: Standard output for task 4461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7c8f9dbc-37b2-4a52-9864-9008a0a49e33_isrForCrosstalkSources_997026_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7c8f9dbc-37b2-4a52-9864-9008a0a49e33_isrForCrosstalkSources_997026_27.stderr parsl.dataflow.dflow DEBUG: Task 4462 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4462 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4462 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4462 has memoization hash 82a9d71eddbec6cdc755d77ecbe2aa48 parsl.dataflow.memoization INFO: Task 4462 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4462 try 0 launched on executor multi with executor id 4463 parsl.dataflow.dflow INFO: Standard output for task 4462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/ebdee1b8-825c-44bf-99d1-633d50a8cab9_isrForCrosstalkSources_996996_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/ebdee1b8-825c-44bf-99d1-633d50a8cab9_isrForCrosstalkSources_996996_10.stderr parsl.dataflow.dflow DEBUG: Task 4463 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4463 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4463 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4463 has memoization hash abacb36069f6494742da607c5c4e54cf parsl.dataflow.memoization INFO: Task 4463 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f86fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4463 try 0 launched on executor multi with executor id 4464 parsl.dataflow.dflow INFO: Standard output for task 4463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/b2232f25-04a5-4b03-bb00-d2c4c51c98ac_isrForCrosstalkSources_996956_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/b2232f25-04a5-4b03-bb00-d2c4c51c98ac_isrForCrosstalkSources_996956_38.stderr parsl.dataflow.dflow DEBUG: Task 4464 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4464 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4464 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4464 has memoization hash 6c5366895035ecf3ca64bdd8192877ce parsl.dataflow.memoization INFO: Task 4464 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f872e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4464 try 0 launched on executor multi with executor id 4465 parsl.dataflow.dflow INFO: Standard output for task 4464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/124b1898-2523-417f-a3f8-8956ed0ee4f6_isrForCrosstalkSources_997022_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/124b1898-2523-417f-a3f8-8956ed0ee4f6_isrForCrosstalkSources_997022_28.stderr parsl.dataflow.dflow DEBUG: Task 4465 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4465 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4465 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4465 has memoization hash e257e3bd0e6c3e7515d158fa64fbd6d9 parsl.dataflow.memoization INFO: Task 4465 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4465 try 0 launched on executor multi with executor id 4466 parsl.dataflow.dflow INFO: Standard output for task 4465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/54bcafc0-40b8-42b0-88b1-dd174d925b63_isrForCrosstalkSources_997017_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/54bcafc0-40b8-42b0-88b1-dd174d925b63_isrForCrosstalkSources_997017_1.stderr parsl.dataflow.dflow DEBUG: Task 4466 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4466 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4466 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4466 has memoization hash 01a26efec9394bda9027360a8d0934e8 parsl.dataflow.memoization INFO: Task 4466 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f84040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4466 try 0 launched on executor multi with executor id 4467 parsl.dataflow.dflow INFO: Standard output for task 4466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f68965bf-d6f9-41ce-b1b5-1c93385b6254_isrForCrosstalkSources_996977_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f68965bf-d6f9-41ce-b1b5-1c93385b6254_isrForCrosstalkSources_996977_33.stderr parsl.dataflow.dflow DEBUG: Task 4467 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4467 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4467 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4467 has memoization hash 70d4d804820b978bfa00c51862701003 parsl.dataflow.memoization INFO: Task 4467 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f871a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4467 try 0 launched on executor multi with executor id 4468 parsl.dataflow.dflow INFO: Standard output for task 4467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/614645f8-38f3-428c-90c1-97983a115c9d_isrForCrosstalkSources_996989_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/614645f8-38f3-428c-90c1-97983a115c9d_isrForCrosstalkSources_996989_40.stderr parsl.dataflow.dflow DEBUG: Task 4468 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4468 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4468 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4468 has memoization hash 5cb4b4e432e7b002b91fcf438353bfad parsl.dataflow.memoization INFO: Task 4468 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4468 try 0 launched on executor multi with executor id 4469 parsl.dataflow.dflow INFO: Standard output for task 4468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/2df5ef37-616f-4e5a-a4df-d414ebd27e3f_isrForCrosstalkSources_997008_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/2df5ef37-616f-4e5a-a4df-d414ebd27e3f_isrForCrosstalkSources_997008_30.stderr parsl.dataflow.dflow DEBUG: Task 4469 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4469 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4469 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4469 has memoization hash bdb3da532cf93bbdf0ca5e823381c648 parsl.dataflow.memoization INFO: Task 4469 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f874c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4469 try 0 launched on executor multi with executor id 4470 parsl.dataflow.dflow INFO: Standard output for task 4469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/9994882d-6655-4f36-8863-b05fa4b0f747_isrForCrosstalkSources_996956_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/9994882d-6655-4f36-8863-b05fa4b0f747_isrForCrosstalkSources_996956_51.stderr parsl.dataflow.dflow DEBUG: Task 4470 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4470 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4470 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4470 has memoization hash 32803cc98e0ae2c484711f07414c7057 parsl.dataflow.memoization INFO: Task 4470 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f876a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4470 try 0 launched on executor multi with executor id 4471 parsl.dataflow.dflow INFO: Standard output for task 4470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/646c251c-814e-4e98-93c6-52097ac47c4f_isrForCrosstalkSources_997016_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/646c251c-814e-4e98-93c6-52097ac47c4f_isrForCrosstalkSources_997016_62.stderr parsl.dataflow.dflow DEBUG: Task 4471 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4471 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4471 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4471 has memoization hash 9154e14faee6483c3d517ad4e6bc4a34 parsl.dataflow.memoization INFO: Task 4471 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4471 try 0 launched on executor multi with executor id 4472 parsl.dataflow.dflow INFO: Standard output for task 4471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ac2b0119-231c-4f8f-b993-e7b85a70f487_isrForCrosstalkSources_997019_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ac2b0119-231c-4f8f-b993-e7b85a70f487_isrForCrosstalkSources_997019_55.stderr parsl.dataflow.dflow DEBUG: Task 4472 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4472 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4472 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4472 has memoization hash c82ad6539ecc1736501e167cfa5caa8b parsl.dataflow.memoization INFO: Task 4472 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4472 try 0 launched on executor multi with executor id 4473 parsl.dataflow.dflow INFO: Standard output for task 4472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/96477658-95e1-4e60-b1f6-640780467e0e_isrForCrosstalkSources_997026_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/96477658-95e1-4e60-b1f6-640780467e0e_isrForCrosstalkSources_997026_39.stderr parsl.dataflow.dflow DEBUG: Task 4473 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4473 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4473 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4473 has memoization hash 3909b8a6c1270db72267e36754acdb1a parsl.dataflow.memoization INFO: Task 4473 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4473 try 0 launched on executor multi with executor id 4474 parsl.dataflow.dflow INFO: Standard output for task 4473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/d7f25149-05e6-45b2-aa45-335fd81dc371_isrForCrosstalkSources_996975_28.stdout parsl.dataflow.dflow INFO: Task 40 completed (launched -> exec_done) parsl.dataflow.dflow INFO: Standard error for task 4473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/d7f25149-05e6-45b2-aa45-335fd81dc371_isrForCrosstalkSources_996975_28.stderr parsl.dataflow.memoization DEBUG: Storing app cache entry 1e894828ed8545a7c7452e403d85739d with result from task 40 parsl.dataflow.dflow DEBUG: Task 4474 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow DEBUG: Task 40 checkpointed parsl.dataflow.dflow INFO: Task 4474 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow DEBUG: Task 4474 set to pending state with AppFuture: parsl.dataflow.dflow INFO: Standard output for task 40 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/dafa327f-7bbc-44b3-a932-1fa003ec5b77_isrForCrosstalkSources_997006_16.stdout parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.dflow INFO: Standard error for task 40 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/dafa327f-7bbc-44b3-a932-1fa003ec5b77_isrForCrosstalkSources_997006_16.stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4474 has memoization hash b7de673d5a88f674452e52070dae940d parsl.dataflow.memoization INFO: Task 4474 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f877e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4474 try 0 launched on executor multi with executor id 4475 parsl.dataflow.dflow INFO: Standard output for task 4474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/eec2344d-9524-40ff-a139-7e344862e906_isrForCrosstalkSources_996994_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/eec2344d-9524-40ff-a139-7e344862e906_isrForCrosstalkSources_996994_9.stderr parsl.dataflow.dflow DEBUG: Task 4475 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4475 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4475 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4475 has memoization hash 23e38391d894260b14251499ec14fd83 parsl.dataflow.memoization INFO: Task 4475 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4475 try 0 launched on executor multi with executor id 4476 parsl.dataflow.dflow INFO: Standard output for task 4475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/57cd6856-5e82-471c-aa69-1cb653ee3845_isrForCrosstalkSources_997010_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/57cd6856-5e82-471c-aa69-1cb653ee3845_isrForCrosstalkSources_997010_18.stderr parsl.dataflow.dflow DEBUG: Task 4476 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4476 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4476 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4476 has memoization hash bff739b3f3c22e660a815a57264f0aad parsl.dataflow.memoization INFO: Task 4476 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4476 try 0 launched on executor multi with executor id 4477 parsl.dataflow.dflow INFO: Standard output for task 4476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/43340da4-fd97-45a9-9ec5-ea4a703443ed_isrForCrosstalkSources_997026_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/43340da4-fd97-45a9-9ec5-ea4a703443ed_isrForCrosstalkSources_997026_23.stderr parsl.dataflow.dflow DEBUG: Task 4477 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4477 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4477 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4477 has memoization hash 9b40b499c932d8783a3c547072af1191 parsl.dataflow.memoization INFO: Task 4477 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4477 try 0 launched on executor multi with executor id 4478 parsl.dataflow.dflow INFO: Standard output for task 4477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e3810314-ba5b-4062-b1ab-7b5561d9edfe_isrForCrosstalkSources_996986_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e3810314-ba5b-4062-b1ab-7b5561d9edfe_isrForCrosstalkSources_996986_23.stderr parsl.dataflow.dflow DEBUG: Task 4478 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4478 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4478 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4478 has memoization hash b87a4d521dcb09eacc7ef7d1da9e7946 parsl.dataflow.memoization INFO: Task 4478 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4478 try 0 launched on executor multi with executor id 4479 parsl.dataflow.dflow INFO: Standard output for task 4478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/0e28ff92-3366-47a0-8496-92a9c4f73449_isrForCrosstalkSources_997017_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/0e28ff92-3366-47a0-8496-92a9c4f73449_isrForCrosstalkSources_997017_43.stderr parsl.dataflow.dflow DEBUG: Task 4479 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4479 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4479 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4479 has memoization hash 562404e858a2d9bdd83b6cfa70654fa3 parsl.dataflow.memoization INFO: Task 4479 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4479 try 0 launched on executor multi with executor id 4480 parsl.dataflow.dflow INFO: Standard output for task 4479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1084a3ce-7e1d-4a69-9ec2-b760cdce1e01_isrForCrosstalkSources_997032_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1084a3ce-7e1d-4a69-9ec2-b760cdce1e01_isrForCrosstalkSources_997032_53.stderr parsl.dataflow.dflow DEBUG: Task 4480 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4480 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4480 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4480 has memoization hash c7809e5dfb5d6ca904bc7aaecd3f41f2 parsl.dataflow.memoization INFO: Task 4480 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4480 try 0 launched on executor multi with executor id 4481 parsl.dataflow.dflow INFO: Standard output for task 4480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3b863b40-a1b3-42fe-9d37-c3d258205951_isrForCrosstalkSources_996967_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3b863b40-a1b3-42fe-9d37-c3d258205951_isrForCrosstalkSources_996967_30.stderr parsl.dataflow.dflow DEBUG: Task 4481 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4481 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4481 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4481 has memoization hash b31dce87dc8ac887a5d62ab0cf68148f parsl.dataflow.memoization INFO: Task 4481 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f879c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4481 try 0 launched on executor multi with executor id 4482 parsl.dataflow.dflow INFO: Standard output for task 4481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4d82dd88-527f-45c8-aaaa-ebcd7a4242fd_isrForCrosstalkSources_997005_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4d82dd88-527f-45c8-aaaa-ebcd7a4242fd_isrForCrosstalkSources_997005_14.stderr parsl.dataflow.dflow DEBUG: Task 4482 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4482 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4482 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4482 has memoization hash 138bbb29a5b240fc8434f3fbb5ed1117 parsl.dataflow.memoization INFO: Task 4482 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4482 try 0 launched on executor multi with executor id 4483 parsl.dataflow.dflow INFO: Standard output for task 4482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/cf1bb616-06fa-4d3e-91c6-3e0fcc1e0731_isrForCrosstalkSources_997026_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/cf1bb616-06fa-4d3e-91c6-3e0fcc1e0731_isrForCrosstalkSources_997026_57.stderr parsl.dataflow.dflow DEBUG: Task 4483 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4483 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4483 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4483 has memoization hash 84386c3252be7f118e79e37916d665a3 parsl.dataflow.memoization INFO: Task 4483 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4483 try 0 launched on executor multi with executor id 4484 parsl.dataflow.dflow INFO: Standard output for task 4483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/141f8555-e669-45b4-960d-bbbbe03ac3b7_isrForCrosstalkSources_996991_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/141f8555-e669-45b4-960d-bbbbe03ac3b7_isrForCrosstalkSources_996991_10.stderr parsl.dataflow.dflow DEBUG: Task 4484 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4484 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4484 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4484 has memoization hash e779fd1733d2a6e30eb0c02a87223d10 parsl.dataflow.memoization INFO: Task 4484 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4484 try 0 launched on executor multi with executor id 4485 parsl.dataflow.dflow INFO: Standard output for task 4484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bf082dc5-22b5-45a6-b842-d268760d43fc_isrForCrosstalkSources_997030_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bf082dc5-22b5-45a6-b842-d268760d43fc_isrForCrosstalkSources_997030_8.stderr parsl.dataflow.dflow DEBUG: Task 4485 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4485 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4485 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4485 has memoization hash 5c56c06a1096323d2707a52e3100f929 parsl.dataflow.memoization INFO: Task 4485 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4485 try 0 launched on executor multi with executor id 4486 parsl.dataflow.dflow INFO: Standard output for task 4485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/15b197d4-6133-471a-bb83-7b71ce32389d_isrForCrosstalkSources_996962_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/15b197d4-6133-471a-bb83-7b71ce32389d_isrForCrosstalkSources_996962_50.stderr parsl.dataflow.dflow DEBUG: Task 4486 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4486 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4486 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4486 has memoization hash c04cd14c071860abd477a73e6fa156b7 parsl.dataflow.memoization INFO: Task 4486 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4486 try 0 launched on executor multi with executor id 4487 parsl.dataflow.dflow INFO: Standard output for task 4486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b20cb7cd-2653-4aa2-b260-80afbff91c40_isrForCrosstalkSources_997004_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b20cb7cd-2653-4aa2-b260-80afbff91c40_isrForCrosstalkSources_997004_23.stderr parsl.dataflow.dflow DEBUG: Task 4487 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4487 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4487 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4487 has memoization hash 463d45b64af6e3d7529d5d6aa35b13ed parsl.dataflow.memoization INFO: Task 4487 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4487 try 0 launched on executor multi with executor id 4488 parsl.dataflow.dflow INFO: Standard output for task 4487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/4971d94b-517f-4fbf-9128-20b06cd1f375_isrForCrosstalkSources_996981_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/4971d94b-517f-4fbf-9128-20b06cd1f375_isrForCrosstalkSources_996981_38.stderr parsl.dataflow.dflow DEBUG: Task 4488 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4488 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4488 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4488 has memoization hash c1a374f2ef27f6174920571cde46f234 parsl.dataflow.memoization INFO: Task 4488 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4488 try 0 launched on executor multi with executor id 4489 parsl.dataflow.dflow INFO: Standard output for task 4488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/e3528487-b9c2-4b97-954a-51df8df41494_isrForCrosstalkSources_996981_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/e3528487-b9c2-4b97-954a-51df8df41494_isrForCrosstalkSources_996981_49.stderr parsl.dataflow.dflow DEBUG: Task 4489 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4489 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4489 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4489 has memoization hash c775e2f51064f38051022ba8b4ef50e0 parsl.dataflow.memoization INFO: Task 4489 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4489 try 0 launched on executor multi with executor id 4490 parsl.dataflow.dflow INFO: Standard output for task 4489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b1e0a762-e13f-40fe-ad8f-f423309fe96d_isrForCrosstalkSources_996999_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b1e0a762-e13f-40fe-ad8f-f423309fe96d_isrForCrosstalkSources_996999_46.stderr parsl.dataflow.dflow DEBUG: Task 4490 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4490 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4490 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4490 has memoization hash c8b58976a3f81371c025b600e41b5dd9 parsl.dataflow.memoization INFO: Task 4490 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4490 try 0 launched on executor multi with executor id 4491 parsl.dataflow.dflow INFO: Standard output for task 4490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8b9332d2-f57d-4a40-86df-4558599c9f69_isrForCrosstalkSources_996984_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8b9332d2-f57d-4a40-86df-4558599c9f69_isrForCrosstalkSources_996984_62.stderr parsl.dataflow.dflow DEBUG: Task 4491 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4491 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4491 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4491 has memoization hash 36215315a905d78265296797f0e4bd8a parsl.dataflow.memoization INFO: Task 4491 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4491 try 0 launched on executor multi with executor id 4492 parsl.dataflow.dflow INFO: Standard output for task 4491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c031d22f-9520-4220-95ac-e504c18b6068_isrForCrosstalkSources_996969_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c031d22f-9520-4220-95ac-e504c18b6068_isrForCrosstalkSources_996969_55.stderr parsl.dataflow.dflow DEBUG: Task 4492 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4492 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4492 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4492 has memoization hash 0cc5c15875da2c403c8976de45a86113 parsl.dataflow.memoization INFO: Task 4492 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4492 try 0 launched on executor multi with executor id 4493 parsl.dataflow.dflow INFO: Standard output for task 4492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/19fd3b51-5996-4108-9a31-04cac7d9695a_isrForCrosstalkSources_997000_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/19fd3b51-5996-4108-9a31-04cac7d9695a_isrForCrosstalkSources_997000_1.stderr parsl.dataflow.dflow DEBUG: Task 4493 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4493 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4493 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4493 has memoization hash fb0aef91e7d9c1d00346a73767b8f9f6 parsl.dataflow.memoization INFO: Task 4493 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4493 try 0 launched on executor multi with executor id 4494 parsl.dataflow.dflow INFO: Standard output for task 4493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/19bf4753-9b53-4ce9-9fd3-8509cc388a5a_isrForCrosstalkSources_996965_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/19bf4753-9b53-4ce9-9fd3-8509cc388a5a_isrForCrosstalkSources_996965_40.stderr parsl.dataflow.dflow DEBUG: Task 4494 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4494 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4494 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4494 has memoization hash 88c89e57b1bce02cb4d66d0cfc1d84e7 parsl.dataflow.memoization INFO: Task 4494 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4494 try 0 launched on executor multi with executor id 4495 parsl.dataflow.dflow INFO: Standard output for task 4494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/2d9c5c30-4bee-44c7-8a60-bc70898bcb6b_isrForCrosstalkSources_996954_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/2d9c5c30-4bee-44c7-8a60-bc70898bcb6b_isrForCrosstalkSources_996954_17.stderr parsl.dataflow.dflow DEBUG: Task 4495 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4495 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4495 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4495 has memoization hash 81e02ed6b76ad3949c07e7f7de49c31a parsl.dataflow.memoization INFO: Task 4495 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4495 try 0 launched on executor multi with executor id 4496 parsl.dataflow.dflow INFO: Standard output for task 4495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/eaeef10b-0b64-42e4-ba61-ca13764bda58_isrForCrosstalkSources_997012_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/eaeef10b-0b64-42e4-ba61-ca13764bda58_isrForCrosstalkSources_997012_33.stderr parsl.dataflow.dflow DEBUG: Task 4496 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4496 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4496 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4496 has memoization hash 1bc8fd54ba9f991d3e5afbe72484fbf5 parsl.dataflow.memoization INFO: Task 4496 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4496 try 0 launched on executor multi with executor id 4497 parsl.dataflow.dflow INFO: Standard output for task 4496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c092cae6-6bfa-4c6a-91e8-4cd6090e8e3d_isrForCrosstalkSources_996973_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c092cae6-6bfa-4c6a-91e8-4cd6090e8e3d_isrForCrosstalkSources_996973_44.stderr parsl.dataflow.dflow DEBUG: Task 4497 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4497 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4497 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4497 has memoization hash aa3e134f961192251c28c1c8cfb4b49e parsl.dataflow.memoization INFO: Task 4497 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4497 try 0 launched on executor multi with executor id 4498 parsl.dataflow.dflow INFO: Standard output for task 4497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/29750e47-42d5-4ea8-972b-7251ca5e7c74_isrForCrosstalkSources_997006_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/29750e47-42d5-4ea8-972b-7251ca5e7c74_isrForCrosstalkSources_997006_1.stderr parsl.dataflow.dflow DEBUG: Task 4498 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4498 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4498 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4498 has memoization hash b8ef2059031708159cd68ea1a371e766 parsl.dataflow.memoization INFO: Task 4498 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4498 try 0 launched on executor multi with executor id 4499 parsl.dataflow.dflow INFO: Standard output for task 4498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9601ddd9-4911-4fc3-b1db-96962c0d7386_isrForCrosstalkSources_997008_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9601ddd9-4911-4fc3-b1db-96962c0d7386_isrForCrosstalkSources_997008_8.stderr parsl.dataflow.dflow DEBUG: Task 4499 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4499 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4499 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4499 has memoization hash 5f87035d772fce6c6d55b510d604828c parsl.dataflow.memoization INFO: Task 4499 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4499 try 0 launched on executor multi with executor id 4500 parsl.dataflow.dflow INFO: Standard output for task 4499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/01ee553a-0093-44fc-b315-fe59de80c9ed_isrForCrosstalkSources_996953_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/01ee553a-0093-44fc-b315-fe59de80c9ed_isrForCrosstalkSources_996953_30.stderr parsl.dataflow.dflow DEBUG: Task 4500 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4500 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4500 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4500 has memoization hash b3153c73937013c52c74a72e14223d91 parsl.dataflow.memoization INFO: Task 4500 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4500 try 0 launched on executor multi with executor id 4501 parsl.dataflow.dflow INFO: Standard output for task 4500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/74d00b04-1f50-4dd5-91dd-c2bb212e8322_isrForCrosstalkSources_996998_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/74d00b04-1f50-4dd5-91dd-c2bb212e8322_isrForCrosstalkSources_996998_8.stderr parsl.dataflow.dflow DEBUG: Task 4501 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4501 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4501 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4501 has memoization hash e9933211df6dba447484a19722926e5a parsl.dataflow.memoization INFO: Task 4501 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4501 try 0 launched on executor multi with executor id 4502 parsl.dataflow.dflow INFO: Standard output for task 4501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/74c6e314-84ae-43a9-b24d-63ac1d665483_isrForCrosstalkSources_996982_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/74c6e314-84ae-43a9-b24d-63ac1d665483_isrForCrosstalkSources_996982_40.stderr parsl.dataflow.dflow DEBUG: Task 4502 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4502 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4502 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4502 has memoization hash 0418486987491d3255990d3b9dda9d45 parsl.dataflow.memoization INFO: Task 4502 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4502 try 0 launched on executor multi with executor id 4503 parsl.dataflow.dflow INFO: Standard output for task 4502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/8b957c19-5aba-451b-b65e-4c97110ac8ee_isrForCrosstalkSources_997012_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/8b957c19-5aba-451b-b65e-4c97110ac8ee_isrForCrosstalkSources_997012_51.stderr parsl.dataflow.dflow DEBUG: Task 4503 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4503 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4503 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4503 has memoization hash 25f1baecc5c52017fde72e608d8d6f87 parsl.dataflow.memoization INFO: Task 4503 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4503 try 0 launched on executor multi with executor id 4504 parsl.dataflow.dflow INFO: Standard output for task 4503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/52e22a4d-446b-41de-b0e2-7c39ab10e2eb_isrForCrosstalkSources_997010_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/52e22a4d-446b-41de-b0e2-7c39ab10e2eb_isrForCrosstalkSources_997010_40.stderr parsl.dataflow.dflow DEBUG: Task 4504 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4504 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4504 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4504 has memoization hash a8e400749ae08e094f7b420d2c262dd8 parsl.dataflow.memoization INFO: Task 4504 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffcae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4504 try 0 launched on executor multi with executor id 4505 parsl.dataflow.dflow INFO: Standard output for task 4504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7eb5eb31-87bb-41e7-84cd-3e54bd50629b_isrForCrosstalkSources_997005_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7eb5eb31-87bb-41e7-84cd-3e54bd50629b_isrForCrosstalkSources_997005_15.stderr parsl.dataflow.dflow DEBUG: Task 4505 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4505 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4505 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4505 has memoization hash 3c188c6a5f3432a1819ad22371884328 parsl.dataflow.memoization INFO: Task 4505 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4505 try 0 launched on executor multi with executor id 4506 parsl.dataflow.dflow INFO: Standard output for task 4505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/4c7aa05e-3bb3-43bb-bc24-b0443504540d_isrForCrosstalkSources_996978_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/4c7aa05e-3bb3-43bb-bc24-b0443504540d_isrForCrosstalkSources_996978_19.stderr parsl.dataflow.dflow DEBUG: Task 4506 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4506 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4506 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4506 has memoization hash 31b19316e707a33b4ebc83350b88caec parsl.dataflow.memoization INFO: Task 4506 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffc900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4506 try 0 launched on executor multi with executor id 4507 parsl.dataflow.dflow INFO: Standard output for task 4506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/eed28bb0-9cb7-4f7b-92ea-a311109575ef_isrForCrosstalkSources_996974_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/eed28bb0-9cb7-4f7b-92ea-a311109575ef_isrForCrosstalkSources_996974_47.stderr parsl.dataflow.dflow DEBUG: Task 4507 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4507 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4507 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4507 has memoization hash 8f5f92d8d49b245ae4d30188117f986b parsl.dataflow.memoization INFO: Task 4507 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffcc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4507 try 0 launched on executor multi with executor id 4508 parsl.dataflow.dflow INFO: Standard output for task 4507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/99ea1653-9e19-4653-b29a-2a7ecbd13f45_isrForCrosstalkSources_997029_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/99ea1653-9e19-4653-b29a-2a7ecbd13f45_isrForCrosstalkSources_997029_30.stderr parsl.dataflow.dflow DEBUG: Task 4508 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4508 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4508 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4508 has memoization hash ca22a3fcc5b80a3ac807de3d2088c2a5 parsl.dataflow.memoization INFO: Task 4508 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4508 try 0 launched on executor multi with executor id 4509 parsl.dataflow.dflow INFO: Standard output for task 4508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b40ad668-05ae-443e-8526-3b4f961bcff2_isrForCrosstalkSources_996975_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b40ad668-05ae-443e-8526-3b4f961bcff2_isrForCrosstalkSources_996975_22.stderr parsl.dataflow.dflow DEBUG: Task 4509 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4509 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4509 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4509 has memoization hash f286d7f4401b6cfbfac400b89cd878a0 parsl.dataflow.memoization INFO: Task 4509 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffcb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4509 try 0 launched on executor multi with executor id 4510 parsl.dataflow.dflow INFO: Standard output for task 4509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f4c05ffe-7c89-481e-bd46-64f9aca430e5_isrForCrosstalkSources_997031_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f4c05ffe-7c89-481e-bd46-64f9aca430e5_isrForCrosstalkSources_997031_14.stderr parsl.dataflow.dflow DEBUG: Task 4510 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4510 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4510 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4510 has memoization hash 16b040a0197b8c316552d249f7c09061 parsl.dataflow.memoization INFO: Task 4510 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffcd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4510 try 0 launched on executor multi with executor id 4511 parsl.dataflow.dflow INFO: Standard output for task 4510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/38068b8c-d7eb-4938-9136-ac984f9ea849_isrForCrosstalkSources_997011_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/38068b8c-d7eb-4938-9136-ac984f9ea849_isrForCrosstalkSources_997011_61.stderr parsl.dataflow.dflow DEBUG: Task 4511 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4511 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4511 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4511 has memoization hash e694a6dc2cb9363bc7af4580b0118a01 parsl.dataflow.memoization INFO: Task 4511 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4511 try 0 launched on executor multi with executor id 4512 parsl.dataflow.dflow INFO: Standard output for task 4511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/14ef5dde-e26b-4565-8dec-fefe4ca146ec_isrForCrosstalkSources_996999_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/14ef5dde-e26b-4565-8dec-fefe4ca146ec_isrForCrosstalkSources_996999_2.stderr parsl.dataflow.dflow DEBUG: Task 4512 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4512 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4512 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4512 has memoization hash f2991f4cdfb385547cf23fb848bd63bc parsl.dataflow.memoization INFO: Task 4512 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffcea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4512 try 0 launched on executor multi with executor id 4513 parsl.dataflow.dflow INFO: Standard output for task 4512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7ac1fa30-5940-4ab2-af44-135a7b909c6a_isrForCrosstalkSources_996990_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7ac1fa30-5940-4ab2-af44-135a7b909c6a_isrForCrosstalkSources_996990_41.stderr parsl.dataflow.dflow DEBUG: Task 4513 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4513 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4513 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4513 has memoization hash 8d5c85610c750b97badc8ada6565dcf2 parsl.dataflow.memoization INFO: Task 4513 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffcfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4513 try 0 launched on executor multi with executor id 4514 parsl.dataflow.dflow INFO: Standard output for task 4513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/78474556-93db-4e91-b494-46c210507d7d_isrForCrosstalkSources_996958_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/78474556-93db-4e91-b494-46c210507d7d_isrForCrosstalkSources_996958_4.stderr parsl.dataflow.dflow DEBUG: Task 4514 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4514 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4514 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4514 has memoization hash d604bc5e9070439aace045bacc42552d parsl.dataflow.memoization INFO: Task 4514 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4514 try 0 launched on executor multi with executor id 4515 parsl.dataflow.dflow INFO: Standard output for task 4514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/9fd0283c-93f4-444f-baf1-f473778f4f8b_isrForCrosstalkSources_996999_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/9fd0283c-93f4-444f-baf1-f473778f4f8b_isrForCrosstalkSources_996999_62.stderr parsl.dataflow.dflow DEBUG: Task 4515 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4515 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4515 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4515 has memoization hash 506e05e597780455dd083ccc9063f5b4 parsl.dataflow.memoization INFO: Task 4515 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4515 try 0 launched on executor multi with executor id 4516 parsl.dataflow.dflow INFO: Standard output for task 4515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/54d1db09-f92b-42bb-ba38-f06938f0dc3e_isrForCrosstalkSources_997011_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/54d1db09-f92b-42bb-ba38-f06938f0dc3e_isrForCrosstalkSources_997011_6.stderr parsl.dataflow.dflow DEBUG: Task 4516 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4516 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4516 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4516 has memoization hash 0c29c4e0590f1ae488f3e1a79b935972 parsl.dataflow.memoization INFO: Task 4516 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffdf80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4516 try 0 launched on executor multi with executor id 4517 parsl.dataflow.dflow INFO: Standard output for task 4516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/3960529c-cbb9-4623-8988-ab44c17d4109_isrForCrosstalkSources_996966_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/3960529c-cbb9-4623-8988-ab44c17d4109_isrForCrosstalkSources_996966_15.stderr parsl.dataflow.dflow DEBUG: Task 4517 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4517 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4517 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4517 has memoization hash f4bc970c5ed00a658eda118800ebaaf2 parsl.dataflow.memoization INFO: Task 4517 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffdee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4517 try 0 launched on executor multi with executor id 4518 parsl.dataflow.dflow INFO: Standard output for task 4517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c2b5932e-e2c7-4890-a210-9be81bf0ad7f_isrForCrosstalkSources_997028_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c2b5932e-e2c7-4890-a210-9be81bf0ad7f_isrForCrosstalkSources_997028_48.stderr parsl.dataflow.dflow DEBUG: Task 4518 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4518 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4518 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4518 has memoization hash c43407f679169bd50a3e1a4cc3cede7c parsl.dataflow.memoization INFO: Task 4518 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffde40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4518 try 0 launched on executor multi with executor id 4519 parsl.dataflow.dflow INFO: Standard output for task 4518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d85622f4-b947-4340-be4a-54dd37c2d7f7_isrForCrosstalkSources_996989_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d85622f4-b947-4340-be4a-54dd37c2d7f7_isrForCrosstalkSources_996989_49.stderr parsl.dataflow.dflow DEBUG: Task 4519 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4519 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4519 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4519 has memoization hash c062c9c78257710326359116967d5d07 parsl.dataflow.memoization INFO: Task 4519 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffdd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4519 try 0 launched on executor multi with executor id 4520 parsl.dataflow.dflow INFO: Standard output for task 4519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ba5dffbe-330b-414c-8be0-8635aa91e7a4_isrForCrosstalkSources_996994_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ba5dffbe-330b-414c-8be0-8635aa91e7a4_isrForCrosstalkSources_996994_56.stderr parsl.dataflow.dflow DEBUG: Task 4520 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4520 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4520 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4520 has memoization hash 8040e234c3438ca0089a7ea30e205a30 parsl.dataflow.memoization INFO: Task 4520 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffdda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4520 try 0 launched on executor multi with executor id 4521 parsl.dataflow.dflow INFO: Standard output for task 4520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/f7b1bc31-59d9-4def-855e-1fe49c85827c_isrForCrosstalkSources_997013_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/f7b1bc31-59d9-4def-855e-1fe49c85827c_isrForCrosstalkSources_997013_31.stderr parsl.dataflow.dflow DEBUG: Task 4521 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4521 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4521 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4521 has memoization hash 7ab4c92a051ec961c0a68226454eeffe parsl.dataflow.memoization INFO: Task 4521 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffdc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4521 try 0 launched on executor multi with executor id 4522 parsl.dataflow.dflow INFO: Standard output for task 4521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ea7a1f44-24f8-4bb4-848f-56f615d4945a_isrForCrosstalkSources_996969_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ea7a1f44-24f8-4bb4-848f-56f615d4945a_isrForCrosstalkSources_996969_45.stderr parsl.dataflow.dflow DEBUG: Task 4522 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4522 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4522 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4522 has memoization hash 1555a88add80a08e3cdde3bffdec66dd parsl.dataflow.memoization INFO: Task 4522 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffdbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4522 try 0 launched on executor multi with executor id 4523 parsl.dataflow.dflow INFO: Standard output for task 4522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/999bb3a6-49de-4cfe-acb9-638b56698a2b_isrForCrosstalkSources_996975_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/999bb3a6-49de-4cfe-acb9-638b56698a2b_isrForCrosstalkSources_996975_52.stderr parsl.dataflow.dflow DEBUG: Task 4523 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4523 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4523 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4523 has memoization hash b1040ee0bbddf4ce72af7f0bae97f60c parsl.dataflow.memoization INFO: Task 4523 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffdb20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4523 try 0 launched on executor multi with executor id 4524 parsl.dataflow.dflow INFO: Standard output for task 4523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/408e36d9-dc10-4d32-96ed-d3aa294588bd_isrForCrosstalkSources_996996_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/408e36d9-dc10-4d32-96ed-d3aa294588bd_isrForCrosstalkSources_996996_19.stderr parsl.dataflow.dflow DEBUG: Task 4524 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4524 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4524 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4524 has memoization hash f29976a957ff354efcfdbed7da9f36af parsl.dataflow.memoization INFO: Task 4524 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4524 try 0 launched on executor multi with executor id 4525 parsl.dataflow.dflow INFO: Standard output for task 4524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/efc1d32c-0ffd-47f8-a954-8dde0c843264_isrForCrosstalkSources_997004_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/efc1d32c-0ffd-47f8-a954-8dde0c843264_isrForCrosstalkSources_997004_2.stderr parsl.dataflow.dflow DEBUG: Task 4525 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4525 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4525 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4525 has memoization hash d184055a3540b24b04207d311d52cd53 parsl.dataflow.memoization INFO: Task 4525 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffda80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4525 try 0 launched on executor multi with executor id 4526 parsl.dataflow.dflow INFO: Standard output for task 4525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/53ec19e0-6dcf-4e5c-83fb-5deee1157f71_isrForCrosstalkSources_996982_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/53ec19e0-6dcf-4e5c-83fb-5deee1157f71_isrForCrosstalkSources_996982_5.stderr parsl.dataflow.dflow DEBUG: Task 4526 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4526 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4526 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4526 has memoization hash f619c80ea3f1e673bf2b69668f15df50 parsl.dataflow.memoization INFO: Task 4526 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4526 try 0 launched on executor multi with executor id 4527 parsl.dataflow.dflow INFO: Standard output for task 4526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/56263901-1e69-4056-862d-1ae1ad736fa6_isrForCrosstalkSources_997015_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/56263901-1e69-4056-862d-1ae1ad736fa6_isrForCrosstalkSources_997015_21.stderr parsl.dataflow.dflow DEBUG: Task 4527 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4527 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4527 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4527 has memoization hash a89c44e7a2974a6bb56583842471b94a parsl.dataflow.memoization INFO: Task 4527 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4527 try 0 launched on executor multi with executor id 4528 parsl.dataflow.dflow INFO: Standard output for task 4527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3a1ee839-4540-46e6-9729-e12b10d4bb3a_isrForCrosstalkSources_996995_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3a1ee839-4540-46e6-9729-e12b10d4bb3a_isrForCrosstalkSources_996995_11.stderr parsl.dataflow.dflow DEBUG: Task 4528 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4528 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4528 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4528 has memoization hash f4c882ffcd7db857e1700e7ffaea0bbe parsl.dataflow.memoization INFO: Task 4528 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4528 try 0 launched on executor multi with executor id 4529 parsl.dataflow.dflow INFO: Standard output for task 4528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/89ea3cf2-cf73-49bb-968d-cabab5862486_isrForCrosstalkSources_996979_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/89ea3cf2-cf73-49bb-968d-cabab5862486_isrForCrosstalkSources_996979_58.stderr parsl.dataflow.dflow DEBUG: Task 4529 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4529 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4529 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4529 has memoization hash 030131a86a68c06be68a7d8521de0e8e parsl.dataflow.memoization INFO: Task 4529 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4529 try 0 launched on executor multi with executor id 4530 parsl.dataflow.dflow INFO: Standard output for task 4529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/664ffb9a-4fbf-4a7d-b5c9-00cac9c19e88_isrForCrosstalkSources_996983_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/664ffb9a-4fbf-4a7d-b5c9-00cac9c19e88_isrForCrosstalkSources_996983_24.stderr parsl.dataflow.dflow DEBUG: Task 4530 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4530 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4530 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4530 has memoization hash ed30302a851e5117fc1cec11f96be6f1 parsl.dataflow.memoization INFO: Task 4530 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4530 try 0 launched on executor multi with executor id 4531 parsl.dataflow.dflow INFO: Standard output for task 4530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c77911a3-ff58-4836-9ede-04a6b06030af_isrForCrosstalkSources_997014_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c77911a3-ff58-4836-9ede-04a6b06030af_isrForCrosstalkSources_997014_14.stderr parsl.dataflow.dflow DEBUG: Task 4531 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4531 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4531 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4531 has memoization hash 734804a58b60c8499ded043fb7361797 parsl.dataflow.memoization INFO: Task 4531 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4531 try 0 launched on executor multi with executor id 4532 parsl.dataflow.dflow INFO: Standard output for task 4531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9e5c3541-745a-4bc2-be70-30dbf300945c_isrForCrosstalkSources_996960_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9e5c3541-745a-4bc2-be70-30dbf300945c_isrForCrosstalkSources_996960_30.stderr parsl.dataflow.dflow DEBUG: Task 4532 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4532 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4532 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4532 has memoization hash bb1d8d6b3604cc6225d5a560b0534190 parsl.dataflow.memoization INFO: Task 4532 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4532 try 0 launched on executor multi with executor id 4533 parsl.dataflow.dflow INFO: Standard output for task 4532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/56dc4c2f-b61c-452f-87b7-5b31aa49c915_isrForCrosstalkSources_996966_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/56dc4c2f-b61c-452f-87b7-5b31aa49c915_isrForCrosstalkSources_996966_7.stderr parsl.dataflow.dflow DEBUG: Task 4533 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4533 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4533 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4533 has memoization hash 06ceb4682193bbcffec9bb84ac0f3c51 parsl.dataflow.memoization INFO: Task 4533 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4533 try 0 launched on executor multi with executor id 4534 parsl.dataflow.dflow INFO: Standard output for task 4533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/ac208d0b-3c0e-4b77-9e15-7b5798da96c0_isrForCrosstalkSources_996995_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/ac208d0b-3c0e-4b77-9e15-7b5798da96c0_isrForCrosstalkSources_996995_54.stderr parsl.dataflow.dflow DEBUG: Task 4534 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4534 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4534 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4534 has memoization hash f46f5fe3942a7615b8ceb3e02b463fd1 parsl.dataflow.memoization INFO: Task 4534 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4534 try 0 launched on executor multi with executor id 4535 parsl.dataflow.dflow INFO: Standard output for task 4534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/0af44847-7ce3-4b2f-baa5-2c8b29b19928_isrForCrosstalkSources_997032_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/0af44847-7ce3-4b2f-baa5-2c8b29b19928_isrForCrosstalkSources_997032_13.stderr parsl.dataflow.dflow DEBUG: Task 4535 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4535 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4535 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4535 has memoization hash a5f38fadb9f12a26433a5f3f291b34b7 parsl.dataflow.memoization INFO: Task 4535 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4535 try 0 launched on executor multi with executor id 4536 parsl.dataflow.dflow INFO: Standard output for task 4535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/8a429e39-34d6-4b12-a5cc-19fc9bf399f0_isrForCrosstalkSources_997000_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/8a429e39-34d6-4b12-a5cc-19fc9bf399f0_isrForCrosstalkSources_997000_33.stderr parsl.dataflow.dflow DEBUG: Task 4536 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4536 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4536 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4536 has memoization hash 94a7ca7774635b2c4567a54112ee4b95 parsl.dataflow.memoization INFO: Task 4536 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4536 try 0 launched on executor multi with executor id 4537 parsl.dataflow.dflow INFO: Standard output for task 4536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a0516b1d-0640-409d-840b-ef6b2fa06d96_isrForCrosstalkSources_996960_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a0516b1d-0640-409d-840b-ef6b2fa06d96_isrForCrosstalkSources_996960_11.stderr parsl.dataflow.dflow DEBUG: Task 4537 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4537 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4537 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4537 has memoization hash a07bf4aec97ad9375a625df6ee5be1fb parsl.dataflow.memoization INFO: Task 4537 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4537 try 0 launched on executor multi with executor id 4538 parsl.dataflow.dflow INFO: Standard output for task 4537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fc736d57-a58b-4a0c-8136-f96c9c7ba207_isrForCrosstalkSources_997012_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fc736d57-a58b-4a0c-8136-f96c9c7ba207_isrForCrosstalkSources_997012_12.stderr parsl.dataflow.dflow DEBUG: Task 4538 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4538 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4538 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4538 has memoization hash a1efc8256e519d41690104e328e89370 parsl.dataflow.memoization INFO: Task 4538 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4538 try 0 launched on executor multi with executor id 4539 parsl.dataflow.dflow INFO: Standard output for task 4538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/58c60c6e-5b94-418f-9a9d-71ae6d28e61b_isrForCrosstalkSources_996984_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/58c60c6e-5b94-418f-9a9d-71ae6d28e61b_isrForCrosstalkSources_996984_27.stderr parsl.dataflow.dflow DEBUG: Task 4539 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4539 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4539 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4539 has memoization hash 2b4e0dfe3e385c80eb1d536282287164 parsl.dataflow.memoization INFO: Task 4539 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4539 try 0 launched on executor multi with executor id 4540 parsl.dataflow.dflow INFO: Standard output for task 4539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a85eca6d-2f55-4539-a9e9-20d928adc8b8_isrForCrosstalkSources_997022_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a85eca6d-2f55-4539-a9e9-20d928adc8b8_isrForCrosstalkSources_997022_46.stderr parsl.dataflow.dflow DEBUG: Task 4540 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4540 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4540 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4540 has memoization hash fa9431838b4e3be91bf788b94c934a90 parsl.dataflow.memoization INFO: Task 4540 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffd120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4540 try 0 launched on executor multi with executor id 4541 parsl.dataflow.dflow INFO: Standard output for task 4540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/174d47ea-18b5-4735-9032-0af0685c38fe_isrForCrosstalkSources_997006_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/174d47ea-18b5-4735-9032-0af0685c38fe_isrForCrosstalkSources_997006_62.stderr parsl.dataflow.dflow DEBUG: Task 4541 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4541 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4541 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4541 has memoization hash 63aeb6dd02e00c5b2bdeb03273a1a089 parsl.dataflow.memoization INFO: Task 4541 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffcf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4541 try 0 launched on executor multi with executor id 4542 parsl.dataflow.dflow INFO: Standard output for task 4541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/377ef795-dd2d-4afc-8b86-b6893f8d9abe_isrForCrosstalkSources_996990_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/377ef795-dd2d-4afc-8b86-b6893f8d9abe_isrForCrosstalkSources_996990_43.stderr parsl.dataflow.dflow DEBUG: Task 4542 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4542 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4542 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4542 has memoization hash 4274fd115cae6589c2b80eb5c1865481 parsl.dataflow.memoization INFO: Task 4542 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4542 try 0 launched on executor multi with executor id 4543 parsl.dataflow.dflow INFO: Standard output for task 4542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/cd684355-0e3c-475e-a3c1-340064b1379b_isrForCrosstalkSources_996962_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/cd684355-0e3c-475e-a3c1-340064b1379b_isrForCrosstalkSources_996962_2.stderr parsl.dataflow.dflow DEBUG: Task 4543 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4543 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4543 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4543 has memoization hash 72c89e7e0287855c76ef53b181602ef2 parsl.dataflow.memoization INFO: Task 4543 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4543 try 0 launched on executor multi with executor id 4544 parsl.dataflow.dflow INFO: Standard output for task 4543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/21034041-8abe-459b-a790-a0403090c5e5_isrForCrosstalkSources_996990_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/21034041-8abe-459b-a790-a0403090c5e5_isrForCrosstalkSources_996990_31.stderr parsl.dataflow.dflow DEBUG: Task 4544 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4544 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4544 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4544 has memoization hash ab10a4b6d2b719b593d039a70d72484f parsl.dataflow.memoization INFO: Task 4544 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4544 try 0 launched on executor multi with executor id 4545 parsl.dataflow.dflow INFO: Standard output for task 4544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/cc3d4e97-33cb-4ee7-9b79-36a8db617b64_isrForCrosstalkSources_996998_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/cc3d4e97-33cb-4ee7-9b79-36a8db617b64_isrForCrosstalkSources_996998_41.stderr parsl.dataflow.dflow DEBUG: Task 4545 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4545 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4545 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4545 has memoization hash 0eeedea1fb93c86d38f28f94c12f9b91 parsl.dataflow.memoization INFO: Task 4545 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4545 try 0 launched on executor multi with executor id 4546 parsl.dataflow.dflow INFO: Standard output for task 4545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ea690f34-da4f-4b46-84e8-b56e9cc9186d_isrForCrosstalkSources_996978_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ea690f34-da4f-4b46-84e8-b56e9cc9186d_isrForCrosstalkSources_996978_6.stderr parsl.dataflow.dflow DEBUG: Task 4546 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4546 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4546 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4546 has memoization hash 0dd6ae8fe5938a409163ae0f1678c2ac parsl.dataflow.memoization INFO: Task 4546 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4546 try 0 launched on executor multi with executor id 4547 parsl.dataflow.dflow INFO: Standard output for task 4546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/0988359b-2882-4c14-9b32-db46f43b1239_isrForCrosstalkSources_997022_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/0988359b-2882-4c14-9b32-db46f43b1239_isrForCrosstalkSources_997022_15.stderr parsl.dataflow.dflow DEBUG: Task 4547 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4547 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4547 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4547 has memoization hash 53c266a5e05cd53cb6a50ed7e8ad6a42 parsl.dataflow.memoization INFO: Task 4547 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4547 try 0 launched on executor multi with executor id 4548 parsl.dataflow.dflow INFO: Standard output for task 4547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/bc03580f-8c79-49bc-82a4-daaeb08324cf_isrForCrosstalkSources_996954_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/bc03580f-8c79-49bc-82a4-daaeb08324cf_isrForCrosstalkSources_996954_8.stderr parsl.dataflow.dflow DEBUG: Task 4548 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4548 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4548 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4548 has memoization hash 53e4862f87000b3bee8a4fc6df24ab0b parsl.dataflow.memoization INFO: Task 4548 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4548 try 0 launched on executor multi with executor id 4549 parsl.dataflow.dflow INFO: Standard output for task 4548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a10e2f9d-c1e1-4c71-82b8-f6ad37ee0f38_isrForCrosstalkSources_996983_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a10e2f9d-c1e1-4c71-82b8-f6ad37ee0f38_isrForCrosstalkSources_996983_22.stderr parsl.dataflow.dflow DEBUG: Task 4549 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4549 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4549 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4549 has memoization hash 266e47ac374a216e75205bac80f9368d parsl.dataflow.memoization INFO: Task 4549 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4549 try 0 launched on executor multi with executor id 4550 parsl.dataflow.dflow INFO: Standard output for task 4549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/52d3ef4c-52f5-4085-b478-20d703d2eeb6_isrForCrosstalkSources_996969_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/52d3ef4c-52f5-4085-b478-20d703d2eeb6_isrForCrosstalkSources_996969_10.stderr parsl.dataflow.dflow DEBUG: Task 4550 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4550 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4550 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4550 has memoization hash 5d044ebe77842088aa6aeb75a6997c08 parsl.dataflow.memoization INFO: Task 4550 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4550 try 0 launched on executor multi with executor id 4551 parsl.dataflow.dflow INFO: Standard output for task 4550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/83043a46-9526-4f11-9ddb-499374d37a58_isrForCrosstalkSources_997022_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/83043a46-9526-4f11-9ddb-499374d37a58_isrForCrosstalkSources_997022_25.stderr parsl.dataflow.dflow DEBUG: Task 4551 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4551 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4551 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4551 has memoization hash 418c52606167698cc76d261decb3090e parsl.dataflow.memoization INFO: Task 4551 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4551 try 0 launched on executor multi with executor id 4552 parsl.dataflow.dflow INFO: Standard output for task 4551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ade2383b-381f-4822-abb1-791115a6b1c7_isrForCrosstalkSources_997016_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ade2383b-381f-4822-abb1-791115a6b1c7_isrForCrosstalkSources_997016_53.stderr parsl.dataflow.dflow DEBUG: Task 4552 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4552 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4552 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4552 has memoization hash a7f9723650c759f0f245d65e89395122 parsl.dataflow.memoization INFO: Task 4552 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffeac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4552 try 0 launched on executor multi with executor id 4553 parsl.dataflow.dflow INFO: Standard output for task 4552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/37b0a483-ef97-46f0-9cb3-8933a6aa3b76_isrForCrosstalkSources_997018_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/37b0a483-ef97-46f0-9cb3-8933a6aa3b76_isrForCrosstalkSources_997018_7.stderr parsl.dataflow.dflow DEBUG: Task 4553 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4553 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4553 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4553 has memoization hash f70699e4962fe8edb63af6ac169157c1 parsl.dataflow.memoization INFO: Task 4553 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffeb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4553 try 0 launched on executor multi with executor id 4554 parsl.dataflow.dflow INFO: Standard output for task 4553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/511aacff-c034-4b3f-b93b-b58fb1ed18fd_isrForCrosstalkSources_997010_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/511aacff-c034-4b3f-b93b-b58fb1ed18fd_isrForCrosstalkSources_997010_53.stderr parsl.dataflow.dflow DEBUG: Task 4554 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4554 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4554 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4554 has memoization hash be69dedcb457c71437611853d968923a parsl.dataflow.memoization INFO: Task 4554 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffeca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4554 try 0 launched on executor multi with executor id 4555 parsl.dataflow.dflow INFO: Standard output for task 4554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/f3fd096f-1b69-4265-99e8-1a6212065fb4_isrForCrosstalkSources_997006_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/f3fd096f-1b69-4265-99e8-1a6212065fb4_isrForCrosstalkSources_997006_50.stderr parsl.dataflow.dflow DEBUG: Task 4555 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4555 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4555 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4555 has memoization hash d697ba71ac65211b6a38df06ff683e01 parsl.dataflow.memoization INFO: Task 4555 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4555 try 0 launched on executor multi with executor id 4556 parsl.dataflow.dflow INFO: Standard output for task 4555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/842ba0fa-1042-465b-8714-3be19d3ac021_isrForCrosstalkSources_996969_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/842ba0fa-1042-465b-8714-3be19d3ac021_isrForCrosstalkSources_996969_51.stderr parsl.dataflow.dflow DEBUG: Task 4556 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4556 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4556 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4556 has memoization hash 6b15884c794519c0141bd77b624394c7 parsl.dataflow.memoization INFO: Task 4556 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4556 try 0 launched on executor multi with executor id 4557 parsl.dataflow.dflow INFO: Standard output for task 4556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/48a0e81c-c4cd-4658-adaa-9c8999a7c059_isrForCrosstalkSources_997006_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/48a0e81c-c4cd-4658-adaa-9c8999a7c059_isrForCrosstalkSources_997006_48.stderr parsl.dataflow.dflow DEBUG: Task 4557 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4557 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4557 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4557 has memoization hash ab4b18cfc6a6ac270780eabdc592909d parsl.dataflow.memoization INFO: Task 4557 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4557 try 0 launched on executor multi with executor id 4558 parsl.dataflow.dflow INFO: Standard output for task 4557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/3a0f1645-4446-4648-8f83-5c9b3508c0e0_isrForCrosstalkSources_996993_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/3a0f1645-4446-4648-8f83-5c9b3508c0e0_isrForCrosstalkSources_996993_32.stderr parsl.dataflow.dflow DEBUG: Task 4558 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4558 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4558 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4558 has memoization hash bb904b7ec5e8000859f3c7d0406d7933 parsl.dataflow.memoization INFO: Task 4558 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4558 try 0 launched on executor multi with executor id 4559 parsl.dataflow.dflow INFO: Standard output for task 4558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ba3976dd-b02f-42fb-9b08-99f854b0db2b_isrForCrosstalkSources_996994_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ba3976dd-b02f-42fb-9b08-99f854b0db2b_isrForCrosstalkSources_996994_35.stderr parsl.dataflow.dflow DEBUG: Task 4559 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4559 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4559 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4559 has memoization hash 0ac2ae1ae073e7d79b2233d39edcda35 parsl.dataflow.memoization INFO: Task 4559 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffefc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4559 try 0 launched on executor multi with executor id 4560 parsl.dataflow.dflow INFO: Standard output for task 4559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/00249d29-ead9-43de-9ab7-8de9816cd442_isrForCrosstalkSources_997016_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/00249d29-ead9-43de-9ab7-8de9816cd442_isrForCrosstalkSources_997016_4.stderr parsl.dataflow.dflow DEBUG: Task 4560 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4560 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4560 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4560 has memoization hash 5853ded2b62883b22da51ef59c3b1635 parsl.dataflow.memoization INFO: Task 4560 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4560 try 0 launched on executor multi with executor id 4561 parsl.dataflow.dflow INFO: Standard output for task 4560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3dea8121-7481-422b-a458-57f8755ae1d4_isrForCrosstalkSources_997014_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3dea8121-7481-422b-a458-57f8755ae1d4_isrForCrosstalkSources_997014_5.stderr parsl.dataflow.dflow DEBUG: Task 4561 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4561 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4561 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4561 has memoization hash 3645470017304a41950f17bd53ba6afe parsl.dataflow.memoization INFO: Task 4561 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4561 try 0 launched on executor multi with executor id 4562 parsl.dataflow.dflow INFO: Standard output for task 4561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ebf34ba6-ff4e-421b-b33c-65d98958622f_isrForCrosstalkSources_996959_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ebf34ba6-ff4e-421b-b33c-65d98958622f_isrForCrosstalkSources_996959_62.stderr parsl.dataflow.dflow DEBUG: Task 4562 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4562 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4562 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4562 has memoization hash 1d70dbe88092811040a3be2f659cdcf0 parsl.dataflow.memoization INFO: Task 4562 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:09 proc_lsst.multi:146 [INFO] found job 17074837 in provider astro proc_lsst.multi INFO: found job 17074837 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4562 try 0 launched on executor multi with executor id 4563 parsl.dataflow.dflow INFO: Standard output for task 4562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/41b841f3-82f0-465f-a8c1-a5385f4f403d_isrForCrosstalkSources_997033_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/41b841f3-82f0-465f-a8c1-a5385f4f403d_isrForCrosstalkSources_997033_12.stderr parsl.dataflow.dflow DEBUG: Task 4563 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4563 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4563 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4563 has memoization hash 144638e36e239a788b8288a9a0eba7b1 parsl.dataflow.memoization INFO: Task 4563 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767f87ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4563 try 0 launched on executor multi with executor id 4564 parsl.dataflow.dflow INFO: Standard output for task 4563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c0ce3481-2983-4199-8802-4a8294496656_isrForCrosstalkSources_997004_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c0ce3481-2983-4199-8802-4a8294496656_isrForCrosstalkSources_997004_29.stderr parsl.dataflow.dflow DEBUG: Task 4564 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4564 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4564 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4564 has memoization hash d205e9e7e201aae43d2f9aac2300005b parsl.dataflow.memoization INFO: Task 4564 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4564 try 0 launched on executor multi with executor id 4565 parsl.dataflow.dflow INFO: Standard output for task 4564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8cde139f-f98f-4c37-a207-8c1bcbd69183_isrForCrosstalkSources_996997_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8cde139f-f98f-4c37-a207-8c1bcbd69183_isrForCrosstalkSources_996997_22.stderr parsl.dataflow.dflow DEBUG: Task 4565 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4565 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4565 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4565 has memoization hash d4e3934c134307f04e818d6d55107504 parsl.dataflow.memoization INFO: Task 4565 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e965c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4565 try 0 launched on executor multi with executor id 4566 parsl.dataflow.dflow INFO: Standard output for task 4565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/bbc082a9-e128-4598-a6eb-b06c111b469b_isrForCrosstalkSources_996999_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/bbc082a9-e128-4598-a6eb-b06c111b469b_isrForCrosstalkSources_996999_11.stderr parsl.dataflow.dflow DEBUG: Task 4566 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4566 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4566 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4566 has memoization hash 182f20eb507ce6b6b809349ee0bd5efd parsl.dataflow.memoization INFO: Task 4566 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4566 try 0 launched on executor multi with executor id 4567 parsl.dataflow.dflow INFO: Standard output for task 4566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3d23b953-c7e0-42fe-aae7-6c74be9b5b93_isrForCrosstalkSources_996968_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3d23b953-c7e0-42fe-aae7-6c74be9b5b93_isrForCrosstalkSources_996968_61.stderr parsl.dataflow.dflow DEBUG: Task 4567 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4567 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4567 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4567 has memoization hash 91db5a4814bfd89fa41ce15851b84b1a parsl.dataflow.memoization INFO: Task 4567 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4567 try 0 launched on executor multi with executor id 4568 parsl.dataflow.dflow INFO: Standard output for task 4567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f5a45dc2-992d-4dc6-8171-8b88896f052e_isrForCrosstalkSources_996990_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f5a45dc2-992d-4dc6-8171-8b88896f052e_isrForCrosstalkSources_996990_11.stderr parsl.dataflow.dflow DEBUG: Task 4568 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4568 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4568 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4568 has memoization hash a5d936306ae0c72008da98a7c9cac564 parsl.dataflow.memoization INFO: Task 4568 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4568 try 0 launched on executor multi with executor id 4569 parsl.dataflow.dflow INFO: Standard output for task 4568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/a6027c55-988b-4ddb-adf9-8754c79f049a_isrForCrosstalkSources_996963_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/a6027c55-988b-4ddb-adf9-8754c79f049a_isrForCrosstalkSources_996963_8.stderr parsl.dataflow.dflow DEBUG: Task 4569 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4569 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4569 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4569 has memoization hash 4ea1f1b53992bb2de95b0f877ab2ab97 parsl.dataflow.memoization INFO: Task 4569 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4569 try 0 launched on executor multi with executor id 4570 parsl.dataflow.dflow INFO: Standard output for task 4569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/49cf20ee-ac31-463d-84d7-08b43d2dc1a4_isrForCrosstalkSources_996970_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/49cf20ee-ac31-463d-84d7-08b43d2dc1a4_isrForCrosstalkSources_996970_46.stderr parsl.dataflow.dflow DEBUG: Task 4570 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4570 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4570 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4570 has memoization hash 4c4cddb4177cf16a2cb5cc55218d8268 parsl.dataflow.memoization INFO: Task 4570 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4570 try 0 launched on executor multi with executor id 4571 parsl.dataflow.dflow INFO: Standard output for task 4570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/b1657f4d-4ea2-4727-9619-04384a563a3c_isrForCrosstalkSources_996987_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/b1657f4d-4ea2-4727-9619-04384a563a3c_isrForCrosstalkSources_996987_27.stderr parsl.dataflow.dflow DEBUG: Task 4571 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4571 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4571 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4571 has memoization hash d602f641b35c99a7d566ea903bee8e56 parsl.dataflow.memoization INFO: Task 4571 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4571 try 0 launched on executor multi with executor id 4572 parsl.dataflow.dflow INFO: Standard output for task 4571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5296c4f8-8ca6-45a7-a50c-053bc38a4655_isrForCrosstalkSources_996979_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5296c4f8-8ca6-45a7-a50c-053bc38a4655_isrForCrosstalkSources_996979_34.stderr parsl.dataflow.dflow DEBUG: Task 4572 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4572 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4572 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4572 has memoization hash fb7412ae894769bd83403bc242b2b17e parsl.dataflow.memoization INFO: Task 4572 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4572 try 0 launched on executor multi with executor id 4573 parsl.dataflow.dflow INFO: Standard output for task 4572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ab1118d9-3370-414c-9610-d947bc966921_isrForCrosstalkSources_997005_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ab1118d9-3370-414c-9610-d947bc966921_isrForCrosstalkSources_997005_17.stderr parsl.dataflow.dflow DEBUG: Task 4573 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4573 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4573 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4573 has memoization hash 792fb5fc72c882889a3354a97da7aa62 parsl.dataflow.memoization INFO: Task 4573 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4573 try 0 launched on executor multi with executor id 4574 parsl.dataflow.dflow INFO: Standard output for task 4573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0e0e6ba7-001a-4c1d-aa5f-646de60312c1_isrForCrosstalkSources_996983_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0e0e6ba7-001a-4c1d-aa5f-646de60312c1_isrForCrosstalkSources_996983_21.stderr parsl.dataflow.dflow DEBUG: Task 4574 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4574 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4574 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4574 has memoization hash ae2e421c8ce2cde5a4243426d37df008 parsl.dataflow.memoization INFO: Task 4574 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4574 try 0 launched on executor multi with executor id 4575 parsl.dataflow.dflow INFO: Standard output for task 4574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/48a309b6-8602-45e0-8f4a-3bd661a7449b_isrForCrosstalkSources_997015_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/48a309b6-8602-45e0-8f4a-3bd661a7449b_isrForCrosstalkSources_997015_43.stderr parsl.dataflow.dflow DEBUG: Task 4575 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4575 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4575 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4575 has memoization hash c9bc355141feda8c9ba60c42d6fd180e parsl.dataflow.memoization INFO: Task 4575 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4575 try 0 launched on executor multi with executor id 4576 parsl.dataflow.dflow INFO: Standard output for task 4575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b64bf3cb-4a6f-43d2-b26e-965720dfd4f2_isrForCrosstalkSources_997015_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b64bf3cb-4a6f-43d2-b26e-965720dfd4f2_isrForCrosstalkSources_997015_18.stderr parsl.dataflow.dflow DEBUG: Task 4576 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4576 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4576 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4576 has memoization hash 0e879f73a06ce3e7d0234c175a1f9fa0 parsl.dataflow.memoization INFO: Task 4576 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4576 try 0 launched on executor multi with executor id 4577 parsl.dataflow.dflow INFO: Standard output for task 4576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5d642915-337b-4209-9cea-7e205c11e354_isrForCrosstalkSources_997018_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5d642915-337b-4209-9cea-7e205c11e354_isrForCrosstalkSources_997018_14.stderr parsl.dataflow.dflow DEBUG: Task 4577 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4577 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4577 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4577 has memoization hash 56586b9571ff74e9521076831e01b944 parsl.dataflow.memoization INFO: Task 4577 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4577 try 0 launched on executor multi with executor id 4578 parsl.dataflow.dflow INFO: Standard output for task 4577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/473e6caa-524f-4a66-aa68-de539ca02081_isrForCrosstalkSources_996971_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/473e6caa-524f-4a66-aa68-de539ca02081_isrForCrosstalkSources_996971_15.stderr parsl.dataflow.dflow DEBUG: Task 4578 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4578 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4578 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4578 has memoization hash 1a353cd25d7a99359c353e40c981d5bc parsl.dataflow.memoization INFO: Task 4578 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fff9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4578 try 0 launched on executor multi with executor id 4579 parsl.dataflow.dflow INFO: Standard output for task 4578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/39915e4b-74bb-4e5e-97bf-d55c41c880d3_isrForCrosstalkSources_996978_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/39915e4b-74bb-4e5e-97bf-d55c41c880d3_isrForCrosstalkSources_996978_53.stderr parsl.dataflow.dflow DEBUG: Task 4579 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4579 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4579 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4579 has memoization hash 154ba2ffb787549ad050321b340bd6f0 parsl.dataflow.memoization INFO: Task 4579 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fffa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4579 try 0 launched on executor multi with executor id 4580 parsl.dataflow.dflow INFO: Standard output for task 4579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5e64222b-ddda-4c08-bf33-2367fd6d137b_isrForCrosstalkSources_996972_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5e64222b-ddda-4c08-bf33-2367fd6d137b_isrForCrosstalkSources_996972_60.stderr parsl.dataflow.dflow DEBUG: Task 4580 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4580 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4580 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4580 has memoization hash 48b183829b3d7b4514f60cbfa10ca007 parsl.dataflow.memoization INFO: Task 4580 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fffc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4580 try 0 launched on executor multi with executor id 4581 parsl.dataflow.dflow INFO: Standard output for task 4580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3a2c8306-4be0-45b4-9bf2-7a68b701c62e_isrForCrosstalkSources_997001_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3a2c8306-4be0-45b4-9bf2-7a68b701c62e_isrForCrosstalkSources_997001_27.stderr parsl.dataflow.dflow DEBUG: Task 4581 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4581 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4581 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4581 has memoization hash 7fb9e6014f622689e234a3b663c65367 parsl.dataflow.memoization INFO: Task 4581 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fffb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4581 try 0 launched on executor multi with executor id 4582 parsl.dataflow.dflow INFO: Standard output for task 4581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3255b58c-40ff-4b43-b764-d53f5f4519b9_isrForCrosstalkSources_996959_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3255b58c-40ff-4b43-b764-d53f5f4519b9_isrForCrosstalkSources_996959_25.stderr parsl.dataflow.dflow DEBUG: Task 4582 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4582 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4582 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4582 has memoization hash 0967b21cb2913408c6240e0a573834b8 parsl.dataflow.memoization INFO: Task 4582 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fffce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4582 try 0 launched on executor multi with executor id 4583 parsl.dataflow.dflow INFO: Standard output for task 4582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/920faa68-da1a-4017-af08-84018f26f304_isrForCrosstalkSources_996954_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/920faa68-da1a-4017-af08-84018f26f304_isrForCrosstalkSources_996954_13.stderr parsl.dataflow.dflow DEBUG: Task 4583 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4583 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4583 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4583 has memoization hash 7df9a1c62d9e1041ee23abb852c5edc3 parsl.dataflow.memoization INFO: Task 4583 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fffd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4583 try 0 launched on executor multi with executor id 4584 parsl.dataflow.dflow INFO: Standard output for task 4583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ad1bc874-1884-4e22-bbb9-b7fecac502a6_isrForCrosstalkSources_996991_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ad1bc874-1884-4e22-bbb9-b7fecac502a6_isrForCrosstalkSources_996991_30.stderr parsl.dataflow.dflow DEBUG: Task 4584 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4584 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4584 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4584 has memoization hash 8aa705bd23354db82416730553b77c91 parsl.dataflow.memoization INFO: Task 4584 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fffba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4584 try 0 launched on executor multi with executor id 4585 parsl.dataflow.dflow INFO: Standard output for task 4584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1d95aa2-5efe-491c-a963-b48f4ee12adb_isrForCrosstalkSources_996975_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1d95aa2-5efe-491c-a963-b48f4ee12adb_isrForCrosstalkSources_996975_59.stderr parsl.dataflow.dflow DEBUG: Task 4585 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4585 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4585 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4585 has memoization hash a8060b1819d90fec219647dcf16ceaea parsl.dataflow.memoization INFO: Task 4585 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4585 try 0 launched on executor multi with executor id 4586 parsl.dataflow.dflow INFO: Standard output for task 4585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/7a2090e3-62a3-4e19-a987-c59ed6c4030c_isrForCrosstalkSources_996958_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/7a2090e3-62a3-4e19-a987-c59ed6c4030c_isrForCrosstalkSources_996958_55.stderr parsl.dataflow.dflow DEBUG: Task 4586 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4586 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4586 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4586 has memoization hash be86b80138558ccc8027a76a12953ec5 parsl.dataflow.memoization INFO: Task 4586 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fffe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4586 try 0 launched on executor multi with executor id 4587 parsl.dataflow.dflow INFO: Standard output for task 4586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/dac41d00-20ca-4ef7-8fa6-2cde21917c11_isrForCrosstalkSources_996993_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/dac41d00-20ca-4ef7-8fa6-2cde21917c11_isrForCrosstalkSources_996993_18.stderr parsl.dataflow.dflow DEBUG: Task 4587 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4587 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4587 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4587 has memoization hash c0ffe22e59a29306d6ce13a52147767d parsl.dataflow.memoization INFO: Task 4587 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767ffe200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4587 try 0 launched on executor multi with executor id 4588 parsl.dataflow.dflow INFO: Standard output for task 4587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8b68af6a-c1b2-4e72-88d2-e426b74e005a_isrForCrosstalkSources_996967_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8b68af6a-c1b2-4e72-88d2-e426b74e005a_isrForCrosstalkSources_996967_17.stderr parsl.dataflow.dflow DEBUG: Task 4588 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4588 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4588 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4588 has memoization hash 11c02431c644151c16886f18ba7f0129 parsl.dataflow.memoization INFO: Task 4588 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4588 try 0 launched on executor multi with executor id 4589 parsl.dataflow.dflow INFO: Standard output for task 4588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ad65a8d0-a20a-44fa-a8d3-451a65814c11_isrForCrosstalkSources_997028_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ad65a8d0-a20a-44fa-a8d3-451a65814c11_isrForCrosstalkSources_997028_53.stderr parsl.dataflow.dflow DEBUG: Task 4589 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4589 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4589 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4589 has memoization hash 95718e27bb758984c6cdd2ed4a4f30f8 parsl.dataflow.memoization INFO: Task 4589 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e940e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4589 try 0 launched on executor multi with executor id 4590 parsl.dataflow.dflow INFO: Standard output for task 4589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/ac19a2a3-7465-4f7e-b6b0-0a1e1c8f77ff_isrForCrosstalkSources_996971_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/ac19a2a3-7465-4f7e-b6b0-0a1e1c8f77ff_isrForCrosstalkSources_996971_43.stderr parsl.dataflow.dflow DEBUG: Task 4590 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4590 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4590 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4590 has memoization hash 838ce56f89136b3c0c7fec1a06d25070 parsl.dataflow.memoization INFO: Task 4590 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4590 try 0 launched on executor multi with executor id 4591 parsl.dataflow.dflow INFO: Standard output for task 4590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/36f293b7-d472-4f23-8955-1c963fac7afa_isrForCrosstalkSources_997023_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/36f293b7-d472-4f23-8955-1c963fac7afa_isrForCrosstalkSources_997023_38.stderr parsl.dataflow.dflow DEBUG: Task 4591 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4591 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4591 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4591 has memoization hash fd0ab147be7080653f737470bc4c4f83 parsl.dataflow.memoization INFO: Task 4591 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4591 try 0 launched on executor multi with executor id 4592 parsl.dataflow.dflow INFO: Standard output for task 4591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/16727007-99ff-4806-9d3d-9a60e453b3f5_isrForCrosstalkSources_996981_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/16727007-99ff-4806-9d3d-9a60e453b3f5_isrForCrosstalkSources_996981_22.stderr parsl.dataflow.dflow DEBUG: Task 4592 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4592 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4592 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4592 has memoization hash df59456d0b7e936ddf04474f579127ac parsl.dataflow.memoization INFO: Task 4592 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4592 try 0 launched on executor multi with executor id 4593 parsl.dataflow.dflow INFO: Standard output for task 4592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/db7fa4dc-eada-4b0d-8de0-3d5960309b9b_isrForCrosstalkSources_996998_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/db7fa4dc-eada-4b0d-8de0-3d5960309b9b_isrForCrosstalkSources_996998_29.stderr parsl.dataflow.dflow DEBUG: Task 4593 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4593 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4593 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4593 has memoization hash 04ada73a6cb85e4a354c71b14433acec parsl.dataflow.memoization INFO: Task 4593 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e942c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4593 try 0 launched on executor multi with executor id 4594 parsl.dataflow.dflow INFO: Standard output for task 4593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f9804382-df39-4d69-8705-e519efb288b5_isrForCrosstalkSources_997019_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f9804382-df39-4d69-8705-e519efb288b5_isrForCrosstalkSources_997019_24.stderr parsl.dataflow.dflow DEBUG: Task 4594 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4594 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4594 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4594 has memoization hash 8d9ab6ce64b789f0c6818a81bb8e06b4 parsl.dataflow.memoization INFO: Task 4594 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4594 try 0 launched on executor multi with executor id 4595 parsl.dataflow.dflow INFO: Standard output for task 4594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e0f5570a-40a5-4d9d-8af2-a9565d0d871c_isrForCrosstalkSources_997026_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e0f5570a-40a5-4d9d-8af2-a9565d0d871c_isrForCrosstalkSources_997026_43.stderr parsl.dataflow.dflow DEBUG: Task 4595 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4595 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4595 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4595 has memoization hash 1b9006296be7f60879d8340020634d17 parsl.dataflow.memoization INFO: Task 4595 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4595 try 0 launched on executor multi with executor id 4596 parsl.dataflow.dflow INFO: Standard output for task 4595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/6b3275f1-9788-463a-8772-24b90b3e4251_isrForCrosstalkSources_996999_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/6b3275f1-9788-463a-8772-24b90b3e4251_isrForCrosstalkSources_996999_14.stderr parsl.dataflow.dflow DEBUG: Task 4596 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4596 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4596 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4596 has memoization hash e5c4756b0d014e3bee1a8b55e5b71001 parsl.dataflow.memoization INFO: Task 4596 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4596 try 0 launched on executor multi with executor id 4597 parsl.dataflow.dflow INFO: Standard output for task 4596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cac4b099-ab73-46a6-a406-37fc67623edb_isrForCrosstalkSources_996978_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cac4b099-ab73-46a6-a406-37fc67623edb_isrForCrosstalkSources_996978_7.stderr parsl.dataflow.dflow DEBUG: Task 4597 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4597 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4597 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4597 has memoization hash b9b8b395647d69c64b9e9145732e8fbd parsl.dataflow.memoization INFO: Task 4597 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e944a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4597 try 0 launched on executor multi with executor id 4598 parsl.dataflow.dflow INFO: Standard output for task 4597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/c5835ab0-786a-4949-9915-f57efd1fa9df_isrForCrosstalkSources_996956_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/c5835ab0-786a-4949-9915-f57efd1fa9df_isrForCrosstalkSources_996956_3.stderr parsl.dataflow.dflow DEBUG: Task 4598 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4598 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4598 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4598 has memoization hash d05846e90c27bab4a91ff2aff7b98a7e parsl.dataflow.memoization INFO: Task 4598 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e945e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4598 try 0 launched on executor multi with executor id 4599 parsl.dataflow.dflow INFO: Standard output for task 4598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/a5f4017b-055e-46bd-bbe7-13b578a6be49_isrForCrosstalkSources_997030_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/a5f4017b-055e-46bd-bbe7-13b578a6be49_isrForCrosstalkSources_997030_47.stderr parsl.dataflow.dflow DEBUG: Task 4599 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4599 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4599 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4599 has memoization hash 0bf19f52c921533ad85975ce240fca92 parsl.dataflow.memoization INFO: Task 4599 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4599 try 0 launched on executor multi with executor id 4600 parsl.dataflow.dflow INFO: Standard output for task 4599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/7f0c9631-85fd-48f5-aacb-9c7833a9ac95_isrForCrosstalkSources_996976_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/7f0c9631-85fd-48f5-aacb-9c7833a9ac95_isrForCrosstalkSources_996976_45.stderr parsl.dataflow.dflow DEBUG: Task 4600 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4600 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4600 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4600 has memoization hash af21d2dbe97d8c1adaf82fcdde6e3013 parsl.dataflow.memoization INFO: Task 4600 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4600 try 0 launched on executor multi with executor id 4601 parsl.dataflow.dflow INFO: Standard output for task 4600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/39c872d3-7ef2-4014-8fce-dd2cea52f3b3_isrForCrosstalkSources_996979_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/39c872d3-7ef2-4014-8fce-dd2cea52f3b3_isrForCrosstalkSources_996979_7.stderr parsl.dataflow.dflow DEBUG: Task 4601 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4601 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4601 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4601 has memoization hash 630640bcb25ae8baf81a1fe615c694d5 parsl.dataflow.memoization INFO: Task 4601 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4601 try 0 launched on executor multi with executor id 4602 parsl.dataflow.dflow INFO: Standard output for task 4601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/24a0ecae-6783-4f1c-a24c-7b1bee977b79_isrForCrosstalkSources_996998_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/24a0ecae-6783-4f1c-a24c-7b1bee977b79_isrForCrosstalkSources_996998_5.stderr parsl.dataflow.dflow DEBUG: Task 4602 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4602 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4602 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4602 has memoization hash c42e8cc43ebaa061c9506edabdb9ce21 parsl.dataflow.memoization INFO: Task 4602 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e947c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4602 try 0 launched on executor multi with executor id 4603 parsl.dataflow.dflow INFO: Standard output for task 4602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/210667da-4692-4534-94ce-9b6257aa4abc_isrForCrosstalkSources_997023_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/210667da-4692-4534-94ce-9b6257aa4abc_isrForCrosstalkSources_997023_20.stderr parsl.dataflow.dflow DEBUG: Task 4603 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4603 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4603 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4603 has memoization hash dccb6e5f77f1c7b36d5cd0aab9bd7839 parsl.dataflow.memoization INFO: Task 4603 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e949a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4603 try 0 launched on executor multi with executor id 4604 parsl.dataflow.dflow INFO: Standard output for task 4603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/aa891d6c-653e-4c12-9e9f-dbf5341293aa_isrForCrosstalkSources_996991_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/aa891d6c-653e-4c12-9e9f-dbf5341293aa_isrForCrosstalkSources_996991_22.stderr parsl.dataflow.dflow DEBUG: Task 4604 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4604 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4604 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4604 has memoization hash 040bb792ad88026e1ed2a9f207333dd6 parsl.dataflow.memoization INFO: Task 4604 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4604 try 0 launched on executor multi with executor id 4605 parsl.dataflow.dflow INFO: Standard output for task 4604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fdd00b2c-a9a6-4969-bc12-1b0c223e28e2_isrForCrosstalkSources_997001_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fdd00b2c-a9a6-4969-bc12-1b0c223e28e2_isrForCrosstalkSources_997001_32.stderr parsl.dataflow.dflow DEBUG: Task 4605 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4605 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4605 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4605 has memoization hash 4ed778cac6bf5afbd3db378327cd9866 parsl.dataflow.memoization INFO: Task 4605 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4605 try 0 launched on executor multi with executor id 4606 parsl.dataflow.dflow INFO: Standard output for task 4605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/da0fd872-8cd2-47b9-9c31-1b9bd9d7b66f_isrForCrosstalkSources_996965_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/da0fd872-8cd2-47b9-9c31-1b9bd9d7b66f_isrForCrosstalkSources_996965_26.stderr parsl.dataflow.dflow DEBUG: Task 4606 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4606 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4606 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4606 has memoization hash f4e76ce0a8f7ce20c9b746fcb6e20b21 parsl.dataflow.memoization INFO: Task 4606 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4606 try 0 launched on executor multi with executor id 4607 parsl.dataflow.dflow INFO: Standard output for task 4606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a9632314-8900-465f-850c-499c4a53c152_isrForCrosstalkSources_996952_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a9632314-8900-465f-850c-499c4a53c152_isrForCrosstalkSources_996952_19.stderr parsl.dataflow.dflow DEBUG: Task 4607 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4607 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4607 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4607 has memoization hash 620ee8c3f245c553f471b54c5bb26553 parsl.dataflow.memoization INFO: Task 4607 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4607 try 0 launched on executor multi with executor id 4608 parsl.dataflow.dflow INFO: Standard output for task 4607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2b176645-17a9-41fe-a28b-61fdb9466480_isrForCrosstalkSources_997005_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2b176645-17a9-41fe-a28b-61fdb9466480_isrForCrosstalkSources_997005_53.stderr parsl.dataflow.dflow DEBUG: Task 4608 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4608 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4608 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4608 has memoization hash edc86d86327bf793113d86888df2122b parsl.dataflow.memoization INFO: Task 4608 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4608 try 0 launched on executor multi with executor id 4609 parsl.dataflow.dflow INFO: Standard output for task 4608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/e1bc9a66-f6be-4a48-bd9b-b8250f8f2202_isrForCrosstalkSources_996971_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/e1bc9a66-f6be-4a48-bd9b-b8250f8f2202_isrForCrosstalkSources_996971_31.stderr parsl.dataflow.dflow DEBUG: Task 4609 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4609 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4609 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4609 has memoization hash 5ab893faf024548e01da805a371649e9 parsl.dataflow.memoization INFO: Task 4609 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4609 try 0 launched on executor multi with executor id 4610 parsl.dataflow.dflow INFO: Standard output for task 4609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/05650d05-ed1f-4a92-911f-4339ca749ff4_isrForCrosstalkSources_996974_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/05650d05-ed1f-4a92-911f-4339ca749ff4_isrForCrosstalkSources_996974_62.stderr parsl.dataflow.dflow DEBUG: Task 4610 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4610 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4610 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4610 has memoization hash 52709b3a2b114d8e23e091c45c2dcf55 parsl.dataflow.memoization INFO: Task 4610 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4610 try 0 launched on executor multi with executor id 4611 parsl.dataflow.dflow INFO: Standard output for task 4610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/5249b3de-3000-41ba-9e76-ff57247ee6a4_isrForCrosstalkSources_996998_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/5249b3de-3000-41ba-9e76-ff57247ee6a4_isrForCrosstalkSources_996998_48.stderr parsl.dataflow.dflow DEBUG: Task 4611 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4611 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4611 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4611 has memoization hash 51b17ae926f57cdbf47e05fe85465f7c parsl.dataflow.memoization INFO: Task 4611 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4611 try 0 launched on executor multi with executor id 4612 parsl.dataflow.dflow INFO: Standard output for task 4611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ff884701-8a7f-41df-bfa2-cd4650d9e556_isrForCrosstalkSources_996988_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ff884701-8a7f-41df-bfa2-cd4650d9e556_isrForCrosstalkSources_996988_37.stderr parsl.dataflow.dflow DEBUG: Task 4612 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4612 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4612 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4612 has memoization hash 14f1a2533a70f41e436f24673765f6b7 parsl.dataflow.memoization INFO: Task 4612 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4612 try 0 launched on executor multi with executor id 4613 parsl.dataflow.dflow INFO: Standard output for task 4612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c3e36a4f-d697-4bd6-bf5c-8b63062015b0_isrForCrosstalkSources_996992_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c3e36a4f-d697-4bd6-bf5c-8b63062015b0_isrForCrosstalkSources_996992_17.stderr parsl.dataflow.dflow DEBUG: Task 4613 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4613 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4613 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4613 has memoization hash fc6e04a7fd480a9d44093fb1998eb914 parsl.dataflow.memoization INFO: Task 4613 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4613 try 0 launched on executor multi with executor id 4614 parsl.dataflow.dflow INFO: Standard output for task 4613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3b221d3d-5568-4ebe-be85-f69538a87e94_isrForCrosstalkSources_996963_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3b221d3d-5568-4ebe-be85-f69538a87e94_isrForCrosstalkSources_996963_21.stderr parsl.dataflow.dflow DEBUG: Task 4614 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4614 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4614 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4614 has memoization hash d16ec0aa755f7c176ab826258f1042ad parsl.dataflow.memoization INFO: Task 4614 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4614 try 0 launched on executor multi with executor id 4615 parsl.dataflow.dflow INFO: Standard output for task 4614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/615c1e2e-efa2-4b17-a634-9fc2d8849bc9_isrForCrosstalkSources_997027_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/615c1e2e-efa2-4b17-a634-9fc2d8849bc9_isrForCrosstalkSources_997027_55.stderr parsl.dataflow.dflow DEBUG: Task 4615 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4615 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4615 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4615 has memoization hash 41b2149b455a1a9d270b165321c0c6bf parsl.dataflow.memoization INFO: Task 4615 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e94fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4615 try 0 launched on executor multi with executor id 4616 parsl.dataflow.dflow INFO: Standard output for task 4615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/fc9496e8-e998-4bec-a0f0-9ac0fa4414fa_isrForCrosstalkSources_997022_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/fc9496e8-e998-4bec-a0f0-9ac0fa4414fa_isrForCrosstalkSources_997022_5.stderr parsl.dataflow.dflow DEBUG: Task 4616 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4616 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4616 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4616 has memoization hash 0ca7ddd70d2a46ff334472b4d6f11319 parsl.dataflow.memoization INFO: Task 4616 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e951c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4616 try 0 launched on executor multi with executor id 4617 parsl.dataflow.dflow INFO: Standard output for task 4616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/aef47b4e-02cf-4812-959e-2e21ad4ab3e2_isrForCrosstalkSources_997023_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/aef47b4e-02cf-4812-959e-2e21ad4ab3e2_isrForCrosstalkSources_997023_12.stderr parsl.dataflow.dflow DEBUG: Task 4617 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4617 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4617 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4617 has memoization hash ce420b5588214dbc6e54fcc62485b735 parsl.dataflow.memoization INFO: Task 4617 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4617 try 0 launched on executor multi with executor id 4618 parsl.dataflow.dflow INFO: Standard output for task 4617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/01ffc209-77ff-41b6-a36d-706d8792e6ee_isrForCrosstalkSources_997025_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/01ffc209-77ff-41b6-a36d-706d8792e6ee_isrForCrosstalkSources_997025_19.stderr parsl.dataflow.dflow DEBUG: Task 4618 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4618 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4618 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4618 has memoization hash 861a71c7ba83453d07abb29eb497740f parsl.dataflow.memoization INFO: Task 4618 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e963e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4618 try 0 launched on executor multi with executor id 4619 parsl.dataflow.dflow INFO: Standard output for task 4618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2a2e5f04-d95f-4a00-98d8-bc75d383a498_isrForCrosstalkSources_996989_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2a2e5f04-d95f-4a00-98d8-bc75d383a498_isrForCrosstalkSources_996989_1.stderr parsl.dataflow.dflow DEBUG: Task 4619 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4619 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4619 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4619 has memoization hash 036eff9a3216e416321843599010e45e parsl.dataflow.memoization INFO: Task 4619 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4619 try 0 launched on executor multi with executor id 4620 parsl.dataflow.dflow INFO: Standard output for task 4619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e4ad0eb6-ea9e-471c-966c-0772a81a3b8c_isrForCrosstalkSources_996952_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e4ad0eb6-ea9e-471c-966c-0772a81a3b8c_isrForCrosstalkSources_996952_38.stderr parsl.dataflow.dflow DEBUG: Task 4620 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4620 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4620 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4620 has memoization hash c34e680cbac7338b465960129c54ee3e parsl.dataflow.memoization INFO: Task 4620 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e962a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4620 try 0 launched on executor multi with executor id 4621 parsl.dataflow.dflow INFO: Standard output for task 4620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/ce65000b-1280-46bc-99e0-6bb147553e92_isrForCrosstalkSources_996968_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/ce65000b-1280-46bc-99e0-6bb147553e92_isrForCrosstalkSources_996968_52.stderr parsl.dataflow.dflow DEBUG: Task 4621 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4621 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4621 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4621 has memoization hash 25b01f1d029dcf730837f1d175c22c8a parsl.dataflow.memoization INFO: Task 4621 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4621 try 0 launched on executor multi with executor id 4622 parsl.dataflow.dflow INFO: Standard output for task 4621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f73343b2-fbd9-4271-9535-e8ee33f360e6_isrForCrosstalkSources_996999_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f73343b2-fbd9-4271-9535-e8ee33f360e6_isrForCrosstalkSources_996999_33.stderr parsl.dataflow.dflow DEBUG: Task 4622 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4622 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4622 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4622 has memoization hash 317336367868ba4b9d7e4c2a6c9c051e parsl.dataflow.memoization INFO: Task 4622 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e960c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4622 try 0 launched on executor multi with executor id 4623 parsl.dataflow.dflow INFO: Standard output for task 4622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/5ed8216e-d0af-4991-9d20-559356342810_isrForCrosstalkSources_997027_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/5ed8216e-d0af-4991-9d20-559356342810_isrForCrosstalkSources_997027_51.stderr parsl.dataflow.dflow DEBUG: Task 4623 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4623 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4623 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4623 has memoization hash 7ad1808a9f959d6e8e50ecef18448bc3 parsl.dataflow.memoization INFO: Task 4623 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4623 try 0 launched on executor multi with executor id 4624 parsl.dataflow.dflow INFO: Standard output for task 4623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/38e6d9bb-784e-4829-b099-56ac448cb35f_isrForCrosstalkSources_997007_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/38e6d9bb-784e-4829-b099-56ac448cb35f_isrForCrosstalkSources_997007_60.stderr parsl.dataflow.dflow DEBUG: Task 4624 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4624 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4624 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4624 has memoization hash af0358652febefc3a3ea7394b1b45569 parsl.dataflow.memoization INFO: Task 4624 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4624 try 0 launched on executor multi with executor id 4625 parsl.dataflow.dflow INFO: Standard output for task 4624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/756c07dc-8cb7-4d96-b903-c61dd7c807ea_isrForCrosstalkSources_996988_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/756c07dc-8cb7-4d96-b903-c61dd7c807ea_isrForCrosstalkSources_996988_31.stderr parsl.dataflow.dflow DEBUG: Task 4625 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4625 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4625 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4625 has memoization hash 5db9849e568a4816bd972204f0ff7f5e parsl.dataflow.memoization INFO: Task 4625 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4625 try 0 launched on executor multi with executor id 4626 parsl.dataflow.dflow INFO: Standard output for task 4625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/db30f475-61fc-4296-89dd-baf77a1a5fd4_isrForCrosstalkSources_996998_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/db30f475-61fc-4296-89dd-baf77a1a5fd4_isrForCrosstalkSources_996998_49.stderr parsl.dataflow.dflow DEBUG: Task 4626 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4626 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4626 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4626 has memoization hash b9c101f01cd7ba221e24e028d46ab075 parsl.dataflow.memoization INFO: Task 4626 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4626 try 0 launched on executor multi with executor id 4627 parsl.dataflow.dflow INFO: Standard output for task 4626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/952a4e13-1af5-44a2-a797-e9530704cf86_isrForCrosstalkSources_997003_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/952a4e13-1af5-44a2-a797-e9530704cf86_isrForCrosstalkSources_997003_2.stderr parsl.dataflow.dflow DEBUG: Task 4627 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4627 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4627 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4627 has memoization hash e4616b39a222947b59190518ddc56dd4 parsl.dataflow.memoization INFO: Task 4627 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4627 try 0 launched on executor multi with executor id 4628 parsl.dataflow.dflow INFO: Standard output for task 4627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/7d78e049-6c32-445b-aae3-3fe72846d1ce_isrForCrosstalkSources_996980_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/7d78e049-6c32-445b-aae3-3fe72846d1ce_isrForCrosstalkSources_996980_45.stderr parsl.dataflow.dflow DEBUG: Task 4628 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4628 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4628 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4628 has memoization hash cca38b947862f8a58bc790beecaa6cec parsl.dataflow.memoization INFO: Task 4628 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4628 try 0 launched on executor multi with executor id 4629 parsl.dataflow.dflow INFO: Standard output for task 4628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eab9c3d-7a08-4de2-95fb-875cf0af85bc_isrForCrosstalkSources_996985_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eab9c3d-7a08-4de2-95fb-875cf0af85bc_isrForCrosstalkSources_996985_60.stderr parsl.dataflow.dflow DEBUG: Task 4629 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4629 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4629 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4629 has memoization hash 1fcbfd9b3439b3b09640411f065e32e7 parsl.dataflow.memoization INFO: Task 4629 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4629 try 0 launched on executor multi with executor id 4630 parsl.dataflow.dflow INFO: Standard output for task 4629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/94053b65-63eb-4762-b0f9-4e25a70db99d_isrForCrosstalkSources_996977_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/94053b65-63eb-4762-b0f9-4e25a70db99d_isrForCrosstalkSources_996977_52.stderr parsl.dataflow.dflow DEBUG: Task 4630 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4630 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4630 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4630 has memoization hash e6f3161c2e2a7fb187562c786cfb3788 parsl.dataflow.memoization INFO: Task 4630 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4630 try 0 launched on executor multi with executor id 4631 parsl.dataflow.dflow INFO: Standard output for task 4630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e8d3a893-8f6b-4b1f-9dc2-966796d59ba9_isrForCrosstalkSources_996978_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e8d3a893-8f6b-4b1f-9dc2-966796d59ba9_isrForCrosstalkSources_996978_9.stderr parsl.dataflow.dflow DEBUG: Task 4631 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4631 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4631 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4631 has memoization hash 0f88db3f831ea9bb4848a1a854df7df9 parsl.dataflow.memoization INFO: Task 4631 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4631 try 0 launched on executor multi with executor id 4632 parsl.dataflow.dflow INFO: Standard output for task 4631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/849dd612-e3a3-47c8-940f-16eb08b1b67f_isrForCrosstalkSources_997013_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/849dd612-e3a3-47c8-940f-16eb08b1b67f_isrForCrosstalkSources_997013_49.stderr parsl.dataflow.dflow DEBUG: Task 4632 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4632 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4632 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4632 has memoization hash d98fd5c527fc5a246ef8eac07da31339 parsl.dataflow.memoization INFO: Task 4632 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4632 try 0 launched on executor multi with executor id 4633 parsl.dataflow.dflow INFO: Standard output for task 4632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/5973f750-05eb-473d-9b29-eead1634baf4_isrForCrosstalkSources_996953_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/5973f750-05eb-473d-9b29-eead1634baf4_isrForCrosstalkSources_996953_24.stderr parsl.dataflow.dflow DEBUG: Task 4633 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4633 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4633 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4633 has memoization hash 37864f37520963b9b38d23aeec299d68 parsl.dataflow.memoization INFO: Task 4633 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e959e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4633 try 0 launched on executor multi with executor id 4634 parsl.dataflow.dflow INFO: Standard output for task 4633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/372a08c5-2767-45f7-97d7-f2aaaafd4539_isrForCrosstalkSources_996999_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/372a08c5-2767-45f7-97d7-f2aaaafd4539_isrForCrosstalkSources_996999_8.stderr parsl.dataflow.dflow DEBUG: Task 4634 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4634 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4634 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4634 has memoization hash 056efd6dee6b5e19461878116d625a9e parsl.dataflow.memoization INFO: Task 4634 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e958a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4634 try 0 launched on executor multi with executor id 4635 parsl.dataflow.dflow INFO: Standard output for task 4634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/3a484663-3377-402f-aed5-5f5791a9f37e_isrForCrosstalkSources_997011_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/3a484663-3377-402f-aed5-5f5791a9f37e_isrForCrosstalkSources_997011_36.stderr parsl.dataflow.dflow DEBUG: Task 4635 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4635 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4635 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4635 has memoization hash 4ba9ec84571a824f3ea5ca86745407d7 parsl.dataflow.memoization INFO: Task 4635 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4635 try 0 launched on executor multi with executor id 4636 parsl.dataflow.dflow INFO: Standard output for task 4635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/603cd3d1-73e3-4b09-aea5-9b0f1e2c9cad_isrForCrosstalkSources_997000_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/603cd3d1-73e3-4b09-aea5-9b0f1e2c9cad_isrForCrosstalkSources_997000_41.stderr parsl.dataflow.dflow DEBUG: Task 4636 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4636 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4636 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4636 has memoization hash 02b7cba786f02c768e537ee3bb16b26d parsl.dataflow.memoization INFO: Task 4636 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4636 try 0 launched on executor multi with executor id 4637 parsl.dataflow.dflow INFO: Standard output for task 4636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/3b702984-18c3-46c7-a4d4-9f3815a9bed3_isrForCrosstalkSources_996998_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/3b702984-18c3-46c7-a4d4-9f3815a9bed3_isrForCrosstalkSources_996998_24.stderr parsl.dataflow.dflow DEBUG: Task 4637 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4637 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4637 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4637 has memoization hash c18b5586160ae92b4ff88ea73e1046b3 parsl.dataflow.memoization INFO: Task 4637 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4637 try 0 launched on executor multi with executor id 4638 parsl.dataflow.dflow INFO: Standard output for task 4637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/711f9fd0-0b6c-4369-859e-1cfc6d8359be_isrForCrosstalkSources_997015_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/711f9fd0-0b6c-4369-859e-1cfc6d8359be_isrForCrosstalkSources_997015_9.stderr parsl.dataflow.dflow DEBUG: Task 4638 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4638 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4638 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4638 has memoization hash 96a75cec1e040a279bcbbb790a218622 parsl.dataflow.memoization INFO: Task 4638 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e956c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4638 try 0 launched on executor multi with executor id 4639 parsl.dataflow.dflow INFO: Standard output for task 4638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/214a3421-40e7-49b2-a094-fc369473278a_isrForCrosstalkSources_997001_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/214a3421-40e7-49b2-a094-fc369473278a_isrForCrosstalkSources_997001_1.stderr parsl.dataflow.dflow DEBUG: Task 4639 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4639 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4639 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4639 has memoization hash c3155ae8379959fdebb8f904d941951e parsl.dataflow.memoization INFO: Task 4639 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4639 try 0 launched on executor multi with executor id 4640 parsl.dataflow.dflow INFO: Standard output for task 4639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/1f1d188a-4b95-413b-825d-49aa687deb49_isrForCrosstalkSources_996999_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/1f1d188a-4b95-413b-825d-49aa687deb49_isrForCrosstalkSources_996999_32.stderr parsl.dataflow.dflow DEBUG: Task 4640 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4640 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4640 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4640 has memoization hash e9ba755da70735c298f716ed60466571 parsl.dataflow.memoization INFO: Task 4640 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4640 try 0 launched on executor multi with executor id 4641 parsl.dataflow.dflow INFO: Standard output for task 4640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3027b5ec-ca59-422b-926c-2ddbd8f8e3c8_isrForCrosstalkSources_996988_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3027b5ec-ca59-422b-926c-2ddbd8f8e3c8_isrForCrosstalkSources_996988_10.stderr parsl.dataflow.dflow DEBUG: Task 4641 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4641 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4641 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4641 has memoization hash a357425da399b8f7288b1db27a3f9927 parsl.dataflow.memoization INFO: Task 4641 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e954e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4641 try 0 launched on executor multi with executor id 4642 parsl.dataflow.dflow INFO: Standard output for task 4641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b888e1db-ddc8-489c-9f01-c1526516b021_isrForCrosstalkSources_997006_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b888e1db-ddc8-489c-9f01-c1526516b021_isrForCrosstalkSources_997006_6.stderr parsl.dataflow.dflow DEBUG: Task 4642 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4642 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4642 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4642 has memoization hash 64159e12f681501c97d7749f00356746 parsl.dataflow.memoization INFO: Task 4642 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4642 try 0 launched on executor multi with executor id 4643 parsl.dataflow.dflow INFO: Standard output for task 4642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c97dec21-67fd-450c-a9bf-0f4f71604ca0_isrForCrosstalkSources_996994_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c97dec21-67fd-450c-a9bf-0f4f71604ca0_isrForCrosstalkSources_996994_26.stderr parsl.dataflow.dflow DEBUG: Task 4643 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4643 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4643 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4643 has memoization hash 8b30b697c958def3dd606fba28ce2b29 parsl.dataflow.memoization INFO: Task 4643 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e953a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4643 try 0 launched on executor multi with executor id 4644 parsl.dataflow.dflow INFO: Standard output for task 4643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e0eee41f-12a7-46e9-948b-d6e3a183bdb6_isrForCrosstalkSources_996997_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e0eee41f-12a7-46e9-948b-d6e3a183bdb6_isrForCrosstalkSources_996997_20.stderr parsl.dataflow.dflow DEBUG: Task 4644 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4644 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4644 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4644 has memoization hash 9566da025c0e3755051eb88cb750081b parsl.dataflow.memoization INFO: Task 4644 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4644 try 0 launched on executor multi with executor id 4645 parsl.dataflow.dflow INFO: Standard output for task 4644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/5f5acaa2-7aff-4926-8d8d-40a3f91cabf3_isrForCrosstalkSources_997019_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/5f5acaa2-7aff-4926-8d8d-40a3f91cabf3_isrForCrosstalkSources_997019_62.stderr parsl.dataflow.dflow DEBUG: Task 4645 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4645 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4645 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4645 has memoization hash f3fe06a1b5cc87001618325829ce4f4a parsl.dataflow.memoization INFO: Task 4645 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4645 try 0 launched on executor multi with executor id 4646 parsl.dataflow.dflow INFO: Standard output for task 4645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a42ae75a-4fbd-45c0-8f71-203b6a8e4720_isrForCrosstalkSources_996981_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a42ae75a-4fbd-45c0-8f71-203b6a8e4720_isrForCrosstalkSources_996981_20.stderr parsl.dataflow.dflow DEBUG: Task 4646 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4646 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4646 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4646 has memoization hash 09dd76efcfc060f80df4a040826ed703 parsl.dataflow.memoization INFO: Task 4646 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e95120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4646 try 0 launched on executor multi with executor id 4647 parsl.dataflow.dflow INFO: Standard output for task 4646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/9cfc607d-5239-4e44-be01-c7b53b880301_isrForCrosstalkSources_997028_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/9cfc607d-5239-4e44-be01-c7b53b880301_isrForCrosstalkSources_997028_56.stderr parsl.dataflow.dflow DEBUG: Task 4647 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4647 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4647 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4647 has memoization hash bf7012010e5db97c12066084b7ebcc36 parsl.dataflow.memoization INFO: Task 4647 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4647 try 0 launched on executor multi with executor id 4648 parsl.dataflow.dflow INFO: Standard output for task 4647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/a487f763-84c0-4ce8-b87b-4676e49a8de5_isrForCrosstalkSources_997016_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/a487f763-84c0-4ce8-b87b-4676e49a8de5_isrForCrosstalkSources_997016_13.stderr parsl.dataflow.dflow DEBUG: Task 4648 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4648 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4648 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4648 has memoization hash 3c2f8bfe3e8e229e8dee7f0396e43aaa parsl.dataflow.memoization INFO: Task 4648 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4648 try 0 launched on executor multi with executor id 4649 parsl.dataflow.dflow INFO: Standard output for task 4648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/947fb6df-cd6a-47f4-abb3-fcfe00a5358c_isrForCrosstalkSources_997005_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/947fb6df-cd6a-47f4-abb3-fcfe00a5358c_isrForCrosstalkSources_997005_19.stderr parsl.dataflow.dflow DEBUG: Task 4649 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4649 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4649 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4649 has memoization hash c8597a7ba589fee765f8319d0fbbc852 parsl.dataflow.memoization INFO: Task 4649 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4649 try 0 launched on executor multi with executor id 4650 parsl.dataflow.dflow INFO: Standard output for task 4649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/fc932746-fc33-4322-9a83-6bff7add5c07_isrForCrosstalkSources_997028_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/fc932746-fc33-4322-9a83-6bff7add5c07_isrForCrosstalkSources_997028_15.stderr parsl.dataflow.dflow DEBUG: Task 4650 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4650 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4650 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4650 has memoization hash 22473bf1b07f637d486b71471f9757d7 parsl.dataflow.memoization INFO: Task 4650 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e967a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4650 try 0 launched on executor multi with executor id 4651 parsl.dataflow.dflow INFO: Standard output for task 4650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0f04d8ce-d2b0-43da-8711-c225705fd684_isrForCrosstalkSources_997007_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0f04d8ce-d2b0-43da-8711-c225705fd684_isrForCrosstalkSources_997007_1.stderr parsl.dataflow.dflow DEBUG: Task 4651 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4651 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4651 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4651 has memoization hash afac47251c3b0626acad3ce27589c44d parsl.dataflow.memoization INFO: Task 4651 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e968e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4651 try 0 launched on executor multi with executor id 4652 parsl.dataflow.dflow INFO: Standard output for task 4651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/789ba7a9-f5dd-47af-a688-f249548a37c2_isrForCrosstalkSources_996967_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/789ba7a9-f5dd-47af-a688-f249548a37c2_isrForCrosstalkSources_996967_48.stderr parsl.dataflow.dflow DEBUG: Task 4652 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4652 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4652 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4652 has memoization hash 1a111a298722d23517fbc5aa0ee2a369 parsl.dataflow.memoization INFO: Task 4652 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4652 try 0 launched on executor multi with executor id 4653 parsl.dataflow.dflow INFO: Standard output for task 4652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c99baec9-14b9-4ff6-85a4-282fed867581_isrForCrosstalkSources_996970_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c99baec9-14b9-4ff6-85a4-282fed867581_isrForCrosstalkSources_996970_34.stderr parsl.dataflow.dflow DEBUG: Task 4653 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4653 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4653 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4653 has memoization hash 54da84fad3ce0cb65e5030a410d864a7 parsl.dataflow.memoization INFO: Task 4653 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4653 try 0 launched on executor multi with executor id 4654 parsl.dataflow.dflow INFO: Standard output for task 4653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/8829c8b4-ae19-402e-9309-28565aae73dc_isrForCrosstalkSources_997020_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/8829c8b4-ae19-402e-9309-28565aae73dc_isrForCrosstalkSources_997020_31.stderr parsl.dataflow.dflow DEBUG: Task 4654 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4654 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4654 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4654 has memoization hash 2457819bd1a2f932b64fa1676c655112 parsl.dataflow.memoization INFO: Task 4654 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4654 try 0 launched on executor multi with executor id 4655 parsl.dataflow.dflow INFO: Standard output for task 4654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/aea3ef04-7f03-485f-85e1-b995f00e324f_isrForCrosstalkSources_996969_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/aea3ef04-7f03-485f-85e1-b995f00e324f_isrForCrosstalkSources_996969_47.stderr parsl.dataflow.dflow DEBUG: Task 4655 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4655 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4655 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4655 has memoization hash d92d42279599f3858857b6f7c9ffd638 parsl.dataflow.memoization INFO: Task 4655 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4655 try 0 launched on executor multi with executor id 4656 parsl.dataflow.dflow INFO: Standard output for task 4655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/94292aaf-47ac-45c0-94d0-b1ec436f475e_isrForCrosstalkSources_996979_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/94292aaf-47ac-45c0-94d0-b1ec436f475e_isrForCrosstalkSources_996979_56.stderr parsl.dataflow.dflow DEBUG: Task 4656 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4656 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4656 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4656 has memoization hash 045803940ab5e339e027772f32f94b92 parsl.dataflow.memoization INFO: Task 4656 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4656 try 0 launched on executor multi with executor id 4657 parsl.dataflow.dflow INFO: Standard output for task 4656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c72911f2-f69f-436e-b420-1479a0d959e3_isrForCrosstalkSources_997001_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c72911f2-f69f-436e-b420-1479a0d959e3_isrForCrosstalkSources_997001_3.stderr parsl.dataflow.dflow DEBUG: Task 4657 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4657 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4657 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4657 has memoization hash d05664e1a248dfafaeb7d7be23f88ab5 parsl.dataflow.memoization INFO: Task 4657 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4657 try 0 launched on executor multi with executor id 4658 parsl.dataflow.dflow INFO: Standard output for task 4657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/40aeb43a-8f41-4485-a7ec-ea4d409c514f_isrForCrosstalkSources_996982_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/40aeb43a-8f41-4485-a7ec-ea4d409c514f_isrForCrosstalkSources_996982_38.stderr parsl.dataflow.dflow DEBUG: Task 4658 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4658 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4658 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4658 has memoization hash 0a8bb686c2cd5433bea528bc916d7506 parsl.dataflow.memoization INFO: Task 4658 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4658 try 0 launched on executor multi with executor id 4659 parsl.dataflow.dflow INFO: Standard output for task 4658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bde1ab4d-a4d5-4aa2-9ed0-b4113dcc2054_isrForCrosstalkSources_996959_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bde1ab4d-a4d5-4aa2-9ed0-b4113dcc2054_isrForCrosstalkSources_996959_56.stderr parsl.dataflow.dflow DEBUG: Task 4659 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4659 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4659 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4659 has memoization hash a462967feafeeeab2a12534cbab87068 parsl.dataflow.memoization INFO: Task 4659 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4659 try 0 launched on executor multi with executor id 4660 parsl.dataflow.dflow INFO: Standard output for task 4659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/efb92a83-5bdc-4959-be37-906a430a2a99_isrForCrosstalkSources_996952_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/efb92a83-5bdc-4959-be37-906a430a2a99_isrForCrosstalkSources_996952_15.stderr parsl.dataflow.dflow DEBUG: Task 4660 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4660 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4660 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4660 has memoization hash 653a565982151521536e1542fd0695d5 parsl.dataflow.memoization INFO: Task 4660 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4660 try 0 launched on executor multi with executor id 4661 parsl.dataflow.dflow INFO: Standard output for task 4660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/90a1370d-5aac-4252-968a-fe585e30e2eb_isrForCrosstalkSources_996959_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/90a1370d-5aac-4252-968a-fe585e30e2eb_isrForCrosstalkSources_996959_57.stderr parsl.dataflow.dflow DEBUG: Task 4661 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4661 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4661 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4661 has memoization hash 696f352113ebb9d1ad041123182678c4 parsl.dataflow.memoization INFO: Task 4661 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4661 try 0 launched on executor multi with executor id 4662 parsl.dataflow.dflow INFO: Standard output for task 4661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a921f02b-30ad-4591-9e0f-f16caf24c8ed_isrForCrosstalkSources_997033_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a921f02b-30ad-4591-9e0f-f16caf24c8ed_isrForCrosstalkSources_997033_2.stderr parsl.dataflow.dflow DEBUG: Task 4662 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4662 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4662 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4662 has memoization hash 198e005f2304918d6697be271ac57c8b parsl.dataflow.memoization INFO: Task 4662 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4662 try 0 launched on executor multi with executor id 4663 parsl.dataflow.dflow INFO: Standard output for task 4662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/dc2e2908-114f-43f3-b856-a970463455d2_isrForCrosstalkSources_996981_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/dc2e2908-114f-43f3-b856-a970463455d2_isrForCrosstalkSources_996981_26.stderr parsl.dataflow.dflow DEBUG: Task 4663 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4663 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4663 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4663 has memoization hash abbf6574c0ad6fa44221f299a38b77bb parsl.dataflow.memoization INFO: Task 4663 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4663 try 0 launched on executor multi with executor id 4664 parsl.dataflow.dflow INFO: Standard output for task 4663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/137a3921-b6d2-44a7-a08b-d0a78bac5068_isrForCrosstalkSources_996954_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/137a3921-b6d2-44a7-a08b-d0a78bac5068_isrForCrosstalkSources_996954_50.stderr parsl.dataflow.dflow DEBUG: Task 4664 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4664 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4664 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4664 has memoization hash dcf9832f74f0962c87e2647bfee1a540 parsl.dataflow.memoization INFO: Task 4664 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e971a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4664 try 0 launched on executor multi with executor id 4665 parsl.dataflow.dflow INFO: Standard output for task 4664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e5f0cddd-2e9f-483e-bcbd-1e1bd118b37c_isrForCrosstalkSources_997021_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e5f0cddd-2e9f-483e-bcbd-1e1bd118b37c_isrForCrosstalkSources_997021_7.stderr parsl.dataflow.dflow DEBUG: Task 4665 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4665 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4665 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4665 has memoization hash 72a2e5e9a288e98e5fd7b2dc0c9e3718 parsl.dataflow.memoization INFO: Task 4665 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4665 try 0 launched on executor multi with executor id 4666 parsl.dataflow.dflow INFO: Standard output for task 4665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/23a139a2-ff08-455b-a0d3-944c8386caca_isrForCrosstalkSources_996959_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/23a139a2-ff08-455b-a0d3-944c8386caca_isrForCrosstalkSources_996959_61.stderr parsl.dataflow.dflow DEBUG: Task 4666 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4666 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4666 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4666 has memoization hash 660f18cece6c03e5cace12f7ceefb82b parsl.dataflow.memoization INFO: Task 4666 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4666 try 0 launched on executor multi with executor id 4667 parsl.dataflow.dflow INFO: Standard output for task 4666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/b719fabc-74ab-4910-b8b7-b1d497762cfd_isrForCrosstalkSources_997017_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/b719fabc-74ab-4910-b8b7-b1d497762cfd_isrForCrosstalkSources_997017_36.stderr parsl.dataflow.dflow DEBUG: Task 4667 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4667 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4667 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4667 has memoization hash 0f03a5b322d0f5431f13ace04a07cd76 parsl.dataflow.memoization INFO: Task 4667 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e972e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4667 try 0 launched on executor multi with executor id 4668 parsl.dataflow.dflow INFO: Standard output for task 4667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/435342d8-e46b-491c-b2e1-0607913cacc1_isrForCrosstalkSources_997031_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/435342d8-e46b-491c-b2e1-0607913cacc1_isrForCrosstalkSources_997031_28.stderr parsl.dataflow.dflow DEBUG: Task 4668 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4668 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4668 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4668 has memoization hash ba5508c6b20a9cadaa55208301e3b659 parsl.dataflow.memoization INFO: Task 4668 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4668 try 0 launched on executor multi with executor id 4669 parsl.dataflow.dflow INFO: Standard output for task 4668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f11c1aa8-1045-4e13-b549-a2577ef404e7_isrForCrosstalkSources_996978_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f11c1aa8-1045-4e13-b549-a2577ef404e7_isrForCrosstalkSources_996978_20.stderr parsl.dataflow.dflow DEBUG: Task 4669 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4669 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4669 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4669 has memoization hash 1348151b0356fe96ed0695fe501707cf parsl.dataflow.memoization INFO: Task 4669 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e974c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4669 try 0 launched on executor multi with executor id 4670 parsl.dataflow.dflow INFO: Standard output for task 4669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/6ca087a6-fc6c-4f4c-953f-11fd73c4fb4a_isrForCrosstalkSources_996972_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/6ca087a6-fc6c-4f4c-953f-11fd73c4fb4a_isrForCrosstalkSources_996972_40.stderr parsl.dataflow.dflow DEBUG: Task 4670 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4670 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4670 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4670 has memoization hash 4e3725eaac46ba29add61007e13c7db9 parsl.dataflow.memoization INFO: Task 4670 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4670 try 0 launched on executor multi with executor id 4671 parsl.dataflow.dflow INFO: Standard output for task 4670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a4352089-796d-4294-95c6-f5b23f81e101_isrForCrosstalkSources_996961_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a4352089-796d-4294-95c6-f5b23f81e101_isrForCrosstalkSources_996961_27.stderr parsl.dataflow.dflow DEBUG: Task 4671 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4671 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4671 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4671 has memoization hash f31c5f798e73208972e2d366f254b587 parsl.dataflow.memoization INFO: Task 4671 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4671 try 0 launched on executor multi with executor id 4672 parsl.dataflow.dflow INFO: Standard output for task 4671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9281d8b6-85ba-4f16-a44f-0cd60ea37cf5_isrForCrosstalkSources_996960_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9281d8b6-85ba-4f16-a44f-0cd60ea37cf5_isrForCrosstalkSources_996960_49.stderr parsl.dataflow.dflow DEBUG: Task 4672 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4672 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4672 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4672 has memoization hash f8c88c77a557cce6503fd062fc7d7593 parsl.dataflow.memoization INFO: Task 4672 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4672 try 0 launched on executor multi with executor id 4673 parsl.dataflow.dflow INFO: Standard output for task 4672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a85c2bcb-d717-4de0-97dd-cffff7f306ab_isrForCrosstalkSources_997006_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a85c2bcb-d717-4de0-97dd-cffff7f306ab_isrForCrosstalkSources_997006_47.stderr parsl.dataflow.dflow DEBUG: Task 4673 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4673 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4673 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4673 has memoization hash 1be62125a5c3fa7a5d9ae4e9899ddf43 parsl.dataflow.memoization INFO: Task 4673 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e976a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4673 try 0 launched on executor multi with executor id 4674 parsl.dataflow.dflow INFO: Standard output for task 4673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4d94cd36-baef-4f1a-bb37-7a3be5b2fcf4_isrForCrosstalkSources_996998_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4d94cd36-baef-4f1a-bb37-7a3be5b2fcf4_isrForCrosstalkSources_996998_46.stderr parsl.dataflow.dflow DEBUG: Task 4674 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4674 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4674 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4674 has memoization hash b44e158c9251d83a99cf3bb3a6f65ac4 parsl.dataflow.memoization INFO: Task 4674 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e977e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4674 try 0 launched on executor multi with executor id 4675 parsl.dataflow.dflow INFO: Standard output for task 4674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/38438d2e-c480-4f91-848e-d20fabb47943_isrForCrosstalkSources_996992_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/38438d2e-c480-4f91-848e-d20fabb47943_isrForCrosstalkSources_996992_56.stderr parsl.dataflow.dflow DEBUG: Task 4675 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4675 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4675 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4675 has memoization hash 53eeee45646091b8ad3bbb0b806c26ae parsl.dataflow.memoization INFO: Task 4675 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4675 try 0 launched on executor multi with executor id 4676 parsl.dataflow.dflow INFO: Standard output for task 4675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/2d68e52c-7df0-4ab6-82d6-7faf7879a021_isrForCrosstalkSources_997014_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/2d68e52c-7df0-4ab6-82d6-7faf7879a021_isrForCrosstalkSources_997014_36.stderr parsl.dataflow.dflow DEBUG: Task 4676 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4676 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4676 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4676 has memoization hash 6ecff01e0d9bbbad9c43c1684c154495 parsl.dataflow.memoization INFO: Task 4676 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4676 try 0 launched on executor multi with executor id 4677 parsl.dataflow.dflow INFO: Standard output for task 4676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5c45ac06-d339-4531-b917-a10b1d194c45_isrForCrosstalkSources_997012_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5c45ac06-d339-4531-b917-a10b1d194c45_isrForCrosstalkSources_997012_10.stderr parsl.dataflow.dflow DEBUG: Task 4677 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4677 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4677 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4677 has memoization hash 37008b4bc1e67ae4355ff34068fd8999 parsl.dataflow.memoization INFO: Task 4677 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4677 try 0 launched on executor multi with executor id 4678 parsl.dataflow.dflow INFO: Standard output for task 4677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b75c58b7-6b6d-4ed2-a54f-75958dfe7f28_isrForCrosstalkSources_996997_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b75c58b7-6b6d-4ed2-a54f-75958dfe7f28_isrForCrosstalkSources_996997_28.stderr parsl.dataflow.dflow DEBUG: Task 4678 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4678 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4678 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4678 has memoization hash 4d27b9d5fb70960feef48490515e83e5 parsl.dataflow.memoization INFO: Task 4678 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e979c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4678 try 0 launched on executor multi with executor id 4679 parsl.dataflow.dflow INFO: Standard output for task 4678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/06533278-32d8-40ce-a703-cbd8b889ef77_isrForCrosstalkSources_997014_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/06533278-32d8-40ce-a703-cbd8b889ef77_isrForCrosstalkSources_997014_28.stderr parsl.dataflow.dflow DEBUG: Task 4679 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4679 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4679 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4679 has memoization hash da6175d18d8eaac4143e2c51f85526de parsl.dataflow.memoization INFO: Task 4679 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4679 try 0 launched on executor multi with executor id 4680 parsl.dataflow.dflow INFO: Standard output for task 4679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f85b2d43-2331-4bd6-a45c-41ee189396fa_isrForCrosstalkSources_996960_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f85b2d43-2331-4bd6-a45c-41ee189396fa_isrForCrosstalkSources_996960_25.stderr parsl.dataflow.dflow DEBUG: Task 4680 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4680 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4680 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4680 has memoization hash 829df41683217236b61900ebedf0b712 parsl.dataflow.memoization INFO: Task 4680 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4680 try 0 launched on executor multi with executor id 4681 parsl.dataflow.dflow INFO: Standard output for task 4680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/14e06baf-379a-4023-93e8-2b310d9f74cf_isrForCrosstalkSources_997014_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/14e06baf-379a-4023-93e8-2b310d9f74cf_isrForCrosstalkSources_997014_34.stderr parsl.dataflow.dflow DEBUG: Task 4681 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4681 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4681 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4681 has memoization hash 1fb4ba7f0bc4aa2e8e2909bc8ba17160 parsl.dataflow.memoization INFO: Task 4681 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4681 try 0 launched on executor multi with executor id 4682 parsl.dataflow.dflow INFO: Standard output for task 4681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c1e9b93a-00ec-4e26-ad94-6afc715a0499_isrForCrosstalkSources_996967_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c1e9b93a-00ec-4e26-ad94-6afc715a0499_isrForCrosstalkSources_996967_50.stderr parsl.dataflow.dflow DEBUG: Task 4682 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4682 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4682 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4682 has memoization hash 2bbe4164c2b01ec3ffc2367ab30fcc36 parsl.dataflow.memoization INFO: Task 4682 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4682 try 0 launched on executor multi with executor id 4683 parsl.dataflow.dflow INFO: Standard output for task 4682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/68605157-f841-4559-a470-83e5f67b4b1d_isrForCrosstalkSources_997010_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/68605157-f841-4559-a470-83e5f67b4b1d_isrForCrosstalkSources_997010_42.stderr parsl.dataflow.dflow DEBUG: Task 4683 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4683 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4683 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4683 has memoization hash aa94b9e74c156ecbd072879489eec8e7 parsl.dataflow.memoization INFO: Task 4683 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4683 try 0 launched on executor multi with executor id 4684 parsl.dataflow.dflow INFO: Standard output for task 4683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a2f594ef-46b5-4a9a-8dab-6efe57a32117_isrForCrosstalkSources_996996_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a2f594ef-46b5-4a9a-8dab-6efe57a32117_isrForCrosstalkSources_996996_33.stderr parsl.dataflow.dflow DEBUG: Task 4684 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4684 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4684 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4684 has memoization hash b505a4e992306b3c036c6c1b4bcdc74d parsl.dataflow.memoization INFO: Task 4684 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4684 try 0 launched on executor multi with executor id 4685 parsl.dataflow.dflow INFO: Standard output for task 4684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8c217af1-d0b1-4c4c-af29-67d8527a1399_isrForCrosstalkSources_996984_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8c217af1-d0b1-4c4c-af29-67d8527a1399_isrForCrosstalkSources_996984_15.stderr parsl.dataflow.dflow DEBUG: Task 4685 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4685 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4685 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4685 has memoization hash f642203801bd478698a0299aed570c23 parsl.dataflow.memoization INFO: Task 4685 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4685 try 0 launched on executor multi with executor id 4686 parsl.dataflow.dflow INFO: Standard output for task 4685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/813febc4-e29c-412b-a518-4ef5b5646cb7_isrForCrosstalkSources_996975_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/813febc4-e29c-412b-a518-4ef5b5646cb7_isrForCrosstalkSources_996975_12.stderr parsl.dataflow.dflow DEBUG: Task 4686 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4686 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4686 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4686 has memoization hash 209e79b1deef3f3d2588974d75fbfcad parsl.dataflow.memoization INFO: Task 4686 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4686 try 0 launched on executor multi with executor id 4687 parsl.dataflow.dflow INFO: Standard output for task 4686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fabc437c-528f-42a6-afc4-8536a51760c1_isrForCrosstalkSources_996955_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fabc437c-528f-42a6-afc4-8536a51760c1_isrForCrosstalkSources_996955_29.stderr parsl.dataflow.dflow DEBUG: Task 4687 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4687 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4687 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4687 has memoization hash 3eeba0846b19c3ad00b2ae49b6f7c12e parsl.dataflow.memoization INFO: Task 4687 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e97f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4687 try 0 launched on executor multi with executor id 4688 parsl.dataflow.dflow INFO: Standard output for task 4687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/079db310-2456-46ad-8f0b-598de1a9c0cb_isrForCrosstalkSources_996989_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/079db310-2456-46ad-8f0b-598de1a9c0cb_isrForCrosstalkSources_996989_50.stderr parsl.dataflow.dflow DEBUG: Task 4688 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4688 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4688 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4688 has memoization hash 256e9e2e24b03180cd02906e7041d26a parsl.dataflow.memoization INFO: Task 4688 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767e96340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4688 try 0 launched on executor multi with executor id 4689 parsl.dataflow.dflow INFO: Standard output for task 4688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7a9932b1-e86a-4963-b106-e947b3edab05_isrForCrosstalkSources_997027_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7a9932b1-e86a-4963-b106-e947b3edab05_isrForCrosstalkSources_997027_48.stderr parsl.dataflow.dflow DEBUG: Task 4689 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4689 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4689 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4689 has memoization hash ad57beffa69aed805585663877803cae parsl.dataflow.memoization INFO: Task 4689 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4689 try 0 launched on executor multi with executor id 4690 parsl.dataflow.dflow INFO: Standard output for task 4689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/402cfe6b-045e-44df-9683-80752a164f39_isrForCrosstalkSources_997004_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/402cfe6b-045e-44df-9683-80752a164f39_isrForCrosstalkSources_997004_15.stderr parsl.dataflow.dflow DEBUG: Task 4690 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4690 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4690 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4690 has memoization hash fbdc20d0719f74e83f83505ebdd1343d parsl.dataflow.memoization INFO: Task 4690 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d480e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4690 try 0 launched on executor multi with executor id 4691 parsl.dataflow.dflow INFO: Standard output for task 4690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a5ee799d-1e1a-4b24-a818-21b73e861cf9_isrForCrosstalkSources_997018_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a5ee799d-1e1a-4b24-a818-21b73e861cf9_isrForCrosstalkSources_997018_60.stderr parsl.dataflow.dflow DEBUG: Task 4691 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4691 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4691 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4691 has memoization hash d6a65f7100cb1fe0b4b12c0788c164c1 parsl.dataflow.memoization INFO: Task 4691 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4691 try 0 launched on executor multi with executor id 4692 parsl.dataflow.dflow INFO: Standard output for task 4691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/f25ad7e6-fc7b-489d-821c-3b4526ec3122_isrForCrosstalkSources_996969_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/f25ad7e6-fc7b-489d-821c-3b4526ec3122_isrForCrosstalkSources_996969_9.stderr parsl.dataflow.dflow DEBUG: Task 4692 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4692 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4692 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4692 has memoization hash 6299f4d222da45671bacb9bc32fd0bec parsl.dataflow.memoization INFO: Task 4692 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d482c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4692 try 0 launched on executor multi with executor id 4693 parsl.dataflow.dflow INFO: Standard output for task 4692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bb70c79f-09e1-4e08-baf1-11e4ad2f05de_isrForCrosstalkSources_997030_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bb70c79f-09e1-4e08-baf1-11e4ad2f05de_isrForCrosstalkSources_997030_56.stderr parsl.dataflow.dflow DEBUG: Task 4693 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4693 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4693 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4693 has memoization hash 3da33a4d6c6a1ab35114ed0e143fae72 parsl.dataflow.memoization INFO: Task 4693 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4693 try 0 launched on executor multi with executor id 4694 parsl.dataflow.dflow INFO: Standard output for task 4693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6f96a91f-3ff6-4cba-8da2-bbbd547c9067_isrForCrosstalkSources_996971_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6f96a91f-3ff6-4cba-8da2-bbbd547c9067_isrForCrosstalkSources_996971_47.stderr parsl.dataflow.dflow DEBUG: Task 4694 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4694 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4694 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4694 has memoization hash 1045db5127fcf8e426b0f329b19cb353 parsl.dataflow.memoization INFO: Task 4694 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d484a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4694 try 0 launched on executor multi with executor id 4695 parsl.dataflow.dflow INFO: Standard output for task 4694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0f61838a-6ef6-45a6-815f-d7096d594921_isrForCrosstalkSources_997012_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0f61838a-6ef6-45a6-815f-d7096d594921_isrForCrosstalkSources_997012_28.stderr parsl.dataflow.dflow DEBUG: Task 4695 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4695 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4695 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4695 has memoization hash a495c257f7c10106cef2445c97402339 parsl.dataflow.memoization INFO: Task 4695 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4695 try 0 launched on executor multi with executor id 4696 parsl.dataflow.dflow INFO: Standard output for task 4695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/a0efdfbe-016c-4a81-89c3-be9a47f0a224_isrForCrosstalkSources_996979_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/a0efdfbe-016c-4a81-89c3-be9a47f0a224_isrForCrosstalkSources_996979_51.stderr parsl.dataflow.dflow DEBUG: Task 4696 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4696 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4696 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4696 has memoization hash 65124a6e4edc376164443156fbb8bcc9 parsl.dataflow.memoization INFO: Task 4696 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4696 try 0 launched on executor multi with executor id 4697 parsl.dataflow.dflow INFO: Standard output for task 4696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/94d02795-3efd-4a37-bc59-0c553216a47d_isrForCrosstalkSources_997012_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/94d02795-3efd-4a37-bc59-0c553216a47d_isrForCrosstalkSources_997012_22.stderr parsl.dataflow.dflow DEBUG: Task 4697 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4697 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4697 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4697 has memoization hash 01008868bbe9861976d4678f7f73c3c9 parsl.dataflow.memoization INFO: Task 4697 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d485e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4697 try 0 launched on executor multi with executor id 4698 parsl.dataflow.dflow INFO: Standard output for task 4697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a44d2656-b814-443a-bfd4-4c0da18fd462_isrForCrosstalkSources_996952_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a44d2656-b814-443a-bfd4-4c0da18fd462_isrForCrosstalkSources_996952_7.stderr parsl.dataflow.dflow DEBUG: Task 4698 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4698 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4698 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4698 has memoization hash be7ab23dca3b2f605106cb43f83b4fcf parsl.dataflow.memoization INFO: Task 4698 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4698 try 0 launched on executor multi with executor id 4699 parsl.dataflow.dflow INFO: Standard output for task 4698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c936105f-0dd5-4829-bf5e-3bb9767234ea_isrForCrosstalkSources_996966_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c936105f-0dd5-4829-bf5e-3bb9767234ea_isrForCrosstalkSources_996966_13.stderr parsl.dataflow.dflow DEBUG: Task 4699 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4699 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4699 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4699 has memoization hash 77b4e466268cf07ae5806aadb73cb0c7 parsl.dataflow.memoization INFO: Task 4699 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d487c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4699 try 0 launched on executor multi with executor id 4700 parsl.dataflow.dflow INFO: Standard output for task 4699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/bd023fdb-3319-49dc-a9dd-c6583542a75b_isrForCrosstalkSources_996976_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/bd023fdb-3319-49dc-a9dd-c6583542a75b_isrForCrosstalkSources_996976_6.stderr parsl.dataflow.dflow DEBUG: Task 4700 will be sent to executor multi parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 4700 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 4700 set to pending state with AppFuture: parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Task 4700 has memoization hash d4a07fab6cde2409a0c3ab8047355480 parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization INFO: Task 4700 had no result in cache parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767fffec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:10 proc_lsst.multi:146 [INFO] found job 17074838 in provider astro proc_lsst.multi INFO: found job 17074838 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4700 try 0 launched on executor multi with executor id 4701 parsl.dataflow.dflow INFO: Standard output for task 4700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c060dc7d-f52c-4606-8d69-bfa1d48bc60f_isrForCrosstalkSources_996967_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c060dc7d-f52c-4606-8d69-bfa1d48bc60f_isrForCrosstalkSources_996967_25.stderr parsl.dataflow.dflow DEBUG: Task 4701 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4701 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4701 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4701 has memoization hash 3f00dc3f61f3de3fa3144e49dad53211 parsl.dataflow.memoization INFO: Task 4701 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4701 try 0 launched on executor multi with executor id 4702 parsl.dataflow.dflow INFO: Standard output for task 4701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/098bf01d-a54d-428e-8ba0-cd7b8934e20a_isrForCrosstalkSources_996973_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/098bf01d-a54d-428e-8ba0-cd7b8934e20a_isrForCrosstalkSources_996973_9.stderr parsl.dataflow.dflow DEBUG: Task 4702 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4702 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4702 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4702 has memoization hash 4137ac720d33ca2bb25c0616d1a32a85 parsl.dataflow.memoization INFO: Task 4702 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4bb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4702 try 0 launched on executor multi with executor id 4703 parsl.dataflow.dflow INFO: Standard output for task 4702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a2e10751-f7b4-4f3a-9d0a-8db3bd01fc84_isrForCrosstalkSources_996956_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a2e10751-f7b4-4f3a-9d0a-8db3bd01fc84_isrForCrosstalkSources_996956_56.stderr parsl.dataflow.dflow DEBUG: Task 4703 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4703 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4703 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4703 has memoization hash f53703ffdd6cd10baa613be6dd254eb4 parsl.dataflow.memoization INFO: Task 4703 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4703 try 0 launched on executor multi with executor id 4704 parsl.dataflow.dflow INFO: Standard output for task 4703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/47a4d2f6-b003-4c34-9a40-70a7789114d0_isrForCrosstalkSources_996982_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/47a4d2f6-b003-4c34-9a40-70a7789114d0_isrForCrosstalkSources_996982_18.stderr parsl.dataflow.dflow DEBUG: Task 4704 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4704 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4704 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4704 has memoization hash 7b484cccde7a54fc6d7ce49c94e39937 parsl.dataflow.memoization INFO: Task 4704 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4704 try 0 launched on executor multi with executor id 4705 parsl.dataflow.dflow INFO: Standard output for task 4704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/b73cf6ba-514b-4f79-9145-9257d7237a8f_isrForCrosstalkSources_996982_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/b73cf6ba-514b-4f79-9145-9257d7237a8f_isrForCrosstalkSources_996982_8.stderr parsl.dataflow.dflow DEBUG: Task 4705 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4705 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4705 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4705 has memoization hash 0f9cc949ce79193865dde8eee37452e5 parsl.dataflow.memoization INFO: Task 4705 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4705 try 0 launched on executor multi with executor id 4706 parsl.dataflow.dflow INFO: Standard output for task 4705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/74f4b4c7-0d05-4b13-852e-904a11f81dd3_isrForCrosstalkSources_996952_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/74f4b4c7-0d05-4b13-852e-904a11f81dd3_isrForCrosstalkSources_996952_20.stderr parsl.dataflow.dflow DEBUG: Task 4706 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4706 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4706 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4706 has memoization hash 2a673f352e80d9766f6961e83d13c45e parsl.dataflow.memoization INFO: Task 4706 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4706 try 0 launched on executor multi with executor id 4707 parsl.dataflow.dflow INFO: Standard output for task 4706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/203dacff-24dd-4f9b-9dee-716f24e879a9_isrForCrosstalkSources_996999_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/203dacff-24dd-4f9b-9dee-716f24e879a9_isrForCrosstalkSources_996999_39.stderr parsl.dataflow.dflow DEBUG: Task 4707 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4707 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4707 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4707 has memoization hash cd3763209e661d7651687aa7cd12cbbc parsl.dataflow.memoization INFO: Task 4707 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4707 try 0 launched on executor multi with executor id 4708 parsl.dataflow.dflow INFO: Standard output for task 4707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/c9d9bd19-337a-4613-8e18-e14410f9c5c0_isrForCrosstalkSources_997022_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/c9d9bd19-337a-4613-8e18-e14410f9c5c0_isrForCrosstalkSources_997022_45.stderr parsl.dataflow.dflow DEBUG: Task 4708 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4708 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4708 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4708 has memoization hash e440777637d55d5e6929cfe64a03724f parsl.dataflow.memoization INFO: Task 4708 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4708 try 0 launched on executor multi with executor id 4709 parsl.dataflow.dflow INFO: Standard output for task 4708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7a85cd27-ea9f-4c9f-9986-7998bc8a917b_isrForCrosstalkSources_996984_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7a85cd27-ea9f-4c9f-9986-7998bc8a917b_isrForCrosstalkSources_996984_49.stderr parsl.dataflow.dflow DEBUG: Task 4709 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4709 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4709 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4709 has memoization hash 2e18c697618fbfc27e052971d4991bb7 parsl.dataflow.memoization INFO: Task 4709 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d489a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4709 try 0 launched on executor multi with executor id 4710 parsl.dataflow.dflow INFO: Standard output for task 4709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b33d5895-a264-46c0-bb3b-9b7001254071_isrForCrosstalkSources_997031_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b33d5895-a264-46c0-bb3b-9b7001254071_isrForCrosstalkSources_997031_42.stderr parsl.dataflow.dflow DEBUG: Task 4710 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4710 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4710 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4710 has memoization hash 244a14d4de53609499822248519cee76 parsl.dataflow.memoization INFO: Task 4710 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4710 try 0 launched on executor multi with executor id 4711 parsl.dataflow.dflow INFO: Standard output for task 4710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/4a398c7e-3161-4fd7-b69e-1d8a854877d8_isrForCrosstalkSources_996952_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/4a398c7e-3161-4fd7-b69e-1d8a854877d8_isrForCrosstalkSources_996952_39.stderr parsl.dataflow.dflow DEBUG: Task 4711 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4711 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4711 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4711 has memoization hash e2627c6435f4435608140bbf40a75374 parsl.dataflow.memoization INFO: Task 4711 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4711 try 0 launched on executor multi with executor id 4712 parsl.dataflow.dflow INFO: Standard output for task 4711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/9a686d53-c1c8-4207-ae8d-75e07846e652_isrForCrosstalkSources_997027_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/9a686d53-c1c8-4207-ae8d-75e07846e652_isrForCrosstalkSources_997027_13.stderr parsl.dataflow.dflow DEBUG: Task 4712 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4712 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4712 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4712 has memoization hash e9d31d98e90e0d21539219c59f2da363 parsl.dataflow.memoization INFO: Task 4712 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4712 try 0 launched on executor multi with executor id 4713 parsl.dataflow.dflow INFO: Standard output for task 4712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/efaa3a93-95b7-40c7-b2dd-cbc5e4d2f03e_isrForCrosstalkSources_997011_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/efaa3a93-95b7-40c7-b2dd-cbc5e4d2f03e_isrForCrosstalkSources_997011_42.stderr parsl.dataflow.dflow DEBUG: Task 4713 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4713 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4713 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4713 has memoization hash 8ff41d92618e7dcdd344e93ea2ee9444 parsl.dataflow.memoization INFO: Task 4713 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4713 try 0 launched on executor multi with executor id 4714 parsl.dataflow.dflow INFO: Standard output for task 4713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/42c9ded1-5588-4471-8cd4-e79823cc5325_isrForCrosstalkSources_996988_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/42c9ded1-5588-4471-8cd4-e79823cc5325_isrForCrosstalkSources_996988_18.stderr parsl.dataflow.dflow DEBUG: Task 4714 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4714 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4714 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4714 has memoization hash 0ea277a1c6d6bde40612be5be87dbcc0 parsl.dataflow.memoization INFO: Task 4714 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4714 try 0 launched on executor multi with executor id 4715 parsl.dataflow.dflow INFO: Standard output for task 4714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/741bdeb7-3a86-46a8-8b11-23ff1f36c169_isrForCrosstalkSources_996984_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/741bdeb7-3a86-46a8-8b11-23ff1f36c169_isrForCrosstalkSources_996984_9.stderr parsl.dataflow.dflow DEBUG: Task 4715 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4715 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4715 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4715 has memoization hash 5071c59f5cbd0470ba03ad1b371bb799 parsl.dataflow.memoization INFO: Task 4715 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4715 try 0 launched on executor multi with executor id 4716 parsl.dataflow.dflow INFO: Standard output for task 4715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/160126b2-460d-4d8c-a3d1-5eb7371f8a65_isrForCrosstalkSources_997029_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/160126b2-460d-4d8c-a3d1-5eb7371f8a65_isrForCrosstalkSources_997029_35.stderr parsl.dataflow.dflow DEBUG: Task 4716 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4716 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4716 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4716 has memoization hash 31e99e4e3a6d91517ff16f20111cd352 parsl.dataflow.memoization INFO: Task 4716 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4716 try 0 launched on executor multi with executor id 4717 parsl.dataflow.dflow INFO: Standard output for task 4716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/c5fd04df-6de1-4939-81cb-684c8d6c2ce4_isrForCrosstalkSources_996979_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/c5fd04df-6de1-4939-81cb-684c8d6c2ce4_isrForCrosstalkSources_996979_20.stderr parsl.dataflow.dflow DEBUG: Task 4717 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4717 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4717 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4717 has memoization hash 9efa502eb89194433a57614f802c74c1 parsl.dataflow.memoization INFO: Task 4717 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4717 try 0 launched on executor multi with executor id 4718 parsl.dataflow.dflow INFO: Standard output for task 4717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/643b1c2f-d554-40f0-9550-a73b54e06f46_isrForCrosstalkSources_997010_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/643b1c2f-d554-40f0-9550-a73b54e06f46_isrForCrosstalkSources_997010_3.stderr parsl.dataflow.dflow DEBUG: Task 4718 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4718 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4718 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4718 has memoization hash c5fe6e10e23e3550241f026462ed06ba parsl.dataflow.memoization INFO: Task 4718 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4718 try 0 launched on executor multi with executor id 4719 parsl.dataflow.dflow INFO: Standard output for task 4718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/3cc1e13e-ad79-4f51-bfab-b38f5688fc48_isrForCrosstalkSources_996991_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/3cc1e13e-ad79-4f51-bfab-b38f5688fc48_isrForCrosstalkSources_996991_9.stderr parsl.dataflow.dflow DEBUG: Task 4719 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4719 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4719 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4719 has memoization hash 09dd852fd8e55c1b095a9b0c5ea44434 parsl.dataflow.memoization INFO: Task 4719 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4719 try 0 launched on executor multi with executor id 4720 parsl.dataflow.dflow INFO: Standard output for task 4719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2bc20b56-b2b0-4399-a74c-cfda6f1b5a14_isrForCrosstalkSources_996970_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2bc20b56-b2b0-4399-a74c-cfda6f1b5a14_isrForCrosstalkSources_996970_55.stderr parsl.dataflow.dflow DEBUG: Task 4720 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4720 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4720 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4720 has memoization hash bafcdfeed783205317a2ac9c276396bf parsl.dataflow.memoization INFO: Task 4720 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4720 try 0 launched on executor multi with executor id 4721 parsl.dataflow.dflow INFO: Standard output for task 4720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6b1bc84e-1a81-442f-9ce9-4f835c87d558_isrForCrosstalkSources_997010_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6b1bc84e-1a81-442f-9ce9-4f835c87d558_isrForCrosstalkSources_997010_34.stderr parsl.dataflow.dflow DEBUG: Task 4721 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4721 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4721 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4721 has memoization hash 1c78cea0f79370c6eec4a0ad7884e67d parsl.dataflow.memoization INFO: Task 4721 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d48fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4721 try 0 launched on executor multi with executor id 4722 parsl.dataflow.dflow INFO: Standard output for task 4721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9fa4c950-7f08-4901-9c86-b3df04082ebd_isrForCrosstalkSources_997023_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9fa4c950-7f08-4901-9c86-b3df04082ebd_isrForCrosstalkSources_997023_32.stderr parsl.dataflow.dflow DEBUG: Task 4722 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4722 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4722 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4722 has memoization hash 5056a52c064dd8fdece555da46896d31 parsl.dataflow.memoization INFO: Task 4722 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d491c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4722 try 0 launched on executor multi with executor id 4723 parsl.dataflow.dflow INFO: Standard output for task 4722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/24e6d407-699e-4414-8314-aacc0f01d645_isrForCrosstalkSources_996959_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/24e6d407-699e-4414-8314-aacc0f01d645_isrForCrosstalkSources_996959_40.stderr parsl.dataflow.dflow DEBUG: Task 4723 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4723 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4723 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4723 has memoization hash ccde9c1a347e4b765a683434b4591890 parsl.dataflow.memoization INFO: Task 4723 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4723 try 0 launched on executor multi with executor id 4724 parsl.dataflow.dflow INFO: Standard output for task 4723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/53d58de9-b54e-4504-8264-a451a0814ecc_isrForCrosstalkSources_997027_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/53d58de9-b54e-4504-8264-a451a0814ecc_isrForCrosstalkSources_997027_34.stderr parsl.dataflow.dflow DEBUG: Task 4724 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4724 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4724 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4724 has memoization hash 29f69a84ca86471d57e112db4f1464e8 parsl.dataflow.memoization INFO: Task 4724 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4724 try 0 launched on executor multi with executor id 4725 parsl.dataflow.dflow INFO: Standard output for task 4724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/5a915cfd-6cd2-4234-8359-f4b3429ee550_isrForCrosstalkSources_997022_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/5a915cfd-6cd2-4234-8359-f4b3429ee550_isrForCrosstalkSources_997022_14.stderr parsl.dataflow.dflow DEBUG: Task 4725 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4725 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4725 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4725 has memoization hash 8a3ee4f7d99d57e22b98e1d697ad1b66 parsl.dataflow.memoization INFO: Task 4725 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d493a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4725 try 0 launched on executor multi with executor id 4726 parsl.dataflow.dflow INFO: Standard output for task 4725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a07c454-7e09-4628-b064-7205d96445c4_isrForCrosstalkSources_997004_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a07c454-7e09-4628-b064-7205d96445c4_isrForCrosstalkSources_997004_33.stderr parsl.dataflow.dflow DEBUG: Task 4726 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4726 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4726 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4726 has memoization hash 8a724a16f87be414bd8cf4b89f6c99cb parsl.dataflow.memoization INFO: Task 4726 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4726 try 0 launched on executor multi with executor id 4727 parsl.dataflow.dflow INFO: Standard output for task 4726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/659077f5-d3ab-4e5a-81a3-bef50a820427_isrForCrosstalkSources_996985_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/659077f5-d3ab-4e5a-81a3-bef50a820427_isrForCrosstalkSources_996985_35.stderr parsl.dataflow.dflow DEBUG: Task 4727 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4727 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4727 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4727 has memoization hash d369333b6113da67904393d6b71e64bf parsl.dataflow.memoization INFO: Task 4727 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d494e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4727 try 0 launched on executor multi with executor id 4728 parsl.dataflow.dflow INFO: Standard output for task 4727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b086b559-4c08-4ec6-97fa-504a063d80b8_isrForCrosstalkSources_997004_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b086b559-4c08-4ec6-97fa-504a063d80b8_isrForCrosstalkSources_997004_24.stderr parsl.dataflow.dflow DEBUG: Task 4728 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4728 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4728 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4728 has memoization hash 99ac3d388365832a6e1cb8eca930f9d0 parsl.dataflow.memoization INFO: Task 4728 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4728 try 0 launched on executor multi with executor id 4729 parsl.dataflow.dflow INFO: Standard output for task 4728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/49ab4feb-9f37-441f-8fe3-38531aa1560c_isrForCrosstalkSources_996985_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/49ab4feb-9f37-441f-8fe3-38531aa1560c_isrForCrosstalkSources_996985_40.stderr parsl.dataflow.dflow DEBUG: Task 4729 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4729 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4729 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4729 has memoization hash 7ef712a8944312461be000fbaf5d4432 parsl.dataflow.memoization INFO: Task 4729 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4729 try 0 launched on executor multi with executor id 4730 parsl.dataflow.dflow INFO: Standard output for task 4729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b053d659-3263-426d-87a8-cba77cd2e384_isrForCrosstalkSources_997006_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b053d659-3263-426d-87a8-cba77cd2e384_isrForCrosstalkSources_997006_36.stderr parsl.dataflow.dflow DEBUG: Task 4730 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4730 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4730 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4730 has memoization hash 99d5d84591abf4c49fdbbe12799af696 parsl.dataflow.memoization INFO: Task 4730 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4730 try 0 launched on executor multi with executor id 4731 parsl.dataflow.dflow INFO: Standard output for task 4730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7468517c-3dfe-4dca-9d4d-2dac660b08b0_isrForCrosstalkSources_997005_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7468517c-3dfe-4dca-9d4d-2dac660b08b0_isrForCrosstalkSources_997005_12.stderr parsl.dataflow.dflow DEBUG: Task 4731 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4731 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4731 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4731 has memoization hash 724e58f6bc470764c34bbd2c9e3c35d0 parsl.dataflow.memoization INFO: Task 4731 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4731 try 0 launched on executor multi with executor id 4732 parsl.dataflow.dflow INFO: Standard output for task 4731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/fc853780-10e0-434f-8bf6-86267b155c69_isrForCrosstalkSources_996959_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/fc853780-10e0-434f-8bf6-86267b155c69_isrForCrosstalkSources_996959_7.stderr parsl.dataflow.dflow DEBUG: Task 4732 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4732 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4732 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4732 has memoization hash 7c495ba56a3765facfe0e178b011bb1c parsl.dataflow.memoization INFO: Task 4732 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d496c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4732 try 0 launched on executor multi with executor id 4733 parsl.dataflow.dflow INFO: Standard output for task 4732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0134c818-d3ec-47ad-a7a3-c9266032b9c5_isrForCrosstalkSources_997025_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0134c818-d3ec-47ad-a7a3-c9266032b9c5_isrForCrosstalkSources_997025_26.stderr parsl.dataflow.dflow DEBUG: Task 4733 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4733 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4733 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4733 has memoization hash 8d50d3702955d2c551d3f820cb7705d8 parsl.dataflow.memoization INFO: Task 4733 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4733 try 0 launched on executor multi with executor id 4734 parsl.dataflow.dflow INFO: Standard output for task 4733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/a440741f-fc45-4551-8d73-cf26622e4c84_isrForCrosstalkSources_997025_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/a440741f-fc45-4551-8d73-cf26622e4c84_isrForCrosstalkSources_997025_7.stderr parsl.dataflow.dflow DEBUG: Task 4734 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4734 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4734 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4734 has memoization hash 896f708dbd3077098c7c22eca31833a2 parsl.dataflow.memoization INFO: Task 4734 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4734 try 0 launched on executor multi with executor id 4735 parsl.dataflow.dflow INFO: Standard output for task 4734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/63144f9f-dd47-4aa4-b13a-b6f3695f220b_isrForCrosstalkSources_996982_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/63144f9f-dd47-4aa4-b13a-b6f3695f220b_isrForCrosstalkSources_996982_46.stderr parsl.dataflow.dflow DEBUG: Task 4735 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4735 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4735 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4735 has memoization hash 50aa183d91fa22d24b0470bbaa8522a1 parsl.dataflow.memoization INFO: Task 4735 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4735 try 0 launched on executor multi with executor id 4736 parsl.dataflow.dflow INFO: Standard output for task 4735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/28a39f36-7aa9-4e14-8612-26280800869c_isrForCrosstalkSources_997012_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/28a39f36-7aa9-4e14-8612-26280800869c_isrForCrosstalkSources_997012_7.stderr parsl.dataflow.dflow DEBUG: Task 4736 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4736 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4736 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4736 has memoization hash 4284935388f66b63374b822c8fca0c58 parsl.dataflow.memoization INFO: Task 4736 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d498a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4736 try 0 launched on executor multi with executor id 4737 parsl.dataflow.dflow INFO: Standard output for task 4736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/c3d3fefe-b0c4-4a25-99d0-04c8f71525f3_isrForCrosstalkSources_997011_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/c3d3fefe-b0c4-4a25-99d0-04c8f71525f3_isrForCrosstalkSources_997011_52.stderr parsl.dataflow.dflow DEBUG: Task 4737 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4737 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4737 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4737 has memoization hash 8cdb3557b71bdb0cce4b5eb7e565f38f parsl.dataflow.memoization INFO: Task 4737 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d499e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4737 try 0 launched on executor multi with executor id 4738 parsl.dataflow.dflow INFO: Standard output for task 4737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/45b107aa-3986-40ed-bf33-6c8633a56fd3_isrForCrosstalkSources_996980_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/45b107aa-3986-40ed-bf33-6c8633a56fd3_isrForCrosstalkSources_996980_62.stderr parsl.dataflow.dflow DEBUG: Task 4738 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4738 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4738 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4738 has memoization hash 05d731541e54902a7f690b2be938c230 parsl.dataflow.memoization INFO: Task 4738 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4738 try 0 launched on executor multi with executor id 4739 parsl.dataflow.dflow INFO: Standard output for task 4738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/302de15a-8731-4a38-bd9b-bdf192c72588_isrForCrosstalkSources_996964_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/302de15a-8731-4a38-bd9b-bdf192c72588_isrForCrosstalkSources_996964_40.stderr parsl.dataflow.dflow DEBUG: Task 4739 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4739 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4739 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4739 has memoization hash 9d2691b8bb921358d6c72817dfa7033d parsl.dataflow.memoization INFO: Task 4739 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4739 try 0 launched on executor multi with executor id 4740 parsl.dataflow.dflow INFO: Standard output for task 4739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/4b4da93f-0dc4-4a4b-937e-a9ddd4ca24f0_isrForCrosstalkSources_997032_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/4b4da93f-0dc4-4a4b-937e-a9ddd4ca24f0_isrForCrosstalkSources_997032_7.stderr parsl.dataflow.dflow DEBUG: Task 4740 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4740 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4740 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4740 has memoization hash ca8d4e862330eae59e2fbaebe114dbf7 parsl.dataflow.memoization INFO: Task 4740 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4740 try 0 launched on executor multi with executor id 4741 parsl.dataflow.dflow INFO: Standard output for task 4740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fab0ebb2-d8e2-4cf6-bb8e-f973da118f70_isrForCrosstalkSources_996990_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fab0ebb2-d8e2-4cf6-bb8e-f973da118f70_isrForCrosstalkSources_996990_16.stderr parsl.dataflow.dflow DEBUG: Task 4741 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4741 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4741 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4741 has memoization hash 259f5b3280a60c46947c0cabc1c192cf parsl.dataflow.memoization INFO: Task 4741 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4741 try 0 launched on executor multi with executor id 4742 parsl.dataflow.dflow INFO: Standard output for task 4741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4ea4d1c2-59d1-44fe-9636-e838e3bc4f67_isrForCrosstalkSources_996995_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4ea4d1c2-59d1-44fe-9636-e838e3bc4f67_isrForCrosstalkSources_996995_18.stderr parsl.dataflow.dflow DEBUG: Task 4742 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4742 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4742 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4742 has memoization hash f65319fbe378a96366706278ef526648 parsl.dataflow.memoization INFO: Task 4742 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4742 try 0 launched on executor multi with executor id 4743 parsl.dataflow.dflow INFO: Standard output for task 4742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f1e46bfc-28ab-4a64-9e63-2ddba3d72411_isrForCrosstalkSources_996955_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f1e46bfc-28ab-4a64-9e63-2ddba3d72411_isrForCrosstalkSources_996955_30.stderr parsl.dataflow.dflow DEBUG: Task 4743 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4743 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4743 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4743 has memoization hash facddfa92f09cdfad62530b28db4cedc parsl.dataflow.memoization INFO: Task 4743 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4743 try 0 launched on executor multi with executor id 4744 parsl.dataflow.dflow INFO: Standard output for task 4743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/23b9b745-cb60-4898-9db1-02084d2032a5_isrForCrosstalkSources_996984_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/23b9b745-cb60-4898-9db1-02084d2032a5_isrForCrosstalkSources_996984_35.stderr parsl.dataflow.dflow DEBUG: Task 4744 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4744 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4744 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4744 has memoization hash c76777e553167b9d7891184fa289033a parsl.dataflow.memoization INFO: Task 4744 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4744 try 0 launched on executor multi with executor id 4745 parsl.dataflow.dflow INFO: Standard output for task 4744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/08bdd6a2-5af3-4e95-bf37-db3d35ca82c4_isrForCrosstalkSources_996994_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/08bdd6a2-5af3-4e95-bf37-db3d35ca82c4_isrForCrosstalkSources_996994_28.stderr parsl.dataflow.dflow DEBUG: Task 4745 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4745 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4745 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4745 has memoization hash 222f6bd35b7a0be8f3b8c349d0ef5317 parsl.dataflow.memoization INFO: Task 4745 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4745 try 0 launched on executor multi with executor id 4746 parsl.dataflow.dflow INFO: Standard output for task 4745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e28e04e1-65a5-4133-afff-baada127eda3_isrForCrosstalkSources_996957_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e28e04e1-65a5-4133-afff-baada127eda3_isrForCrosstalkSources_996957_42.stderr parsl.dataflow.dflow DEBUG: Task 4746 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4746 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4746 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4746 has memoization hash 7302a81070692849be91f12c4975a6d1 parsl.dataflow.memoization INFO: Task 4746 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d49e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4746 try 0 launched on executor multi with executor id 4747 parsl.dataflow.dflow INFO: Standard output for task 4746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/09f8800d-5088-4632-865e-b1cc90e8c5fb_isrForCrosstalkSources_997012_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/09f8800d-5088-4632-865e-b1cc90e8c5fb_isrForCrosstalkSources_997012_46.stderr parsl.dataflow.dflow DEBUG: Task 4747 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4747 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4747 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4747 has memoization hash 6fc05bd02b1f4af0b874019fa00aa633 parsl.dataflow.memoization INFO: Task 4747 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4747 try 0 launched on executor multi with executor id 4748 parsl.dataflow.dflow INFO: Standard output for task 4747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/288a71ae-9d00-4a5a-9ec6-273258a6a4b5_isrForCrosstalkSources_996974_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/288a71ae-9d00-4a5a-9ec6-273258a6a4b5_isrForCrosstalkSources_996974_20.stderr parsl.dataflow.dflow DEBUG: Task 4748 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4748 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4748 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4748 has memoization hash abba48c2df605b4f702d135811822fc4 parsl.dataflow.memoization INFO: Task 4748 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4748 try 0 launched on executor multi with executor id 4749 parsl.dataflow.dflow INFO: Standard output for task 4748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/5c8b2e0b-1674-4c91-8d53-7130deef92fe_isrForCrosstalkSources_997028_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/5c8b2e0b-1674-4c91-8d53-7130deef92fe_isrForCrosstalkSources_997028_44.stderr parsl.dataflow.dflow DEBUG: Task 4749 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4749 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4749 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4749 has memoization hash 3b4fb41c65506eab1fe882e34c3c1ae8 parsl.dataflow.memoization INFO: Task 4749 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4749 try 0 launched on executor multi with executor id 4750 parsl.dataflow.dflow INFO: Standard output for task 4749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/05946fe8-13b0-4022-a21d-e1ced42d768c_isrForCrosstalkSources_997022_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/05946fe8-13b0-4022-a21d-e1ced42d768c_isrForCrosstalkSources_997022_24.stderr parsl.dataflow.dflow DEBUG: Task 4750 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4750 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4750 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4750 has memoization hash 7e7928cb0d96401788800d4f50feeb1e parsl.dataflow.memoization INFO: Task 4750 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4750 try 0 launched on executor multi with executor id 4751 parsl.dataflow.dflow INFO: Standard output for task 4750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/4b90119e-8e19-4d0f-ac37-e576b4968f11_isrForCrosstalkSources_996973_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/4b90119e-8e19-4d0f-ac37-e576b4968f11_isrForCrosstalkSources_996973_51.stderr parsl.dataflow.dflow DEBUG: Task 4751 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4751 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4751 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4751 has memoization hash 5df9bef9f46cb2dba6b003d88b528529 parsl.dataflow.memoization INFO: Task 4751 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4751 try 0 launched on executor multi with executor id 4752 parsl.dataflow.dflow INFO: Standard output for task 4751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f9adef0b-fc29-4bbf-888d-78df9809996e_isrForCrosstalkSources_996993_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f9adef0b-fc29-4bbf-888d-78df9809996e_isrForCrosstalkSources_996993_55.stderr parsl.dataflow.dflow DEBUG: Task 4752 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4752 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4752 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4752 has memoization hash 0bc5d9637665651a71876c8f8eb75938 parsl.dataflow.memoization INFO: Task 4752 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4752 try 0 launched on executor multi with executor id 4753 parsl.dataflow.dflow INFO: Standard output for task 4752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/4fa6cc0c-8573-4e31-9bd3-53e02188860b_isrForCrosstalkSources_996981_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/4fa6cc0c-8573-4e31-9bd3-53e02188860b_isrForCrosstalkSources_996981_47.stderr parsl.dataflow.dflow DEBUG: Task 4753 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4753 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4753 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4753 has memoization hash df455c77c77307042831100619972266 parsl.dataflow.memoization INFO: Task 4753 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4753 try 0 launched on executor multi with executor id 4754 parsl.dataflow.dflow INFO: Standard output for task 4753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/674ee716-6a33-4390-99b3-79913d14db0e_isrForCrosstalkSources_996958_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/674ee716-6a33-4390-99b3-79913d14db0e_isrForCrosstalkSources_996958_31.stderr parsl.dataflow.dflow DEBUG: Task 4754 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4754 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4754 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4754 has memoization hash a307c08731529bc0281cb1d3e82b1cf2 parsl.dataflow.memoization INFO: Task 4754 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4754 try 0 launched on executor multi with executor id 4755 parsl.dataflow.dflow INFO: Standard output for task 4754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/cfb9691c-64fe-42a8-84c5-4c83420d6235_isrForCrosstalkSources_996976_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/cfb9691c-64fe-42a8-84c5-4c83420d6235_isrForCrosstalkSources_996976_47.stderr parsl.dataflow.dflow DEBUG: Task 4755 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4755 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4755 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4755 has memoization hash d742791b80133b8a651a06d8b66c3f75 parsl.dataflow.memoization INFO: Task 4755 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4755 try 0 launched on executor multi with executor id 4756 parsl.dataflow.dflow INFO: Standard output for task 4755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c08bd6c5-e2b1-4fbd-934e-9aa9dc6c799b_isrForCrosstalkSources_996973_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c08bd6c5-e2b1-4fbd-934e-9aa9dc6c799b_isrForCrosstalkSources_996973_2.stderr parsl.dataflow.dflow DEBUG: Task 4756 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4756 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4756 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4756 has memoization hash 02af1547ee02c2a18f16e0997f93bb76 parsl.dataflow.memoization INFO: Task 4756 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4756 try 0 launched on executor multi with executor id 4757 parsl.dataflow.dflow INFO: Standard output for task 4756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/fceae346-14ce-46b2-aeea-d0f8d1dc41e4_isrForCrosstalkSources_996952_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/fceae346-14ce-46b2-aeea-d0f8d1dc41e4_isrForCrosstalkSources_996952_14.stderr parsl.dataflow.dflow DEBUG: Task 4757 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4757 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4757 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4757 has memoization hash cc9884dcfd37c4230432a6b2c0766571 parsl.dataflow.memoization INFO: Task 4757 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4757 try 0 launched on executor multi with executor id 4758 parsl.dataflow.dflow INFO: Standard output for task 4757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0666bbc9-c8a1-4866-8673-af4fe6d63488_isrForCrosstalkSources_996997_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0666bbc9-c8a1-4866-8673-af4fe6d63488_isrForCrosstalkSources_996997_59.stderr parsl.dataflow.dflow DEBUG: Task 4758 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4758 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4758 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4758 has memoization hash fd0e07a938e25a8c915b88907d082958 parsl.dataflow.memoization INFO: Task 4758 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4758 try 0 launched on executor multi with executor id 4759 parsl.dataflow.dflow INFO: Standard output for task 4758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/3a026ef4-f910-484e-99f5-b25ca316353c_isrForCrosstalkSources_997027_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/3a026ef4-f910-484e-99f5-b25ca316353c_isrForCrosstalkSources_997027_24.stderr parsl.dataflow.dflow DEBUG: Task 4759 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4759 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4759 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4759 has memoization hash 65ed0eb24ef72ec846cecdd7fb8e3aa7 parsl.dataflow.memoization INFO: Task 4759 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4759 try 0 launched on executor multi with executor id 4760 parsl.dataflow.dflow INFO: Standard output for task 4759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/42c68201-dac8-495a-a559-3e7836002ce8_isrForCrosstalkSources_996978_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/42c68201-dac8-495a-a559-3e7836002ce8_isrForCrosstalkSources_996978_5.stderr parsl.dataflow.dflow DEBUG: Task 4760 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4760 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4760 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4760 has memoization hash eddf44658e738a653ab1b5c0cdc340d1 parsl.dataflow.memoization INFO: Task 4760 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4760 try 0 launched on executor multi with executor id 4761 parsl.dataflow.dflow INFO: Standard output for task 4760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9ba11b09-ec88-4221-b0c5-9f7b66aa669a_isrForCrosstalkSources_996971_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9ba11b09-ec88-4221-b0c5-9f7b66aa669a_isrForCrosstalkSources_996971_21.stderr parsl.dataflow.dflow DEBUG: Task 4761 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4761 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4761 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4761 has memoization hash 2ceb53fcbbdeed68ade046c6135130e4 parsl.dataflow.memoization INFO: Task 4761 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4761 try 0 launched on executor multi with executor id 4762 parsl.dataflow.dflow INFO: Standard output for task 4761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b450fee8-c754-4e65-b34d-5ccec9f1047a_isrForCrosstalkSources_996998_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b450fee8-c754-4e65-b34d-5ccec9f1047a_isrForCrosstalkSources_996998_23.stderr parsl.dataflow.dflow DEBUG: Task 4762 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4762 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4762 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4762 has memoization hash a24eaffa64b9d2ebb903bb19a279855a parsl.dataflow.memoization INFO: Task 4762 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4762 try 0 launched on executor multi with executor id 4763 parsl.dataflow.dflow INFO: Standard output for task 4762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/94a527de-3642-459e-8942-ded8a49a4abe_isrForCrosstalkSources_996960_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/94a527de-3642-459e-8942-ded8a49a4abe_isrForCrosstalkSources_996960_31.stderr parsl.dataflow.dflow DEBUG: Task 4763 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4763 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4763 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4763 has memoization hash 5233a35f874f8be76a7f39f351bfb153 parsl.dataflow.memoization INFO: Task 4763 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4763 try 0 launched on executor multi with executor id 4764 parsl.dataflow.dflow INFO: Standard output for task 4763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/34a7fec2-35cf-49ce-9793-5bc705fea95a_isrForCrosstalkSources_996969_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/34a7fec2-35cf-49ce-9793-5bc705fea95a_isrForCrosstalkSources_996969_17.stderr parsl.dataflow.dflow DEBUG: Task 4764 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4764 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4764 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4764 has memoization hash 5c5589d62bea03c9f17feb8f7db7dd82 parsl.dataflow.memoization INFO: Task 4764 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4764 try 0 launched on executor multi with executor id 4765 parsl.dataflow.dflow INFO: Standard output for task 4764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ca9e5713-805e-4107-8ea0-febb81096f73_isrForCrosstalkSources_996982_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ca9e5713-805e-4107-8ea0-febb81096f73_isrForCrosstalkSources_996982_50.stderr parsl.dataflow.dflow DEBUG: Task 4765 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4765 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4765 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4765 has memoization hash 2b84d48f4ef9056cb6f46da4a14fa5f7 parsl.dataflow.memoization INFO: Task 4765 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4765 try 0 launched on executor multi with executor id 4766 parsl.dataflow.dflow INFO: Standard output for task 4765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/34654c7b-9da6-441e-ae37-8407383cd6e5_isrForCrosstalkSources_996999_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/34654c7b-9da6-441e-ae37-8407383cd6e5_isrForCrosstalkSources_996999_36.stderr parsl.dataflow.dflow DEBUG: Task 4766 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4766 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4766 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4766 has memoization hash 01edb9e1f571b45994cdab2f8a57606e parsl.dataflow.memoization INFO: Task 4766 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4766 try 0 launched on executor multi with executor id 4767 parsl.dataflow.dflow INFO: Standard output for task 4766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d45a8a2d-e8a1-4988-a9a0-5093baf7ab6e_isrForCrosstalkSources_996981_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d45a8a2d-e8a1-4988-a9a0-5093baf7ab6e_isrForCrosstalkSources_996981_28.stderr parsl.dataflow.dflow DEBUG: Task 4767 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4767 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4767 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4767 has memoization hash 23b5817bbfe158d6e1f191a0215f5a9c parsl.dataflow.memoization INFO: Task 4767 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4767 try 0 launched on executor multi with executor id 4768 parsl.dataflow.dflow INFO: Standard output for task 4767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3d660650-f1cb-462b-9190-2b357fb14683_isrForCrosstalkSources_996984_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3d660650-f1cb-462b-9190-2b357fb14683_isrForCrosstalkSources_996984_41.stderr parsl.dataflow.dflow DEBUG: Task 4768 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4768 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4768 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4768 has memoization hash 4fb7ba7d11ca22959e9cf6a0d27fb5e2 parsl.dataflow.memoization INFO: Task 4768 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4b060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4768 try 0 launched on executor multi with executor id 4769 parsl.dataflow.dflow INFO: Standard output for task 4768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/90dcd1b1-1213-4329-9a3f-94c6e6b85e7d_isrForCrosstalkSources_996960_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/90dcd1b1-1213-4329-9a3f-94c6e6b85e7d_isrForCrosstalkSources_996960_28.stderr parsl.dataflow.dflow DEBUG: Task 4769 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4769 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4769 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4769 has memoization hash 66d647bf352c9177ee23d228abae1e76 parsl.dataflow.memoization INFO: Task 4769 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4afc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4769 try 0 launched on executor multi with executor id 4770 parsl.dataflow.dflow INFO: Standard output for task 4769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0d8d5805-f21c-4882-996b-928a6e6933d7_isrForCrosstalkSources_996966_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0d8d5805-f21c-4882-996b-928a6e6933d7_isrForCrosstalkSources_996966_11.stderr parsl.dataflow.dflow DEBUG: Task 4770 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4770 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4770 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4770 has memoization hash 4b3990b09d27a8d73ded54ecf467f7ec parsl.dataflow.memoization INFO: Task 4770 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4ae80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4770 try 0 launched on executor multi with executor id 4771 parsl.dataflow.dflow INFO: Standard output for task 4770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e16d8268-0419-4ce7-962e-d8917dbb107b_isrForCrosstalkSources_997005_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e16d8268-0419-4ce7-962e-d8917dbb107b_isrForCrosstalkSources_997005_45.stderr parsl.dataflow.dflow DEBUG: Task 4771 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4771 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4771 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4771 has memoization hash de7e1f6531087383ea9266709ccb67ea parsl.dataflow.memoization INFO: Task 4771 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4af20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:10 proc_lsst.multi:146 [INFO] found job 17074839 in provider astro proc_lsst.multi INFO: found job 17074839 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4771 try 0 launched on executor multi with executor id 4772 parsl.dataflow.dflow INFO: Standard output for task 4771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/3ba58704-102b-431b-b7d2-2a6976685f03_isrForCrosstalkSources_997020_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/3ba58704-102b-431b-b7d2-2a6976685f03_isrForCrosstalkSources_997020_14.stderr parsl.dataflow.dflow DEBUG: Task 4772 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4772 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4772 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4772 has memoization hash c79d5122bb1ec3238dd98bc4a91749d9 parsl.dataflow.memoization INFO: Task 4772 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4ad40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4772 try 0 launched on executor multi with executor id 4773 parsl.dataflow.dflow INFO: Standard output for task 4772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5243bd19-f8ea-4413-bbb4-efe14284d74d_isrForCrosstalkSources_996954_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5243bd19-f8ea-4413-bbb4-efe14284d74d_isrForCrosstalkSources_996954_39.stderr parsl.dataflow.dflow DEBUG: Task 4773 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4773 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4773 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4773 has memoization hash 6ebdb315d684d9eb9db193308c0620a9 parsl.dataflow.memoization INFO: Task 4773 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4aca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4773 try 0 launched on executor multi with executor id 4774 parsl.dataflow.dflow INFO: Standard output for task 4773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/1bb4588b-5f06-4486-ac82-32e65c0553d1_isrForCrosstalkSources_997002_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/1bb4588b-5f06-4486-ac82-32e65c0553d1_isrForCrosstalkSources_997002_30.stderr parsl.dataflow.dflow DEBUG: Task 4774 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4774 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4774 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4774 has memoization hash b0c41eb45aafb5b82a06e163271e86ff parsl.dataflow.memoization INFO: Task 4774 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a780e74400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4774 try 0 launched on executor multi with executor id 4775 parsl.dataflow.dflow INFO: Standard output for task 4774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b033272c-1a03-4c26-a264-03cfb99596b3_isrForCrosstalkSources_997001_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b033272c-1a03-4c26-a264-03cfb99596b3_isrForCrosstalkSources_997001_23.stderr parsl.dataflow.dflow DEBUG: Task 4775 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4775 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4775 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4775 has memoization hash 996733964046ff480e1c506e2cdd9f2d parsl.dataflow.memoization INFO: Task 4775 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4ade0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4775 try 0 launched on executor multi with executor id 4776 parsl.dataflow.dflow INFO: Standard output for task 4775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9277410-5a42-4bbe-a09c-981df26d19ef_isrForCrosstalkSources_997009_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9277410-5a42-4bbe-a09c-981df26d19ef_isrForCrosstalkSources_997009_48.stderr parsl.dataflow.dflow DEBUG: Task 4776 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4776 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4776 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4776 has memoization hash 36cb5328f267e8c5ab60c7609b3d916b parsl.dataflow.memoization INFO: Task 4776 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4ab60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4776 try 0 launched on executor multi with executor id 4777 parsl.dataflow.dflow INFO: Standard output for task 4776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/f214442b-4c04-4ed0-846f-9f089f2c0166_isrForCrosstalkSources_996952_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/f214442b-4c04-4ed0-846f-9f089f2c0166_isrForCrosstalkSources_996952_47.stderr parsl.dataflow.dflow DEBUG: Task 4777 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4777 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4777 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4777 has memoization hash b6547901947c574d4f212f49d0670c97 parsl.dataflow.memoization INFO: Task 4777 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4777 try 0 launched on executor multi with executor id 4778 parsl.dataflow.dflow INFO: Standard output for task 4777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/7b3514a2-ee69-43a9-b436-a088aa3f5abc_isrForCrosstalkSources_996982_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/7b3514a2-ee69-43a9-b436-a088aa3f5abc_isrForCrosstalkSources_996982_1.stderr parsl.dataflow.dflow DEBUG: Task 4778 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4778 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4778 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4778 has memoization hash 7b27f94dce951058353e04ef8226d6e4 parsl.dataflow.memoization INFO: Task 4778 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4778 try 0 launched on executor multi with executor id 4779 parsl.dataflow.dflow INFO: Standard output for task 4778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7fa6c051-78e3-45ba-8ab8-80068e26ca35_isrForCrosstalkSources_996971_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7fa6c051-78e3-45ba-8ab8-80068e26ca35_isrForCrosstalkSources_996971_20.stderr parsl.dataflow.dflow DEBUG: Task 4779 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4779 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4779 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4779 has memoization hash 04bba4e9f52d96b2cb9b2e0995509e31 parsl.dataflow.memoization INFO: Task 4779 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4aa20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4779 try 0 launched on executor multi with executor id 4780 parsl.dataflow.dflow INFO: Standard output for task 4779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/ff0f1246-213b-494c-8a36-7d05955b2d9c_isrForCrosstalkSources_997011_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/ff0f1246-213b-494c-8a36-7d05955b2d9c_isrForCrosstalkSources_997011_55.stderr parsl.dataflow.dflow DEBUG: Task 4780 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4780 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4780 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4780 has memoization hash c7939b62057ea5a6d4b5fa08db2732f5 parsl.dataflow.memoization INFO: Task 4780 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4780 try 0 launched on executor multi with executor id 4781 parsl.dataflow.dflow INFO: Standard output for task 4780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d90d15f2-0f57-422f-b81a-b32ff486e691_isrForCrosstalkSources_996982_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d90d15f2-0f57-422f-b81a-b32ff486e691_isrForCrosstalkSources_996982_41.stderr parsl.dataflow.dflow DEBUG: Task 4781 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4781 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4781 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4781 has memoization hash 5842eda515ea4f86f6873ebe45212511 parsl.dataflow.memoization INFO: Task 4781 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4781 try 0 launched on executor multi with executor id 4782 parsl.dataflow.dflow INFO: Standard output for task 4781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/2547de06-3557-43b9-8981-c0fff41dda87_isrForCrosstalkSources_996954_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/2547de06-3557-43b9-8981-c0fff41dda87_isrForCrosstalkSources_996954_27.stderr parsl.dataflow.dflow DEBUG: Task 4782 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4782 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4782 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4782 has memoization hash 29c7fcc9f6b2200408317a24450bbb11 parsl.dataflow.memoization INFO: Task 4782 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4a660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4782 try 0 launched on executor multi with executor id 4783 parsl.dataflow.dflow INFO: Standard output for task 4782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/b8b429f9-b990-4d82-aeba-f036a6478c33_isrForCrosstalkSources_996968_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/b8b429f9-b990-4d82-aeba-f036a6478c33_isrForCrosstalkSources_996968_17.stderr parsl.dataflow.dflow DEBUG: Task 4783 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4783 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4783 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4783 has memoization hash 899aa96deffcf5485a2ae5df1cebc448 parsl.dataflow.memoization INFO: Task 4783 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4bba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4783 try 0 launched on executor multi with executor id 4784 parsl.dataflow.dflow INFO: Standard output for task 4783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/4d991ba3-bb7b-4eed-a9a0-8be2c43700f9_isrForCrosstalkSources_997010_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/4d991ba3-bb7b-4eed-a9a0-8be2c43700f9_isrForCrosstalkSources_997010_36.stderr parsl.dataflow.dflow DEBUG: Task 4784 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4784 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4784 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4784 has memoization hash f4066e8fddcafe1c5bf4a9b02f69578e parsl.dataflow.memoization INFO: Task 4784 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4bd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4784 try 0 launched on executor multi with executor id 4785 parsl.dataflow.dflow INFO: Standard output for task 4784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e82a1d63-5d3c-4dc5-b8d5-44b8f9698a25_isrForCrosstalkSources_997002_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e82a1d63-5d3c-4dc5-b8d5-44b8f9698a25_isrForCrosstalkSources_997002_59.stderr parsl.dataflow.dflow DEBUG: Task 4785 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4785 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4785 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4785 has memoization hash af6b12d5c01731094749fbb8d9b80de9 parsl.dataflow.memoization INFO: Task 4785 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4bce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4785 try 0 launched on executor multi with executor id 4786 parsl.dataflow.dflow INFO: Standard output for task 4785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ef1d40e0-dfe9-469e-bcd9-7ae1f3d54367_isrForCrosstalkSources_997016_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ef1d40e0-dfe9-469e-bcd9-7ae1f3d54367_isrForCrosstalkSources_997016_12.stderr parsl.dataflow.dflow DEBUG: Task 4786 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4786 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4786 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4786 has memoization hash 198c9d789b1ca6b2dd17fba69d42e39c parsl.dataflow.memoization INFO: Task 4786 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4bc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4786 try 0 launched on executor multi with executor id 4787 parsl.dataflow.dflow INFO: Standard output for task 4786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/5367b1ba-1ef0-4ba4-bc92-9c96ae679b5f_isrForCrosstalkSources_996957_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/5367b1ba-1ef0-4ba4-bc92-9c96ae679b5f_isrForCrosstalkSources_996957_34.stderr parsl.dataflow.dflow DEBUG: Task 4787 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4787 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4787 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4787 has memoization hash 27cc17dcfb18e9148c4d4940f0036c8b parsl.dataflow.memoization INFO: Task 4787 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4be20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4787 try 0 launched on executor multi with executor id 4788 parsl.dataflow.dflow INFO: Standard output for task 4787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/95ed680b-f5d7-4578-bff3-694b89b74caf_isrForCrosstalkSources_996990_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/95ed680b-f5d7-4578-bff3-694b89b74caf_isrForCrosstalkSources_996990_12.stderr parsl.dataflow.dflow DEBUG: Task 4788 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4788 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4788 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4788 has memoization hash 771dda9e9895294b76f644ce9088a6ad parsl.dataflow.memoization INFO: Task 4788 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4bec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4788 try 0 launched on executor multi with executor id 4789 parsl.dataflow.dflow INFO: Standard output for task 4788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/59154914-99ec-4d55-b40e-171b6e920dd6_isrForCrosstalkSources_997012_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/59154914-99ec-4d55-b40e-171b6e920dd6_isrForCrosstalkSources_997012_62.stderr parsl.dataflow.dflow DEBUG: Task 4789 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4789 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4789 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4789 has memoization hash 38dfee8fc7ebfaf3d200b3f089be3a11 parsl.dataflow.memoization INFO: Task 4789 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4aac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4789 try 0 launched on executor multi with executor id 4790 parsl.dataflow.dflow INFO: Standard output for task 4789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d1a65c13-5878-4414-a268-9de23aba57d7_isrForCrosstalkSources_996980_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d1a65c13-5878-4414-a268-9de23aba57d7_isrForCrosstalkSources_996980_60.stderr parsl.dataflow.dflow DEBUG: Task 4790 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4790 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4790 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4790 has memoization hash a5fa4e748fc85ba96bac2fa38d11f13a parsl.dataflow.memoization INFO: Task 4790 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4bf60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4790 try 0 launched on executor multi with executor id 4791 parsl.dataflow.dflow INFO: Standard output for task 4790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9424f062-afb4-4933-86b6-0d200eb82ef0_isrForCrosstalkSources_997018_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9424f062-afb4-4933-86b6-0d200eb82ef0_isrForCrosstalkSources_997018_24.stderr parsl.dataflow.dflow DEBUG: Task 4791 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4791 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4791 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4791 has memoization hash 7ffa2c93431e4d858c19e8616955e6eb parsl.dataflow.memoization INFO: Task 4791 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de00e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4791 try 0 launched on executor multi with executor id 4792 parsl.dataflow.dflow INFO: Standard output for task 4791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2d7c59b2-956a-45be-849f-c1d7f932cac1_isrForCrosstalkSources_996956_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2d7c59b2-956a-45be-849f-c1d7f932cac1_isrForCrosstalkSources_996956_33.stderr parsl.dataflow.dflow DEBUG: Task 4792 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4792 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4792 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4792 has memoization hash 03e3b4cd266fdc32231b276f2b2c863f parsl.dataflow.memoization INFO: Task 4792 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4792 try 0 launched on executor multi with executor id 4793 parsl.dataflow.dflow INFO: Standard output for task 4792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/a8b85d1a-2074-4869-9f17-b17235e33549_isrForCrosstalkSources_996987_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/a8b85d1a-2074-4869-9f17-b17235e33549_isrForCrosstalkSources_996987_32.stderr parsl.dataflow.dflow DEBUG: Task 4793 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4793 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4793 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4793 has memoization hash ab3f71e98fbe4675a524733afbf6b0a1 parsl.dataflow.memoization INFO: Task 4793 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4793 try 0 launched on executor multi with executor id 4794 parsl.dataflow.dflow INFO: Standard output for task 4793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a5512902-ec3a-4984-83c4-3e4155142c7f_isrForCrosstalkSources_996999_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a5512902-ec3a-4984-83c4-3e4155142c7f_isrForCrosstalkSources_996999_60.stderr parsl.dataflow.dflow DEBUG: Task 4794 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4794 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4794 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4794 has memoization hash d277c14b7cacfb6f8537e62c211c5724 parsl.dataflow.memoization INFO: Task 4794 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4794 try 0 launched on executor multi with executor id 4795 parsl.dataflow.dflow INFO: Standard output for task 4794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/2eab5ced-e763-41f9-940a-e35fb3166909_isrForCrosstalkSources_996991_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/2eab5ced-e763-41f9-940a-e35fb3166909_isrForCrosstalkSources_996991_60.stderr parsl.dataflow.dflow DEBUG: Task 4795 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4795 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4795 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4795 has memoization hash 23e0a511eef3b3519b8cdb4480bd9517 parsl.dataflow.memoization INFO: Task 4795 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de02c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4795 try 0 launched on executor multi with executor id 4796 parsl.dataflow.dflow INFO: Standard output for task 4795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/49fdecdd-f247-428b-b230-ad716093e476_isrForCrosstalkSources_997029_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/49fdecdd-f247-428b-b230-ad716093e476_isrForCrosstalkSources_997029_34.stderr parsl.dataflow.dflow DEBUG: Task 4796 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4796 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4796 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4796 has memoization hash 9d44dbb4c4dd0df86b90859a04c3720b parsl.dataflow.memoization INFO: Task 4796 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4796 try 0 launched on executor multi with executor id 4797 parsl.dataflow.dflow INFO: Standard output for task 4796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b81aee93-1d5d-41c3-a3e3-3fd57b1bea9b_isrForCrosstalkSources_997003_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b81aee93-1d5d-41c3-a3e3-3fd57b1bea9b_isrForCrosstalkSources_997003_16.stderr parsl.dataflow.dflow DEBUG: Task 4797 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4797 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4797 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4797 has memoization hash 60c2582222e057db6b605def0de6aaa4 parsl.dataflow.memoization INFO: Task 4797 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de04a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4797 try 0 launched on executor multi with executor id 4798 parsl.dataflow.dflow INFO: Standard output for task 4797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/cc1aa648-a89b-4111-bc55-6b780064702b_isrForCrosstalkSources_996961_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/cc1aa648-a89b-4111-bc55-6b780064702b_isrForCrosstalkSources_996961_12.stderr parsl.dataflow.dflow DEBUG: Task 4798 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4798 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4798 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4798 has memoization hash cb9e2c27b6df9146c9a474d27f120d9b parsl.dataflow.memoization INFO: Task 4798 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4798 try 0 launched on executor multi with executor id 4799 parsl.dataflow.dflow INFO: Standard output for task 4798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4ccbb321-10f3-4cf9-a357-6be685c31110_isrForCrosstalkSources_996976_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4ccbb321-10f3-4cf9-a357-6be685c31110_isrForCrosstalkSources_996976_59.stderr parsl.dataflow.dflow DEBUG: Task 4799 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4799 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4799 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4799 has memoization hash 8b8d0687ef6717079f665e176029d830 parsl.dataflow.memoization INFO: Task 4799 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4799 try 0 launched on executor multi with executor id 4800 parsl.dataflow.dflow INFO: Standard output for task 4799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/bf3261b9-c171-4052-ba90-7fe346ca7633_isrForCrosstalkSources_996953_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/bf3261b9-c171-4052-ba90-7fe346ca7633_isrForCrosstalkSources_996953_47.stderr parsl.dataflow.dflow DEBUG: Task 4800 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4800 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4800 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4800 has memoization hash de9133980a07cc64abf294c2eebbb527 parsl.dataflow.memoization INFO: Task 4800 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de05e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4800 try 0 launched on executor multi with executor id 4801 parsl.dataflow.dflow INFO: Standard output for task 4800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a3d8b0d4-1d25-4358-9160-bc37f1f45e0f_isrForCrosstalkSources_996956_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a3d8b0d4-1d25-4358-9160-bc37f1f45e0f_isrForCrosstalkSources_996956_10.stderr parsl.dataflow.dflow DEBUG: Task 4801 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4801 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4801 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4801 has memoization hash 62d973069c72b6e23a2dd6e542c8a316 parsl.dataflow.memoization INFO: Task 4801 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4801 try 0 launched on executor multi with executor id 4802 parsl.dataflow.dflow INFO: Standard output for task 4801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/84fdc6e0-79fe-4c4e-9c93-ae1f324f1667_isrForCrosstalkSources_997033_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/84fdc6e0-79fe-4c4e-9c93-ae1f324f1667_isrForCrosstalkSources_997033_61.stderr parsl.dataflow.dflow DEBUG: Task 4802 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4802 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4802 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4802 has memoization hash 8ab96c54e9ca8c9db74843cf3feb7459 parsl.dataflow.memoization INFO: Task 4802 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de07c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4802 try 0 launched on executor multi with executor id 4803 parsl.dataflow.dflow INFO: Standard output for task 4802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/45e1a12b-214c-44f4-a673-5a997b91d152_isrForCrosstalkSources_996978_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/45e1a12b-214c-44f4-a673-5a997b91d152_isrForCrosstalkSources_996978_21.stderr parsl.dataflow.dflow DEBUG: Task 4803 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4803 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4803 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4803 has memoization hash 47f19f944909bee835466676a151af4f parsl.dataflow.memoization INFO: Task 4803 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4803 try 0 launched on executor multi with executor id 4804 parsl.dataflow.dflow INFO: Standard output for task 4803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/11c35dfe-3b4e-40d2-8511-9187a615f271_isrForCrosstalkSources_996967_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/11c35dfe-3b4e-40d2-8511-9187a615f271_isrForCrosstalkSources_996967_19.stderr parsl.dataflow.dflow DEBUG: Task 4804 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4804 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4804 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4804 has memoization hash 56d2303ca0bcab58ee3e19c26fe96ebe parsl.dataflow.memoization INFO: Task 4804 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de09a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4804 try 0 launched on executor multi with executor id 4805 parsl.dataflow.dflow INFO: Standard output for task 4804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/ebde189f-f0f0-4dd4-af70-8865fd8a8a2c_isrForCrosstalkSources_997003_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/ebde189f-f0f0-4dd4-af70-8865fd8a8a2c_isrForCrosstalkSources_997003_24.stderr parsl.dataflow.dflow DEBUG: Task 4805 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4805 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4805 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4805 has memoization hash a8f8472af76b8ba4f477e8d0bffbd37c parsl.dataflow.memoization INFO: Task 4805 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4805 try 0 launched on executor multi with executor id 4806 parsl.dataflow.dflow INFO: Standard output for task 4805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/5f809ef4-58f0-4d03-8095-c7f2b70d90da_isrForCrosstalkSources_997023_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/5f809ef4-58f0-4d03-8095-c7f2b70d90da_isrForCrosstalkSources_997023_6.stderr parsl.dataflow.dflow DEBUG: Task 4806 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4806 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4806 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4806 has memoization hash fb44d2e3b98852ee245437c0c3a9d3d1 parsl.dataflow.memoization INFO: Task 4806 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4806 try 0 launched on executor multi with executor id 4807 parsl.dataflow.dflow INFO: Standard output for task 4806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4e763486-5a91-4ee4-9def-89c179e681df_isrForCrosstalkSources_997014_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4e763486-5a91-4ee4-9def-89c179e681df_isrForCrosstalkSources_997014_1.stderr parsl.dataflow.dflow DEBUG: Task 4807 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4807 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4807 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4807 has memoization hash 642d15f004c9ea34be11406471dd160a parsl.dataflow.memoization INFO: Task 4807 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4807 try 0 launched on executor multi with executor id 4808 parsl.dataflow.dflow INFO: Standard output for task 4807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/236dcf1f-0e01-4358-9258-4e4dc1375e4c_isrForCrosstalkSources_996960_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/236dcf1f-0e01-4358-9258-4e4dc1375e4c_isrForCrosstalkSources_996960_20.stderr parsl.dataflow.dflow DEBUG: Task 4808 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4808 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4808 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4808 has memoization hash a1ac1f8c8ebfb8b802e6ebc9c6519465 parsl.dataflow.memoization INFO: Task 4808 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4808 try 0 launched on executor multi with executor id 4809 parsl.dataflow.dflow INFO: Standard output for task 4808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/6ab39b32-d9a9-4213-abf7-bf7c19256863_isrForCrosstalkSources_997020_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/6ab39b32-d9a9-4213-abf7-bf7c19256863_isrForCrosstalkSources_997020_8.stderr parsl.dataflow.dflow DEBUG: Task 4809 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4809 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4809 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4809 has memoization hash 9ea245bb48004ba13c0cd986b8744ac4 parsl.dataflow.memoization INFO: Task 4809 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4809 try 0 launched on executor multi with executor id 4810 parsl.dataflow.dflow INFO: Standard output for task 4809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/53c6479b-38e9-47b3-8f67-7b664218bb6b_isrForCrosstalkSources_997018_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/53c6479b-38e9-47b3-8f67-7b664218bb6b_isrForCrosstalkSources_997018_32.stderr parsl.dataflow.dflow DEBUG: Task 4810 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4810 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4810 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4810 has memoization hash 0dd064a447408ddb46e581c60bb6fa67 parsl.dataflow.memoization INFO: Task 4810 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4810 try 0 launched on executor multi with executor id 4811 parsl.dataflow.dflow INFO: Standard output for task 4810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1ef2c8b7-3cf8-42b3-9ce5-914f6c28e379_isrForCrosstalkSources_997004_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1ef2c8b7-3cf8-42b3-9ce5-914f6c28e379_isrForCrosstalkSources_997004_62.stderr parsl.dataflow.dflow DEBUG: Task 4811 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4811 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4811 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4811 has memoization hash c14906e1deade80ce86b7324a7c9829f parsl.dataflow.memoization INFO: Task 4811 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4811 try 0 launched on executor multi with executor id 4812 parsl.dataflow.dflow INFO: Standard output for task 4811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a82beef8-147e-445e-9d00-4a8f635e5606_isrForCrosstalkSources_996956_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a82beef8-147e-445e-9d00-4a8f635e5606_isrForCrosstalkSources_996956_15.stderr parsl.dataflow.dflow DEBUG: Task 4812 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4812 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4812 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4812 has memoization hash 849f29d4280801a3e75c8c0ee81682c1 parsl.dataflow.memoization INFO: Task 4812 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4812 try 0 launched on executor multi with executor id 4813 parsl.dataflow.dflow INFO: Standard output for task 4812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/da4318bd-da56-4775-bdf4-b06cacb73d45_isrForCrosstalkSources_997012_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/da4318bd-da56-4775-bdf4-b06cacb73d45_isrForCrosstalkSources_997012_52.stderr parsl.dataflow.dflow DEBUG: Task 4813 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4813 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4813 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4813 has memoization hash e00469b0b254d8889f8f783785e686c8 parsl.dataflow.memoization INFO: Task 4813 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4813 try 0 launched on executor multi with executor id 4814 parsl.dataflow.dflow INFO: Standard output for task 4813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bdc5dac7-ac9f-4d2c-aa72-36a6bd268f9a_isrForCrosstalkSources_997030_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bdc5dac7-ac9f-4d2c-aa72-36a6bd268f9a_isrForCrosstalkSources_997030_59.stderr parsl.dataflow.dflow DEBUG: Task 4814 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4814 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4814 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4814 has memoization hash 19760ccd1a318225fe73693cd33de327 parsl.dataflow.memoization INFO: Task 4814 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4814 try 0 launched on executor multi with executor id 4815 parsl.dataflow.dflow INFO: Standard output for task 4814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f7998868-6567-4482-bdf6-b071cf94d4de_isrForCrosstalkSources_996989_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f7998868-6567-4482-bdf6-b071cf94d4de_isrForCrosstalkSources_996989_16.stderr parsl.dataflow.dflow DEBUG: Task 4815 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4815 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4815 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4815 has memoization hash a7900aedb9303f46393afc094dd5c6fd parsl.dataflow.memoization INFO: Task 4815 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4815 try 0 launched on executor multi with executor id 4816 parsl.dataflow.dflow INFO: Standard output for task 4815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/7a0bb4bd-7257-4743-9602-5bef11f0bef7_isrForCrosstalkSources_996999_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/7a0bb4bd-7257-4743-9602-5bef11f0bef7_isrForCrosstalkSources_996999_50.stderr parsl.dataflow.dflow DEBUG: Task 4816 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4816 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4816 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4816 has memoization hash 2bbed5e7c3b66905e9bf9ef7bf2c1a54 parsl.dataflow.memoization INFO: Task 4816 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4816 try 0 launched on executor multi with executor id 4817 parsl.dataflow.dflow INFO: Standard output for task 4816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/5c515fff-e439-4be7-9abe-e4a0013a99e1_isrForCrosstalkSources_996976_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/5c515fff-e439-4be7-9abe-e4a0013a99e1_isrForCrosstalkSources_996976_56.stderr parsl.dataflow.dflow DEBUG: Task 4817 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4817 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4817 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4817 has memoization hash 661bf8a78e08efe642a4953540b0648a parsl.dataflow.memoization INFO: Task 4817 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4817 try 0 launched on executor multi with executor id 4818 parsl.dataflow.dflow INFO: Standard output for task 4817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c59fc0ae-2080-4154-b33c-517714d06e5e_isrForCrosstalkSources_997016_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c59fc0ae-2080-4154-b33c-517714d06e5e_isrForCrosstalkSources_997016_54.stderr parsl.dataflow.dflow DEBUG: Task 4818 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4818 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4818 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4818 has memoization hash 9ae9d595ab207db7ec59580c5456b763 parsl.dataflow.memoization INFO: Task 4818 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de11c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4818 try 0 launched on executor multi with executor id 4819 parsl.dataflow.dflow INFO: Standard output for task 4818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/aff4f67a-2d14-44cd-b0e6-29af457bd2a7_isrForCrosstalkSources_997012_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/aff4f67a-2d14-44cd-b0e6-29af457bd2a7_isrForCrosstalkSources_997012_2.stderr parsl.dataflow.dflow DEBUG: Task 4819 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4819 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4819 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4819 has memoization hash dc5a77e08a0b91c613d786819161445c parsl.dataflow.memoization INFO: Task 4819 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4819 try 0 launched on executor multi with executor id 4820 parsl.dataflow.dflow INFO: Standard output for task 4819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c0c39e1d-f687-4c3f-a8fd-e0e965c79713_isrForCrosstalkSources_996962_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c0c39e1d-f687-4c3f-a8fd-e0e965c79713_isrForCrosstalkSources_996962_37.stderr parsl.dataflow.dflow DEBUG: Task 4820 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4820 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4820 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4820 has memoization hash c78ef33fa7a1f59a01d3a1b064e790e7 parsl.dataflow.memoization INFO: Task 4820 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4820 try 0 launched on executor multi with executor id 4821 parsl.dataflow.dflow INFO: Standard output for task 4820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/5ffeb5fc-ea2f-4bcf-b262-cecd3e8e9aad_isrForCrosstalkSources_996975_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/5ffeb5fc-ea2f-4bcf-b262-cecd3e8e9aad_isrForCrosstalkSources_996975_37.stderr parsl.dataflow.dflow DEBUG: Task 4821 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4821 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4821 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4821 has memoization hash 5bde44d965b641b6f7c22e34ddf549c3 parsl.dataflow.memoization INFO: Task 4821 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de13a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4821 try 0 launched on executor multi with executor id 4822 parsl.dataflow.dflow INFO: Standard output for task 4821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a70b1ec5-526e-4309-aa69-2e4166f1e0d8_isrForCrosstalkSources_996954_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a70b1ec5-526e-4309-aa69-2e4166f1e0d8_isrForCrosstalkSources_996954_29.stderr parsl.dataflow.dflow DEBUG: Task 4822 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4822 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4822 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4822 has memoization hash 9a9cac2346a11c909d0a6aa3a725db0c parsl.dataflow.memoization INFO: Task 4822 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4822 try 0 launched on executor multi with executor id 4823 parsl.dataflow.dflow INFO: Standard output for task 4822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/42b64a96-8541-4abd-ab33-9e2626a82352_isrForCrosstalkSources_997018_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/42b64a96-8541-4abd-ab33-9e2626a82352_isrForCrosstalkSources_997018_21.stderr parsl.dataflow.dflow DEBUG: Task 4823 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4823 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4823 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4823 has memoization hash 3fa24af1cdc063607ac52a995a47d34c parsl.dataflow.memoization INFO: Task 4823 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de14e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4823 try 0 launched on executor multi with executor id 4824 parsl.dataflow.dflow INFO: Standard output for task 4823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5ccf8a06-5a1b-4ffd-9008-a08a22947950_isrForCrosstalkSources_996985_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5ccf8a06-5a1b-4ffd-9008-a08a22947950_isrForCrosstalkSources_996985_10.stderr parsl.dataflow.dflow DEBUG: Task 4824 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4824 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4824 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4824 has memoization hash d71dc7a051b44a0dda22a0c017e4004b parsl.dataflow.memoization INFO: Task 4824 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4824 try 0 launched on executor multi with executor id 4825 parsl.dataflow.dflow INFO: Standard output for task 4824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0e95d1d3-a30f-41e5-9cd4-3f48011aea2d_isrForCrosstalkSources_996985_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0e95d1d3-a30f-41e5-9cd4-3f48011aea2d_isrForCrosstalkSources_996985_37.stderr parsl.dataflow.dflow DEBUG: Task 4825 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4825 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4825 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4825 has memoization hash 25ded45107323d88c4a237e7c0301e16 parsl.dataflow.memoization INFO: Task 4825 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4825 try 0 launched on executor multi with executor id 4826 parsl.dataflow.dflow INFO: Standard output for task 4825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/2e9fae70-1d68-4086-81f4-a15d2f457520_isrForCrosstalkSources_997017_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/2e9fae70-1d68-4086-81f4-a15d2f457520_isrForCrosstalkSources_997017_15.stderr parsl.dataflow.dflow DEBUG: Task 4826 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4826 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4826 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4826 has memoization hash e31bb7b6e2321157c0d4948ec2a5c110 parsl.dataflow.memoization INFO: Task 4826 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de16c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4826 try 0 launched on executor multi with executor id 4827 parsl.dataflow.dflow INFO: Standard output for task 4826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/2baff398-d696-424e-a761-17a6807b66e6_isrForCrosstalkSources_996976_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/2baff398-d696-424e-a761-17a6807b66e6_isrForCrosstalkSources_996976_43.stderr parsl.dataflow.dflow DEBUG: Task 4827 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4827 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4827 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4827 has memoization hash 3322c3073374527622afa30dbefadad8 parsl.dataflow.memoization INFO: Task 4827 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4827 try 0 launched on executor multi with executor id 4828 parsl.dataflow.dflow INFO: Standard output for task 4827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/324906f8-e6f2-45e1-972d-c3241b304a4d_isrForCrosstalkSources_996982_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/324906f8-e6f2-45e1-972d-c3241b304a4d_isrForCrosstalkSources_996982_44.stderr parsl.dataflow.dflow DEBUG: Task 4828 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4828 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4828 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4828 has memoization hash 2da06926975f941a9a1b160b48b85739 parsl.dataflow.memoization INFO: Task 4828 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4828 try 0 launched on executor multi with executor id 4829 parsl.dataflow.dflow INFO: Standard output for task 4828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/47cdf5dc-47da-4bc6-9745-dac574079b24_isrForCrosstalkSources_997007_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/47cdf5dc-47da-4bc6-9745-dac574079b24_isrForCrosstalkSources_997007_30.stderr parsl.dataflow.dflow DEBUG: Task 4829 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4829 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4829 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4829 has memoization hash b215bfd3ef695e275fc109e913cec6fe parsl.dataflow.memoization INFO: Task 4829 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4829 try 0 launched on executor multi with executor id 4830 parsl.dataflow.dflow INFO: Standard output for task 4829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/de583e6e-e657-4a2e-924a-b842200f27c2_isrForCrosstalkSources_997032_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/de583e6e-e657-4a2e-924a-b842200f27c2_isrForCrosstalkSources_997032_16.stderr parsl.dataflow.dflow DEBUG: Task 4830 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4830 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4830 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4830 has memoization hash 9c568dcb5c3bb466428b2a3080c9deab parsl.dataflow.memoization INFO: Task 4830 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de18a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4830 try 0 launched on executor multi with executor id 4831 parsl.dataflow.dflow INFO: Standard output for task 4830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8a0fae67-b371-4f10-85f1-824df8eb00ca_isrForCrosstalkSources_996977_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8a0fae67-b371-4f10-85f1-824df8eb00ca_isrForCrosstalkSources_996977_62.stderr parsl.dataflow.dflow DEBUG: Task 4831 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4831 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4831 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4831 has memoization hash 42353360cbc4abe011161e64965ce6e3 parsl.dataflow.memoization INFO: Task 4831 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de19e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4831 try 0 launched on executor multi with executor id 4832 parsl.dataflow.dflow INFO: Standard output for task 4831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/684534c7-9f23-405f-9f65-73f1beaaf65c_isrForCrosstalkSources_997010_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/684534c7-9f23-405f-9f65-73f1beaaf65c_isrForCrosstalkSources_997010_48.stderr parsl.dataflow.dflow DEBUG: Task 4832 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4832 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4832 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4832 has memoization hash 4e437f9b6b8d068dd494316d7737fb5f parsl.dataflow.memoization INFO: Task 4832 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4832 try 0 launched on executor multi with executor id 4833 parsl.dataflow.dflow INFO: Standard output for task 4832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/fea1923d-b180-47ab-a25b-2891491cbbdd_isrForCrosstalkSources_997005_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/fea1923d-b180-47ab-a25b-2891491cbbdd_isrForCrosstalkSources_997005_11.stderr parsl.dataflow.dflow DEBUG: Task 4833 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4833 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4833 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4833 has memoization hash d0760720adb6ce93bd3aee05217ed7c0 parsl.dataflow.memoization INFO: Task 4833 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4833 try 0 launched on executor multi with executor id 4834 parsl.dataflow.dflow INFO: Standard output for task 4833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/967532dc-78ee-41b2-b39f-ca253e47ebc3_isrForCrosstalkSources_997031_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/967532dc-78ee-41b2-b39f-ca253e47ebc3_isrForCrosstalkSources_997031_10.stderr parsl.dataflow.dflow DEBUG: Task 4834 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4834 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4834 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4834 has memoization hash 88c5d51d25a406d6854f4b1eb3adf82e parsl.dataflow.memoization INFO: Task 4834 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4834 try 0 launched on executor multi with executor id 4835 parsl.dataflow.dflow INFO: Standard output for task 4834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/59ecab66-c62f-4928-95d6-7634dc8ab023_isrForCrosstalkSources_996968_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/59ecab66-c62f-4928-95d6-7634dc8ab023_isrForCrosstalkSources_996968_10.stderr parsl.dataflow.dflow DEBUG: Task 4835 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4835 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4835 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4835 has memoization hash c9580ca544f2e27335548c2c9331e3d8 parsl.dataflow.memoization INFO: Task 4835 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4835 try 0 launched on executor multi with executor id 4836 parsl.dataflow.dflow INFO: Standard output for task 4835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/1ed953c9-901e-44a7-9542-343bcd545442_isrForCrosstalkSources_996999_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/1ed953c9-901e-44a7-9542-343bcd545442_isrForCrosstalkSources_996999_4.stderr parsl.dataflow.dflow DEBUG: Task 4836 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4836 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4836 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4836 has memoization hash 2ae0475d6424683829691415fbbeb0dc parsl.dataflow.memoization INFO: Task 4836 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4836 try 0 launched on executor multi with executor id 4837 parsl.dataflow.dflow INFO: Standard output for task 4836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9da524f2-0dd6-4d5e-9781-bc7760560e73_isrForCrosstalkSources_996960_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9da524f2-0dd6-4d5e-9781-bc7760560e73_isrForCrosstalkSources_996960_38.stderr parsl.dataflow.dflow DEBUG: Task 4837 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4837 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4837 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4837 has memoization hash 7dc7727083f7661c8e08d405914df670 parsl.dataflow.memoization INFO: Task 4837 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4837 try 0 launched on executor multi with executor id 4838 parsl.dataflow.dflow INFO: Standard output for task 4837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8cd51ecd-9426-4660-8c1d-0ad74d3d466f_isrForCrosstalkSources_996977_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8cd51ecd-9426-4660-8c1d-0ad74d3d466f_isrForCrosstalkSources_996977_10.stderr parsl.dataflow.dflow DEBUG: Task 4838 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4838 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4838 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4838 has memoization hash 3bb9285dea6be423752d09356f5ca9a3 parsl.dataflow.memoization INFO: Task 4838 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4838 try 0 launched on executor multi with executor id 4839 parsl.dataflow.dflow INFO: Standard output for task 4838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/74d06804-736d-49fb-af8f-47aa089629a1_isrForCrosstalkSources_996978_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/74d06804-736d-49fb-af8f-47aa089629a1_isrForCrosstalkSources_996978_24.stderr parsl.dataflow.dflow DEBUG: Task 4839 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4839 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4839 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4839 has memoization hash b870536c1cd419045537b01f42fef87a parsl.dataflow.memoization INFO: Task 4839 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4839 try 0 launched on executor multi with executor id 4840 parsl.dataflow.dflow INFO: Standard output for task 4839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fb32cfbe-03c7-4cd5-a375-79eab16192bc_isrForCrosstalkSources_996961_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fb32cfbe-03c7-4cd5-a375-79eab16192bc_isrForCrosstalkSources_996961_45.stderr parsl.dataflow.dflow DEBUG: Task 4840 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4840 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4840 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4840 has memoization hash 392e3b5f89c96fa6f6bbbd2bd1f1696c parsl.dataflow.memoization INFO: Task 4840 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de1ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4840 try 0 launched on executor multi with executor id 4841 parsl.dataflow.dflow INFO: Standard output for task 4840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/d5e5a495-f4bf-44e9-9dff-5a9dc5f9d226_isrForCrosstalkSources_996956_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/d5e5a495-f4bf-44e9-9dff-5a9dc5f9d226_isrForCrosstalkSources_996956_24.stderr parsl.dataflow.dflow DEBUG: Task 4841 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4841 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4841 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4841 has memoization hash 994738a3476f2fa269a1bc1c4c3249bc parsl.dataflow.memoization INFO: Task 4841 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4841 try 0 launched on executor multi with executor id 4842 parsl.dataflow.dflow INFO: Standard output for task 4841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/714525c9-4894-47f3-8d8f-719756cf39fd_isrForCrosstalkSources_996968_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/714525c9-4894-47f3-8d8f-719756cf39fd_isrForCrosstalkSources_996968_41.stderr parsl.dataflow.dflow DEBUG: Task 4842 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4842 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4842 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4842 has memoization hash e9289a91b9a0614f13c7aeb29f73fb93 parsl.dataflow.memoization INFO: Task 4842 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de20c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4842 try 0 launched on executor multi with executor id 4843 parsl.dataflow.dflow INFO: Standard output for task 4842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/509c5d9b-0708-4437-922b-c421cc1ed7ea_isrForCrosstalkSources_996977_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/509c5d9b-0708-4437-922b-c421cc1ed7ea_isrForCrosstalkSources_996977_28.stderr parsl.dataflow.dflow DEBUG: Task 4843 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4843 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4843 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4843 has memoization hash e2d5866a7d79c4c563b19458fb260ac3 parsl.dataflow.memoization INFO: Task 4843 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4843 try 0 launched on executor multi with executor id 4844 parsl.dataflow.dflow INFO: Standard output for task 4843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c33fc834-4714-47b5-9962-03a8d2113be5_isrForCrosstalkSources_997006_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c33fc834-4714-47b5-9962-03a8d2113be5_isrForCrosstalkSources_997006_53.stderr parsl.dataflow.dflow DEBUG: Task 4844 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4844 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4844 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4844 has memoization hash 0daca9c05cf626beae29e3bc17a3c6b4 parsl.dataflow.memoization INFO: Task 4844 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de22a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4844 try 0 launched on executor multi with executor id 4845 parsl.dataflow.dflow INFO: Standard output for task 4844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/be930fe0-76f6-4a5d-93fd-a690d1aca6c5_isrForCrosstalkSources_996962_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/be930fe0-76f6-4a5d-93fd-a690d1aca6c5_isrForCrosstalkSources_996962_5.stderr parsl.dataflow.dflow DEBUG: Task 4845 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4845 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4845 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4845 has memoization hash 077bd7ad3e91f2cbe4bd5297832b2489 parsl.dataflow.memoization INFO: Task 4845 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4845 try 0 launched on executor multi with executor id 4846 parsl.dataflow.dflow INFO: Standard output for task 4845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/74101d64-436a-48f5-bca8-f63c2025b222_isrForCrosstalkSources_996986_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/74101d64-436a-48f5-bca8-f63c2025b222_isrForCrosstalkSources_996986_18.stderr parsl.dataflow.dflow DEBUG: Task 4846 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4846 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4846 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4846 has memoization hash 2e980df614a56311b6e0dcb69dfa1247 parsl.dataflow.memoization INFO: Task 4846 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4846 try 0 launched on executor multi with executor id 4847 parsl.dataflow.dflow INFO: Standard output for task 4846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/d8ad6cf1-6e33-4b72-816a-99c2606d9e2c_isrForCrosstalkSources_996977_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/d8ad6cf1-6e33-4b72-816a-99c2606d9e2c_isrForCrosstalkSources_996977_14.stderr parsl.dataflow.dflow DEBUG: Task 4847 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4847 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4847 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4847 has memoization hash 66cd989023a616fc3aaf11e9518c61ae parsl.dataflow.memoization INFO: Task 4847 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de23e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4847 try 0 launched on executor multi with executor id 4848 parsl.dataflow.dflow INFO: Standard output for task 4847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d72acd6a-a4e5-4a5d-be89-5e3cbcc826de_isrForCrosstalkSources_997030_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d72acd6a-a4e5-4a5d-be89-5e3cbcc826de_isrForCrosstalkSources_997030_1.stderr parsl.dataflow.dflow DEBUG: Task 4848 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4848 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4848 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4848 has memoization hash 29c0987946d36a4f909860003b6a522f parsl.dataflow.memoization INFO: Task 4848 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4848 try 0 launched on executor multi with executor id 4849 parsl.dataflow.dflow INFO: Standard output for task 4848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/ba7c868c-821a-40ed-a48a-1ff65c94f278_isrForCrosstalkSources_996970_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/ba7c868c-821a-40ed-a48a-1ff65c94f278_isrForCrosstalkSources_996970_56.stderr parsl.dataflow.dflow DEBUG: Task 4849 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4849 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4849 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4849 has memoization hash 2bdda7bb2c9587da5fa9aa4dd26cf14b parsl.dataflow.memoization INFO: Task 4849 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de25c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:11 proc_lsst.multi:146 [INFO] found job 17074841 in provider astro proc_lsst.multi INFO: found job 17074841 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4849 try 0 launched on executor multi with executor id 4850 parsl.dataflow.dflow INFO: Standard output for task 4849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/913e98e9-f594-4fd4-9783-18b62b617b4d_isrForCrosstalkSources_996960_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/913e98e9-f594-4fd4-9783-18b62b617b4d_isrForCrosstalkSources_996960_16.stderr parsl.dataflow.dflow DEBUG: Task 4850 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4850 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4850 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4850 has memoization hash 4a6e8a76f1f7ad2687696cf13c6dbec5 parsl.dataflow.memoization INFO: Task 4850 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4850 try 0 launched on executor multi with executor id 4851 parsl.dataflow.dflow INFO: Standard output for task 4850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1723d15d-ca3b-46a7-815f-799fdc1fc4dd_isrForCrosstalkSources_997017_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1723d15d-ca3b-46a7-815f-799fdc1fc4dd_isrForCrosstalkSources_997017_50.stderr parsl.dataflow.dflow DEBUG: Task 4851 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4851 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4851 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4851 has memoization hash 6f2b45c61298441b72f47ea2ae7a58cd parsl.dataflow.memoization INFO: Task 4851 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de0040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4851 try 0 launched on executor multi with executor id 4852 parsl.dataflow.dflow INFO: Standard output for task 4851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/901c8aa9-c6e8-4aa3-aabf-7f4d65cb24e0_isrForCrosstalkSources_997005_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/901c8aa9-c6e8-4aa3-aabf-7f4d65cb24e0_isrForCrosstalkSources_997005_56.stderr parsl.dataflow.dflow DEBUG: Task 4852 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4852 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4852 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4852 has memoization hash b331de0127706ef588196141fcdc71d8 parsl.dataflow.memoization INFO: Task 4852 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4852 try 0 launched on executor multi with executor id 4853 parsl.dataflow.dflow INFO: Standard output for task 4852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7267ffef-01af-429d-8fe5-f86cdefce52a_isrForCrosstalkSources_997026_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7267ffef-01af-429d-8fe5-f86cdefce52a_isrForCrosstalkSources_997026_42.stderr parsl.dataflow.dflow DEBUG: Task 4853 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4853 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4853 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4853 has memoization hash a0322e05acaa80e9c77bd4442d799bcc parsl.dataflow.memoization INFO: Task 4853 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de27a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4853 try 0 launched on executor multi with executor id 4854 parsl.dataflow.dflow INFO: Standard output for task 4853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/9637e321-7a20-43b0-9bfb-866bd7e7dc8c_isrForCrosstalkSources_997017_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/9637e321-7a20-43b0-9bfb-866bd7e7dc8c_isrForCrosstalkSources_997017_55.stderr parsl.dataflow.dflow DEBUG: Task 4854 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4854 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4854 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4854 has memoization hash ee1c6730314bb6a9b03194893dd3899a parsl.dataflow.memoization INFO: Task 4854 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de28e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4854 try 0 launched on executor multi with executor id 4855 parsl.dataflow.dflow INFO: Standard output for task 4854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/461210a5-4f4f-4217-b019-f873b4b7696b_isrForCrosstalkSources_997011_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/461210a5-4f4f-4217-b019-f873b4b7696b_isrForCrosstalkSources_997011_34.stderr parsl.dataflow.dflow DEBUG: Task 4855 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4855 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4855 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4855 has memoization hash ac2a4d24d744aed87dbbe9fd4013dd76 parsl.dataflow.memoization INFO: Task 4855 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2a20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4855 try 0 launched on executor multi with executor id 4856 parsl.dataflow.dflow INFO: Standard output for task 4855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/3b2c707a-daec-4f68-a6ec-e0fadca47ee8_isrForCrosstalkSources_997023_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/3b2c707a-daec-4f68-a6ec-e0fadca47ee8_isrForCrosstalkSources_997023_51.stderr parsl.dataflow.dflow DEBUG: Task 4856 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4856 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4856 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4856 has memoization hash 26495f7afdd9fbb7b3b6a683a7f8c21b parsl.dataflow.memoization INFO: Task 4856 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4856 try 0 launched on executor multi with executor id 4857 parsl.dataflow.dflow INFO: Standard output for task 4856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/1c2f889a-c7df-4cdf-a747-c24158191340_isrForCrosstalkSources_997016_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/1c2f889a-c7df-4cdf-a747-c24158191340_isrForCrosstalkSources_997016_44.stderr parsl.dataflow.dflow DEBUG: Task 4857 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4857 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4857 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4857 has memoization hash 01803a52899fabe292afa454574b35ea parsl.dataflow.memoization INFO: Task 4857 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4857 try 0 launched on executor multi with executor id 4858 parsl.dataflow.dflow INFO: Standard output for task 4857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/0af8e2db-4557-4350-a77f-68634768dfc1_isrForCrosstalkSources_996989_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/0af8e2db-4557-4350-a77f-68634768dfc1_isrForCrosstalkSources_996989_38.stderr parsl.dataflow.dflow DEBUG: Task 4858 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4858 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4858 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4858 has memoization hash 9fbdf167f72c49a9c696080ae86d986e parsl.dataflow.memoization INFO: Task 4858 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4858 try 0 launched on executor multi with executor id 4859 parsl.dataflow.dflow INFO: Standard output for task 4858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/a29ea7ff-9b99-460f-ab21-a87ab8b0c9ac_isrForCrosstalkSources_996975_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/a29ea7ff-9b99-460f-ab21-a87ab8b0c9ac_isrForCrosstalkSources_996975_57.stderr parsl.dataflow.dflow DEBUG: Task 4859 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4859 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4859 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4859 has memoization hash 10643daaf6fd0f7c0b15bea03b21debb parsl.dataflow.memoization INFO: Task 4859 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4859 try 0 launched on executor multi with executor id 4860 parsl.dataflow.dflow INFO: Standard output for task 4859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/515faa0d-ab74-447c-9112-f6c18a8ed663_isrForCrosstalkSources_996988_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/515faa0d-ab74-447c-9112-f6c18a8ed663_isrForCrosstalkSources_996988_7.stderr parsl.dataflow.dflow DEBUG: Task 4860 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4860 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4860 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4860 has memoization hash 133ffe5134e518420be119628919fd4b parsl.dataflow.memoization INFO: Task 4860 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4860 try 0 launched on executor multi with executor id 4861 parsl.dataflow.dflow INFO: Standard output for task 4860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/aca07a5c-1f40-48b4-84e1-dc6b297b349c_isrForCrosstalkSources_996962_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/aca07a5c-1f40-48b4-84e1-dc6b297b349c_isrForCrosstalkSources_996962_32.stderr parsl.dataflow.dflow DEBUG: Task 4861 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4861 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4861 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4861 has memoization hash 860720b5a9642845d9c4a249562eca84 parsl.dataflow.memoization INFO: Task 4861 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4861 try 0 launched on executor multi with executor id 4862 parsl.dataflow.dflow INFO: Standard output for task 4861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/63c96795-3ff8-4be8-9915-c3d6ec489b1c_isrForCrosstalkSources_997018_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/63c96795-3ff8-4be8-9915-c3d6ec489b1c_isrForCrosstalkSources_997018_39.stderr parsl.dataflow.dflow DEBUG: Task 4862 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4862 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4862 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4862 has memoization hash 9a4a8975312bde53663b6d670c37260b parsl.dataflow.memoization INFO: Task 4862 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4862 try 0 launched on executor multi with executor id 4863 parsl.dataflow.dflow INFO: Standard output for task 4862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b455cfb8-1a4c-4c65-bf30-118c52a3171e_isrForCrosstalkSources_997016_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b455cfb8-1a4c-4c65-bf30-118c52a3171e_isrForCrosstalkSources_997016_57.stderr parsl.dataflow.dflow DEBUG: Task 4863 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4863 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4863 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4863 has memoization hash 439b0338dcf178f28dcc2b3456722d5a parsl.dataflow.memoization INFO: Task 4863 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4863 try 0 launched on executor multi with executor id 4864 parsl.dataflow.dflow INFO: Standard output for task 4863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/87e07dee-a22d-4a31-ae90-9250d514a8ce_isrForCrosstalkSources_996993_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/87e07dee-a22d-4a31-ae90-9250d514a8ce_isrForCrosstalkSources_996993_21.stderr parsl.dataflow.dflow DEBUG: Task 4864 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4864 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4864 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4864 has memoization hash 8fa797d294c6dd25cb86bcec5748da00 parsl.dataflow.memoization INFO: Task 4864 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4864 try 0 launched on executor multi with executor id 4865 parsl.dataflow.dflow INFO: Standard output for task 4864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7a136b8c-811e-4918-8834-0a096b56319b_isrForCrosstalkSources_997025_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7a136b8c-811e-4918-8834-0a096b56319b_isrForCrosstalkSources_997025_53.stderr parsl.dataflow.dflow DEBUG: Task 4865 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4865 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4865 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4865 has memoization hash eaadffc22bfcf784a7c2fdc5500f41c9 parsl.dataflow.memoization INFO: Task 4865 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4865 try 0 launched on executor multi with executor id 4866 parsl.dataflow.dflow INFO: Standard output for task 4865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ec80df07-ff55-477e-b5e7-abbc3a87be50_isrForCrosstalkSources_996954_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ec80df07-ff55-477e-b5e7-abbc3a87be50_isrForCrosstalkSources_996954_19.stderr parsl.dataflow.dflow DEBUG: Task 4866 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4866 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4866 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4866 has memoization hash 8d5de7a1e347e59fde25b6c5fd8b9c2f parsl.dataflow.memoization INFO: Task 4866 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4866 try 0 launched on executor multi with executor id 4867 parsl.dataflow.dflow INFO: Standard output for task 4866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/ea60fdc9-fa45-4678-adb9-396177bdc689_isrForCrosstalkSources_996963_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/ea60fdc9-fa45-4678-adb9-396177bdc689_isrForCrosstalkSources_996963_31.stderr parsl.dataflow.dflow DEBUG: Task 4867 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4867 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4867 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4867 has memoization hash c3f6d52143313236c4af26f98051a660 parsl.dataflow.memoization INFO: Task 4867 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4867 try 0 launched on executor multi with executor id 4868 parsl.dataflow.dflow INFO: Standard output for task 4867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/de90c2c5-2258-4868-b4d4-691786f1b715_isrForCrosstalkSources_997020_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/de90c2c5-2258-4868-b4d4-691786f1b715_isrForCrosstalkSources_997020_40.stderr parsl.dataflow.dflow DEBUG: Task 4868 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4868 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4868 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4868 has memoization hash 9cea4c989387dc2527e12537b053a908 parsl.dataflow.memoization INFO: Task 4868 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de31a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4868 try 0 launched on executor multi with executor id 4869 parsl.dataflow.dflow INFO: Standard output for task 4868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/7aa437d6-79e6-497f-a9ee-92cd9d272e77_isrForCrosstalkSources_997031_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/7aa437d6-79e6-497f-a9ee-92cd9d272e77_isrForCrosstalkSources_997031_36.stderr parsl.dataflow.dflow DEBUG: Task 4869 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4869 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4869 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4869 has memoization hash 6e2051911066271705fb9c93b0f0c8f6 parsl.dataflow.memoization INFO: Task 4869 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4869 try 0 launched on executor multi with executor id 4870 parsl.dataflow.dflow INFO: Standard output for task 4869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d664f007-9ffd-4a75-ba87-362b456aa6b8_isrForCrosstalkSources_996982_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d664f007-9ffd-4a75-ba87-362b456aa6b8_isrForCrosstalkSources_996982_31.stderr parsl.dataflow.dflow DEBUG: Task 4870 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4870 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4870 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4870 has memoization hash 35a2e59c7b9abc19558b2497ca0c87fb parsl.dataflow.memoization INFO: Task 4870 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4870 try 0 launched on executor multi with executor id 4871 parsl.dataflow.dflow INFO: Standard output for task 4870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/13b340fa-e993-4fd5-bd3f-a59389c07beb_isrForCrosstalkSources_996977_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/13b340fa-e993-4fd5-bd3f-a59389c07beb_isrForCrosstalkSources_996977_61.stderr parsl.dataflow.dflow DEBUG: Task 4871 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4871 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4871 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4871 has memoization hash e00fdfecbd795c832baa0632576ef624 parsl.dataflow.memoization INFO: Task 4871 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de32e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4871 try 0 launched on executor multi with executor id 4872 parsl.dataflow.dflow INFO: Standard output for task 4871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/de339a2e-4a5c-4ade-bdd5-077a44e82373_isrForCrosstalkSources_996974_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/de339a2e-4a5c-4ade-bdd5-077a44e82373_isrForCrosstalkSources_996974_17.stderr parsl.dataflow.dflow DEBUG: Task 4872 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4872 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4872 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4872 has memoization hash a595b60e53a48624192f35576e349c69 parsl.dataflow.memoization INFO: Task 4872 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4872 try 0 launched on executor multi with executor id 4873 parsl.dataflow.dflow INFO: Standard output for task 4872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/89d9acef-6c9e-4ddf-a76a-686e13570b89_isrForCrosstalkSources_996987_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/89d9acef-6c9e-4ddf-a76a-686e13570b89_isrForCrosstalkSources_996987_45.stderr parsl.dataflow.dflow DEBUG: Task 4873 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4873 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4873 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4873 has memoization hash fd26cc9b959d4392c6f53ecedd4c7451 parsl.dataflow.memoization INFO: Task 4873 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de34c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4873 try 0 launched on executor multi with executor id 4874 parsl.dataflow.dflow INFO: Standard output for task 4873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/549f1d73-dc5a-4b28-a55c-70014509215b_isrForCrosstalkSources_996962_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/549f1d73-dc5a-4b28-a55c-70014509215b_isrForCrosstalkSources_996962_15.stderr parsl.dataflow.dflow DEBUG: Task 4874 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4874 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4874 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4874 has memoization hash e24e187bf15c8efe90e7e7b6a4e8c1b4 parsl.dataflow.memoization INFO: Task 4874 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4874 try 0 launched on executor multi with executor id 4875 parsl.dataflow.dflow INFO: Standard output for task 4874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b5b7eb2b-99f1-43b5-860f-0c5792abdce3_isrForCrosstalkSources_997030_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b5b7eb2b-99f1-43b5-860f-0c5792abdce3_isrForCrosstalkSources_997030_5.stderr parsl.dataflow.dflow DEBUG: Task 4875 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4875 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4875 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4875 has memoization hash 6c3c5725db8afecef0b0ecf8bdcf0719 parsl.dataflow.memoization INFO: Task 4875 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4875 try 0 launched on executor multi with executor id 4876 parsl.dataflow.dflow INFO: Standard output for task 4875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c65837e6-2eee-4e20-a048-30def3cf0987_isrForCrosstalkSources_997025_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c65837e6-2eee-4e20-a048-30def3cf0987_isrForCrosstalkSources_997025_20.stderr parsl.dataflow.dflow DEBUG: Task 4876 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4876 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4876 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4876 has memoization hash a8e86eebf533d255a765c313693fb1ac parsl.dataflow.memoization INFO: Task 4876 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de36a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4876 try 0 launched on executor multi with executor id 4877 parsl.dataflow.dflow INFO: Standard output for task 4876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/80290703-9328-4809-ace0-aab95801ba51_isrForCrosstalkSources_997018_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/80290703-9328-4809-ace0-aab95801ba51_isrForCrosstalkSources_997018_45.stderr parsl.dataflow.dflow DEBUG: Task 4877 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4877 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4877 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4877 has memoization hash 9bef2a30331c92993d330278f305dd57 parsl.dataflow.memoization INFO: Task 4877 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4877 try 0 launched on executor multi with executor id 4878 parsl.dataflow.dflow INFO: Standard output for task 4877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/cc7c8e67-8e4d-4503-8be0-bb6c37bbd282_isrForCrosstalkSources_996991_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/cc7c8e67-8e4d-4503-8be0-bb6c37bbd282_isrForCrosstalkSources_996991_15.stderr parsl.dataflow.dflow DEBUG: Task 4878 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4878 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4878 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4878 has memoization hash 47ab0504700bfac61efe98cf33a56097 parsl.dataflow.memoization INFO: Task 4878 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4878 try 0 launched on executor multi with executor id 4879 parsl.dataflow.dflow INFO: Standard output for task 4878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/75f1c6b9-1ab0-4d8f-9212-0524dcab0e3f_isrForCrosstalkSources_996976_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/75f1c6b9-1ab0-4d8f-9212-0524dcab0e3f_isrForCrosstalkSources_996976_4.stderr parsl.dataflow.dflow DEBUG: Task 4879 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4879 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4879 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4879 has memoization hash eb186ba41920a807cade99e0f8411e48 parsl.dataflow.memoization INFO: Task 4879 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de37e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4879 try 0 launched on executor multi with executor id 4880 parsl.dataflow.dflow INFO: Standard output for task 4879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ba35184e-5f6c-4e48-86ef-b87f6c5d0873_isrForCrosstalkSources_997015_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ba35184e-5f6c-4e48-86ef-b87f6c5d0873_isrForCrosstalkSources_997015_15.stderr parsl.dataflow.dflow DEBUG: Task 4880 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4880 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4880 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4880 has memoization hash 8f1f1b269d9df4485f1fbd8fa0df932e parsl.dataflow.memoization INFO: Task 4880 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4880 try 0 launched on executor multi with executor id 4881 parsl.dataflow.dflow INFO: Standard output for task 4880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b960d7b7-7fc4-40d4-b77c-1148fe977419_isrForCrosstalkSources_997008_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b960d7b7-7fc4-40d4-b77c-1148fe977419_isrForCrosstalkSources_997008_41.stderr parsl.dataflow.dflow DEBUG: Task 4881 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4881 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4881 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4881 has memoization hash 2d914d75733782eeee4f14b40942dc6b parsl.dataflow.memoization INFO: Task 4881 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de39c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4881 try 0 launched on executor multi with executor id 4882 parsl.dataflow.dflow INFO: Standard output for task 4881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7659ef5f-43ad-47de-a238-a744f64f3bdd_isrForCrosstalkSources_996975_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7659ef5f-43ad-47de-a238-a744f64f3bdd_isrForCrosstalkSources_996975_18.stderr parsl.dataflow.dflow DEBUG: Task 4882 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4882 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4882 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4882 has memoization hash a8858fadcb81433ec5d1753e50299d04 parsl.dataflow.memoization INFO: Task 4882 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4882 try 0 launched on executor multi with executor id 4883 parsl.dataflow.dflow INFO: Standard output for task 4882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/dc9d1b16-ef70-4d59-95db-7e57e55f0fed_isrForCrosstalkSources_997017_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/dc9d1b16-ef70-4d59-95db-7e57e55f0fed_isrForCrosstalkSources_997017_27.stderr parsl.dataflow.dflow DEBUG: Task 4883 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4883 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4883 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4883 has memoization hash f3f71a0e99d02024f580669d44ddebf6 parsl.dataflow.memoization INFO: Task 4883 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3a60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4883 try 0 launched on executor multi with executor id 4884 parsl.dataflow.dflow INFO: Standard output for task 4883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/1c6abc8a-b89d-482b-969d-7d8a2336b993_isrForCrosstalkSources_996987_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/1c6abc8a-b89d-482b-969d-7d8a2336b993_isrForCrosstalkSources_996987_50.stderr parsl.dataflow.dflow DEBUG: Task 4884 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4884 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4884 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4884 has memoization hash e5fabc108f0194e5dae9771eda0c66dd parsl.dataflow.memoization INFO: Task 4884 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3b00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4884 try 0 launched on executor multi with executor id 4885 parsl.dataflow.dflow INFO: Standard output for task 4884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/3433fd14-8aa2-41b2-b4ff-cdfe0e02ed5d_isrForCrosstalkSources_997007_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/3433fd14-8aa2-41b2-b4ff-cdfe0e02ed5d_isrForCrosstalkSources_997007_48.stderr parsl.dataflow.dflow DEBUG: Task 4885 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4885 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4885 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4885 has memoization hash 42a63c9ef42b573929115d0644d89d1b parsl.dataflow.memoization INFO: Task 4885 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3ba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4885 try 0 launched on executor multi with executor id 4886 parsl.dataflow.dflow INFO: Standard output for task 4885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2775c0f8-929c-4f95-b590-20a3cba77e4d_isrForCrosstalkSources_996962_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2775c0f8-929c-4f95-b590-20a3cba77e4d_isrForCrosstalkSources_996962_51.stderr parsl.dataflow.dflow DEBUG: Task 4886 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4886 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4886 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4886 has memoization hash 834cdf175bda2da175780a7b815e8e72 parsl.dataflow.memoization INFO: Task 4886 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3ce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4886 try 0 launched on executor multi with executor id 4887 parsl.dataflow.dflow INFO: Standard output for task 4886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/9d5e14f3-5d6e-4a11-b41e-20862bb8aa65_isrForCrosstalkSources_997021_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/9d5e14f3-5d6e-4a11-b41e-20862bb8aa65_isrForCrosstalkSources_997021_45.stderr parsl.dataflow.dflow DEBUG: Task 4887 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4887 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4887 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4887 has memoization hash e7c3930a478c746e4575fbe2e9a8c7d1 parsl.dataflow.memoization INFO: Task 4887 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3c40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4887 try 0 launched on executor multi with executor id 4888 parsl.dataflow.dflow INFO: Standard output for task 4887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f1152981-5493-48fd-9a4f-d1444415b2f1_isrForCrosstalkSources_997005_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f1152981-5493-48fd-9a4f-d1444415b2f1_isrForCrosstalkSources_997005_38.stderr parsl.dataflow.dflow DEBUG: Task 4888 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4888 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4888 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4888 has memoization hash 109716ce1c6a76ffc5a3f5d2d8674ce5 parsl.dataflow.memoization INFO: Task 4888 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3e20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4888 try 0 launched on executor multi with executor id 4889 parsl.dataflow.dflow INFO: Standard output for task 4888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/58aaa06e-87aa-4c63-a938-3e3a0b8617f1_isrForCrosstalkSources_997028_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/58aaa06e-87aa-4c63-a938-3e3a0b8617f1_isrForCrosstalkSources_997028_5.stderr parsl.dataflow.dflow DEBUG: Task 4889 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4889 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4889 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4889 has memoization hash c9b020f95c8d27c828e6228eb3ef48af parsl.dataflow.memoization INFO: Task 4889 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3d80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4889 try 0 launched on executor multi with executor id 4890 parsl.dataflow.dflow INFO: Standard output for task 4889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/fbfdab72-87bf-4c5d-a790-f1a1df1397fc_isrForCrosstalkSources_997021_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/fbfdab72-87bf-4c5d-a790-f1a1df1397fc_isrForCrosstalkSources_997021_36.stderr parsl.dataflow.dflow DEBUG: Task 4890 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4890 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4890 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4890 has memoization hash c8fefc769781f5ecb70e63ac87d541aa parsl.dataflow.memoization INFO: Task 4890 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3ec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4890 try 0 launched on executor multi with executor id 4891 parsl.dataflow.dflow INFO: Standard output for task 4890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/df44e15d-e7e7-4f67-b2cb-31f8e712ffc6_isrForCrosstalkSources_997025_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/df44e15d-e7e7-4f67-b2cb-31f8e712ffc6_isrForCrosstalkSources_997025_2.stderr parsl.dataflow.dflow DEBUG: Task 4891 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4891 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4891 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4891 has memoization hash a0b0a635f9f56e7ea6468fdcd8daa41a parsl.dataflow.memoization INFO: Task 4891 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de3f60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4891 try 0 launched on executor multi with executor id 4892 parsl.dataflow.dflow INFO: Standard output for task 4891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8df19e22-1f35-4085-bf65-2333e65076f8_isrForCrosstalkSources_997027_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8df19e22-1f35-4085-bf65-2333e65076f8_isrForCrosstalkSources_997027_60.stderr parsl.dataflow.dflow DEBUG: Task 4892 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4892 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4892 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4892 has memoization hash 00b6a931e71680dd307bf11cbf7d4a02 parsl.dataflow.memoization INFO: Task 4892 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767de2660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4892 try 0 launched on executor multi with executor id 4893 parsl.dataflow.dflow INFO: Standard output for task 4892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/81a8a68b-d858-48d3-aa57-0b2de4feb496_isrForCrosstalkSources_997020_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/81a8a68b-d858-48d3-aa57-0b2de4feb496_isrForCrosstalkSources_997020_4.stderr parsl.dataflow.dflow DEBUG: Task 4893 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4893 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4893 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4893 has memoization hash 52b8d269de5577720f05757a3a75dd78 parsl.dataflow.memoization INFO: Task 4893 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c0e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4893 try 0 launched on executor multi with executor id 4894 parsl.dataflow.dflow INFO: Standard output for task 4893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/2449369e-4d49-41fa-849b-008f3fbf6b77_isrForCrosstalkSources_997018_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/2449369e-4d49-41fa-849b-008f3fbf6b77_isrForCrosstalkSources_997018_26.stderr parsl.dataflow.dflow DEBUG: Task 4894 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4894 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4894 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4894 has memoization hash 5f2c3ed8724565a39a123c5e5ba3deba parsl.dataflow.memoization INFO: Task 4894 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4894 try 0 launched on executor multi with executor id 4895 parsl.dataflow.dflow INFO: Standard output for task 4894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/4fd2ace2-b453-482a-8b97-40e4c77eb9ed_isrForCrosstalkSources_996990_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/4fd2ace2-b453-482a-8b97-40e4c77eb9ed_isrForCrosstalkSources_996990_30.stderr parsl.dataflow.dflow DEBUG: Task 4895 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4895 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4895 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4895 has memoization hash c0c4014339bb68fa523e3c4cb06de5d5 parsl.dataflow.memoization INFO: Task 4895 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4895 try 0 launched on executor multi with executor id 4896 parsl.dataflow.dflow INFO: Standard output for task 4895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/2516195f-0706-4da8-b180-fed2fafe0e13_isrForCrosstalkSources_997028_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/2516195f-0706-4da8-b180-fed2fafe0e13_isrForCrosstalkSources_997028_3.stderr parsl.dataflow.dflow DEBUG: Task 4896 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4896 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4896 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4896 has memoization hash 883ba7fe1de4ac2c13ad8c38591eae0e parsl.dataflow.memoization INFO: Task 4896 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c2c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4896 try 0 launched on executor multi with executor id 4897 parsl.dataflow.dflow INFO: Standard output for task 4896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d4da0a8c-8768-4f49-891d-25de523000e5_isrForCrosstalkSources_996997_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d4da0a8c-8768-4f49-891d-25de523000e5_isrForCrosstalkSources_996997_37.stderr parsl.dataflow.dflow DEBUG: Task 4897 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4897 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4897 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4897 has memoization hash fbeebe9b6f7572e7798155c00b33bb98 parsl.dataflow.memoization INFO: Task 4897 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4897 try 0 launched on executor multi with executor id 4898 parsl.dataflow.dflow INFO: Standard output for task 4897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/e142ef6d-7478-44cd-be8d-ed95884b54e0_isrForCrosstalkSources_996967_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/e142ef6d-7478-44cd-be8d-ed95884b54e0_isrForCrosstalkSources_996967_61.stderr parsl.dataflow.dflow DEBUG: Task 4898 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4898 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4898 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4898 has memoization hash d1379656b37634e2acb41872b84ca2ff parsl.dataflow.memoization INFO: Task 4898 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4898 try 0 launched on executor multi with executor id 4899 parsl.dataflow.dflow INFO: Standard output for task 4898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/50553538-638d-42cf-85df-b3dd71a064f9_isrForCrosstalkSources_996974_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/50553538-638d-42cf-85df-b3dd71a064f9_isrForCrosstalkSources_996974_15.stderr parsl.dataflow.dflow DEBUG: Task 4899 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4899 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4899 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4899 has memoization hash cf5bcea1e046d55ca27c098016ff1912 parsl.dataflow.memoization INFO: Task 4899 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4899 try 0 launched on executor multi with executor id 4900 parsl.dataflow.dflow INFO: Standard output for task 4899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c2f9b681-164e-4677-94c1-f40d04b1c7a2_isrForCrosstalkSources_996995_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c2f9b681-164e-4677-94c1-f40d04b1c7a2_isrForCrosstalkSources_996995_15.stderr parsl.dataflow.dflow DEBUG: Task 4900 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4900 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4900 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4900 has memoization hash e27dd5de2150bde1cc22965ab4df8e94 parsl.dataflow.memoization INFO: Task 4900 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c4a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4900 try 0 launched on executor multi with executor id 4901 parsl.dataflow.dflow INFO: Standard output for task 4900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/390e9d8f-e982-42dc-88ab-a6570915e4ac_isrForCrosstalkSources_996970_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/390e9d8f-e982-42dc-88ab-a6570915e4ac_isrForCrosstalkSources_996970_39.stderr parsl.dataflow.dflow DEBUG: Task 4901 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4901 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4901 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4901 has memoization hash 48c80bbe9deba580b42703055da46c9b parsl.dataflow.memoization INFO: Task 4901 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c5e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4901 try 0 launched on executor multi with executor id 4902 parsl.dataflow.dflow INFO: Standard output for task 4901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/4ef14c5a-77a0-4b51-977a-642567c32103_isrForCrosstalkSources_996988_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/4ef14c5a-77a0-4b51-977a-642567c32103_isrForCrosstalkSources_996988_5.stderr parsl.dataflow.dflow DEBUG: Task 4902 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4902 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4902 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4902 has memoization hash cb15b4a5aa8a212573f171827122bc55 parsl.dataflow.memoization INFO: Task 4902 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c680> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4902 try 0 launched on executor multi with executor id 4903 parsl.dataflow.dflow INFO: Standard output for task 4902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/22c9b2f0-062b-4d62-85d5-d0724d541091_isrForCrosstalkSources_997030_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/22c9b2f0-062b-4d62-85d5-d0724d541091_isrForCrosstalkSources_997030_52.stderr parsl.dataflow.dflow DEBUG: Task 4903 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4903 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4903 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4903 has memoization hash efddda1ebe4020a4ec0b77913ae30553 parsl.dataflow.memoization INFO: Task 4903 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4903 try 0 launched on executor multi with executor id 4904 parsl.dataflow.dflow INFO: Standard output for task 4903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/461d13b3-7a28-49d3-8ba3-a7ec57162d01_isrForCrosstalkSources_997033_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/461d13b3-7a28-49d3-8ba3-a7ec57162d01_isrForCrosstalkSources_997033_43.stderr parsl.dataflow.dflow DEBUG: Task 4904 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4904 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4904 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4904 has memoization hash 45fb94fb525428c07cde2ea474c127fa parsl.dataflow.memoization INFO: Task 4904 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4904 try 0 launched on executor multi with executor id 4905 parsl.dataflow.dflow INFO: Standard output for task 4904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/973e6b60-f0b4-4515-8b02-cdf7b8ded2ec_isrForCrosstalkSources_997023_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/973e6b60-f0b4-4515-8b02-cdf7b8ded2ec_isrForCrosstalkSources_997023_40.stderr parsl.dataflow.dflow DEBUG: Task 4905 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4905 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4905 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4905 has memoization hash 03a146b1fcefdd7e5c11d8be3543eed8 parsl.dataflow.memoization INFO: Task 4905 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c7c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4905 try 0 launched on executor multi with executor id 4906 parsl.dataflow.dflow INFO: Standard output for task 4905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/b5cd0d18-106a-4de1-a6d0-c95cf1eb0c8d_isrForCrosstalkSources_996988_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/b5cd0d18-106a-4de1-a6d0-c95cf1eb0c8d_isrForCrosstalkSources_996988_51.stderr parsl.dataflow.dflow DEBUG: Task 4906 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4906 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4906 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4906 has memoization hash bfc8086285f6d5b968d2d817776d7966 parsl.dataflow.memoization INFO: Task 4906 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4906 try 0 launched on executor multi with executor id 4907 parsl.dataflow.dflow INFO: Standard output for task 4906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/fc8557e3-2514-441e-a4ae-78bacbf75a94_isrForCrosstalkSources_996972_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/fc8557e3-2514-441e-a4ae-78bacbf75a94_isrForCrosstalkSources_996972_26.stderr parsl.dataflow.dflow DEBUG: Task 4907 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4907 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4907 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4907 has memoization hash bd38a5f74223c935944e156e480b80d2 parsl.dataflow.memoization INFO: Task 4907 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c9a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4907 try 0 launched on executor multi with executor id 4908 parsl.dataflow.dflow INFO: Standard output for task 4907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/97ec02d2-472b-48eb-bdc3-91ca30b0b44a_isrForCrosstalkSources_996964_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/97ec02d2-472b-48eb-bdc3-91ca30b0b44a_isrForCrosstalkSources_996964_23.stderr parsl.dataflow.dflow DEBUG: Task 4908 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4908 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4908 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4908 has memoization hash 194d2f6fd643482fd6e02f30c6dc2aac parsl.dataflow.memoization INFO: Task 4908 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ca40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4908 try 0 launched on executor multi with executor id 4909 parsl.dataflow.dflow INFO: Standard output for task 4908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/90864c8b-24d3-4e0f-856a-9f2ef66e8b38_isrForCrosstalkSources_997015_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/90864c8b-24d3-4e0f-856a-9f2ef66e8b38_isrForCrosstalkSources_997015_23.stderr parsl.dataflow.dflow DEBUG: Task 4909 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4909 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4909 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4909 has memoization hash efddc89d339400d258ab313ed273d6b4 parsl.dataflow.memoization INFO: Task 4909 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6cb80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4909 try 0 launched on executor multi with executor id 4910 parsl.dataflow.dflow INFO: Standard output for task 4909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/52add8eb-5885-4d5b-872d-069df61190f2_isrForCrosstalkSources_997017_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/52add8eb-5885-4d5b-872d-069df61190f2_isrForCrosstalkSources_997017_9.stderr parsl.dataflow.dflow DEBUG: Task 4910 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4910 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4910 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4910 has memoization hash c3c276f0e992b2f1bd94e194d947125b parsl.dataflow.memoization INFO: Task 4910 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6cae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4910 try 0 launched on executor multi with executor id 4911 parsl.dataflow.dflow INFO: Standard output for task 4910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a1eb21eb-cec1-42e3-8f59-59bba775649e_isrForCrosstalkSources_997011_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a1eb21eb-cec1-42e3-8f59-59bba775649e_isrForCrosstalkSources_997011_56.stderr parsl.dataflow.dflow DEBUG: Task 4911 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4911 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4911 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4911 has memoization hash a4c8b45dd0336625cd5447cb7da7182a parsl.dataflow.memoization INFO: Task 4911 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6cc20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4911 try 0 launched on executor multi with executor id 4912 parsl.dataflow.dflow INFO: Standard output for task 4911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/25c4553a-16a9-4f2a-803a-03b00daf0292_isrForCrosstalkSources_996967_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/25c4553a-16a9-4f2a-803a-03b00daf0292_isrForCrosstalkSources_996967_49.stderr parsl.dataflow.dflow DEBUG: Task 4912 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4912 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4912 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4912 has memoization hash 5de1c80e18f0714f164b60f1885f0f9a parsl.dataflow.memoization INFO: Task 4912 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ccc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4912 try 0 launched on executor multi with executor id 4913 parsl.dataflow.dflow INFO: Standard output for task 4912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/619d40ce-6585-4e6b-945c-2f980bd50885_isrForCrosstalkSources_996963_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/619d40ce-6585-4e6b-945c-2f980bd50885_isrForCrosstalkSources_996963_13.stderr parsl.dataflow.dflow DEBUG: Task 4913 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4913 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4913 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4913 has memoization hash 53acd412f0a5b902e8163e2d46bf4522 parsl.dataflow.memoization INFO: Task 4913 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6cd60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4913 try 0 launched on executor multi with executor id 4914 parsl.dataflow.dflow INFO: Standard output for task 4913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/281b2ad1-d141-4be2-8f34-2134a20c9cdd_isrForCrosstalkSources_997025_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/281b2ad1-d141-4be2-8f34-2134a20c9cdd_isrForCrosstalkSources_997025_18.stderr parsl.dataflow.dflow DEBUG: Task 4914 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4914 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4914 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4914 has memoization hash 24745cae888d54cfbed58e0716888404 parsl.dataflow.memoization INFO: Task 4914 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6cea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4914 try 0 launched on executor multi with executor id 4915 parsl.dataflow.dflow INFO: Standard output for task 4914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/acb4e012-17fb-4154-b557-f3aae791712a_isrForCrosstalkSources_997002_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/acb4e012-17fb-4154-b557-f3aae791712a_isrForCrosstalkSources_997002_12.stderr parsl.dataflow.dflow DEBUG: Task 4915 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4915 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4915 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4915 has memoization hash e100f5c37ddff4dd808cee7782048d4b parsl.dataflow.memoization INFO: Task 4915 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ce00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4915 try 0 launched on executor multi with executor id 4916 parsl.dataflow.dflow INFO: Standard output for task 4915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8b073c14-6335-49bb-841b-bfaeabff2e00_isrForCrosstalkSources_996997_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8b073c14-6335-49bb-841b-bfaeabff2e00_isrForCrosstalkSources_996997_3.stderr parsl.dataflow.dflow DEBUG: Task 4916 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4916 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4916 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4916 has memoization hash b5c880db4f780950f5d411d6b4f21e81 parsl.dataflow.memoization INFO: Task 4916 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6cf40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4916 try 0 launched on executor multi with executor id 4917 parsl.dataflow.dflow INFO: Standard output for task 4916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/e9f6a769-f8ba-448f-be27-6de44baed749_isrForCrosstalkSources_996991_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/e9f6a769-f8ba-448f-be27-6de44baed749_isrForCrosstalkSources_996991_50.stderr parsl.dataflow.dflow DEBUG: Task 4917 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4917 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4917 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4917 has memoization hash fa30116e0a4cefe3bb0bd056ee6e6862 parsl.dataflow.memoization INFO: Task 4917 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6cfe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4917 try 0 launched on executor multi with executor id 4918 parsl.dataflow.dflow INFO: Standard output for task 4917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/dc5a0731-8e68-4d36-b0e8-b368a1799219_isrForCrosstalkSources_997027_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/dc5a0731-8e68-4d36-b0e8-b368a1799219_isrForCrosstalkSources_997027_42.stderr parsl.dataflow.dflow DEBUG: Task 4918 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4918 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4918 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4918 has memoization hash fcd4921571459fa886368d61cb2b97b5 parsl.dataflow.memoization INFO: Task 4918 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4918 try 0 launched on executor multi with executor id 4919 parsl.dataflow.dflow INFO: Standard output for task 4918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f66c4f76-efdc-4e66-9afd-6e0a99db88a1_isrForCrosstalkSources_996978_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f66c4f76-efdc-4e66-9afd-6e0a99db88a1_isrForCrosstalkSources_996978_34.stderr parsl.dataflow.dflow DEBUG: Task 4919 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4919 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4919 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4919 has memoization hash 697266183593dab53c4f7a06e66fcde6 parsl.dataflow.memoization INFO: Task 4919 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d1c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4919 try 0 launched on executor multi with executor id 4920 parsl.dataflow.dflow INFO: Standard output for task 4919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7f2f35f4-8053-4c8b-86ef-3089293697a6_isrForCrosstalkSources_997011_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7f2f35f4-8053-4c8b-86ef-3089293697a6_isrForCrosstalkSources_997011_15.stderr parsl.dataflow.dflow DEBUG: Task 4920 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4920 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4920 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4920 has memoization hash b7fdeb0c79d6699c9025f0e9f35bff7e parsl.dataflow.memoization INFO: Task 4920 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4920 try 0 launched on executor multi with executor id 4921 parsl.dataflow.dflow INFO: Standard output for task 4920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7dddedd8-b6a9-4ce2-ac0a-5834316294fe_isrForCrosstalkSources_997005_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7dddedd8-b6a9-4ce2-ac0a-5834316294fe_isrForCrosstalkSources_997005_16.stderr parsl.dataflow.dflow DEBUG: Task 4921 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4921 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4921 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4921 has memoization hash b5ebbb07fe3893fc419a3926b7167c67 parsl.dataflow.memoization INFO: Task 4921 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4921 try 0 launched on executor multi with executor id 4922 parsl.dataflow.dflow INFO: Standard output for task 4921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0c6489a3-7aa7-4c30-afd9-b750d7409ab2_isrForCrosstalkSources_996997_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0c6489a3-7aa7-4c30-afd9-b750d7409ab2_isrForCrosstalkSources_996997_2.stderr parsl.dataflow.dflow DEBUG: Task 4922 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4922 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4922 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4922 has memoization hash 65088e00f82c1cca7bf23c95c2b92628 parsl.dataflow.memoization INFO: Task 4922 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4922 try 0 launched on executor multi with executor id 4923 parsl.dataflow.dflow INFO: Standard output for task 4922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/75c56022-3198-46ef-85a3-8ce460afc4b3_isrForCrosstalkSources_996958_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/75c56022-3198-46ef-85a3-8ce460afc4b3_isrForCrosstalkSources_996958_10.stderr parsl.dataflow.dflow DEBUG: Task 4923 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4923 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4923 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4923 has memoization hash f1f951f7d294a7cbc0df69b34d61a16b parsl.dataflow.memoization INFO: Task 4923 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d3a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4923 try 0 launched on executor multi with executor id 4924 parsl.dataflow.dflow INFO: Standard output for task 4923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/49bbb278-1d14-44b0-a954-cdfb4e8956b6_isrForCrosstalkSources_996973_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/49bbb278-1d14-44b0-a954-cdfb4e8956b6_isrForCrosstalkSources_996973_38.stderr parsl.dataflow.dflow DEBUG: Task 4924 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4924 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4924 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4924 has memoization hash 3941e01787102e26ff8b6162426bd253 parsl.dataflow.memoization INFO: Task 4924 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d4e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4924 try 0 launched on executor multi with executor id 4925 parsl.dataflow.dflow INFO: Standard output for task 4924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e3226332-5718-4cb8-8e41-fbbf057e1659_isrForCrosstalkSources_996993_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e3226332-5718-4cb8-8e41-fbbf057e1659_isrForCrosstalkSources_996993_53.stderr parsl.dataflow.dflow DEBUG: Task 4925 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4925 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4925 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4925 has memoization hash 52d185486b69e0cab7c478a56abf01ad parsl.dataflow.memoization INFO: Task 4925 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4925 try 0 launched on executor multi with executor id 4926 parsl.dataflow.dflow INFO: Standard output for task 4925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/00b78541-0aa0-4183-8790-8fc7bbdb703d_isrForCrosstalkSources_997016_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/00b78541-0aa0-4183-8790-8fc7bbdb703d_isrForCrosstalkSources_997016_9.stderr parsl.dataflow.dflow DEBUG: Task 4926 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4926 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4926 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4926 has memoization hash a4c39c96b4810ea4951549fde1927671 parsl.dataflow.memoization INFO: Task 4926 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4926 try 0 launched on executor multi with executor id 4927 parsl.dataflow.dflow INFO: Standard output for task 4926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4776aab2-3028-409c-ab09-eee1d744776b_isrForCrosstalkSources_996957_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4776aab2-3028-409c-ab09-eee1d744776b_isrForCrosstalkSources_996957_40.stderr parsl.dataflow.dflow DEBUG: Task 4927 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4927 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4927 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4927 has memoization hash 2118b89ea9335d381594e0cb6a521604 parsl.dataflow.memoization INFO: Task 4927 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4927 try 0 launched on executor multi with executor id 4928 parsl.dataflow.dflow INFO: Standard output for task 4927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/42d0dc47-74c9-4232-a9d8-1ca7de92eac3_isrForCrosstalkSources_996971_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/42d0dc47-74c9-4232-a9d8-1ca7de92eac3_isrForCrosstalkSources_996971_27.stderr parsl.dataflow.dflow DEBUG: Task 4928 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4928 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4928 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4928 has memoization hash d8ba6d5c17bfa4c599397d6e435cf269 parsl.dataflow.memoization INFO: Task 4928 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d6c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4928 try 0 launched on executor multi with executor id 4929 parsl.dataflow.dflow INFO: Standard output for task 4928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ad49c6f0-08a9-4c3a-a488-9872a2451a6f_isrForCrosstalkSources_997002_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ad49c6f0-08a9-4c3a-a488-9872a2451a6f_isrForCrosstalkSources_997002_53.stderr parsl.dataflow.dflow DEBUG: Task 4929 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4929 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4929 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4929 has memoization hash 568ab67759add8ec5119cbeae85e871f parsl.dataflow.memoization INFO: Task 4929 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4929 try 0 launched on executor multi with executor id 4930 parsl.dataflow.dflow INFO: Standard output for task 4929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/82b1896a-a608-40b8-85b8-7c8db851f68b_isrForCrosstalkSources_997003_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/82b1896a-a608-40b8-85b8-7c8db851f68b_isrForCrosstalkSources_997003_26.stderr parsl.dataflow.dflow DEBUG: Task 4930 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4930 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4930 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4930 has memoization hash e095706399f1867133731eb3f2a241e6 parsl.dataflow.memoization INFO: Task 4930 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4930 try 0 launched on executor multi with executor id 4931 parsl.dataflow.dflow INFO: Standard output for task 4930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/d89395d8-0741-49f8-9234-413c33d62c77_isrForCrosstalkSources_997033_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/d89395d8-0741-49f8-9234-413c33d62c77_isrForCrosstalkSources_997033_27.stderr parsl.dataflow.dflow DEBUG: Task 4931 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4931 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4931 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4931 has memoization hash 8c020467f2c7305f3be9bedf43b2e76e parsl.dataflow.memoization INFO: Task 4931 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d8a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4931 try 0 launched on executor multi with executor id 4932 parsl.dataflow.dflow INFO: Standard output for task 4931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7ffb6109-7f17-457c-b3a5-0f7f085cd071_isrForCrosstalkSources_996974_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7ffb6109-7f17-457c-b3a5-0f7f085cd071_isrForCrosstalkSources_996974_58.stderr parsl.dataflow.dflow DEBUG: Task 4932 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4932 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.dataflow.dflow DEBUG: Task 4932 set to pending state with AppFuture: parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Task 4932 has memoization hash ffb05414cf589d478b61edf3b232c16d parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization INFO: Task 4932 had no result in cache parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:11 proc_lsst.multi:146 [INFO] found job 17074842 in provider astro proc_lsst.multi INFO: found job 17074842 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4932 try 0 launched on executor multi with executor id 4933 parsl.dataflow.dflow INFO: Standard output for task 4932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4fae4627-297b-4d65-8e39-bf6b008f1137_isrForCrosstalkSources_997015_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4fae4627-297b-4d65-8e39-bf6b008f1137_isrForCrosstalkSources_997015_26.stderr parsl.dataflow.dflow DEBUG: Task 4933 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4933 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4933 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4933 has memoization hash 3c784a644341b903b9878f86932fba6e parsl.dataflow.memoization INFO: Task 4933 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4ba60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4933 try 0 launched on executor multi with executor id 4934 parsl.dataflow.dflow INFO: Standard output for task 4933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a48bfe39-e8a2-44dc-8302-434a8e2cd56f_isrForCrosstalkSources_997015_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a48bfe39-e8a2-44dc-8302-434a8e2cd56f_isrForCrosstalkSources_997015_3.stderr parsl.dataflow.dflow DEBUG: Task 4934 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4934 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4934 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4934 has memoization hash 054cc3f9af5450fe1a29b03569f322ed parsl.dataflow.memoization INFO: Task 4934 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767d4ac00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4934 try 0 launched on executor multi with executor id 4935 parsl.dataflow.dflow INFO: Standard output for task 4934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/ce77481d-b2cb-4972-87ec-2a2672dfb0b5_isrForCrosstalkSources_997014_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/ce77481d-b2cb-4972-87ec-2a2672dfb0b5_isrForCrosstalkSources_997014_58.stderr parsl.dataflow.dflow DEBUG: Task 4935 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4935 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4935 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4935 has memoization hash d9ab695bc995e5f3c618b557db1567d4 parsl.dataflow.memoization INFO: Task 4935 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6da80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4935 try 0 launched on executor multi with executor id 4936 parsl.dataflow.dflow INFO: Standard output for task 4935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/1a2f32bf-7479-4f00-990a-439ed3590484_isrForCrosstalkSources_996974_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/1a2f32bf-7479-4f00-990a-439ed3590484_isrForCrosstalkSources_996974_45.stderr parsl.dataflow.dflow DEBUG: Task 4936 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4936 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4936 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4936 has memoization hash c976c9e38b6cba2a1f8b3a91719246e8 parsl.dataflow.memoization INFO: Task 4936 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6fe20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4936 try 0 launched on executor multi with executor id 4937 parsl.dataflow.dflow INFO: Standard output for task 4936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/33fb6ffe-bd91-46d5-9382-86b360af45ea_isrForCrosstalkSources_996970_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/33fb6ffe-bd91-46d5-9382-86b360af45ea_isrForCrosstalkSources_996970_43.stderr parsl.dataflow.dflow DEBUG: Task 4937 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4937 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4937 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4937 has memoization hash 1fa52042dc69d629401b324fe2e64c38 parsl.dataflow.memoization INFO: Task 4937 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6fec0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4937 try 0 launched on executor multi with executor id 4938 parsl.dataflow.dflow INFO: Standard output for task 4937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/21b9699a-b1b4-4366-90d6-f79bcbcc6173_isrForCrosstalkSources_996991_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/21b9699a-b1b4-4366-90d6-f79bcbcc6173_isrForCrosstalkSources_996991_54.stderr parsl.dataflow.dflow DEBUG: Task 4938 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4938 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4938 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4938 has memoization hash 2d1380783cdf611524073c58201c8628 parsl.dataflow.memoization INFO: Task 4938 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6fd80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4938 try 0 launched on executor multi with executor id 4939 parsl.dataflow.dflow INFO: Standard output for task 4938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/571664c2-752e-4824-9111-fe5f2ba3bb78_isrForCrosstalkSources_996963_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/571664c2-752e-4824-9111-fe5f2ba3bb78_isrForCrosstalkSources_996963_25.stderr parsl.dataflow.dflow DEBUG: Task 4939 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4939 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4939 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4939 has memoization hash e8717ceeec0a367fde9da02bda4e1eda parsl.dataflow.memoization INFO: Task 4939 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6db20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4939 try 0 launched on executor multi with executor id 4940 parsl.dataflow.dflow INFO: Standard output for task 4939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/aa2d3eff-2e98-4e57-8925-e8a4e88889cc_isrForCrosstalkSources_996983_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/aa2d3eff-2e98-4e57-8925-e8a4e88889cc_isrForCrosstalkSources_996983_17.stderr parsl.dataflow.dflow DEBUG: Task 4940 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4940 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4940 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4940 has memoization hash cfb164ac33df3d16c43ab2eeb5314151 parsl.dataflow.memoization INFO: Task 4940 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6dbc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4940 try 0 launched on executor multi with executor id 4941 parsl.dataflow.dflow INFO: Standard output for task 4940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f80dc8de-d66f-479e-b5db-e987f7e53f8f_isrForCrosstalkSources_997031_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f80dc8de-d66f-479e-b5db-e987f7e53f8f_isrForCrosstalkSources_997031_1.stderr parsl.dataflow.dflow DEBUG: Task 4941 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4941 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4941 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4941 has memoization hash fcf7ba183e028e237458dc4ed3f1722b parsl.dataflow.memoization INFO: Task 4941 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6dd00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4941 try 0 launched on executor multi with executor id 4942 parsl.dataflow.dflow INFO: Standard output for task 4941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b3037a65-bbba-4c69-bb41-6598b1c7194b_isrForCrosstalkSources_997008_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b3037a65-bbba-4c69-bb41-6598b1c7194b_isrForCrosstalkSources_997008_48.stderr parsl.dataflow.dflow DEBUG: Task 4942 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4942 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4942 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4942 has memoization hash 0935a922fdac3eb6efe6ebed8ed05b07 parsl.dataflow.memoization INFO: Task 4942 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6dda0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4942 try 0 launched on executor multi with executor id 4943 parsl.dataflow.dflow INFO: Standard output for task 4942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5dbb3d6f-c032-422d-bdba-db45b21f6b46_isrForCrosstalkSources_996991_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5dbb3d6f-c032-422d-bdba-db45b21f6b46_isrForCrosstalkSources_996991_6.stderr parsl.dataflow.dflow DEBUG: Task 4943 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4943 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4943 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4943 has memoization hash 793970748d13257ae8805d73727cf1e5 parsl.dataflow.memoization INFO: Task 4943 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6de40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4943 try 0 launched on executor multi with executor id 4944 parsl.dataflow.dflow INFO: Standard output for task 4943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/c20f1c22-5bd6-46af-9adb-6098cf454720_isrForCrosstalkSources_996978_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/c20f1c22-5bd6-46af-9adb-6098cf454720_isrForCrosstalkSources_996978_10.stderr parsl.dataflow.dflow DEBUG: Task 4944 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4944 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4944 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4944 has memoization hash 15eea248b04d591bff3248939b5ffbd5 parsl.dataflow.memoization INFO: Task 4944 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6dee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4944 try 0 launched on executor multi with executor id 4945 parsl.dataflow.dflow INFO: Standard output for task 4944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a57e7056-f2cd-4b73-95a5-85c37fd42110_isrForCrosstalkSources_997033_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a57e7056-f2cd-4b73-95a5-85c37fd42110_isrForCrosstalkSources_997033_4.stderr parsl.dataflow.dflow DEBUG: Task 4945 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4945 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4945 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4945 has memoization hash 86b60cced4116311e7a060f43053c529 parsl.dataflow.memoization INFO: Task 4945 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6dc60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4945 try 0 launched on executor multi with executor id 4946 parsl.dataflow.dflow INFO: Standard output for task 4945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a5e0dd4d-f43b-4efa-82b2-941fd94784de_isrForCrosstalkSources_997007_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a5e0dd4d-f43b-4efa-82b2-941fd94784de_isrForCrosstalkSources_997007_55.stderr parsl.dataflow.dflow DEBUG: Task 4946 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4946 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4946 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4946 has memoization hash 16fe66dfa58434e25baee440b3c0846f parsl.dataflow.memoization INFO: Task 4946 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6df80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4946 try 0 launched on executor multi with executor id 4947 parsl.dataflow.dflow INFO: Standard output for task 4946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5f43eab8-126a-47d4-ae03-4e4ea0df3e26_isrForCrosstalkSources_996991_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5f43eab8-126a-47d4-ae03-4e4ea0df3e26_isrForCrosstalkSources_996991_58.stderr parsl.dataflow.dflow DEBUG: Task 4947 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4947 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4947 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4947 has memoization hash 56306f1db520038813d492b12a0460b6 parsl.dataflow.memoization INFO: Task 4947 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4947 try 0 launched on executor multi with executor id 4948 parsl.dataflow.dflow INFO: Standard output for task 4947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/123405ee-ca5e-4b54-bc57-32528372223b_isrForCrosstalkSources_996969_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/123405ee-ca5e-4b54-bc57-32528372223b_isrForCrosstalkSources_996969_2.stderr parsl.dataflow.dflow DEBUG: Task 4948 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4948 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4948 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4948 has memoization hash 8b48c5055178e8163952780502184bb2 parsl.dataflow.memoization INFO: Task 4948 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4948 try 0 launched on executor multi with executor id 4949 parsl.dataflow.dflow INFO: Standard output for task 4948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a921803e-82ed-47e7-b84a-52034c863d83_isrForCrosstalkSources_996994_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a921803e-82ed-47e7-b84a-52034c863d83_isrForCrosstalkSources_996994_20.stderr parsl.dataflow.dflow DEBUG: Task 4949 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4949 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4949 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4949 has memoization hash b36087d4cbaf42e49c947e1dbfc0ac6b parsl.dataflow.memoization INFO: Task 4949 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e0c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4949 try 0 launched on executor multi with executor id 4950 parsl.dataflow.dflow INFO: Standard output for task 4949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a5fb8e34-080b-4ef1-a4be-000910358540_isrForCrosstalkSources_997007_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a5fb8e34-080b-4ef1-a4be-000910358540_isrForCrosstalkSources_997007_10.stderr parsl.dataflow.dflow DEBUG: Task 4950 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4950 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4950 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4950 has memoization hash c9b4d3261046661f50698d3e9c596d78 parsl.dataflow.memoization INFO: Task 4950 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4950 try 0 launched on executor multi with executor id 4951 parsl.dataflow.dflow INFO: Standard output for task 4950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3248d48a-4048-4cec-9921-4c65e2c9158b_isrForCrosstalkSources_997001_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3248d48a-4048-4cec-9921-4c65e2c9158b_isrForCrosstalkSources_997001_14.stderr parsl.dataflow.dflow DEBUG: Task 4951 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4951 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4951 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4951 has memoization hash a224a773b7f73e02bb3a1338c8ead304 parsl.dataflow.memoization INFO: Task 4951 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e2a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4951 try 0 launched on executor multi with executor id 4952 parsl.dataflow.dflow INFO: Standard output for task 4951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/dd5df0c5-4f77-4e58-8e9c-654a227a0a05_isrForCrosstalkSources_997022_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/dd5df0c5-4f77-4e58-8e9c-654a227a0a05_isrForCrosstalkSources_997022_7.stderr parsl.dataflow.dflow DEBUG: Task 4952 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4952 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4952 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4952 has memoization hash eb365cef957e87572ed6b12be0c3468f parsl.dataflow.memoization INFO: Task 4952 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4952 try 0 launched on executor multi with executor id 4953 parsl.dataflow.dflow INFO: Standard output for task 4952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/103bbf15-3378-4d70-b967-a3f418ea1177_isrForCrosstalkSources_997021_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/103bbf15-3378-4d70-b967-a3f418ea1177_isrForCrosstalkSources_997021_49.stderr parsl.dataflow.dflow DEBUG: Task 4953 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4953 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4953 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4953 has memoization hash a2bea23d83d18dd6cda509b336585022 parsl.dataflow.memoization INFO: Task 4953 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e3e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4953 try 0 launched on executor multi with executor id 4954 parsl.dataflow.dflow INFO: Standard output for task 4953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f50d2c37-d952-416f-ba61-8b794ed7f14f_isrForCrosstalkSources_996985_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f50d2c37-d952-416f-ba61-8b794ed7f14f_isrForCrosstalkSources_996985_11.stderr parsl.dataflow.dflow DEBUG: Task 4954 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4954 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4954 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4954 has memoization hash 50b79d783762f33dfd264b89255d2565 parsl.dataflow.memoization INFO: Task 4954 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e5c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4954 try 0 launched on executor multi with executor id 4955 parsl.dataflow.dflow INFO: Standard output for task 4954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/39aacde7-cf5d-4fee-910c-11d6ea65fd8c_isrForCrosstalkSources_997008_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/39aacde7-cf5d-4fee-910c-11d6ea65fd8c_isrForCrosstalkSources_997008_19.stderr parsl.dataflow.dflow DEBUG: Task 4955 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4955 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4955 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4955 has memoization hash 650d0fcc5508f9a06fb3b84b09d16290 parsl.dataflow.memoization INFO: Task 4955 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4955 try 0 launched on executor multi with executor id 4956 parsl.dataflow.dflow INFO: Standard output for task 4955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ba88985c-70a5-43c7-9478-bb9fc96d5ea4_isrForCrosstalkSources_996976_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ba88985c-70a5-43c7-9478-bb9fc96d5ea4_isrForCrosstalkSources_996976_18.stderr parsl.dataflow.dflow DEBUG: Task 4956 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4956 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4956 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4956 has memoization hash 577ed3183c589481459632c56ca6d343 parsl.dataflow.memoization INFO: Task 4956 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e660> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4956 try 0 launched on executor multi with executor id 4957 parsl.dataflow.dflow INFO: Standard output for task 4956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f22722b1-7cc0-4d5f-b48e-b9c11a0111db_isrForCrosstalkSources_996993_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f22722b1-7cc0-4d5f-b48e-b9c11a0111db_isrForCrosstalkSources_996993_56.stderr parsl.dataflow.dflow DEBUG: Task 4957 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4957 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4957 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4957 has memoization hash 3f8a1315e3146677cd2083ead118b827 parsl.dataflow.memoization INFO: Task 4957 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e480> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4957 try 0 launched on executor multi with executor id 4958 parsl.dataflow.dflow INFO: Standard output for task 4957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b57b4c00-2769-417e-8f30-292cc3c670fa_isrForCrosstalkSources_996998_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b57b4c00-2769-417e-8f30-292cc3c670fa_isrForCrosstalkSources_996998_20.stderr parsl.dataflow.dflow DEBUG: Task 4958 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4958 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4958 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4958 has memoization hash d21efcf2300773972b20a2aac4f6ab94 parsl.dataflow.memoization INFO: Task 4958 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e700> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4958 try 0 launched on executor multi with executor id 4959 parsl.dataflow.dflow INFO: Standard output for task 4958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e5c0deec-eecd-4305-a062-3863979564ba_isrForCrosstalkSources_997032_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e5c0deec-eecd-4305-a062-3863979564ba_isrForCrosstalkSources_997032_38.stderr parsl.dataflow.dflow DEBUG: Task 4959 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4959 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4959 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4959 has memoization hash 8cd64ad3b4b8a52a1e4da4ecfbb98a9f parsl.dataflow.memoization INFO: Task 4959 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e8e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4959 try 0 launched on executor multi with executor id 4960 parsl.dataflow.dflow INFO: Standard output for task 4959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/04f2de67-0580-4d08-af10-8410240d94de_isrForCrosstalkSources_996964_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/04f2de67-0580-4d08-af10-8410240d94de_isrForCrosstalkSources_996964_36.stderr parsl.dataflow.dflow DEBUG: Task 4960 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4960 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4960 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4960 has memoization hash 847ea697062e2271fb4f9d2aa9d2084f parsl.dataflow.memoization INFO: Task 4960 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4960 try 0 launched on executor multi with executor id 4961 parsl.dataflow.dflow INFO: Standard output for task 4960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/86f97569-30a2-422d-94ba-b75a19d86b97_isrForCrosstalkSources_996976_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/86f97569-30a2-422d-94ba-b75a19d86b97_isrForCrosstalkSources_996976_58.stderr parsl.dataflow.dflow DEBUG: Task 4961 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4961 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4961 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4961 has memoization hash 306d4db939a5b7eaaaa5cb000825abb4 parsl.dataflow.memoization INFO: Task 4961 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4961 try 0 launched on executor multi with executor id 4962 parsl.dataflow.dflow INFO: Standard output for task 4961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/51915ae6-f1e1-4ac0-a64a-d6d2b7a0270e_isrForCrosstalkSources_996975_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/51915ae6-f1e1-4ac0-a64a-d6d2b7a0270e_isrForCrosstalkSources_996975_23.stderr parsl.dataflow.dflow DEBUG: Task 4962 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4962 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4962 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4962 has memoization hash 8895cd78d040f4924441a3013d68a9a9 parsl.dataflow.memoization INFO: Task 4962 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6e7a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4962 try 0 launched on executor multi with executor id 4963 parsl.dataflow.dflow INFO: Standard output for task 4962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e37aa698-d90c-44ee-9aa5-6498263e5592_isrForCrosstalkSources_997023_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e37aa698-d90c-44ee-9aa5-6498263e5592_isrForCrosstalkSources_997023_42.stderr parsl.dataflow.dflow DEBUG: Task 4963 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4963 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4963 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4963 has memoization hash 517b41888cbe7af0f78e9749977444b5 parsl.dataflow.memoization INFO: Task 4963 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6eac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4963 try 0 launched on executor multi with executor id 4964 parsl.dataflow.dflow INFO: Standard output for task 4963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d8aeca74-e26d-4ef8-bdc3-9a1130890544_isrForCrosstalkSources_997017_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d8aeca74-e26d-4ef8-bdc3-9a1130890544_isrForCrosstalkSources_997017_3.stderr parsl.dataflow.dflow DEBUG: Task 4964 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4964 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4964 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4964 has memoization hash 65fd4cb91296c61c0f07a7c7cd6360c0 parsl.dataflow.memoization INFO: Task 4964 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6eb60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4964 try 0 launched on executor multi with executor id 4965 parsl.dataflow.dflow INFO: Standard output for task 4964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/31eb6657-1929-4afc-b8ce-ddc0652ea54f_isrForCrosstalkSources_996996_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/31eb6657-1929-4afc-b8ce-ddc0652ea54f_isrForCrosstalkSources_996996_50.stderr parsl.dataflow.dflow DEBUG: Task 4965 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4965 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4965 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4965 has memoization hash fb1d61db9d062fad62b6bb07990fe2d1 parsl.dataflow.memoization INFO: Task 4965 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ea20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4965 try 0 launched on executor multi with executor id 4966 parsl.dataflow.dflow INFO: Standard output for task 4965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/bccc67bd-5350-4d56-975c-84acdffc926b_isrForCrosstalkSources_996994_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/bccc67bd-5350-4d56-975c-84acdffc926b_isrForCrosstalkSources_996994_34.stderr parsl.dataflow.dflow DEBUG: Task 4966 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4966 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4966 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4966 has memoization hash ce79982783d46e5aed65604b0d4afb49 parsl.dataflow.memoization INFO: Task 4966 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6eca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4966 try 0 launched on executor multi with executor id 4967 parsl.dataflow.dflow INFO: Standard output for task 4966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a5cde737-adc0-402a-9552-73187a8e7bd2_isrForCrosstalkSources_997001_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a5cde737-adc0-402a-9552-73187a8e7bd2_isrForCrosstalkSources_997001_7.stderr parsl.dataflow.dflow DEBUG: Task 4967 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4967 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4967 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4967 has memoization hash e6e920deee3c4800310858be4585fde8 parsl.dataflow.memoization INFO: Task 4967 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ec00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4967 try 0 launched on executor multi with executor id 4968 parsl.dataflow.dflow INFO: Standard output for task 4967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/ae8f94e1-4aee-45d8-a21b-8ceebf15ce6a_isrForCrosstalkSources_996996_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/ae8f94e1-4aee-45d8-a21b-8ceebf15ce6a_isrForCrosstalkSources_996996_52.stderr parsl.dataflow.dflow DEBUG: Task 4968 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4968 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4968 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4968 has memoization hash bebbaf2ec1da4a468deaaa610792102c parsl.dataflow.memoization INFO: Task 4968 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ede0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4968 try 0 launched on executor multi with executor id 4969 parsl.dataflow.dflow INFO: Standard output for task 4968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4ba671ab-8fe2-4d2c-b6cf-2d25faf193e5_isrForCrosstalkSources_996999_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4ba671ab-8fe2-4d2c-b6cf-2d25faf193e5_isrForCrosstalkSources_996999_28.stderr parsl.dataflow.dflow DEBUG: Task 4969 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4969 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4969 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4969 has memoization hash 232538f470e23fceb78921f5d406191e parsl.dataflow.memoization INFO: Task 4969 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ee80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4969 try 0 launched on executor multi with executor id 4970 parsl.dataflow.dflow INFO: Standard output for task 4969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/52006b67-e71c-41c2-89be-fce49a01ef3c_isrForCrosstalkSources_996984_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/52006b67-e71c-41c2-89be-fce49a01ef3c_isrForCrosstalkSources_996984_14.stderr parsl.dataflow.dflow DEBUG: Task 4970 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4970 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4970 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4970 has memoization hash 45b7d4abdbdd4268ed0a6272a3b9c874 parsl.dataflow.memoization INFO: Task 4970 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ed40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4970 try 0 launched on executor multi with executor id 4971 parsl.dataflow.dflow INFO: Standard output for task 4970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ea5e6aaf-a722-41db-bc18-9637d2286e50_isrForCrosstalkSources_996981_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ea5e6aaf-a722-41db-bc18-9637d2286e50_isrForCrosstalkSources_996981_53.stderr parsl.dataflow.dflow DEBUG: Task 4971 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4971 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4971 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4971 has memoization hash ca488654bb44a86557f7ff62eec5aac5 parsl.dataflow.memoization INFO: Task 4971 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6efc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4971 try 0 launched on executor multi with executor id 4972 parsl.dataflow.dflow INFO: Standard output for task 4971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1e65a889-7ad6-448a-8496-eb134e42d924_isrForCrosstalkSources_996958_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1e65a889-7ad6-448a-8496-eb134e42d924_isrForCrosstalkSources_996958_24.stderr parsl.dataflow.dflow DEBUG: Task 4972 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4972 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4972 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4972 has memoization hash feb31a8ed15fce53f62bb71da0465e2b parsl.dataflow.memoization INFO: Task 4972 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ef20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4972 try 0 launched on executor multi with executor id 4973 parsl.dataflow.dflow INFO: Standard output for task 4972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/423f2ddc-9184-4841-bfd3-6a4abfa2bb63_isrForCrosstalkSources_997029_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/423f2ddc-9184-4841-bfd3-6a4abfa2bb63_isrForCrosstalkSources_997029_58.stderr parsl.dataflow.dflow DEBUG: Task 4973 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4973 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4973 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4973 has memoization hash e86d39df429eecf21bb9a4132c4b8c74 parsl.dataflow.memoization INFO: Task 4973 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4973 try 0 launched on executor multi with executor id 4974 parsl.dataflow.dflow INFO: Standard output for task 4973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e36a2d4d-9203-4899-b89e-9ec7c397b87d_isrForCrosstalkSources_997001_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e36a2d4d-9203-4899-b89e-9ec7c397b87d_isrForCrosstalkSources_997001_37.stderr parsl.dataflow.dflow DEBUG: Task 4974 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4974 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4974 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4974 has memoization hash 5cc240efbc258b4e2ac90753ed8c7790 parsl.dataflow.memoization INFO: Task 4974 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4974 try 0 launched on executor multi with executor id 4975 parsl.dataflow.dflow INFO: Standard output for task 4974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/be5f8e99-8371-4ee5-9e7c-a4d64a98062b_isrForCrosstalkSources_996981_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/be5f8e99-8371-4ee5-9e7c-a4d64a98062b_isrForCrosstalkSources_996981_61.stderr parsl.dataflow.dflow DEBUG: Task 4975 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4975 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4975 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4975 has memoization hash 7c70f0641c012fd4f413b7a14f820c00 parsl.dataflow.memoization INFO: Task 4975 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4975 try 0 launched on executor multi with executor id 4976 parsl.dataflow.dflow INFO: Standard output for task 4975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d5c4cfba-dc0b-476f-81aa-f05b32ecc1d8_isrForCrosstalkSources_996973_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d5c4cfba-dc0b-476f-81aa-f05b32ecc1d8_isrForCrosstalkSources_996973_41.stderr parsl.dataflow.dflow DEBUG: Task 4976 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4976 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4976 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4976 has memoization hash 6baf1a4e6c96dae70ad04a83812af9b8 parsl.dataflow.memoization INFO: Task 4976 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f2e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4976 try 0 launched on executor multi with executor id 4977 parsl.dataflow.dflow INFO: Standard output for task 4976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/0459e4d3-f5a2-422d-a22a-d0608db1ec1c_isrForCrosstalkSources_996952_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/0459e4d3-f5a2-422d-a22a-d0608db1ec1c_isrForCrosstalkSources_996952_58.stderr parsl.dataflow.dflow DEBUG: Task 4977 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4977 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4977 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4977 has memoization hash 11935ee2a94e4246a9b3235df8dd2ab1 parsl.dataflow.memoization INFO: Task 4977 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f1a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4977 try 0 launched on executor multi with executor id 4978 parsl.dataflow.dflow INFO: Standard output for task 4977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7b8bf87f-6d07-4469-a4b2-05b23af3da21_isrForCrosstalkSources_996993_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7b8bf87f-6d07-4469-a4b2-05b23af3da21_isrForCrosstalkSources_996993_49.stderr parsl.dataflow.dflow DEBUG: Task 4978 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4978 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4978 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4978 has memoization hash 2fc36cf7c45f3f227a2f87c9d0b6731b parsl.dataflow.memoization INFO: Task 4978 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4978 try 0 launched on executor multi with executor id 4979 parsl.dataflow.dflow INFO: Standard output for task 4978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c5517f74-2e4a-46c1-8e25-00163fa94750_isrForCrosstalkSources_996966_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c5517f74-2e4a-46c1-8e25-00163fa94750_isrForCrosstalkSources_996966_20.stderr parsl.dataflow.dflow DEBUG: Task 4979 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4979 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4979 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4979 has memoization hash bfedad5d6ef8d0a28dc2f6b02764b520 parsl.dataflow.memoization INFO: Task 4979 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4979 try 0 launched on executor multi with executor id 4980 parsl.dataflow.dflow INFO: Standard output for task 4979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/77947747-57b0-4f35-b7c7-feb0677c03c4_isrForCrosstalkSources_996958_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/77947747-57b0-4f35-b7c7-feb0677c03c4_isrForCrosstalkSources_996958_54.stderr parsl.dataflow.dflow DEBUG: Task 4980 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4980 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4980 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4980 has memoization hash c38e620b085679deab07bbc912c0e4d8 parsl.dataflow.memoization INFO: Task 4980 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4980 try 0 launched on executor multi with executor id 4981 parsl.dataflow.dflow INFO: Standard output for task 4980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/181b69fc-6f12-4241-9afd-16bf1e81eb7f_isrForCrosstalkSources_996981_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/181b69fc-6f12-4241-9afd-16bf1e81eb7f_isrForCrosstalkSources_996981_58.stderr parsl.dataflow.dflow DEBUG: Task 4981 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4981 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4981 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4981 has memoization hash a26e57fb196675d75816e08a97f9fbdd parsl.dataflow.memoization INFO: Task 4981 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f380> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4981 try 0 launched on executor multi with executor id 4982 parsl.dataflow.dflow INFO: Standard output for task 4981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/41cfd45a-5820-49ce-8e60-9a1f0a8eaf0e_isrForCrosstalkSources_996999_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/41cfd45a-5820-49ce-8e60-9a1f0a8eaf0e_isrForCrosstalkSources_996999_20.stderr parsl.dataflow.dflow DEBUG: Task 4982 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4982 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4982 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4982 has memoization hash 4a7a3f8574f0ac9cc168a97d1b44a19f parsl.dataflow.memoization INFO: Task 4982 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f6a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4982 try 0 launched on executor multi with executor id 4983 parsl.dataflow.dflow INFO: Standard output for task 4982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b76cf203-be82-446a-9b83-72132c3dda26_isrForCrosstalkSources_996997_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b76cf203-be82-446a-9b83-72132c3dda26_isrForCrosstalkSources_996997_21.stderr parsl.dataflow.dflow DEBUG: Task 4983 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4983 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4983 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4983 has memoization hash ddd47665c9de3155b95189d294c41b71 parsl.dataflow.memoization INFO: Task 4983 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4983 try 0 launched on executor multi with executor id 4984 parsl.dataflow.dflow INFO: Standard output for task 4983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/bd85914c-dca6-4123-925f-7c4f6bdbf45d_isrForCrosstalkSources_996976_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/bd85914c-dca6-4123-925f-7c4f6bdbf45d_isrForCrosstalkSources_996976_19.stderr parsl.dataflow.dflow DEBUG: Task 4984 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4984 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4984 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4984 has memoization hash 94c6bf294ccd6f0f4b11d986232ca090 parsl.dataflow.memoization INFO: Task 4984 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4984 try 0 launched on executor multi with executor id 4985 parsl.dataflow.dflow INFO: Standard output for task 4984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/123e4ecd-1cb5-4713-acef-2eb1c4adc511_isrForCrosstalkSources_996980_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/123e4ecd-1cb5-4713-acef-2eb1c4adc511_isrForCrosstalkSources_996980_9.stderr parsl.dataflow.dflow DEBUG: Task 4985 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4985 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4985 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4985 has memoization hash d4b346acb013bb939e193fb5d046058f parsl.dataflow.memoization INFO: Task 4985 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f7e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4985 try 0 launched on executor multi with executor id 4986 parsl.dataflow.dflow INFO: Standard output for task 4985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d4d2c8ba-4fb5-4917-8fc8-f949841b34ca_isrForCrosstalkSources_997023_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d4d2c8ba-4fb5-4917-8fc8-f949841b34ca_isrForCrosstalkSources_997023_19.stderr parsl.dataflow.dflow DEBUG: Task 4986 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4986 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4986 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4986 has memoization hash e4d2fc266cace071725063eec21a12a4 parsl.dataflow.memoization INFO: Task 4986 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4986 try 0 launched on executor multi with executor id 4987 parsl.dataflow.dflow INFO: Standard output for task 4986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/4ee42cb6-dacf-417d-b138-ea9bf053f1c9_isrForCrosstalkSources_997025_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/4ee42cb6-dacf-417d-b138-ea9bf053f1c9_isrForCrosstalkSources_997025_44.stderr parsl.dataflow.dflow DEBUG: Task 4987 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4987 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4987 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4987 has memoization hash 5d2faa1548cd23feefd3a06173c5296b parsl.dataflow.memoization INFO: Task 4987 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4987 try 0 launched on executor multi with executor id 4988 parsl.dataflow.dflow INFO: Standard output for task 4987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/891a8797-b31c-4165-a0af-db743f1bf27d_isrForCrosstalkSources_997009_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/891a8797-b31c-4165-a0af-db743f1bf27d_isrForCrosstalkSources_997009_38.stderr parsl.dataflow.dflow DEBUG: Task 4988 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4988 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4988 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4988 has memoization hash 247a11367889facf79726314095b493b parsl.dataflow.memoization INFO: Task 4988 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6fa60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4988 try 0 launched on executor multi with executor id 4989 parsl.dataflow.dflow INFO: Standard output for task 4988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e51bcad0-bc27-40b5-a1f7-7a86317b77f8_isrForCrosstalkSources_996970_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e51bcad0-bc27-40b5-a1f7-7a86317b77f8_isrForCrosstalkSources_996970_29.stderr parsl.dataflow.dflow DEBUG: Task 4989 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4989 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4989 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4989 has memoization hash 599372c7184f012c0d2acb5fe7c3e49c parsl.dataflow.memoization INFO: Task 4989 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6fc40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4989 try 0 launched on executor multi with executor id 4990 parsl.dataflow.dflow INFO: Standard output for task 4989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/3d4bbcdd-e477-4f02-a458-01fa1bdf05bd_isrForCrosstalkSources_997012_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/3d4bbcdd-e477-4f02-a458-01fa1bdf05bd_isrForCrosstalkSources_997012_16.stderr parsl.dataflow.dflow DEBUG: Task 4990 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4990 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4990 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4990 has memoization hash e532e13a7a3c415a2e6ce9f70b7a9929 parsl.dataflow.memoization INFO: Task 4990 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6fce0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4990 try 0 launched on executor multi with executor id 4991 parsl.dataflow.dflow INFO: Standard output for task 4990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/1d386aaa-d4d8-411c-9d43-b693eaa674f5_isrForCrosstalkSources_996991_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/1d386aaa-d4d8-411c-9d43-b693eaa674f5_isrForCrosstalkSources_996991_29.stderr parsl.dataflow.dflow DEBUG: Task 4991 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4991 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4991 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4991 has memoization hash 3c04e7e49f699050a116379f0d793304 parsl.dataflow.memoization INFO: Task 4991 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6fba0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4991 try 0 launched on executor multi with executor id 4992 parsl.dataflow.dflow INFO: Standard output for task 4991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/cf3acd91-d6e8-44c9-99b4-e908e3c8d765_isrForCrosstalkSources_996966_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/cf3acd91-d6e8-44c9-99b4-e908e3c8d765_isrForCrosstalkSources_996966_59.stderr parsl.dataflow.dflow DEBUG: Task 4992 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4992 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4992 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4992 has memoization hash 16a490fa810627bfe760bb30cad1a344 parsl.dataflow.memoization INFO: Task 4992 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6fb00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4992 try 0 launched on executor multi with executor id 4993 parsl.dataflow.dflow INFO: Standard output for task 4992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/7247f74e-93c2-40fc-9ec4-39508355f5e7_isrForCrosstalkSources_996958_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/7247f74e-93c2-40fc-9ec4-39508355f5e7_isrForCrosstalkSources_996958_9.stderr parsl.dataflow.dflow DEBUG: Task 4993 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4993 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4993 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4993 has memoization hash 76ea9f3d3200aa656b338cdbcd84f513 parsl.dataflow.memoization INFO: Task 4993 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6f9c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4993 try 0 launched on executor multi with executor id 4994 parsl.dataflow.dflow INFO: Standard output for task 4993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/edb88570-bac2-4613-a27b-df5ff13ccd14_isrForCrosstalkSources_996996_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/edb88570-bac2-4613-a27b-df5ff13ccd14_isrForCrosstalkSources_996996_4.stderr parsl.dataflow.dflow DEBUG: Task 4994 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4994 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4994 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4994 has memoization hash 27fc6cb4f5bd1cfd30e3f2d233ec79e0 parsl.dataflow.memoization INFO: Task 4994 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6ff60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4994 try 0 launched on executor multi with executor id 4995 parsl.dataflow.dflow INFO: Standard output for task 4994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ac2014e5-1011-4a5c-ad3b-34fc70dcaade_isrForCrosstalkSources_997013_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ac2014e5-1011-4a5c-ad3b-34fc70dcaade_isrForCrosstalkSources_997013_2.stderr parsl.dataflow.dflow DEBUG: Task 4995 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4995 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4995 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4995 has memoization hash 5f1d9d8254e1b0642ea4dedf370dabab parsl.dataflow.memoization INFO: Task 4995 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4995 try 0 launched on executor multi with executor id 4996 parsl.dataflow.dflow INFO: Standard output for task 4995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8c20d3d5-82ac-4c3b-8748-21d033281bf3_isrForCrosstalkSources_997027_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8c20d3d5-82ac-4c3b-8748-21d033281bf3_isrForCrosstalkSources_997027_20.stderr parsl.dataflow.dflow DEBUG: Task 4996 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4996 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4996 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4996 has memoization hash 51868f8bf2c36a55ef5f83c1404128ba parsl.dataflow.memoization INFO: Task 4996 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04180> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4996 try 0 launched on executor multi with executor id 4997 parsl.dataflow.dflow INFO: Standard output for task 4996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e7a7b29d-3d73-48b0-939f-253ffe5d3846_isrForCrosstalkSources_996956_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e7a7b29d-3d73-48b0-939f-253ffe5d3846_isrForCrosstalkSources_996956_50.stderr parsl.dataflow.dflow DEBUG: Task 4997 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4997 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4997 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4997 has memoization hash 1fc970260159f6af5eb255fd48a69641 parsl.dataflow.memoization INFO: Task 4997 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04220> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4997 try 0 launched on executor multi with executor id 4998 parsl.dataflow.dflow INFO: Standard output for task 4997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/dfc93ba7-58cc-493e-8139-8ffbe08133de_isrForCrosstalkSources_996955_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/dfc93ba7-58cc-493e-8139-8ffbe08133de_isrForCrosstalkSources_996955_50.stderr parsl.dataflow.dflow DEBUG: Task 4998 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4998 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4998 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4998 has memoization hash 8d307bab97d591a5419465fcd36efc6e parsl.dataflow.memoization INFO: Task 4998 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b042c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 4998 try 0 launched on executor multi with executor id 4999 parsl.dataflow.dflow INFO: Standard output for task 4998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f8d2771a-13dd-4b24-918d-655c115124ad_isrForCrosstalkSources_996987_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f8d2771a-13dd-4b24-918d-655c115124ad_isrForCrosstalkSources_996987_15.stderr parsl.dataflow.dflow DEBUG: Task 4999 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 4999 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 4999 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 4999 has memoization hash f463e182c526cbd056c2bf6b03f2b5e9 parsl.dataflow.memoization INFO: Task 4999 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04400> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:12 proc_lsst.multi:146 [INFO] found job 17074843 in provider astro proc_lsst.multi INFO: found job 17074843 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Parsl task 4999 try 0 launched on executor multi with executor id 5000 parsl.dataflow.dflow INFO: Standard output for task 4999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0568dc0b-ad31-4d4c-a3a5-8412c2ce13b9_isrForCrosstalkSources_996975_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0568dc0b-ad31-4d4c-a3a5-8412c2ce13b9_isrForCrosstalkSources_996975_4.stderr parsl.dataflow.dflow DEBUG: Task 5000 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5000 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5000 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5000 has memoization hash 72dba624ab99d23e000c815fc16adc4d parsl.dataflow.memoization INFO: Task 5000 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c48f4c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5000 try 0 launched on executor multi with executor id 5001 parsl.dataflow.dflow INFO: Standard output for task 5000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/91a77a0c-2bf7-4f25-b1fb-a3d3afe484d9_isrForCrosstalkSources_996992_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/91a77a0c-2bf7-4f25-b1fb-a3d3afe484d9_isrForCrosstalkSources_996992_8.stderr parsl.dataflow.dflow DEBUG: Task 5001 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5001 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5001 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5001 has memoization hash 39e22972809d3851690a19f61520f930 parsl.dataflow.memoization INFO: Task 5001 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6d9e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5001 try 0 launched on executor multi with executor id 5002 parsl.dataflow.dflow INFO: Standard output for task 5001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/efcbf166-1103-41ad-9f4a-e229e73c708a_isrForCrosstalkSources_996983_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/efcbf166-1103-41ad-9f4a-e229e73c708a_isrForCrosstalkSources_996983_23.stderr parsl.dataflow.dflow DEBUG: Task 5002 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5002 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5002 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5002 has memoization hash aae1db6277d8297ea6f1de5bc6533a47 parsl.dataflow.memoization INFO: Task 5002 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04540> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5002 try 0 launched on executor multi with executor id 5003 parsl.dataflow.dflow INFO: Standard output for task 5002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/8818c06d-75ad-4cb6-a3fc-d87b1c011862_isrForCrosstalkSources_996990_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/8818c06d-75ad-4cb6-a3fc-d87b1c011862_isrForCrosstalkSources_996990_23.stderr parsl.dataflow.dflow DEBUG: Task 5003 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5003 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5003 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5003 has memoization hash 5fed8b3c74bbe9c5e612f2ff3cc4cdff parsl.dataflow.memoization INFO: Task 5003 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07880> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5003 try 0 launched on executor multi with executor id 5004 parsl.dataflow.dflow INFO: Standard output for task 5003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7d834461-981c-4490-801e-9613a8b6d997_isrForCrosstalkSources_997026_36.stdout parsl.dataflow.dflow INFO: Standard error for task 5003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7d834461-981c-4490-801e-9613a8b6d997_isrForCrosstalkSources_997026_36.stderr parsl.dataflow.dflow DEBUG: Task 5004 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5004 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5004 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5004 has memoization hash 2a3978cb720a287c7da032b6175398e1 parsl.dataflow.memoization INFO: Task 5004 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b045e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5004 try 0 launched on executor multi with executor id 5005 parsl.dataflow.dflow INFO: Standard output for task 5004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/255f521b-3f95-459b-8219-5b7f4137d154_isrForCrosstalkSources_997016_49.stdout parsl.dataflow.dflow INFO: Standard error for task 5004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/255f521b-3f95-459b-8219-5b7f4137d154_isrForCrosstalkSources_997016_49.stderr parsl.dataflow.dflow DEBUG: Task 5005 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5005 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5005 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5005 has memoization hash ef7d68d891e8589d49b197d79fdd6e98 parsl.dataflow.memoization INFO: Task 5005 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07920> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5005 try 0 launched on executor multi with executor id 5006 parsl.dataflow.dflow INFO: Standard output for task 5005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/863ecf3e-6722-4679-b4fb-bbf338c2ca53_isrForCrosstalkSources_996988_58.stdout parsl.dataflow.dflow INFO: Standard error for task 5005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/863ecf3e-6722-4679-b4fb-bbf338c2ca53_isrForCrosstalkSources_996988_58.stderr parsl.dataflow.dflow DEBUG: Task 5006 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5006 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5006 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5006 has memoization hash 71f4a2bf4b1876dcf65e389ab007dd49 parsl.dataflow.memoization INFO: Task 5006 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b047c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5006 try 0 launched on executor multi with executor id 5007 parsl.dataflow.dflow INFO: Standard output for task 5006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/a84513e9-1517-49a8-9b24-77ef03f3b0d5_isrForCrosstalkSources_996988_27.stdout parsl.dataflow.dflow INFO: Standard error for task 5006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/a84513e9-1517-49a8-9b24-77ef03f3b0d5_isrForCrosstalkSources_996988_27.stderr parsl.dataflow.dflow DEBUG: Task 5007 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5007 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5007 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5007 has memoization hash 0c3750d52c911f5bcddee50eb69d3197 parsl.dataflow.memoization INFO: Task 5007 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04860> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5007 try 0 launched on executor multi with executor id 5008 parsl.dataflow.dflow INFO: Standard output for task 5007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/10dcbfc3-28aa-4a9d-81eb-6ee05d27295d_isrForCrosstalkSources_997007_29.stdout parsl.dataflow.dflow INFO: Standard error for task 5007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/10dcbfc3-28aa-4a9d-81eb-6ee05d27295d_isrForCrosstalkSources_997007_29.stderr parsl.dataflow.dflow DEBUG: Task 5008 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5008 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5008 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5008 has memoization hash b09393502490784bc584729693002113 parsl.dataflow.memoization INFO: Task 5008 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5008 try 0 launched on executor multi with executor id 5009 parsl.dataflow.dflow INFO: Standard output for task 5008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/da48764b-584e-42e3-b15b-78309b37e700_isrForCrosstalkSources_996986_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/da48764b-584e-42e3-b15b-78309b37e700_isrForCrosstalkSources_996986_8.stderr parsl.dataflow.dflow DEBUG: Task 5009 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5009 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5009 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5009 has memoization hash b9aa8eaaf8714eef9f48ea992876a946 parsl.dataflow.memoization INFO: Task 5009 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b049a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5009 try 0 launched on executor multi with executor id 5010 parsl.dataflow.dflow INFO: Standard output for task 5009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b4996b9f-cc50-45aa-92ef-174ad92fd0d4_isrForCrosstalkSources_996983_16.stdout parsl.dataflow.dflow INFO: Standard error for task 5009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b4996b9f-cc50-45aa-92ef-174ad92fd0d4_isrForCrosstalkSources_996983_16.stderr parsl.dataflow.dflow DEBUG: Task 5010 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5010 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5010 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5010 has memoization hash 58377774491fe98dbcd0cb5c1d1b73cf parsl.dataflow.memoization INFO: Task 5010 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04720> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5010 try 0 launched on executor multi with executor id 5011 parsl.dataflow.dflow INFO: Standard output for task 5010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d21474e5-81e0-4d08-9de3-d615ed5c15df_isrForCrosstalkSources_996992_57.stdout parsl.dataflow.dflow INFO: Standard error for task 5010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d21474e5-81e0-4d08-9de3-d615ed5c15df_isrForCrosstalkSources_996992_57.stderr parsl.dataflow.dflow DEBUG: Task 5011 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5011 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5011 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5011 has memoization hash 3ede90da9f680b7eab56ae6d9476e391 parsl.dataflow.memoization INFO: Task 5011 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5011 try 0 launched on executor multi with executor id 5012 parsl.dataflow.dflow INFO: Standard output for task 5011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/fb01ce7e-da52-4db4-8076-5644df6b9079_isrForCrosstalkSources_996974_3.stdout parsl.dataflow.dflow INFO: Standard error for task 5011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/fb01ce7e-da52-4db4-8076-5644df6b9079_isrForCrosstalkSources_996974_3.stderr parsl.dataflow.dflow DEBUG: Task 5012 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5012 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5012 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5012 has memoization hash 1304159ee2b17bba34195f153c94f092 parsl.dataflow.memoization INFO: Task 5012 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5012 try 0 launched on executor multi with executor id 5013 parsl.dataflow.dflow INFO: Standard output for task 5012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5c147f8d-b89d-49c2-b030-8c8cce0ab03b_isrForCrosstalkSources_996978_18.stdout parsl.dataflow.dflow INFO: Standard error for task 5012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5c147f8d-b89d-49c2-b030-8c8cce0ab03b_isrForCrosstalkSources_996978_18.stderr parsl.dataflow.dflow DEBUG: Task 5013 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5013 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5013 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5013 has memoization hash f070520d4a7ff24686f43c965c0a8a01 parsl.dataflow.memoization INFO: Task 5013 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04c20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5013 try 0 launched on executor multi with executor id 5014 parsl.dataflow.dflow INFO: Standard output for task 5013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/862cb02c-212a-451f-bfc2-d927a435554a_isrForCrosstalkSources_996954_15.stdout parsl.dataflow.dflow INFO: Standard error for task 5013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/862cb02c-212a-451f-bfc2-d927a435554a_isrForCrosstalkSources_996954_15.stderr parsl.dataflow.dflow DEBUG: Task 5014 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5014 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5014 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5014 has memoization hash c27f522229672361c6551b89df175bd6 parsl.dataflow.memoization INFO: Task 5014 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04b80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5014 try 0 launched on executor multi with executor id 5015 parsl.dataflow.dflow INFO: Standard output for task 5014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/901d8d09-51f2-49e3-a277-b3263506af2d_isrForCrosstalkSources_996971_45.stdout parsl.dataflow.dflow INFO: Standard error for task 5014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/901d8d09-51f2-49e3-a277-b3263506af2d_isrForCrosstalkSources_996971_45.stderr parsl.dataflow.dflow DEBUG: Task 5015 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5015 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5015 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5015 has memoization hash 4c4217c089d6ac3a77cf0ec6b33d71fa parsl.dataflow.memoization INFO: Task 5015 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04cc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5015 try 0 launched on executor multi with executor id 5016 parsl.dataflow.dflow INFO: Standard output for task 5015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/35b16825-8e1c-4a8b-a0e7-ed59e1bbaa07_isrForCrosstalkSources_996963_12.stdout parsl.dataflow.dflow INFO: Standard error for task 5015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/35b16825-8e1c-4a8b-a0e7-ed59e1bbaa07_isrForCrosstalkSources_996963_12.stderr parsl.dataflow.dflow DEBUG: Task 5016 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5016 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5016 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5016 has memoization hash 4616a5ada10f9b73f18733e2a69e8217 parsl.dataflow.memoization INFO: Task 5016 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04d60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5016 try 0 launched on executor multi with executor id 5017 parsl.dataflow.dflow INFO: Standard output for task 5016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/78fd9e6a-c649-44d4-a8ae-4a917d33d7b1_isrForCrosstalkSources_997029_42.stdout parsl.dataflow.dflow INFO: Standard error for task 5016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/78fd9e6a-c649-44d4-a8ae-4a917d33d7b1_isrForCrosstalkSources_997029_42.stderr parsl.dataflow.dflow DEBUG: Task 5017 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5017 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5017 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5017 has memoization hash 636b39dfa1929c5c6eac61ff582dc4fc parsl.dataflow.memoization INFO: Task 5017 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04e00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5017 try 0 launched on executor multi with executor id 5018 parsl.dataflow.dflow INFO: Standard output for task 5017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/877330a6-be6e-4164-be95-19aa343bf32e_isrForCrosstalkSources_997022_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/877330a6-be6e-4164-be95-19aa343bf32e_isrForCrosstalkSources_997022_23.stderr parsl.dataflow.dflow DEBUG: Task 5018 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5018 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5018 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5018 has memoization hash a00a5c713bb5f8dae2b67663a36345a2 parsl.dataflow.memoization INFO: Task 5018 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04ea0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5018 try 0 launched on executor multi with executor id 5019 parsl.dataflow.dflow INFO: Standard output for task 5018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e45fd657-67b6-4f75-8c96-a0b306278297_isrForCrosstalkSources_996989_11.stdout parsl.dataflow.dflow INFO: Standard error for task 5018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e45fd657-67b6-4f75-8c96-a0b306278297_isrForCrosstalkSources_996989_11.stderr parsl.dataflow.dflow DEBUG: Task 5019 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5019 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5019 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5019 has memoization hash 6694b22e02b089972cd715a6b381e235 parsl.dataflow.memoization INFO: Task 5019 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05080> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5019 try 0 launched on executor multi with executor id 5020 parsl.dataflow.dflow INFO: Standard output for task 5019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/857fb17c-f8ef-48e8-8c64-fc382d28b71c_isrForCrosstalkSources_996977_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/857fb17c-f8ef-48e8-8c64-fc382d28b71c_isrForCrosstalkSources_996977_23.stderr parsl.dataflow.dflow DEBUG: Task 5020 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5020 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5020 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5020 has memoization hash 23ae9e43add366a2cfe3c06ff54345d6 parsl.dataflow.memoization INFO: Task 5020 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04fe0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5020 try 0 launched on executor multi with executor id 5021 parsl.dataflow.dflow INFO: Standard output for task 5020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/4edc9036-a140-49a4-9aae-c7c0a7bdb14b_isrForCrosstalkSources_996993_24.stdout parsl.dataflow.dflow INFO: Standard error for task 5020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/4edc9036-a140-49a4-9aae-c7c0a7bdb14b_isrForCrosstalkSources_996993_24.stderr parsl.dataflow.dflow DEBUG: Task 5021 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5021 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5021 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5021 has memoization hash 9464f7965702cc65dab104f58cebbd08 parsl.dataflow.memoization INFO: Task 5021 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05120> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5021 try 0 launched on executor multi with executor id 5022 parsl.dataflow.dflow INFO: Standard output for task 5021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bf4c7109-62ca-4578-8372-d0bb2c715291_isrForCrosstalkSources_997014_51.stdout parsl.dataflow.dflow INFO: Standard error for task 5021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bf4c7109-62ca-4578-8372-d0bb2c715291_isrForCrosstalkSources_997014_51.stderr parsl.dataflow.dflow DEBUG: Task 5022 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5022 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5022 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5022 has memoization hash 48b31a8771ea57fc25a62b0d97594a26 parsl.dataflow.memoization INFO: Task 5022 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04f40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5022 try 0 launched on executor multi with executor id 5023 parsl.dataflow.dflow INFO: Standard output for task 5022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f2e2bd15-3ed6-46ca-b96c-4c6365c247f1_isrForCrosstalkSources_996960_62.stdout parsl.dataflow.dflow INFO: Standard error for task 5022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f2e2bd15-3ed6-46ca-b96c-4c6365c247f1_isrForCrosstalkSources_996960_62.stderr parsl.dataflow.dflow DEBUG: Task 5023 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5023 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5023 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5023 has memoization hash 42118a2180d4fd52ecc68d3d86467fee parsl.dataflow.memoization INFO: Task 5023 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b051c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5023 try 0 launched on executor multi with executor id 5024 parsl.dataflow.dflow INFO: Standard output for task 5023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/496a8a03-dc7e-47fe-841b-98f4dad63e8b_isrForCrosstalkSources_996988_56.stdout parsl.dataflow.dflow INFO: Standard error for task 5023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/496a8a03-dc7e-47fe-841b-98f4dad63e8b_isrForCrosstalkSources_996988_56.stderr parsl.dataflow.dflow DEBUG: Task 5024 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5024 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5024 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5024 has memoization hash 927661e3a47757a28bc18a88e693a06a parsl.dataflow.memoization INFO: Task 5024 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b053a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5024 try 0 launched on executor multi with executor id 5025 parsl.dataflow.dflow INFO: Standard output for task 5024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/674cbf07-0ce2-4fe0-a07c-1ee07661ff78_isrForCrosstalkSources_996989_57.stdout parsl.dataflow.dflow INFO: Standard error for task 5024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/674cbf07-0ce2-4fe0-a07c-1ee07661ff78_isrForCrosstalkSources_996989_57.stderr parsl.dataflow.dflow DEBUG: Task 5025 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5025 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5025 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5025 has memoization hash 3838736245e04cc48a431314a65102b9 parsl.dataflow.memoization INFO: Task 5025 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05300> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5025 try 0 launched on executor multi with executor id 5026 parsl.dataflow.dflow INFO: Standard output for task 5025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/4d9d889c-c7e2-4f0b-9bc8-c62fb58f062d_isrForCrosstalkSources_997022_43.stdout parsl.dataflow.dflow INFO: Standard error for task 5025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/4d9d889c-c7e2-4f0b-9bc8-c62fb58f062d_isrForCrosstalkSources_997022_43.stderr parsl.dataflow.dflow DEBUG: Task 5026 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5026 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5026 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5026 has memoization hash 7b46f073457a6439b98895e92d950ce5 parsl.dataflow.memoization INFO: Task 5026 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05440> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5026 try 0 launched on executor multi with executor id 5027 parsl.dataflow.dflow INFO: Standard output for task 5026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/33a82eb2-5f15-4f40-8e4f-1010e754848f_isrForCrosstalkSources_996987_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/33a82eb2-5f15-4f40-8e4f-1010e754848f_isrForCrosstalkSources_996987_8.stderr parsl.dataflow.dflow DEBUG: Task 5027 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5027 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5027 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5027 has memoization hash a6b36995bf647bdf9fe188df44088185 parsl.dataflow.memoization INFO: Task 5027 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05260> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5027 try 0 launched on executor multi with executor id 5028 parsl.dataflow.dflow INFO: Standard output for task 5027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/00f587f6-89f8-44cc-8a1f-dc9e639cd83d_isrForCrosstalkSources_996974_22.stdout parsl.dataflow.dflow INFO: Standard error for task 5027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/00f587f6-89f8-44cc-8a1f-dc9e639cd83d_isrForCrosstalkSources_996974_22.stderr parsl.dataflow.dflow DEBUG: Task 5028 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5028 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5028 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5028 has memoization hash 7fb38ff3c49012f3aa183022c2cb69ee parsl.dataflow.memoization INFO: Task 5028 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5028 try 0 launched on executor multi with executor id 5029 parsl.dataflow.dflow INFO: Standard output for task 5028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/165a8d3e-cd83-45c3-9194-fc1235c3d084_isrForCrosstalkSources_996960_15.stdout parsl.dataflow.dflow INFO: Standard error for task 5028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/165a8d3e-cd83-45c3-9194-fc1235c3d084_isrForCrosstalkSources_996960_15.stderr parsl.dataflow.dflow DEBUG: Task 5029 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5029 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5029 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5029 has memoization hash 59f7f481c2152b33a2c89067ff432f63 parsl.dataflow.memoization INFO: Task 5029 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05620> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5029 try 0 launched on executor multi with executor id 5030 parsl.dataflow.dflow INFO: Standard output for task 5029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/006575f2-d065-4be9-9eff-7f320cf39f5b_isrForCrosstalkSources_996965_34.stdout parsl.dataflow.dflow INFO: Standard error for task 5029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/006575f2-d065-4be9-9eff-7f320cf39f5b_isrForCrosstalkSources_996965_34.stderr parsl.dataflow.dflow DEBUG: Task 5030 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5030 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5030 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5030 has memoization hash 9424f05c8a5c8ab360c9dd71eb681d17 parsl.dataflow.memoization INFO: Task 5030 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b054e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5030 try 0 launched on executor multi with executor id 5031 parsl.dataflow.dflow INFO: Standard output for task 5030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/b060026e-266e-4105-8db9-f0c509347655_isrForCrosstalkSources_997028_17.stdout parsl.dataflow.dflow INFO: Standard error for task 5030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/b060026e-266e-4105-8db9-f0c509347655_isrForCrosstalkSources_997028_17.stderr parsl.dataflow.dflow DEBUG: Task 5031 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5031 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5031 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5031 has memoization hash c72d7a3fa7eaee7d5371837d51b16a71 parsl.dataflow.memoization INFO: Task 5031 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05760> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5031 try 0 launched on executor multi with executor id 5032 parsl.dataflow.dflow INFO: Standard output for task 5031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f38f50b5-1f72-4739-95a5-4a3dd7aec4aa_isrForCrosstalkSources_996971_39.stdout parsl.dataflow.dflow INFO: Standard error for task 5031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f38f50b5-1f72-4739-95a5-4a3dd7aec4aa_isrForCrosstalkSources_996971_39.stderr parsl.dataflow.dflow DEBUG: Task 5032 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5032 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5032 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5032 has memoization hash bff525030e67215a0a13b07ce37993cd parsl.dataflow.memoization INFO: Task 5032 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b056c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5032 try 0 launched on executor multi with executor id 5033 parsl.dataflow.dflow INFO: Standard output for task 5032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/eef1d50d-edc3-4381-b352-2d63421ae26d_isrForCrosstalkSources_996982_34.stdout parsl.dataflow.dflow INFO: Standard error for task 5032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/eef1d50d-edc3-4381-b352-2d63421ae26d_isrForCrosstalkSources_996982_34.stderr parsl.dataflow.dflow DEBUG: Task 5033 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5033 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5033 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5033 has memoization hash 81b4590a204a6e073e69668f57bab6c4 parsl.dataflow.memoization INFO: Task 5033 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b058a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5033 try 0 launched on executor multi with executor id 5034 parsl.dataflow.dflow INFO: Standard output for task 5033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/f5f5f7d8-e523-4883-8ba4-f9f73172ba60_isrForCrosstalkSources_996964_28.stdout parsl.dataflow.dflow INFO: Standard error for task 5033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/f5f5f7d8-e523-4883-8ba4-f9f73172ba60_isrForCrosstalkSources_996964_28.stderr parsl.dataflow.dflow DEBUG: Task 5034 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5034 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5034 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5034 has memoization hash fd1e4f2115265bb6f386f0cc62493ad4 parsl.dataflow.memoization INFO: Task 5034 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05940> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5034 try 0 launched on executor multi with executor id 5035 parsl.dataflow.dflow INFO: Standard output for task 5034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/20f7522c-57cd-455a-ae92-aeef462cd3e6_isrForCrosstalkSources_997004_52.stdout parsl.dataflow.dflow INFO: Standard error for task 5034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/20f7522c-57cd-455a-ae92-aeef462cd3e6_isrForCrosstalkSources_997004_52.stderr parsl.dataflow.dflow DEBUG: Task 5035 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5035 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5035 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5035 has memoization hash 36cb8e1ed7cf9aa1e2c689bfcf53586a parsl.dataflow.memoization INFO: Task 5035 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05800> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5035 try 0 launched on executor multi with executor id 5036 parsl.dataflow.dflow INFO: Standard output for task 5035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/b93be41d-406c-4dc1-b1a1-7739c325908b_isrForCrosstalkSources_996960_51.stdout parsl.dataflow.dflow INFO: Standard error for task 5035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/b93be41d-406c-4dc1-b1a1-7739c325908b_isrForCrosstalkSources_996960_51.stderr parsl.dataflow.dflow DEBUG: Task 5036 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5036 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5036 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5036 has memoization hash a9e66dfb68055fb9cccd8a1db82ca6aa parsl.dataflow.memoization INFO: Task 5036 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05a80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5036 try 0 launched on executor multi with executor id 5037 parsl.dataflow.dflow INFO: Standard output for task 5036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/189c2320-d604-459a-880e-b830a9b3d1ef_isrForCrosstalkSources_997017_19.stdout parsl.dataflow.dflow INFO: Standard error for task 5036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/189c2320-d604-459a-880e-b830a9b3d1ef_isrForCrosstalkSources_997017_19.stderr parsl.dataflow.dflow DEBUG: Task 5037 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5037 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5037 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5037 has memoization hash 13325ca46048833baf81c6b6cc579028 parsl.dataflow.memoization INFO: Task 5037 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b059e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5037 try 0 launched on executor multi with executor id 5038 parsl.dataflow.dflow INFO: Standard output for task 5037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8d67e711-1a22-4f91-a2bc-1e376b11021c_isrForCrosstalkSources_997032_9.stdout parsl.dataflow.dflow INFO: Standard error for task 5037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8d67e711-1a22-4f91-a2bc-1e376b11021c_isrForCrosstalkSources_997032_9.stderr parsl.dataflow.dflow DEBUG: Task 5038 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5038 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5038 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5038 has memoization hash d7aaa3bde7439add7835cf86e777a143 parsl.dataflow.memoization INFO: Task 5038 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05bc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5038 try 0 launched on executor multi with executor id 5039 parsl.dataflow.dflow INFO: Standard output for task 5038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2c70e87d-7c03-4baa-9788-cfa6551ea170_isrForCrosstalkSources_996974_12.stdout parsl.dataflow.dflow INFO: Standard error for task 5038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2c70e87d-7c03-4baa-9788-cfa6551ea170_isrForCrosstalkSources_996974_12.stderr parsl.dataflow.dflow DEBUG: Task 5039 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5039 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5039 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5039 has memoization hash 4039834e930b77a951e672383530dabb parsl.dataflow.memoization INFO: Task 5039 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5039 try 0 launched on executor multi with executor id 5040 parsl.dataflow.dflow INFO: Standard output for task 5039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/06ea4c39-60af-4d77-8a78-a1cdd48e9319_isrForCrosstalkSources_996996_9.stdout parsl.dataflow.dflow INFO: Standard error for task 5039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/06ea4c39-60af-4d77-8a78-a1cdd48e9319_isrForCrosstalkSources_996996_9.stderr parsl.dataflow.dflow DEBUG: Task 5040 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5040 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5040 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5040 has memoization hash 459b5c16d3af49cb2247e4fef93d56e8 parsl.dataflow.memoization INFO: Task 5040 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05b20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5040 try 0 launched on executor multi with executor id 5041 parsl.dataflow.dflow INFO: Standard output for task 5040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/9ac9997d-80f1-4b50-b336-7f711dab242e_isrForCrosstalkSources_997011_57.stdout parsl.dataflow.dflow INFO: Standard error for task 5040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/9ac9997d-80f1-4b50-b336-7f711dab242e_isrForCrosstalkSources_997011_57.stderr parsl.dataflow.dflow DEBUG: Task 5041 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5041 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5041 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5041 has memoization hash 6b16f24a766bd4ae6affa454efe31de8 parsl.dataflow.memoization INFO: Task 5041 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05da0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5041 try 0 launched on executor multi with executor id 5042 parsl.dataflow.dflow INFO: Standard output for task 5041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/302fd97d-869f-4ffa-9a39-b2c43a917efe_isrForCrosstalkSources_997029_48.stdout parsl.dataflow.dflow INFO: Standard error for task 5041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/302fd97d-869f-4ffa-9a39-b2c43a917efe_isrForCrosstalkSources_997029_48.stderr parsl.dataflow.dflow DEBUG: Task 5042 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5042 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5042 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5042 has memoization hash a74b2d7b822704b427c59fbd9e20911b parsl.dataflow.memoization INFO: Task 5042 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05c60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5042 try 0 launched on executor multi with executor id 5043 parsl.dataflow.dflow INFO: Standard output for task 5042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/91e5d330-8f8d-4328-915e-794a9427e577_isrForCrosstalkSources_996952_34.stdout parsl.dataflow.dflow INFO: Standard error for task 5042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/91e5d330-8f8d-4328-915e-794a9427e577_isrForCrosstalkSources_996952_34.stderr parsl.dataflow.dflow DEBUG: Task 5043 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5043 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5043 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5043 has memoization hash 6ee8105319f2145cd41651e0f33ebca3 parsl.dataflow.memoization INFO: Task 5043 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5043 try 0 launched on executor multi with executor id 5044 parsl.dataflow.dflow INFO: Standard output for task 5043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/86215e29-39c7-40d2-8dca-f24c3786f835_isrForCrosstalkSources_997008_58.stdout parsl.dataflow.dflow INFO: Standard error for task 5043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/86215e29-39c7-40d2-8dca-f24c3786f835_isrForCrosstalkSources_997008_58.stderr parsl.dataflow.dflow DEBUG: Task 5044 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5044 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5044 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5044 has memoization hash afa25d607f3d564c643f2eaf496bce65 parsl.dataflow.memoization INFO: Task 5044 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06020> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5044 try 0 launched on executor multi with executor id 5045 parsl.dataflow.dflow INFO: Standard output for task 5044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/154377d1-4122-4b06-bb7d-5f58113ea6b2_isrForCrosstalkSources_997012_42.stdout parsl.dataflow.dflow INFO: Standard error for task 5044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/154377d1-4122-4b06-bb7d-5f58113ea6b2_isrForCrosstalkSources_997012_42.stderr parsl.dataflow.dflow DEBUG: Task 5045 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5045 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5045 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5045 has memoization hash b8c247ae007478e8485fb33da9df5524 parsl.dataflow.memoization INFO: Task 5045 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05f80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5045 try 0 launched on executor multi with executor id 5046 parsl.dataflow.dflow INFO: Standard output for task 5045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/8fa379e6-2175-4c78-bcc3-e197b2069b0c_isrForCrosstalkSources_996982_36.stdout parsl.dataflow.dflow INFO: Standard error for task 5045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/8fa379e6-2175-4c78-bcc3-e197b2069b0c_isrForCrosstalkSources_996982_36.stderr parsl.dataflow.dflow DEBUG: Task 5046 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5046 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5046 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5046 has memoization hash a172f6c8007ad7604cafd27d19cd7fcd parsl.dataflow.memoization INFO: Task 5046 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b05e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5046 try 0 launched on executor multi with executor id 5047 parsl.dataflow.dflow INFO: Standard output for task 5046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e44cfccb-b2f5-4e05-97c5-9f6229740e93_isrForCrosstalkSources_996970_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e44cfccb-b2f5-4e05-97c5-9f6229740e93_isrForCrosstalkSources_996970_8.stderr parsl.dataflow.dflow DEBUG: Task 5047 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5047 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5047 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5047 has memoization hash cc9d4e069c74b3e1a163a22aa88dc766 parsl.dataflow.memoization INFO: Task 5047 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06160> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5047 try 0 launched on executor multi with executor id 5048 parsl.dataflow.dflow INFO: Standard output for task 5047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e0a51ac5-66c9-4740-a704-b7ce8560c4bd_isrForCrosstalkSources_996956_14.stdout parsl.dataflow.dflow INFO: Standard error for task 5047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e0a51ac5-66c9-4740-a704-b7ce8560c4bd_isrForCrosstalkSources_996956_14.stderr parsl.dataflow.dflow DEBUG: Task 5048 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5048 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5048 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5048 has memoization hash fac223ed75dec3189464b2bdae754244 parsl.dataflow.memoization INFO: Task 5048 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06200> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5048 try 0 launched on executor multi with executor id 5049 parsl.dataflow.dflow INFO: Standard output for task 5048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d84e23bb-2686-411d-8007-92012cf1f5df_isrForCrosstalkSources_997027_33.stdout parsl.dataflow.dflow INFO: Standard error for task 5048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d84e23bb-2686-411d-8007-92012cf1f5df_isrForCrosstalkSources_997027_33.stderr parsl.dataflow.dflow DEBUG: Task 5049 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5049 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5049 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5049 has memoization hash 39c381c67eae3ad8b459253e514083fa parsl.dataflow.memoization INFO: Task 5049 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b060c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5049 try 0 launched on executor multi with executor id 5050 parsl.dataflow.dflow INFO: Standard output for task 5049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/38b545a0-1918-44ef-bd54-f9d38761d778_isrForCrosstalkSources_996957_6.stdout parsl.dataflow.dflow INFO: Standard error for task 5049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/38b545a0-1918-44ef-bd54-f9d38761d778_isrForCrosstalkSources_996957_6.stderr parsl.dataflow.dflow DEBUG: Task 5050 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5050 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5050 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5050 has memoization hash e88010e841536c531ba70631bc0794f8 parsl.dataflow.memoization INFO: Task 5050 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b062a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5050 try 0 launched on executor multi with executor id 5051 parsl.dataflow.dflow INFO: Standard output for task 5050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/086bf235-659c-4413-b2e9-41e3c31e5e67_isrForCrosstalkSources_997024_51.stdout parsl.dataflow.dflow INFO: Standard error for task 5050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/086bf235-659c-4413-b2e9-41e3c31e5e67_isrForCrosstalkSources_997024_51.stderr parsl.dataflow.dflow DEBUG: Task 5051 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5051 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5051 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5051 has memoization hash 91876d8b78691a9a92b96bfc897a3ee8 parsl.dataflow.memoization INFO: Task 5051 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06340> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5051 try 0 launched on executor multi with executor id 5052 parsl.dataflow.dflow INFO: Standard output for task 5051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/ebf3a113-bd8d-4e18-ba87-92c96d1b0e0b_isrForCrosstalkSources_996983_50.stdout parsl.dataflow.dflow INFO: Standard error for task 5051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/ebf3a113-bd8d-4e18-ba87-92c96d1b0e0b_isrForCrosstalkSources_996983_50.stderr parsl.dataflow.dflow DEBUG: Task 5052 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5052 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5052 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5052 has memoization hash b809dbe883607b7c154a969f148df529 parsl.dataflow.memoization INFO: Task 5052 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b063e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5052 try 0 launched on executor multi with executor id 5053 parsl.dataflow.dflow INFO: Standard output for task 5052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e61f810c-5c05-491c-8717-31053cb035f3_isrForCrosstalkSources_996955_31.stdout parsl.dataflow.dflow INFO: Standard error for task 5052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e61f810c-5c05-491c-8717-31053cb035f3_isrForCrosstalkSources_996955_31.stderr parsl.dataflow.dflow DEBUG: Task 5053 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5053 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5053 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5053 has memoization hash 1118ed7dd8c21ce3c7417bb1914183a6 parsl.dataflow.memoization INFO: Task 5053 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06520> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5053 try 0 launched on executor multi with executor id 5054 parsl.dataflow.dflow INFO: Standard output for task 5053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/38f5956a-ac16-4274-a134-8d77ef6b6b61_isrForCrosstalkSources_996995_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/38f5956a-ac16-4274-a134-8d77ef6b6b61_isrForCrosstalkSources_996995_8.stderr parsl.dataflow.dflow DEBUG: Task 5054 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5054 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5054 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5054 has memoization hash c07e86cf1da485d03293d5d65378a606 parsl.dataflow.memoization INFO: Task 5054 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b077e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5054 try 0 launched on executor multi with executor id 5055 parsl.dataflow.dflow INFO: Standard output for task 5054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/89dc1ece-171b-4410-bc47-de10adae5e6d_isrForCrosstalkSources_997009_29.stdout parsl.dataflow.dflow INFO: Standard error for task 5054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/89dc1ece-171b-4410-bc47-de10adae5e6d_isrForCrosstalkSources_997009_29.stderr parsl.dataflow.dflow DEBUG: Task 5055 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5055 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5055 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5055 has memoization hash 1fa2bdc36d3e15ab114e38a67e01014e parsl.dataflow.memoization INFO: Task 5055 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b04360> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5055 try 0 launched on executor multi with executor id 5056 parsl.dataflow.dflow INFO: Standard output for task 5055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/bd1a4a4f-7865-42f3-9e56-4c0c2566e752_isrForCrosstalkSources_996978_33.stdout parsl.dataflow.dflow INFO: Standard error for task 5055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/bd1a4a4f-7865-42f3-9e56-4c0c2566e752_isrForCrosstalkSources_996978_33.stderr parsl.dataflow.dflow DEBUG: Task 5056 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5056 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5056 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5056 has memoization hash a081b3898dc5b3c132b0f095720a82f9 parsl.dataflow.memoization INFO: Task 5056 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07740> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5056 try 0 launched on executor multi with executor id 5057 parsl.dataflow.dflow INFO: Standard output for task 5056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/1e3bc188-4a07-4527-a07c-082edbf855fe_isrForCrosstalkSources_997013_20.stdout parsl.dataflow.dflow INFO: Standard error for task 5056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/1e3bc188-4a07-4527-a07c-082edbf855fe_isrForCrosstalkSources_997013_20.stderr parsl.dataflow.dflow DEBUG: Task 5057 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5057 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5057 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5057 has memoization hash 10697f915532af533be23f5f2f91f3ff parsl.dataflow.memoization INFO: Task 5057 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b076a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5057 try 0 launched on executor multi with executor id 5058 parsl.dataflow.dflow INFO: Standard output for task 5057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cc9deb70-8f6c-4faf-a6c1-f5bc8c738192_isrForCrosstalkSources_996978_60.stdout parsl.dataflow.dflow INFO: Standard error for task 5057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cc9deb70-8f6c-4faf-a6c1-f5bc8c738192_isrForCrosstalkSources_996978_60.stderr parsl.dataflow.dflow DEBUG: Task 5058 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5058 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5058 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5058 has memoization hash 837944f024b6bad9fd2a01553a77664f parsl.dataflow.memoization INFO: Task 5058 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07600> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5058 try 0 launched on executor multi with executor id 5059 parsl.dataflow.dflow INFO: Standard output for task 5058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/4c0d7fbb-1324-4e3a-bb01-b2d5375a8d65_isrForCrosstalkSources_997001_4.stdout parsl.dataflow.dflow INFO: Standard error for task 5058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/4c0d7fbb-1324-4e3a-bb01-b2d5375a8d65_isrForCrosstalkSources_997001_4.stderr parsl.dataflow.dflow DEBUG: Task 5059 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5059 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5059 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5059 has memoization hash 88cc7315750564cf2fdb644a2c60a4e1 parsl.dataflow.memoization INFO: Task 5059 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b074c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5059 try 0 launched on executor multi with executor id 5060 parsl.dataflow.dflow INFO: Standard output for task 5059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f01af821-e907-4bb9-829c-89918178a153_isrForCrosstalkSources_996989_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f01af821-e907-4bb9-829c-89918178a153_isrForCrosstalkSources_996989_8.stderr parsl.dataflow.dflow DEBUG: Task 5060 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5060 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5060 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5060 has memoization hash bcf8763008e01bdc916ad85bfbb2e977 parsl.dataflow.memoization INFO: Task 5060 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07560> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5060 try 0 launched on executor multi with executor id 5061 parsl.dataflow.dflow INFO: Standard output for task 5060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d27c231c-660d-4b51-acd6-8bd2b21524a1_isrForCrosstalkSources_996993_44.stdout parsl.dataflow.dflow INFO: Standard error for task 5060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d27c231c-660d-4b51-acd6-8bd2b21524a1_isrForCrosstalkSources_996993_44.stderr parsl.dataflow.dflow DEBUG: Task 5061 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5061 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5061 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5061 has memoization hash 5fb074a7687fb10e2542ea54eddaaa3a parsl.dataflow.memoization INFO: Task 5061 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07420> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Parsl task 5061 try 0 launched on executor multi with executor id 5062 parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 5061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/b31a8430-fc24-44a8-b8e2-54f6a53c59f2_isrForCrosstalkSources_996994_46.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 5061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/b31a8430-fc24-44a8-b8e2-54f6a53c59f2_isrForCrosstalkSources_996994_46.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 5062 will be sent to executor multi parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:12 proc_lsst.multi:146 [INFO] found job 17074850 in provider astro parsl.dataflow.dflow DEBUG: Adding output dependencies proc_lsst.multi INFO: found job 17074850 in provider astro parsl.dataflow.dflow INFO: Task 5062 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow DEBUG: Task 5062 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5062 has memoization hash ab19c11f5ef0a31159ac5e1eb5d55b7b parsl.dataflow.memoization INFO: Task 5062 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767c6c040> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5062 try 0 launched on executor multi with executor id 5063 parsl.dataflow.dflow INFO: Standard output for task 5062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/42ad355e-7de5-49d9-a5aa-03fc78d0b091_isrForCrosstalkSources_997004_40.stdout parsl.dataflow.dflow INFO: Standard error for task 5062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/42ad355e-7de5-49d9-a5aa-03fc78d0b091_isrForCrosstalkSources_997004_40.stderr parsl.dataflow.dflow DEBUG: Task 5063 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5063 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5063 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5063 has memoization hash cca2fed0ac8aee0ca3bcf384d28e9a9a parsl.dataflow.memoization INFO: Task 5063 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b79ee0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5063 try 0 launched on executor multi with executor id 5064 parsl.dataflow.dflow INFO: Standard output for task 5063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/009cd290-21df-44f4-a032-a7a5597bd7cc_isrForCrosstalkSources_996967_60.stdout parsl.dataflow.dflow INFO: Standard error for task 5063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/009cd290-21df-44f4-a032-a7a5597bd7cc_isrForCrosstalkSources_996967_60.stderr parsl.dataflow.dflow DEBUG: Task 5064 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5064 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5064 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5064 has memoization hash 95a6c55a07810fdd9bd4293ed52b66e3 parsl.dataflow.memoization INFO: Task 5064 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b79d00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5064 try 0 launched on executor multi with executor id 5065 parsl.dataflow.dflow INFO: Standard output for task 5064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e7b6d8fb-34c1-4836-925a-ec6c44f9bd88_isrForCrosstalkSources_996952_48.stdout parsl.dataflow.dflow INFO: Standard error for task 5064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e7b6d8fb-34c1-4836-925a-ec6c44f9bd88_isrForCrosstalkSources_996952_48.stderr parsl.dataflow.dflow DEBUG: Task 5065 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5065 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5065 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5065 has memoization hash 6be4a885ef22cc758c73bdfd015eac2f parsl.dataflow.memoization INFO: Task 5065 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b79e40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5065 try 0 launched on executor multi with executor id 5066 parsl.dataflow.dflow INFO: Standard output for task 5065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c4429549-7334-43f2-8320-b8dbaf39dfc1_isrForCrosstalkSources_997033_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c4429549-7334-43f2-8320-b8dbaf39dfc1_isrForCrosstalkSources_997033_46.stderr parsl.dataflow.dflow DEBUG: Task 5066 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5066 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5066 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5066 has memoization hash e26325852d12919f00989d67ad44a549 parsl.dataflow.memoization INFO: Task 5066 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b040e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5066 try 0 launched on executor multi with executor id 5067 parsl.dataflow.dflow INFO: Standard output for task 5066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/cee20815-c9d1-4493-96e0-763f8100daba_isrForCrosstalkSources_997003_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/cee20815-c9d1-4493-96e0-763f8100daba_isrForCrosstalkSources_997003_8.stderr parsl.dataflow.dflow DEBUG: Task 5067 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5067 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5067 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5067 has memoization hash e8682aef0136f9a351dd620e27a93671 parsl.dataflow.memoization INFO: Task 5067 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07240> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5067 try 0 launched on executor multi with executor id 5068 parsl.dataflow.dflow INFO: Standard output for task 5067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/82c25e13-7eb6-476f-a9c2-dc61e7dc43fb_isrForCrosstalkSources_997025_1.stdout parsl.dataflow.dflow INFO: Standard error for task 5067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/82c25e13-7eb6-476f-a9c2-dc61e7dc43fb_isrForCrosstalkSources_997025_1.stderr parsl.dataflow.dflow DEBUG: Task 5068 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5068 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5068 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5068 has memoization hash 3a13c82a22452448481c3cbad69219a0 parsl.dataflow.memoization INFO: Task 5068 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07100> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5068 try 0 launched on executor multi with executor id 5069 parsl.dataflow.dflow INFO: Standard output for task 5068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c953cf3b-3247-4099-8ba6-a944c9077aa0_isrForCrosstalkSources_996965_15.stdout parsl.dataflow.dflow INFO: Standard error for task 5068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c953cf3b-3247-4099-8ba6-a944c9077aa0_isrForCrosstalkSources_996965_15.stderr parsl.dataflow.dflow DEBUG: Task 5069 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5069 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5069 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5069 has memoization hash 2f1026cc7661d2e671a36be9dd20c968 parsl.dataflow.memoization INFO: Task 5069 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b044a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5069 try 0 launched on executor multi with executor id 5070 parsl.dataflow.dflow INFO: Standard output for task 5069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/cf5eb713-d074-45b7-a445-86cbd18cd1b2_isrForCrosstalkSources_996979_61.stdout parsl.dataflow.dflow INFO: Standard error for task 5069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/cf5eb713-d074-45b7-a445-86cbd18cd1b2_isrForCrosstalkSources_996979_61.stderr parsl.dataflow.dflow DEBUG: Task 5070 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5070 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5070 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5070 has memoization hash a30b8929bce66d0248ce03997eba6cc7 parsl.dataflow.memoization INFO: Task 5070 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b072e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5070 try 0 launched on executor multi with executor id 5071 parsl.dataflow.dflow INFO: Standard output for task 5070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/656bfdb3-4b62-4387-a966-aeb399efe34e_isrForCrosstalkSources_997003_4.stdout parsl.dataflow.dflow INFO: Standard error for task 5070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/656bfdb3-4b62-4387-a966-aeb399efe34e_isrForCrosstalkSources_997003_4.stderr parsl.dataflow.dflow DEBUG: Task 5071 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5071 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5071 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5071 has memoization hash 9db79731ad6df9636a15641f254ffffb parsl.dataflow.memoization INFO: Task 5071 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b07060> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5071 try 0 launched on executor multi with executor id 5072 parsl.dataflow.dflow INFO: Standard output for task 5071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/77a9606f-7d0e-4df8-aa77-0c467d150fc3_isrForCrosstalkSources_996989_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/77a9606f-7d0e-4df8-aa77-0c467d150fc3_isrForCrosstalkSources_996989_46.stderr parsl.dataflow.dflow DEBUG: Task 5072 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5072 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5072 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5072 has memoization hash 5558564b3748788d20f535d8abb5afe0 parsl.dataflow.memoization INFO: Task 5072 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06fc0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5072 try 0 launched on executor multi with executor id 5073 parsl.dataflow.dflow INFO: Standard output for task 5072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/176f5245-88be-4d27-bc84-f545bfe21666_isrForCrosstalkSources_996968_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/176f5245-88be-4d27-bc84-f545bfe21666_isrForCrosstalkSources_996968_23.stderr parsl.dataflow.dflow DEBUG: Task 5073 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5073 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5073 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5073 has memoization hash 718be5989251fcd2f59befb7fffc562f parsl.dataflow.memoization INFO: Task 5073 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06f20> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5073 try 0 launched on executor multi with executor id 5074 parsl.dataflow.dflow INFO: Standard output for task 5073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/380aaf61-dbcc-46c9-b747-6c175629c95d_isrForCrosstalkSources_996987_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/380aaf61-dbcc-46c9-b747-6c175629c95d_isrForCrosstalkSources_996987_46.stderr parsl.dataflow.dflow DEBUG: Task 5074 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5074 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5074 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5074 has memoization hash e0d31d9d13f322c131eeeeb881362b24 parsl.dataflow.memoization INFO: Task 5074 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06e80> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5074 try 0 launched on executor multi with executor id 5075 parsl.dataflow.dflow INFO: Standard output for task 5074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/6fefeacf-50f1-4653-9c3d-05506bbc4d01_isrForCrosstalkSources_997022_62.stdout parsl.dataflow.dflow INFO: Standard error for task 5074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/6fefeacf-50f1-4653-9c3d-05506bbc4d01_isrForCrosstalkSources_997022_62.stderr parsl.dataflow.dflow DEBUG: Task 5075 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5075 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5075 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5075 has memoization hash 20aeb1f0956f8f3c4cc5250daca3506a parsl.dataflow.memoization INFO: Task 5075 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06ca0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5075 try 0 launched on executor multi with executor id 5076 parsl.dataflow.dflow INFO: Standard output for task 5075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/583eac7d-13cd-4730-8b12-4bfa2eb5c620_isrForCrosstalkSources_997029_7.stdout parsl.dataflow.dflow INFO: Standard error for task 5075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/583eac7d-13cd-4730-8b12-4bfa2eb5c620_isrForCrosstalkSources_997029_7.stderr parsl.dataflow.dflow DEBUG: Task 5076 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5076 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5076 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5076 has memoization hash 5cebcfdc8e10e1806ac27213f8ad4f5c parsl.dataflow.memoization INFO: Task 5076 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06d40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5076 try 0 launched on executor multi with executor id 5077 parsl.dataflow.dflow INFO: Standard output for task 5076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f27a10a0-8d14-428d-9c62-30e2ab06eb1c_isrForCrosstalkSources_996957_11.stdout parsl.dataflow.dflow INFO: Standard error for task 5076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f27a10a0-8d14-428d-9c62-30e2ab06eb1c_isrForCrosstalkSources_996957_11.stderr parsl.dataflow.dflow DEBUG: Task 5077 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5077 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5077 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5077 has memoization hash 527bd3095b5af62af2aab0109e71e514 parsl.dataflow.memoization INFO: Task 5077 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06de0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5077 try 0 launched on executor multi with executor id 5078 parsl.dataflow.dflow INFO: Standard output for task 5077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e052d490-560c-424e-8bc7-96f98b736d1b_isrForCrosstalkSources_997020_39.stdout parsl.dataflow.dflow INFO: Standard error for task 5077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e052d490-560c-424e-8bc7-96f98b736d1b_isrForCrosstalkSources_997020_39.stderr parsl.dataflow.dflow DEBUG: Task 5078 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5078 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5078 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5078 has memoization hash 7742c0ed0de3cb5aa72cf732a37cb441 parsl.dataflow.memoization INFO: Task 5078 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06c00> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5078 try 0 launched on executor multi with executor id 5079 parsl.dataflow.dflow INFO: Standard output for task 5078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/943c7e17-48e1-49fa-ac2d-f50859822fe0_isrForCrosstalkSources_996984_52.stdout parsl.dataflow.dflow INFO: Standard error for task 5078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/943c7e17-48e1-49fa-ac2d-f50859822fe0_isrForCrosstalkSources_996984_52.stderr parsl.dataflow.dflow DEBUG: Task 5079 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5079 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5079 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5079 has memoization hash a5711657fa75eacdf9ea4f1ed4fd2b6e parsl.dataflow.memoization INFO: Task 5079 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06b60> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5079 try 0 launched on executor multi with executor id 5080 parsl.dataflow.dflow INFO: Standard output for task 5079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/9abf096f-04e3-4b0f-af70-59c95a22a73e_isrForCrosstalkSources_996961_4.stdout parsl.dataflow.dflow INFO: Standard error for task 5079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/9abf096f-04e3-4b0f-af70-59c95a22a73e_isrForCrosstalkSources_996961_4.stderr parsl.dataflow.dflow DEBUG: Task 5080 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5080 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5080 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5080 has memoization hash 7aa0d63e8599384b43f1d4ecd788e22f parsl.dataflow.memoization INFO: Task 5080 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06980> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5080 try 0 launched on executor multi with executor id 5081 parsl.dataflow.dflow INFO: Standard output for task 5080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/aa4d7fe6-badf-46d9-8bf4-a76a97c8c8f0_isrForCrosstalkSources_997030_40.stdout parsl.dataflow.dflow INFO: Standard error for task 5080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/aa4d7fe6-badf-46d9-8bf4-a76a97c8c8f0_isrForCrosstalkSources_997030_40.stderr parsl.dataflow.dflow DEBUG: Task 5081 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5081 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5081 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5081 has memoization hash 42d4ae2333690d7c242576465e46160f parsl.dataflow.memoization INFO: Task 5081 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06ac0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5081 try 0 launched on executor multi with executor id 5082 parsl.dataflow.dflow INFO: Standard output for task 5081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ef2d23cf-04f4-4e22-a2d0-c7d1b1aa8e9b_isrForCrosstalkSources_997005_4.stdout parsl.dataflow.dflow INFO: Standard error for task 5081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ef2d23cf-04f4-4e22-a2d0-c7d1b1aa8e9b_isrForCrosstalkSources_997005_4.stderr parsl.dataflow.dflow DEBUG: Task 5082 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5082 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5082 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5082 has memoization hash f495816daf3da4b7b98ddd94a860af5d parsl.dataflow.memoization INFO: Task 5082 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b068e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5082 try 0 launched on executor multi with executor id 5083 parsl.dataflow.dflow INFO: Standard output for task 5082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b1cf1aa9-0a59-4e75-82e6-375f940d0c19_isrForCrosstalkSources_996962_34.stdout parsl.dataflow.dflow INFO: Standard error for task 5082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b1cf1aa9-0a59-4e75-82e6-375f940d0c19_isrForCrosstalkSources_996962_34.stderr parsl.dataflow.dflow DEBUG: Task 5083 will be sent to executor multi parsl.dataflow.dflow DEBUG: Adding output dependencies parsl.dataflow.dflow INFO: Task 5083 submitted for App isrForCrosstalkSources, not waiting on any dependency parsl.dataflow.dflow DEBUG: Task 5083 set to pending state with AppFuture: parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 5083 has memoization hash 693b0eb2f416939fef1435464e354130 parsl.dataflow.memoization INFO: Task 5083 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a767b06840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 5083 try 0 launched on executor multi with executor id 5084 parsl.dataflow.dflow INFO: Standard output for task 5083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/90fc5198-fd88-42c1-a930-129399d7e9c7_isrForCrosstalkSources_997002_28.stdout parsl.dataflow.dflow INFO: Standard error for task 5083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/90fc5198-fd88-42c1-a930-129399d7e9c7_isrForCrosstalkSources_997002_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:13 proc_lsst.multi:146 [INFO] found job 17074851 in provider astro proc_lsst.multi INFO: found job 17074851 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:13 proc_lsst.multi:146 [INFO] found job 17074852 in provider astro proc_lsst.multi INFO: found job 17074852 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:13 proc_lsst.multi:146 [INFO] found job 17074853 in provider astro proc_lsst.multi INFO: found job 17074853 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:14 proc_lsst.multi:146 [INFO] found job 17074854 in provider astro proc_lsst.multi INFO: found job 17074854 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:14 proc_lsst.multi:146 [INFO] found job 17074855 in provider astro proc_lsst.multi INFO: found job 17074855 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:15 proc_lsst.multi:146 [INFO] found job 17074856 in provider astro proc_lsst.multi INFO: found job 17074856 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:15 proc_lsst.multi:146 [INFO] found job 17074857 in provider astro proc_lsst.multi INFO: found job 17074857 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:16 proc_lsst.multi:146 [INFO] found job 17074859 in provider astro proc_lsst.multi INFO: found job 17074859 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:16 proc_lsst.multi:146 [INFO] found job 17074861 in provider astro proc_lsst.multi INFO: found job 17074861 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:17 proc_lsst.multi:146 [INFO] found job 17074863 in provider astro proc_lsst.multi INFO: found job 17074863 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:17 proc_lsst.multi:146 [INFO] found job 17074865 in provider astro proc_lsst.multi INFO: found job 17074865 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:18 proc_lsst.multi:146 [INFO] found job 17074869 in provider astro proc_lsst.multi INFO: found job 17074869 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 56 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39f9043e9a368a32b4c0de18b432e96f with result from task 56 parsl.dataflow.dflow DEBUG: Task 56 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 56 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/c2af5f87-30ed-4a52-aa57-992997ddaf0c_isrForCrosstalkSources_996957_28.stdout parsl.dataflow.dflow INFO: Standard error for task 56 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/c2af5f87-30ed-4a52-aa57-992997ddaf0c_isrForCrosstalkSources_996957_28.stderr parsl.dataflow.dflow INFO: Task 57 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88e231a0bed8bf613290396e88971743 with result from task 57 parsl.dataflow.dflow DEBUG: Task 57 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 57 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2958b64f-c2dc-40d4-8796-69628c667b2c_isrForCrosstalkSources_996961_7.stdout parsl.dataflow.dflow INFO: Standard error for task 57 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2958b64f-c2dc-40d4-8796-69628c667b2c_isrForCrosstalkSources_996961_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:19 proc_lsst.multi:146 [INFO] found job 17074870 in provider astro proc_lsst.multi INFO: found job 17074870 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 58 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a696ccae5463aeb776215e52af2d250 with result from task 58 parsl.dataflow.dflow DEBUG: Task 58 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 58 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/f8a165e6-dbda-4ae2-8c1e-c45544442a8d_isrForCrosstalkSources_997030_50.stdout parsl.dataflow.dflow INFO: Standard error for task 58 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/f8a165e6-dbda-4ae2-8c1e-c45544442a8d_isrForCrosstalkSources_997030_50.stderr parsl.dataflow.dflow INFO: Task 59 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de81bf7989da01f9e647aa6a156105fd with result from task 59 parsl.dataflow.dflow DEBUG: Task 59 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 59 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/909d631e-91aa-46d3-90b7-69df88074b22_isrForCrosstalkSources_996985_29.stdout parsl.dataflow.dflow INFO: Standard error for task 59 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/909d631e-91aa-46d3-90b7-69df88074b22_isrForCrosstalkSources_996985_29.stderr parsl.dataflow.dflow INFO: Task 60 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 808162cf48b228888bbe608d793e3f4d with result from task 60 parsl.dataflow.dflow DEBUG: Task 60 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 60 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7be3be1c-591a-404e-a97e-5dc03be6ff53_isrForCrosstalkSources_996988_60.stdout parsl.dataflow.dflow INFO: Standard error for task 60 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7be3be1c-591a-404e-a97e-5dc03be6ff53_isrForCrosstalkSources_996988_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:20 proc_lsst.multi:146 [INFO] found job 17074871 in provider astro proc_lsst.multi INFO: found job 17074871 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:20 proc_lsst.multi:146 [INFO] found job 17074872 in provider astro proc_lsst.multi INFO: found job 17074872 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:21 proc_lsst.multi:146 [INFO] found job 17074873 in provider astro proc_lsst.multi INFO: found job 17074873 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 33 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fedbd5207f369e92a36e9718aabf8136 with result from task 33 parsl.dataflow.dflow DEBUG: Task 33 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 33 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/326f25b8-c6ce-467c-a9e9-4a7a4db54d38_isrForCrosstalkSources_997019_26.stdout parsl.dataflow.dflow INFO: Standard error for task 33 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/326f25b8-c6ce-467c-a9e9-4a7a4db54d38_isrForCrosstalkSources_997019_26.stderr parsl.dataflow.dflow INFO: Task 35 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5dc95572a6e766df170dcfb5c369036b with result from task 35 parsl.dataflow.dflow DEBUG: Task 35 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 35 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/37a5e84f-b245-4095-9c6b-988bcb217c79_isrForCrosstalkSources_997015_51.stdout parsl.dataflow.dflow INFO: Standard error for task 35 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/37a5e84f-b245-4095-9c6b-988bcb217c79_isrForCrosstalkSources_997015_51.stderr parsl.dataflow.dflow INFO: Task 34 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4218e543b8cf5752278c24efab7fa480 with result from task 34 parsl.dataflow.dflow DEBUG: Task 34 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 34 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/23df6717-b6f0-4e8f-816b-28c9388e0cab_isrForCrosstalkSources_997010_15.stdout parsl.dataflow.dflow INFO: Standard error for task 34 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/23df6717-b6f0-4e8f-816b-28c9388e0cab_isrForCrosstalkSources_997010_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:21 proc_lsst.multi:146 [INFO] found job 17074876 in provider astro proc_lsst.multi INFO: found job 17074876 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:22 proc_lsst.multi:146 [INFO] found job 17074877 in provider astro proc_lsst.multi INFO: found job 17074877 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:22 proc_lsst.multi:146 [INFO] found job 17074879 in provider astro proc_lsst.multi INFO: found job 17074879 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:23 proc_lsst.multi:146 [INFO] found job 17074880 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074880 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:24 proc_lsst.multi:146 [INFO] found job 17074881 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074881 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:24 proc_lsst.multi:146 [INFO] found job 17074887 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074887 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:25 proc_lsst.multi:146 [INFO] found job 17074888 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074888 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 55 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89360378947b4e11242b73e46dddedc0 with result from task 55 parsl.dataflow.dflow DEBUG: Task 55 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 55 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5ecf84a0-34af-413b-a968-60c2a32bf7d7_isrForCrosstalkSources_996972_14.stdout parsl.dataflow.dflow INFO: Standard error for task 55 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5ecf84a0-34af-413b-a968-60c2a32bf7d7_isrForCrosstalkSources_996972_14.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:26 proc_lsst.multi:146 [INFO] found job 17074889 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074889 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 61 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b51d62babfbac966bb1c77eea6688e00 with result from task 61 parsl.dataflow.dflow DEBUG: Task 61 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 61 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/69443999-cf7a-43f8-a553-0d0318e26210_isrForCrosstalkSources_996987_13.stdout parsl.dataflow.dflow INFO: Standard error for task 61 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/69443999-cf7a-43f8-a553-0d0318e26210_isrForCrosstalkSources_996987_13.stderr parsl.dataflow.dflow INFO: Task 62 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db6494c2816370f599141b311d7b7070 with result from task 62 parsl.dataflow.dflow DEBUG: Task 62 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 62 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/4e73a223-b353-4064-af7f-edbb56ff7ebb_isrForCrosstalkSources_996990_51.stdout parsl.dataflow.dflow INFO: Standard error for task 62 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/4e73a223-b353-4064-af7f-edbb56ff7ebb_isrForCrosstalkSources_996990_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:27 proc_lsst.multi:146 [INFO] found job 17074894 in provider escience_a40 proc_lsst.multi INFO: found job 17074894 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898' parsl.dataflow.dflow INFO: Task 48 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7dec3c3bc322ab6fe13f582e220650bd with result from task 48 parsl.dataflow.dflow DEBUG: Task 48 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 48 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/83741167-45e4-40e2-b9b5-83a4dc357a74_isrForCrosstalkSources_997024_16.stdout parsl.dataflow.dflow INFO: Standard error for task 48 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/83741167-45e4-40e2-b9b5-83a4dc357a74_isrForCrosstalkSources_997024_16.stderr parsl.dataflow.dflow INFO: Task 47 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2aa11ba4689a206627063423f62d6c72 with result from task 47 parsl.dataflow.dflow DEBUG: Task 47 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 47 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/fd3afc48-2174-43a5-a4f5-5c617441e437_isrForCrosstalkSources_996952_16.stdout parsl.dataflow.dflow INFO: Standard error for task 47 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/fd3afc48-2174-43a5-a4f5-5c617441e437_isrForCrosstalkSources_996952_16.stderr parsl.dataflow.dflow INFO: Task 53 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5624af16cbdd7273e01ab51ba0e7d854 with result from task 53 parsl.dataflow.dflow DEBUG: Task 53 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 53 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/48231e4e-5ebc-497f-af14-07b3e91435b4_isrForCrosstalkSources_997003_10.stdout parsl.dataflow.dflow INFO: Standard error for task 53 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/48231e4e-5ebc-497f-af14-07b3e91435b4_isrForCrosstalkSources_997003_10.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:29 proc_lsst.multi:146 [INFO] found job 17074895 in provider escience_a40 proc_lsst.multi INFO: found job 17074895 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898' parsl.dataflow.dflow INFO: Task 44 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca647dfe59ab359eac5625903413a2b4 with result from task 44 parsl.dataflow.dflow DEBUG: Task 44 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 44 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6394f85e-884d-4744-a1ed-fa04bd304750_isrForCrosstalkSources_997006_57.stdout parsl.dataflow.dflow INFO: Standard error for task 44 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6394f85e-884d-4744-a1ed-fa04bd304750_isrForCrosstalkSources_997006_57.stderr parsl.dataflow.dflow INFO: Task 45 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56ee85fb9d0b1b8cda1a0132be2066d7 with result from task 45 parsl.dataflow.dflow DEBUG: Task 45 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 45 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6ace84a6-4415-4e10-8211-57bb438090f5_isrForCrosstalkSources_996953_48.stdout parsl.dataflow.dflow INFO: Standard error for task 45 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6ace84a6-4415-4e10-8211-57bb438090f5_isrForCrosstalkSources_996953_48.stderr parsl.dataflow.dflow INFO: Task 63 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 156a1da3b892693f7958238c16b69ea8 with result from task 63 parsl.providers.slurm.slurm DEBUG: squeue returned 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:30 proc_lsst.multi:146 [INFO] found job 17074897 in provider escience_a40 proc_lsst.multi INFO: found job 17074897 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898' parsl.dataflow.dflow DEBUG: Task 63 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 63 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4f3eb4fa-529c-4eb4-9c2a-0877743a8c3e_isrForCrosstalkSources_996972_53.stdout parsl.dataflow.dflow INFO: Standard error for task 63 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4f3eb4fa-529c-4eb4-9c2a-0877743a8c3e_isrForCrosstalkSources_996972_53.stderr parsl.dataflow.dflow INFO: Task 76 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30756c6d0c39ff1833d9827633bb9b9f with result from task 76 parsl.dataflow.dflow DEBUG: Task 76 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 76 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/473f451a-0148-4778-b462-2df0b128c66b_isrForCrosstalkSources_997017_51.stdout parsl.dataflow.dflow INFO: Standard error for task 76 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/473f451a-0148-4778-b462-2df0b128c66b_isrForCrosstalkSources_997017_51.stderr parsl.dataflow.dflow INFO: Task 69 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c62fd04979093720bfbde4c42b1018a9 with result from task 69 parsl.dataflow.dflow DEBUG: Task 69 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 69 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/9d681ffa-be5d-4793-9e75-8e09d80ec554_isrForCrosstalkSources_996967_36.stdout parsl.dataflow.dflow INFO: Standard error for task 69 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/9d681ffa-be5d-4793-9e75-8e09d80ec554_isrForCrosstalkSources_996967_36.stderr parsl.dataflow.dflow INFO: Task 74 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 69d14c7c37778c5d380bf2b2728efc45 with result from task 74 parsl.dataflow.dflow DEBUG: Task 74 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 74 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f9dcf0e7-4ef2-461d-be65-b784bb59d0f9_isrForCrosstalkSources_996956_40.stdout parsl.dataflow.dflow INFO: Standard error for task 74 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f9dcf0e7-4ef2-461d-be65-b784bb59d0f9_isrForCrosstalkSources_996956_40.stderr parsl.dataflow.dflow INFO: Task 75 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7519557c116e2d80b6f869239c48ae74 with result from task 75 parsl.dataflow.dflow DEBUG: Task 75 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 75 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e3a1fcd6-62f4-4a4f-8812-e26eb6eb2f85_isrForCrosstalkSources_996964_39.stdout parsl.dataflow.dflow INFO: Standard error for task 75 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e3a1fcd6-62f4-4a4f-8812-e26eb6eb2f85_isrForCrosstalkSources_996964_39.stderr parsl.dataflow.dflow INFO: Task 46 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6fbec2255a166632a87d32829ef3c55 with result from task 46 parsl.dataflow.dflow DEBUG: Task 46 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 46 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/cb78d91f-8720-4ece-96ce-530da8cb8af7_isrForCrosstalkSources_996970_54.stdout parsl.dataflow.dflow INFO: Standard error for task 46 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/cb78d91f-8720-4ece-96ce-530da8cb8af7_isrForCrosstalkSources_996970_54.stderr parsl.dataflow.dflow INFO: Task 54 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f0c02f37f5c331c0a4e24e14877cc3a8 with result from task 54 parsl.dataflow.dflow DEBUG: Task 54 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 54 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d6676acd-ed5c-46cc-8fb8-3927f419cb29_isrForCrosstalkSources_996954_47.stdout parsl.dataflow.dflow INFO: Standard error for task 54 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d6676acd-ed5c-46cc-8fb8-3927f419cb29_isrForCrosstalkSources_996954_47.stderr parsl.dataflow.dflow INFO: Task 49 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c38f08ddad2aa0f2b7dbf0ce821b20b0 with result from task 49 parsl.dataflow.dflow DEBUG: Task 49 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 49 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e7d3441f-d0df-486f-a43a-a5584f80bdfe_isrForCrosstalkSources_996986_33.stdout parsl.dataflow.dflow INFO: Standard error for task 49 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e7d3441f-d0df-486f-a43a-a5584f80bdfe_isrForCrosstalkSources_996986_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:50:31 proc_lsst.multi:146 [INFO] found job 17074898 in provider escience_a40 proc_lsst.multi INFO: found job 17074898 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898' parsl.dataflow.dflow INFO: Task 50 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7384ddf5d5ff407879202a1ebbee4ff0 with result from task 50 parsl.dataflow.dflow DEBUG: Task 50 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 50 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/7a2cecc4-ede4-4ff7-b882-8f286eb281d9_isrForCrosstalkSources_997023_4.stdout parsl.dataflow.dflow INFO: Standard error for task 50 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/7a2cecc4-ede4-4ff7-b882-8f286eb281d9_isrForCrosstalkSources_997023_4.stderr parsl.dataflow.dflow INFO: Task 70 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83422729fa032ccaaa245dafcaee1dea with result from task 70 parsl.dataflow.dflow DEBUG: Task 70 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 70 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/03d74eeb-7388-4196-b3c2-eca8e38af562_isrForCrosstalkSources_996963_39.stdout parsl.dataflow.dflow INFO: Standard error for task 70 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/03d74eeb-7388-4196-b3c2-eca8e38af562_isrForCrosstalkSources_996963_39.stderr parsl.dataflow.dflow INFO: Task 52 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 52a84d834c0871421b45085927321eb6 with result from task 52 parsl.dataflow.dflow DEBUG: Task 52 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 52 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d5aba35f-2244-4118-8cdb-432c12d078c9_isrForCrosstalkSources_996998_15.stdout parsl.dataflow.dflow INFO: Standard error for task 52 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d5aba35f-2244-4118-8cdb-432c12d078c9_isrForCrosstalkSources_996998_15.stderr parsl.dataflow.dflow INFO: Task 72 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 949cddc32175547066349f8b6f8052c1 with result from task 72 parsl.dataflow.dflow DEBUG: Task 72 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 72 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/0fe94d2e-e048-4623-b59e-fa3db23dd0ff_isrForCrosstalkSources_996961_60.stdout parsl.dataflow.dflow INFO: Standard error for task 72 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/0fe94d2e-e048-4623-b59e-fa3db23dd0ff_isrForCrosstalkSources_996961_60.stderr parsl.dataflow.dflow INFO: Task 71 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 71b92a704c533b6a3740248bde0b55b0 with result from task 71 parsl.dataflow.dflow DEBUG: Task 71 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 71 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9829c48a-7666-4daf-bd6b-889343b5947a_isrForCrosstalkSources_996959_19.stdout parsl.dataflow.dflow INFO: Standard error for task 71 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9829c48a-7666-4daf-bd6b-889343b5947a_isrForCrosstalkSources_996959_19.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.strategy DEBUG: general strategy starting with strategy_type simple for 1 executors parsl.dataflow.strategy DEBUG: Strategizing for executor multi parsl.dataflow.strategy DEBUG: Slot ratio calculation: active_slots = 40, active_tasks = 5027 parsl.dataflow.strategy DEBUG: Executor multi has 5027 active tasks, 40/0 running/pending blocks, and 41 connected workers parsl.dataflow.strategy DEBUG: Strategy case 2: slots are overloaded - (slot_ratio = active_slots/active_tasks) < parallelism parsl.dataflow.strategy DEBUG: Strategy case 2b: active_blocks 40 < max_blocks 312 so scaling out parsl.dataflow.strategy DEBUG: Requesting 272 more blocks parsl.executors.status_handling INFO: Scaling out by 272 blocks parsl.executors.status_handling INFO: Allocated block ID 40 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-40 2024-03-15 13:50:32 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-40 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-40 2024-03-15 13:50:32 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:32 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:32 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:32 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 4 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 4 2024-03-15 13:50:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:32 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=40 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 51 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fac930ff3766b69fb7094a53bc6f4ad with result from task 51 parsl.dataflow.dflow DEBUG: Task 51 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 51 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4d3f6cb0-286b-4204-b762-a2207a53f17c_isrForCrosstalkSources_996976_57.stdout parsl.dataflow.dflow INFO: Standard error for task 51 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4d3f6cb0-286b-4204-b762-a2207a53f17c_isrForCrosstalkSources_996976_57.stderr parsl.dataflow.dflow INFO: Task 73 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e191b40a9a762701a3633d80e49e596 with result from task 73 parsl.dataflow.dflow DEBUG: Task 73 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 73 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/6eb2a6d8-9dc7-4a18-bfe4-24f427f53143_isrForCrosstalkSources_996955_18.stdout parsl.dataflow.dflow INFO: Standard error for task 73 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/6eb2a6d8-9dc7-4a18-bfe4-24f427f53143_isrForCrosstalkSources_996955_18.stderr 2024-03-15 13:50:32 proc_lsst.multi:170 [INFO] [multi] job_id 17074932 proc_lsst.multi INFO: [multi] job_id 17074932 2024-03-15 13:50:32 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:32 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:32 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074932 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074932 parsl.executors.status_handling DEBUG: Launched block 40 on executor multi with job ID 17074932 parsl.executors.status_handling INFO: Allocated block ID 41 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-41 2024-03-15 13:50:32 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-41 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-41 2024-03-15 13:50:32 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:32 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:32 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:32 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:32 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=41 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:33 proc_lsst.multi:170 [INFO] [multi] job_id 17074933 proc_lsst.multi INFO: [multi] job_id 17074933 2024-03-15 13:50:33 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 6 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 6 2024-03-15 13:50:33 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:33 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074933 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074933 parsl.executors.status_handling DEBUG: Launched block 41 on executor multi with job ID 17074933 parsl.executors.status_handling INFO: Allocated block ID 42 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-42 2024-03-15 13:50:33 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-42 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-42 2024-03-15 13:50:33 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:33 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:33 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:33 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 6 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 6 2024-03-15 13:50:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:33 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=42 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:33 proc_lsst.multi:170 [INFO] [multi] job_id 17074934 proc_lsst.multi INFO: [multi] job_id 17074934 2024-03-15 13:50:33 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 7 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 7 2024-03-15 13:50:33 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:33 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074934 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074934 parsl.executors.status_handling DEBUG: Launched block 42 on executor multi with job ID 17074934 parsl.executors.status_handling INFO: Allocated block ID 43 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-43 2024-03-15 13:50:33 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-43 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-43 2024-03-15 13:50:33 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:33 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:33 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:33 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 7 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 7 2024-03-15 13:50:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:33 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=43 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 25 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55dcd0cf2d234694c57ca05450f108c6 with result from task 25 parsl.dataflow.dflow DEBUG: Task 25 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 25 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/df05ae00-5c52-4ec8-b82c-4b1850b20a5d_isrForCrosstalkSources_996997_52.stdout parsl.dataflow.dflow INFO: Standard error for task 25 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/df05ae00-5c52-4ec8-b82c-4b1850b20a5d_isrForCrosstalkSources_996997_52.stderr parsl.dataflow.dflow INFO: Task 28 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 66a784096ccb4b9dd0e8231f8de6e2f1 with result from task 28 parsl.dataflow.dflow DEBUG: Task 28 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 28 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b9dac3b1-014b-4a3b-b3ed-24111a092ee9_isrForCrosstalkSources_997021_61.stdout parsl.dataflow.dflow INFO: Standard error for task 28 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b9dac3b1-014b-4a3b-b3ed-24111a092ee9_isrForCrosstalkSources_997021_61.stderr parsl.dataflow.dflow INFO: Task 39 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ebf8d3fe58a908192d5e677d51ea34b with result from task 39 parsl.dataflow.dflow DEBUG: Task 39 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 39 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/79b4c3d3-649c-43f5-9de0-cd892fe7e17e_isrForCrosstalkSources_996999_18.stdout parsl.dataflow.dflow INFO: Standard error for task 39 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/79b4c3d3-649c-43f5-9de0-cd892fe7e17e_isrForCrosstalkSources_996999_18.stderr parsl.dataflow.dflow INFO: Task 41 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4538b73023eb13e4267f620caa69e7b with result from task 41 parsl.dataflow.dflow DEBUG: Task 41 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 41 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/44cb1c17-7beb-44ba-9027-df768437db73_isrForCrosstalkSources_997019_53.stdout parsl.dataflow.dflow INFO: Standard error for task 41 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/44cb1c17-7beb-44ba-9027-df768437db73_isrForCrosstalkSources_997019_53.stderr parsl.dataflow.dflow INFO: Task 43 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 716e44646e9ce03a86a4eaeee802854b with result from task 43 parsl.dataflow.dflow DEBUG: Task 43 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 43 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ad277931-73d9-4f90-bb23-11ac940d84c4_isrForCrosstalkSources_997013_21.stdout parsl.dataflow.dflow INFO: Standard error for task 43 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ad277931-73d9-4f90-bb23-11ac940d84c4_isrForCrosstalkSources_997013_21.stderr parsl.dataflow.dflow INFO: Task 32 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86a1a89abe1bba127d064894a9e7e983 with result from task 32 parsl.dataflow.dflow DEBUG: Task 32 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 32 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/610ca1be-a28e-4d64-8d65-42fe24bc15ea_isrForCrosstalkSources_996974_42.stdout parsl.dataflow.dflow INFO: Standard error for task 32 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/610ca1be-a28e-4d64-8d65-42fe24bc15ea_isrForCrosstalkSources_996974_42.stderr parsl.dataflow.dflow INFO: Task 27 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a36ee2f0f4e894903b20f520cb3692f with result from task 27 parsl.dataflow.dflow DEBUG: Task 27 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 27 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/0b3ed825-85da-4e93-85d1-07f4c7c98c4f_isrForCrosstalkSources_996952_61.stdout parsl.dataflow.dflow INFO: Standard error for task 27 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/0b3ed825-85da-4e93-85d1-07f4c7c98c4f_isrForCrosstalkSources_996952_61.stderr parsl.dataflow.dflow INFO: Task 31 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e764d14e5f925a8272be0f0607163bb5 with result from task 31 parsl.dataflow.dflow DEBUG: Task 31 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 31 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/8c852584-a025-4036-a50e-f9f5765cff47_isrForCrosstalkSources_996998_38.stdout parsl.dataflow.dflow INFO: Standard error for task 31 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/8c852584-a025-4036-a50e-f9f5765cff47_isrForCrosstalkSources_996998_38.stderr parsl.dataflow.dflow INFO: Task 42 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b1ac4d50fda7cc0fe10f1c1ef5e2b51 with result from task 42 parsl.dataflow.dflow DEBUG: Task 42 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 42 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cc0e9506-2a62-40c7-a44f-a946384a7aff_isrForCrosstalkSources_996957_62.stdout parsl.dataflow.dflow INFO: Standard error for task 42 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cc0e9506-2a62-40c7-a44f-a946384a7aff_isrForCrosstalkSources_996957_62.stderr parsl.dataflow.dflow INFO: Task 29 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry faf6965c005e545f450965b71b8843f5 with result from task 29 parsl.dataflow.dflow DEBUG: Task 29 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 29 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/94a8a546-d420-4c8e-ae1a-ef2396509c79_isrForCrosstalkSources_996967_62.stdout parsl.dataflow.dflow INFO: Standard error for task 29 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/94a8a546-d420-4c8e-ae1a-ef2396509c79_isrForCrosstalkSources_996967_62.stderr parsl.dataflow.dflow INFO: Task 30 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 172ee54fbc0ffc33a34ad9c8e197ae9e with result from task 30 parsl.dataflow.dflow DEBUG: Task 30 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 30 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a65b1606-0079-4b6a-a114-1485315fa59b_isrForCrosstalkSources_997006_60.stdout parsl.dataflow.dflow INFO: Standard error for task 30 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a65b1606-0079-4b6a-a114-1485315fa59b_isrForCrosstalkSources_997006_60.stderr parsl.dataflow.dflow INFO: Task 26 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25387a4d7a3e966e12711ccb8ef066ff with result from task 26 parsl.dataflow.dflow DEBUG: Task 26 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 26 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/70c568aa-4f83-47f5-9b59-d09b787ec11f_isrForCrosstalkSources_997009_31.stdout parsl.dataflow.dflow INFO: Standard error for task 26 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/70c568aa-4f83-47f5-9b59-d09b787ec11f_isrForCrosstalkSources_997009_31.stderr parsl.dataflow.dflow INFO: Task 37 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3cfc13b94be90dd65e11109952cacdfb with result from task 37 parsl.dataflow.dflow DEBUG: Task 37 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 37 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/deaecffd-bd08-469a-b168-55190f404c2c_isrForCrosstalkSources_996980_54.stdout parsl.dataflow.dflow INFO: Standard error for task 37 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/deaecffd-bd08-469a-b168-55190f404c2c_isrForCrosstalkSources_996980_54.stderr 2024-03-15 13:50:34 proc_lsst.multi:170 [INFO] [multi] job_id 17074935 proc_lsst.multi INFO: [multi] job_id 17074935 2024-03-15 13:50:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 8 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 8 2024-03-15 13:50:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:34 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074935 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074935 parsl.executors.status_handling DEBUG: Launched block 43 on executor multi with job ID 17074935 parsl.executors.status_handling INFO: Allocated block ID 44 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-44 2024-03-15 13:50:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-44 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-44 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 8 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 8 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=44 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:34 proc_lsst.multi:170 [INFO] [multi] job_id 17074936 proc_lsst.multi INFO: [multi] job_id 17074936 2024-03-15 13:50:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 9 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 9 2024-03-15 13:50:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:34 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074936 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074936 parsl.executors.status_handling DEBUG: Launched block 44 on executor multi with job ID 17074936 parsl.executors.status_handling INFO: Allocated block ID 45 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-45 2024-03-15 13:50:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-45 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-45 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 9 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 9 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=45 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:34 proc_lsst.multi:170 [INFO] [multi] job_id 17074937 proc_lsst.multi INFO: [multi] job_id 17074937 2024-03-15 13:50:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 10 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 10 2024-03-15 13:50:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:34 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074937 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074937 parsl.executors.status_handling DEBUG: Launched block 45 on executor multi with job ID 17074937 parsl.executors.status_handling INFO: Allocated block ID 46 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-46 2024-03-15 13:50:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-46 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-46 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 10 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 10 2024-03-15 13:50:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=46 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:37 proc_lsst.multi:170 [INFO] [multi] job_id 17074938 proc_lsst.multi INFO: [multi] job_id 17074938 2024-03-15 13:50:37 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 11 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 11 2024-03-15 13:50:37 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:37 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074938 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074938 parsl.executors.status_handling DEBUG: Launched block 46 on executor multi with job ID 17074938 parsl.executors.status_handling INFO: Allocated block ID 47 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-47 2024-03-15 13:50:37 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-47 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-47 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 11 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 11 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:37 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=47 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:37 proc_lsst.multi:170 [INFO] [multi] job_id 17074939 proc_lsst.multi INFO: [multi] job_id 17074939 2024-03-15 13:50:37 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 12 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 12 2024-03-15 13:50:37 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:37 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074939 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074939 parsl.executors.status_handling DEBUG: Launched block 47 on executor multi with job ID 17074939 parsl.executors.status_handling INFO: Allocated block ID 48 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-48 2024-03-15 13:50:37 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-48 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-48 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 12 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 12 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:37 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=48 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:37 proc_lsst.multi:170 [INFO] [multi] job_id 17074940 proc_lsst.multi INFO: [multi] job_id 17074940 2024-03-15 13:50:37 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 13 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 13 2024-03-15 13:50:37 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:37 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074940 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074940 parsl.executors.status_handling DEBUG: Launched block 48 on executor multi with job ID 17074940 parsl.executors.status_handling INFO: Allocated block ID 49 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-49 2024-03-15 13:50:37 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-49 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-49 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:37 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 13 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 13 2024-03-15 13:50:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:37 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=49 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 65 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2746b14ef84dded70d0475724c088349 with result from task 65 parsl.dataflow.dflow DEBUG: Task 65 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 65 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/59177452-c882-473e-9915-c875c09447f1_isrForCrosstalkSources_996962_14.stdout parsl.dataflow.dflow INFO: Standard error for task 65 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/59177452-c882-473e-9915-c875c09447f1_isrForCrosstalkSources_996962_14.stderr parsl.dataflow.dflow INFO: Task 64 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2052354eca088925aed54a0976586778 with result from task 64 parsl.dataflow.dflow DEBUG: Task 64 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 64 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9b0e8c43-d55a-4f22-a2f1-0148b97488bf_isrForCrosstalkSources_996995_30.stdout parsl.dataflow.dflow INFO: Standard error for task 64 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9b0e8c43-d55a-4f22-a2f1-0148b97488bf_isrForCrosstalkSources_996995_30.stderr parsl.dataflow.dflow INFO: Task 67 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b184ad672cb8d70b65499339b1bb8797 with result from task 67 parsl.dataflow.dflow DEBUG: Task 67 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 67 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/12fc536e-a669-4f54-a111-b97702bd8768_isrForCrosstalkSources_997025_35.stdout parsl.dataflow.dflow INFO: Standard error for task 67 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/12fc536e-a669-4f54-a111-b97702bd8768_isrForCrosstalkSources_997025_35.stderr parsl.dataflow.dflow INFO: Task 66 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 24417ff488aa2d8840063824b662e186 with result from task 66 parsl.dataflow.dflow DEBUG: Task 66 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 66 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/6e2e4b77-0c69-4e4f-bc75-9e9eb4410f73_isrForCrosstalkSources_996997_60.stdout parsl.dataflow.dflow INFO: Standard error for task 66 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/6e2e4b77-0c69-4e4f-bc75-9e9eb4410f73_isrForCrosstalkSources_996997_60.stderr 2024-03-15 13:50:39 proc_lsst.multi:170 [INFO] [multi] job_id 17074942 proc_lsst.multi INFO: [multi] job_id 17074942 2024-03-15 13:50:39 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 14 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 14 2024-03-15 13:50:39 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:39 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074942 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074942 parsl.executors.status_handling DEBUG: Launched block 49 on executor multi with job ID 17074942 parsl.executors.status_handling INFO: Allocated block ID 50 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-50 2024-03-15 13:50:39 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-50 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-50 2024-03-15 13:50:39 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:39 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:39 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:39 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 14 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 14 2024-03-15 13:50:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:39 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=50 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 86 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae916a5d24b029cfe7d936b4c9118c38 with result from task 86 parsl.dataflow.dflow DEBUG: Task 86 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 86 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/f40c8a2a-a00d-4128-ba7e-86ac8c2c78ee_isrForCrosstalkSources_996994_33.stdout parsl.dataflow.dflow INFO: Standard error for task 86 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/f40c8a2a-a00d-4128-ba7e-86ac8c2c78ee_isrForCrosstalkSources_996994_33.stderr 2024-03-15 13:50:40 proc_lsst.multi:170 [INFO] [multi] job_id 17074974 proc_lsst.multi INFO: [multi] job_id 17074974 2024-03-15 13:50:40 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 15 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 15 2024-03-15 13:50:40 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:40 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17074974 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17074974 parsl.executors.status_handling DEBUG: Launched block 50 on executor multi with job ID 17074974 parsl.executors.status_handling INFO: Allocated block ID 51 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-51 2024-03-15 13:50:40 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-51 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-51 2024-03-15 13:50:40 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:40 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:40 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:40 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 15 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 15 2024-03-15 13:50:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:40 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=51 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:40 proc_lsst.multi:170 [INFO] [multi] job_id 17075003 proc_lsst.multi INFO: [multi] job_id 17075003 2024-03-15 13:50:40 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 16 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 16 2024-03-15 13:50:40 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:40 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17075003 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17075003 parsl.executors.status_handling DEBUG: Launched block 51 on executor multi with job ID 17075003 parsl.executors.status_handling INFO: Allocated block ID 52 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-52 2024-03-15 13:50:40 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-52 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-52 2024-03-15 13:50:40 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:40 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:40 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:40 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 16 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 16 2024-03-15 13:50:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:40 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=52 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 88 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4130b967586a26def3207597bb2363a0 with result from task 88 parsl.dataflow.dflow DEBUG: Task 88 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 88 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b5d7e425-8206-4210-bc8f-c2b01237f1c5_isrForCrosstalkSources_997013_29.stdout parsl.dataflow.dflow INFO: Standard error for task 88 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b5d7e425-8206-4210-bc8f-c2b01237f1c5_isrForCrosstalkSources_997013_29.stderr 2024-03-15 13:50:41 proc_lsst.multi:170 [INFO] [multi] job_id 17075018 proc_lsst.multi INFO: [multi] job_id 17075018 2024-03-15 13:50:41 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 17 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 17 2024-03-15 13:50:41 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:41 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17075018 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17075018 parsl.executors.status_handling DEBUG: Launched block 52 on executor multi with job ID 17075018 parsl.executors.status_handling INFO: Allocated block ID 53 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-53 2024-03-15 13:50:41 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-53 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-53 2024-03-15 13:50:41 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:41 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:41 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:41 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 17 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 17 2024-03-15 13:50:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:41 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=53 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 89 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 98bd46cf91df4efd9757be9592a7233d with result from task 89 parsl.dataflow.dflow DEBUG: Task 89 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 89 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4f4fdb78-bf2a-4794-b643-7ad566e145bf_isrForCrosstalkSources_996998_53.stdout parsl.dataflow.dflow INFO: Standard error for task 89 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4f4fdb78-bf2a-4794-b643-7ad566e145bf_isrForCrosstalkSources_996998_53.stderr 2024-03-15 13:50:41 proc_lsst.multi:170 [INFO] [multi] job_id 17075019 proc_lsst.multi INFO: [multi] job_id 17075019 2024-03-15 13:50:41 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 18 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 18 2024-03-15 13:50:41 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:41 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17075019 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17075019 parsl.executors.status_handling DEBUG: Launched block 53 on executor multi with job ID 17075019 parsl.executors.status_handling INFO: Allocated block ID 54 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-54 2024-03-15 13:50:41 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-54 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-54 2024-03-15 13:50:41 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:41 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:41 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:41 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 18 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 18 2024-03-15 13:50:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:41 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=54 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:42 proc_lsst.multi:170 [INFO] [multi] job_id 17075020 proc_lsst.multi INFO: [multi] job_id 17075020 2024-03-15 13:50:42 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 19 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 19 2024-03-15 13:50:42 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:42 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17075020 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17075020 parsl.executors.status_handling DEBUG: Launched block 54 on executor multi with job ID 17075020 parsl.executors.status_handling INFO: Allocated block ID 55 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-55 2024-03-15 13:50:42 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-55 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-55 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 19 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 19 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:42 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=55 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to escience_a40 parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 87 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 40f99d5bf38e8bae1e6dda20fe3c10e1 with result from task 87 parsl.dataflow.dflow DEBUG: Task 87 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 87 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/090c00a5-f758-4475-8db0-4788e643dec5_isrForCrosstalkSources_996966_37.stdout parsl.dataflow.dflow INFO: Standard error for task 87 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/090c00a5-f758-4475-8db0-4788e643dec5_isrForCrosstalkSources_996966_37.stderr 2024-03-15 13:50:42 proc_lsst.multi:170 [INFO] [multi] job_id 17075021 proc_lsst.multi INFO: [multi] job_id 17075021 2024-03-15 13:50:42 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:42 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:42 proc_lsst.multi:178 [INFO] [multi] provider escience_a40 accepted submit and returned 17075021 proc_lsst.multi INFO: [multi] provider escience_a40 accepted submit and returned 17075021 parsl.executors.status_handling DEBUG: Launched block 55 on executor multi with job ID 17075021 parsl.executors.status_handling INFO: Allocated block ID 56 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-56 2024-03-15 13:50:42 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-56 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-56 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:42 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 0 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 0 2024-03-15 13:50:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:42 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=56 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:43 proc_lsst.multi:170 [INFO] [multi] job_id 17075022 proc_lsst.multi INFO: [multi] job_id 17075022 2024-03-15 13:50:43 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:43 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:43 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075022 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075022 parsl.executors.status_handling DEBUG: Launched block 56 on executor multi with job ID 17075022 parsl.executors.status_handling INFO: Allocated block ID 57 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-57 2024-03-15 13:50:43 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-57 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-57 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:43 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=57 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:43 proc_lsst.multi:170 [INFO] [multi] job_id 17075030 proc_lsst.multi INFO: [multi] job_id 17075030 2024-03-15 13:50:43 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 2 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 2 2024-03-15 13:50:43 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:43 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075030 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075030 parsl.executors.status_handling DEBUG: Launched block 57 on executor multi with job ID 17075030 parsl.executors.status_handling INFO: Allocated block ID 58 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-58 2024-03-15 13:50:43 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-58 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-58 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:43 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 2 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 2 2024-03-15 13:50:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:43 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=58 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 98 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e872d28080a105935b50f629e6e5239c with result from task 98 parsl.dataflow.dflow DEBUG: Task 98 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 98 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68ae1b2a-a115-43b7-a1a8-ca55ebe8e510_isrForCrosstalkSources_996985_1.stdout parsl.dataflow.dflow INFO: Standard error for task 98 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68ae1b2a-a115-43b7-a1a8-ca55ebe8e510_isrForCrosstalkSources_996985_1.stderr parsl.dataflow.dflow INFO: Task 92 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a5b0997e6a7e9ea0e11df2c3ccf5168 with result from task 92 parsl.dataflow.dflow DEBUG: Task 92 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 92 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/4c3cdf26-9c39-4ed0-a85e-55fcfa850bf0_isrForCrosstalkSources_996985_48.stdout parsl.dataflow.dflow INFO: Standard error for task 92 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/4c3cdf26-9c39-4ed0-a85e-55fcfa850bf0_isrForCrosstalkSources_996985_48.stderr 2024-03-15 13:50:44 proc_lsst.multi:170 [INFO] [multi] job_id 17075031 proc_lsst.multi INFO: [multi] job_id 17075031 2024-03-15 13:50:44 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 3 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 3 2024-03-15 13:50:44 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:44 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075031 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075031 parsl.executors.status_handling DEBUG: Launched block 58 on executor multi with job ID 17075031 parsl.executors.status_handling INFO: Allocated block ID 59 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-59 2024-03-15 13:50:44 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-59 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-59 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 3 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 3 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:44 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=59 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 94 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c720224bcd1628d7cd915ce373ce121f with result from task 94 parsl.dataflow.dflow DEBUG: Task 94 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 94 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/ba482c0f-9b9e-4250-a394-5e26435cf00a_isrForCrosstalkSources_996956_49.stdout parsl.dataflow.dflow INFO: Standard error for task 94 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/ba482c0f-9b9e-4250-a394-5e26435cf00a_isrForCrosstalkSources_996956_49.stderr parsl.dataflow.dflow INFO: Task 96 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84e11ddf8a92673c0295eb86b67ea2e3 with result from task 96 parsl.dataflow.dflow DEBUG: Task 96 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 96 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/634a01cf-9a39-4517-8e26-5a0b3661da09_isrForCrosstalkSources_997000_57.stdout parsl.dataflow.dflow INFO: Standard error for task 96 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/634a01cf-9a39-4517-8e26-5a0b3661da09_isrForCrosstalkSources_997000_57.stderr 2024-03-15 13:50:44 proc_lsst.multi:170 [INFO] [multi] job_id 17075032 proc_lsst.multi INFO: [multi] job_id 17075032 2024-03-15 13:50:44 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 4 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 4 2024-03-15 13:50:44 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:44 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075032 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075032 parsl.executors.status_handling DEBUG: Launched block 59 on executor multi with job ID 17075032 parsl.executors.status_handling INFO: Allocated block ID 60 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-60 2024-03-15 13:50:44 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-60 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-60 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:44 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 4 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 4 2024-03-15 13:50:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:44 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=60 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 93 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec66f11c5a5eb1bdb22198fafc6740b0 with result from task 93 parsl.dataflow.dflow DEBUG: Task 93 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 93 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f5e93a6f-21e7-4f8e-8ce1-3ad0cef885eb_isrForCrosstalkSources_996995_6.stdout parsl.dataflow.dflow INFO: Standard error for task 93 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f5e93a6f-21e7-4f8e-8ce1-3ad0cef885eb_isrForCrosstalkSources_996995_6.stderr parsl.dataflow.dflow INFO: Task 99 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02435cea2230a42a584e4534491fcfea with result from task 99 parsl.dataflow.dflow DEBUG: Task 99 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 99 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/23f02009-c9ce-4a48-8e50-dfcf0d9eb09d_isrForCrosstalkSources_997032_27.stdout parsl.dataflow.dflow INFO: Standard error for task 99 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/23f02009-c9ce-4a48-8e50-dfcf0d9eb09d_isrForCrosstalkSources_997032_27.stderr parsl.dataflow.dflow INFO: Task 97 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7b3b0bb89f59e4c22f07098924c97ad with result from task 97 parsl.dataflow.dflow DEBUG: Task 97 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 97 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/b0896d75-450f-4787-ad4e-1d15df974675_isrForCrosstalkSources_997005_59.stdout parsl.dataflow.dflow INFO: Standard error for task 97 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/b0896d75-450f-4787-ad4e-1d15df974675_isrForCrosstalkSources_997005_59.stderr parsl.dataflow.dflow INFO: Task 95 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 228f88c1316bf83f563229f0cd3f0979 with result from task 95 parsl.dataflow.dflow DEBUG: Task 95 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 95 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fb31705d-fe5f-4fd0-9d9e-49f24db30404_isrForCrosstalkSources_997002_47.stdout parsl.dataflow.dflow INFO: Standard error for task 95 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fb31705d-fe5f-4fd0-9d9e-49f24db30404_isrForCrosstalkSources_997002_47.stderr 2024-03-15 13:50:45 proc_lsst.multi:170 [INFO] [multi] job_id 17075033 proc_lsst.multi INFO: [multi] job_id 17075033 2024-03-15 13:50:45 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:45 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:45 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075033 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075033 parsl.executors.status_handling DEBUG: Launched block 60 on executor multi with job ID 17075033 parsl.executors.status_handling INFO: Allocated block ID 61 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-61 2024-03-15 13:50:45 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-61 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-61 2024-03-15 13:50:45 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:45 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:45 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:45 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:45 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:45 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=61 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:46 proc_lsst.multi:170 [INFO] [multi] job_id 17075034 proc_lsst.multi INFO: [multi] job_id 17075034 2024-03-15 13:50:46 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 6 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 6 2024-03-15 13:50:46 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:46 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075034 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075034 parsl.executors.status_handling DEBUG: Launched block 61 on executor multi with job ID 17075034 parsl.executors.status_handling INFO: Allocated block ID 62 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-62 2024-03-15 13:50:46 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-62 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-62 2024-03-15 13:50:46 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:46 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:46 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:46 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:46 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 6 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 6 2024-03-15 13:50:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:46 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=62 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:47 proc_lsst.multi:170 [INFO] [multi] job_id 17075035 proc_lsst.multi INFO: [multi] job_id 17075035 2024-03-15 13:50:47 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 7 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 7 2024-03-15 13:50:47 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:47 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075035 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075035 parsl.executors.status_handling DEBUG: Launched block 62 on executor multi with job ID 17075035 parsl.executors.status_handling INFO: Allocated block ID 63 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-63 2024-03-15 13:50:47 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-63 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-63 2024-03-15 13:50:47 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:47 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:47 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:47 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:47 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 7 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 7 2024-03-15 13:50:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:47 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=63 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 114 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b08306ee7d2d0de3b48c7c08d68fbb2d with result from task 114 parsl.dataflow.dflow DEBUG: Task 114 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c54f1ece-7134-447c-8224-c94a960a5f24_isrForCrosstalkSources_996961_8.stdout parsl.dataflow.dflow INFO: Standard error for task 114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c54f1ece-7134-447c-8224-c94a960a5f24_isrForCrosstalkSources_996961_8.stderr parsl.dataflow.dflow INFO: Task 79 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fc5454b2edc55f06aae59960fa824120 with result from task 79 parsl.dataflow.dflow DEBUG: Task 79 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 79 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f728ee0b-a439-46f5-aa3d-c61572dfca6e_isrForCrosstalkSources_996985_46.stdout parsl.dataflow.dflow INFO: Standard error for task 79 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f728ee0b-a439-46f5-aa3d-c61572dfca6e_isrForCrosstalkSources_996985_46.stderr parsl.dataflow.dflow INFO: Task 78 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b32fcd4e429a2aa663dd5d74319f6208 with result from task 78 parsl.dataflow.dflow DEBUG: Task 78 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 78 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3553abe3-a95f-4a7c-94e5-ed8741102e5b_isrForCrosstalkSources_996999_22.stdout parsl.dataflow.dflow INFO: Standard error for task 78 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3553abe3-a95f-4a7c-94e5-ed8741102e5b_isrForCrosstalkSources_996999_22.stderr parsl.dataflow.dflow INFO: Task 116 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 313b75e61a7ee74e56cedc1d85289299 with result from task 116 parsl.dataflow.dflow DEBUG: Task 116 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/97c692f5-10bd-436d-a790-f79e9a1be7a7_isrForCrosstalkSources_997033_32.stdout parsl.dataflow.dflow INFO: Standard error for task 116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/97c692f5-10bd-436d-a790-f79e9a1be7a7_isrForCrosstalkSources_997033_32.stderr parsl.dataflow.dflow INFO: Task 85 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dfebdaa6ff146f98fd16751ea2440e26 with result from task 85 parsl.dataflow.dflow DEBUG: Task 85 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 85 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c17cbef9-aacb-4752-b80d-aaf1397dc7a5_isrForCrosstalkSources_996973_6.stdout parsl.dataflow.dflow INFO: Standard error for task 85 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c17cbef9-aacb-4752-b80d-aaf1397dc7a5_isrForCrosstalkSources_996973_6.stderr parsl.dataflow.dflow INFO: Task 115 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f2ce15a3850b6c9b5e056a9479b7934 with result from task 115 parsl.dataflow.dflow DEBUG: Task 115 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/49f57fd2-7bff-4a57-8f9f-e2bafcd2306c_isrForCrosstalkSources_996953_46.stdout parsl.dataflow.dflow INFO: Standard error for task 115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/49f57fd2-7bff-4a57-8f9f-e2bafcd2306c_isrForCrosstalkSources_996953_46.stderr 2024-03-15 13:50:48 proc_lsst.multi:170 [INFO] [multi] job_id 17075038 proc_lsst.multi INFO: [multi] job_id 17075038 2024-03-15 13:50:48 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 8 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 8 2024-03-15 13:50:48 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:48 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075038 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075038 parsl.executors.status_handling DEBUG: Launched block 63 on executor multi with job ID 17075038 parsl.executors.status_handling INFO: Allocated block ID 64 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-64 2024-03-15 13:50:48 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-64 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-64 2024-03-15 13:50:48 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:48 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:48 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:48 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:48 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 8 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 8 2024-03-15 13:50:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:48 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=64 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 113 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e500889f3f8a804a0c99b9dff9c8471a with result from task 113 parsl.dataflow.dflow DEBUG: Task 113 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0227d6d3-7f5f-4c71-a332-033cc5919fa5_isrForCrosstalkSources_996981_31.stdout parsl.dataflow.dflow INFO: Standard error for task 113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0227d6d3-7f5f-4c71-a332-033cc5919fa5_isrForCrosstalkSources_996981_31.stderr parsl.dataflow.dflow INFO: Task 82 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6acd51c273c5a43adb8c109237428ad5 with result from task 82 parsl.dataflow.dflow DEBUG: Task 82 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 82 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/413b8973-9a62-49f6-87b4-12dad535329b_isrForCrosstalkSources_996994_10.stdout parsl.dataflow.dflow INFO: Standard error for task 82 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/413b8973-9a62-49f6-87b4-12dad535329b_isrForCrosstalkSources_996994_10.stderr parsl.dataflow.dflow INFO: Task 83 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed850339454c26f707311d93f129d08c with result from task 83 parsl.dataflow.dflow DEBUG: Task 83 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 83 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/3413f2aa-e21b-4836-9858-b90e177b40a1_isrForCrosstalkSources_997016_1.stdout parsl.dataflow.dflow INFO: Standard error for task 83 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/3413f2aa-e21b-4836-9858-b90e177b40a1_isrForCrosstalkSources_997016_1.stderr 2024-03-15 13:50:49 proc_lsst.multi:170 [INFO] [multi] job_id 17075041 proc_lsst.multi INFO: [multi] job_id 17075041 2024-03-15 13:50:49 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 9 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 9 2024-03-15 13:50:49 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:49 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075041 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075041 parsl.executors.status_handling DEBUG: Launched block 64 on executor multi with job ID 17075041 parsl.executors.status_handling INFO: Allocated block ID 65 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-65 2024-03-15 13:50:49 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-65 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-65 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 9 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 9 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:49 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=65 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 91 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 14d18171bc6da7b22ac2b1b40351f486 with result from task 91 parsl.dataflow.dflow DEBUG: Task 91 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 91 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/311616ee-1fdc-4ed0-8bed-a2fe97500bf3_isrForCrosstalkSources_997002_7.stdout parsl.dataflow.dflow INFO: Standard error for task 91 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/311616ee-1fdc-4ed0-8bed-a2fe97500bf3_isrForCrosstalkSources_997002_7.stderr parsl.dataflow.dflow INFO: Task 77 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 22438b48dbfd3964bfe0515db98125d2 with result from task 77 parsl.dataflow.dflow DEBUG: Task 77 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 77 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c7189b5b-d3d8-45a3-a092-b42a7508d4f7_isrForCrosstalkSources_997007_3.stdout parsl.dataflow.dflow INFO: Standard error for task 77 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c7189b5b-d3d8-45a3-a092-b42a7508d4f7_isrForCrosstalkSources_997007_3.stderr parsl.dataflow.dflow INFO: Task 119 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2eb95725fde4673e05523ea19725c37f with result from task 119 parsl.dataflow.dflow DEBUG: Task 119 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/5d437e0a-16b3-4132-b251-b349a07a79a3_isrForCrosstalkSources_997011_8.stdout parsl.dataflow.dflow INFO: Standard error for task 119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/5d437e0a-16b3-4132-b251-b349a07a79a3_isrForCrosstalkSources_997011_8.stderr 2024-03-15 13:50:49 proc_lsst.multi:170 [INFO] [multi] job_id 17075042 proc_lsst.multi INFO: [multi] job_id 17075042 2024-03-15 13:50:49 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 10 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 10 2024-03-15 13:50:49 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:49 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075042 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075042 parsl.executors.status_handling DEBUG: Launched block 65 on executor multi with job ID 17075042 parsl.executors.status_handling INFO: Allocated block ID 66 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-66 2024-03-15 13:50:49 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-66 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-66 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:49 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 10 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 10 2024-03-15 13:50:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:49 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=66 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 84 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7852832d28ab1af13aa9ac2e3193638 with result from task 84 parsl.dataflow.dflow DEBUG: Task 84 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 84 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/fcf1a4a4-7426-47d1-a768-0840f8574399_isrForCrosstalkSources_996987_60.stdout parsl.dataflow.dflow INFO: Standard error for task 84 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/fcf1a4a4-7426-47d1-a768-0840f8574399_isrForCrosstalkSources_996987_60.stderr parsl.dataflow.dflow INFO: Task 125 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ddc470688bb5443e40390d40defa3861 with result from task 125 parsl.dataflow.dflow DEBUG: Task 125 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/561e693e-8aad-4ff6-991e-1ba5b3fd03a4_isrForCrosstalkSources_996958_20.stdout parsl.dataflow.dflow INFO: Standard error for task 125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/561e693e-8aad-4ff6-991e-1ba5b3fd03a4_isrForCrosstalkSources_996958_20.stderr 2024-03-15 13:50:50 proc_lsst.multi:170 [INFO] [multi] job_id 17075046 proc_lsst.multi INFO: [multi] job_id 17075046 2024-03-15 13:50:50 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 11 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 11 2024-03-15 13:50:50 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:50 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075046 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075046 parsl.executors.status_handling DEBUG: Launched block 66 on executor multi with job ID 17075046 parsl.executors.status_handling INFO: Allocated block ID 67 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-67 2024-03-15 13:50:50 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-67 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-67 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 11 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 11 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:50 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=67 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 90 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 60c06ff9ab5f117d0b6062494a0416e5 with result from task 90 parsl.dataflow.dflow DEBUG: Task 90 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 90 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/89c97da1-ca50-4d5b-91ee-6c35ce8c1782_isrForCrosstalkSources_997013_42.stdout parsl.dataflow.dflow INFO: Standard error for task 90 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/89c97da1-ca50-4d5b-91ee-6c35ce8c1782_isrForCrosstalkSources_997013_42.stderr parsl.dataflow.dflow INFO: Task 122 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8905255393440e4252dc1680a0ce7fb0 with result from task 122 parsl.dataflow.dflow DEBUG: Task 122 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c90b80d7-a453-4f38-9f87-e5fd13fb895d_isrForCrosstalkSources_996955_56.stdout parsl.dataflow.dflow INFO: Standard error for task 122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c90b80d7-a453-4f38-9f87-e5fd13fb895d_isrForCrosstalkSources_996955_56.stderr parsl.dataflow.dflow INFO: Task 80 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d89ef47f9446491fc282a71bf7c43c9f with result from task 80 parsl.dataflow.dflow DEBUG: Task 80 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 80 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f5f9a6ed-9d3c-4bc8-801b-a183ab1ca3d9_isrForCrosstalkSources_996974_49.stdout parsl.dataflow.dflow INFO: Standard error for task 80 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f5f9a6ed-9d3c-4bc8-801b-a183ab1ca3d9_isrForCrosstalkSources_996974_49.stderr parsl.dataflow.dflow INFO: Task 120 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 017a23a876f1168300e799395c40f2a4 with result from task 120 parsl.dataflow.dflow DEBUG: Task 120 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c20d9e14-bc0b-4073-9420-9f208149ff81_isrForCrosstalkSources_996993_17.stdout parsl.dataflow.dflow INFO: Standard error for task 120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c20d9e14-bc0b-4073-9420-9f208149ff81_isrForCrosstalkSources_996993_17.stderr parsl.dataflow.dflow INFO: Task 123 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bcbfc096cdb79d11231f2333da52aee9 with result from task 123 parsl.dataflow.dflow DEBUG: Task 123 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/2894e442-8d1f-4dc4-8dc4-aa8ba218152b_isrForCrosstalkSources_997018_10.stdout parsl.dataflow.dflow INFO: Standard error for task 123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/2894e442-8d1f-4dc4-8dc4-aa8ba218152b_isrForCrosstalkSources_997018_10.stderr parsl.dataflow.dflow INFO: Task 101 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10c29c90f743db4cdd64b6f4c7a570ce with result from task 101 parsl.dataflow.dflow DEBUG: Task 101 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/713bdaac-b205-4418-b9fc-37a401f36211_isrForCrosstalkSources_996971_25.stdout parsl.dataflow.dflow INFO: Standard error for task 101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/713bdaac-b205-4418-b9fc-37a401f36211_isrForCrosstalkSources_996971_25.stderr parsl.dataflow.dflow INFO: Task 118 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f9e658452d15f672201398ca64d017a6 with result from task 118 parsl.dataflow.dflow DEBUG: Task 118 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1fc92050-7c34-4fbe-9768-f6eed0cc0552_isrForCrosstalkSources_996964_6.stdout parsl.dataflow.dflow INFO: Standard error for task 118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1fc92050-7c34-4fbe-9768-f6eed0cc0552_isrForCrosstalkSources_996964_6.stderr 2024-03-15 13:50:50 proc_lsst.multi:170 [INFO] [multi] job_id 17075047 proc_lsst.multi INFO: [multi] job_id 17075047 2024-03-15 13:50:50 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 12 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 12 2024-03-15 13:50:50 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:50 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075047 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075047 parsl.executors.status_handling DEBUG: Launched block 67 on executor multi with job ID 17075047 parsl.executors.status_handling INFO: Allocated block ID 68 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-68 2024-03-15 13:50:50 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-68 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-68 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:50 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 12 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 12 2024-03-15 13:50:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:50 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=68 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 100 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c6ac99ccea5ceb3b411ddb831f1dd423 with result from task 100 parsl.dataflow.dflow DEBUG: Task 100 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/1feacbec-b11a-4b57-aef9-ce30e05fa82d_isrForCrosstalkSources_996986_25.stdout parsl.dataflow.dflow INFO: Standard error for task 100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/1feacbec-b11a-4b57-aef9-ce30e05fa82d_isrForCrosstalkSources_996986_25.stderr parsl.dataflow.dflow INFO: Task 81 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6541b0f4c45dace5711c5a533b4d7892 with result from task 81 parsl.dataflow.dflow DEBUG: Task 81 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 81 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/be373964-0175-4859-8ca0-0b6785553f61_isrForCrosstalkSources_996997_44.stdout parsl.dataflow.dflow INFO: Standard error for task 81 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/be373964-0175-4859-8ca0-0b6785553f61_isrForCrosstalkSources_996997_44.stderr parsl.dataflow.dflow INFO: Task 121 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89da1e49d15cafe7c4a28b5e90f95d5c with result from task 121 parsl.dataflow.dflow DEBUG: Task 121 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/535d81ae-f011-4dd8-9aaa-8e88d0911479_isrForCrosstalkSources_996971_6.stdout parsl.dataflow.dflow INFO: Standard error for task 121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/535d81ae-f011-4dd8-9aaa-8e88d0911479_isrForCrosstalkSources_996971_6.stderr 2024-03-15 13:50:51 proc_lsst.multi:170 [INFO] [multi] job_id 17075048 proc_lsst.multi INFO: [multi] job_id 17075048 2024-03-15 13:50:51 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 13 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 13 2024-03-15 13:50:51 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:51 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075048 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075048 parsl.executors.status_handling DEBUG: Launched block 68 on executor multi with job ID 17075048 parsl.executors.status_handling INFO: Allocated block ID 69 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-69 2024-03-15 13:50:51 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-69 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-69 2024-03-15 13:50:51 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:51 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:51 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:51 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:51 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 13 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 13 2024-03-15 13:50:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:51 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=69 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 107 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96f7b996723e623ed9b9cf4e34214a8e with result from task 107 parsl.dataflow.dflow DEBUG: Task 107 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cf20fafe-976e-438a-9032-f0a9f0656692_isrForCrosstalkSources_997024_8.stdout parsl.dataflow.dflow INFO: Standard error for task 107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cf20fafe-976e-438a-9032-f0a9f0656692_isrForCrosstalkSources_997024_8.stderr parsl.dataflow.dflow INFO: Task 124 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ef6d4b4ebfa09226f340042a85609d4 with result from task 124 parsl.dataflow.dflow DEBUG: Task 124 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/88aaceea-5b6c-4d88-ba64-918da8291703_isrForCrosstalkSources_997014_44.stdout parsl.dataflow.dflow INFO: Standard error for task 124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/88aaceea-5b6c-4d88-ba64-918da8291703_isrForCrosstalkSources_997014_44.stderr 2024-03-15 13:50:52 proc_lsst.multi:170 [INFO] [multi] job_id 17075049 proc_lsst.multi INFO: [multi] job_id 17075049 2024-03-15 13:50:52 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 14 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 14 2024-03-15 13:50:52 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:52 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075049 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075049 parsl.executors.status_handling DEBUG: Launched block 69 on executor multi with job ID 17075049 parsl.executors.status_handling INFO: Allocated block ID 70 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-70 2024-03-15 13:50:52 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-70 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-70 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 14 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 14 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:52 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=70 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 117 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54ecdb941fc14267235d747f6a5573ea with result from task 117 parsl.dataflow.dflow DEBUG: Task 117 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5916c9a8-54db-450c-9356-3fa2dcd8dac3_isrForCrosstalkSources_996968_47.stdout parsl.dataflow.dflow INFO: Standard error for task 117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5916c9a8-54db-450c-9356-3fa2dcd8dac3_isrForCrosstalkSources_996968_47.stderr 2024-03-15 13:50:52 proc_lsst.multi:170 [INFO] [multi] job_id 17075050 proc_lsst.multi INFO: [multi] job_id 17075050 2024-03-15 13:50:52 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 15 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 15 2024-03-15 13:50:52 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:52 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075050 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075050 parsl.executors.status_handling DEBUG: Launched block 70 on executor multi with job ID 17075050 parsl.executors.status_handling INFO: Allocated block ID 71 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-71 2024-03-15 13:50:52 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-71 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-71 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 15 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 15 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:52 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=71 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:52 proc_lsst.multi:170 [INFO] [multi] job_id 17075052 proc_lsst.multi INFO: [multi] job_id 17075052 2024-03-15 13:50:52 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 16 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 16 2024-03-15 13:50:52 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:52 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075052 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075052 parsl.executors.status_handling DEBUG: Launched block 71 on executor multi with job ID 17075052 parsl.executors.status_handling INFO: Allocated block ID 72 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-72 2024-03-15 13:50:52 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-72 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-72 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:52 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 16 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 16 2024-03-15 13:50:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:52 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=72 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 110 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44a4d70c87b069c97ba0f14566cfebb1 with result from task 110 parsl.dataflow.dflow DEBUG: Task 110 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b00777c1-8bd8-43c9-9fb8-0ae58f88e4ad_isrForCrosstalkSources_996997_39.stdout parsl.dataflow.dflow INFO: Standard error for task 110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b00777c1-8bd8-43c9-9fb8-0ae58f88e4ad_isrForCrosstalkSources_996997_39.stderr parsl.dataflow.dflow INFO: Task 106 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b48ca35e1ee98a3a2207cf8dc602664b with result from task 106 parsl.dataflow.dflow DEBUG: Task 106 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/312595b0-4d49-47fe-ad5b-58c589ac4314_isrForCrosstalkSources_996981_17.stdout parsl.dataflow.dflow INFO: Standard error for task 106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/312595b0-4d49-47fe-ad5b-58c589ac4314_isrForCrosstalkSources_996981_17.stderr parsl.dataflow.dflow INFO: Task 103 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7cca8de6c3860073b94944dd1fb530d with result from task 103 parsl.dataflow.dflow DEBUG: Task 103 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2f725ab0-a311-4628-9450-5727c62351a8_isrForCrosstalkSources_996974_9.stdout parsl.dataflow.dflow INFO: Standard error for task 103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2f725ab0-a311-4628-9450-5727c62351a8_isrForCrosstalkSources_996974_9.stderr 2024-03-15 13:50:53 proc_lsst.multi:170 [INFO] [multi] job_id 17075053 proc_lsst.multi INFO: [multi] job_id 17075053 2024-03-15 13:50:53 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 17 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 17 2024-03-15 13:50:53 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:53 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075053 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075053 parsl.executors.status_handling DEBUG: Launched block 72 on executor multi with job ID 17075053 parsl.executors.status_handling INFO: Allocated block ID 73 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-73 2024-03-15 13:50:53 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-73 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-73 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 17 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 17 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:53 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=73 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 127 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b6c9bf5bbc4ceed02661321fd8443d1 with result from task 127 parsl.dataflow.dflow DEBUG: Task 127 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c6f89d07-c99e-4b6b-96cb-b584ef2ca7ea_isrForCrosstalkSources_996966_12.stdout parsl.dataflow.dflow INFO: Standard error for task 127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c6f89d07-c99e-4b6b-96cb-b584ef2ca7ea_isrForCrosstalkSources_996966_12.stderr parsl.dataflow.dflow INFO: Task 109 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44b6f21012b0113e651e1a8089b661ab with result from task 109 parsl.dataflow.dflow DEBUG: Task 109 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/a9d09f05-fb54-4d94-ae6a-60a08518964f_isrForCrosstalkSources_997024_38.stdout parsl.dataflow.dflow INFO: Standard error for task 109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/a9d09f05-fb54-4d94-ae6a-60a08518964f_isrForCrosstalkSources_997024_38.stderr parsl.dataflow.dflow INFO: Task 112 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f74c13535b10e8202fa718b0c32bf32b with result from task 112 parsl.dataflow.dflow DEBUG: Task 112 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/d34c8067-d669-48f6-9d33-7a07ccd43b45_isrForCrosstalkSources_997018_12.stdout parsl.dataflow.dflow INFO: Standard error for task 112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/d34c8067-d669-48f6-9d33-7a07ccd43b45_isrForCrosstalkSources_997018_12.stderr 2024-03-15 13:50:53 proc_lsst.multi:170 [INFO] [multi] job_id 17075054 proc_lsst.multi INFO: [multi] job_id 17075054 2024-03-15 13:50:53 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 18 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 18 2024-03-15 13:50:53 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:53 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075054 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075054 parsl.executors.status_handling DEBUG: Launched block 73 on executor multi with job ID 17075054 parsl.executors.status_handling INFO: Allocated block ID 74 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-74 2024-03-15 13:50:53 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-74 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-74 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:53 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 18 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 18 2024-03-15 13:50:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:53 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=74 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 102 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7dd9fe157d4da1ed96f6d811d7c9180 with result from task 102 parsl.dataflow.dflow DEBUG: Task 102 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/e1eb5408-aa7e-4e5e-b929-d0ae45bf0fc4_isrForCrosstalkSources_996973_58.stdout parsl.dataflow.dflow INFO: Standard error for task 102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/e1eb5408-aa7e-4e5e-b929-d0ae45bf0fc4_isrForCrosstalkSources_996973_58.stderr parsl.dataflow.dflow INFO: Task 111 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dfc0522b952a3e3d069affbefdfd887f with result from task 111 parsl.dataflow.dflow DEBUG: Task 111 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/60f490e6-1613-48c6-8926-c7483f90bce0_isrForCrosstalkSources_996959_9.stdout parsl.dataflow.dflow INFO: Standard error for task 111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/60f490e6-1613-48c6-8926-c7483f90bce0_isrForCrosstalkSources_996959_9.stderr parsl.dataflow.dflow INFO: Task 108 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fdb727f5d1be7c9f04870efabaf56f15 with result from task 108 parsl.dataflow.dflow DEBUG: Task 108 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/927cb936-48c8-4e0b-acaf-7ecb9ade929a_isrForCrosstalkSources_997032_22.stdout parsl.dataflow.dflow INFO: Standard error for task 108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/927cb936-48c8-4e0b-acaf-7ecb9ade929a_isrForCrosstalkSources_997032_22.stderr 2024-03-15 13:50:54 proc_lsst.multi:170 [INFO] [multi] job_id 17075055 proc_lsst.multi INFO: [multi] job_id 17075055 2024-03-15 13:50:54 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 19 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 19 2024-03-15 13:50:54 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:54 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075055 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075055 parsl.executors.status_handling DEBUG: Launched block 74 on executor multi with job ID 17075055 parsl.executors.status_handling INFO: Allocated block ID 75 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-75 2024-03-15 13:50:54 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-75 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-75 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 19 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 19 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:54 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=75 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 104 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d02dac4c6d80d39dd6061510a600792 with result from task 104 parsl.dataflow.dflow DEBUG: Task 104 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ddc5b242-a550-4d1b-8c05-a26f3d5e99f0_isrForCrosstalkSources_997020_55.stdout parsl.dataflow.dflow INFO: Standard error for task 104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ddc5b242-a550-4d1b-8c05-a26f3d5e99f0_isrForCrosstalkSources_997020_55.stderr parsl.dataflow.dflow INFO: Task 128 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2af846a7ae6d55a32aaad5738fc65c99 with result from task 128 parsl.dataflow.dflow DEBUG: Task 128 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/0368705d-b686-4046-b784-b80fdc9f8144_isrForCrosstalkSources_996995_34.stdout parsl.dataflow.dflow INFO: Standard error for task 128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/0368705d-b686-4046-b784-b80fdc9f8144_isrForCrosstalkSources_996995_34.stderr 2024-03-15 13:50:54 proc_lsst.multi:170 [INFO] [multi] job_id 17075056 proc_lsst.multi INFO: [multi] job_id 17075056 2024-03-15 13:50:54 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:54 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:54 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075056 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075056 parsl.executors.status_handling DEBUG: Launched block 75 on executor multi with job ID 17075056 parsl.executors.status_handling INFO: Allocated block ID 76 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-76 2024-03-15 13:50:54 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-76 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-76 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:54 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:54 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=76 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 129 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 081cf076af672236374bdbd20fbde6d1 with result from task 129 parsl.dataflow.dflow DEBUG: Task 129 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7728f295-4697-4a29-a43d-700ad042b218_isrForCrosstalkSources_996993_23.stdout parsl.dataflow.dflow INFO: Standard error for task 129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7728f295-4697-4a29-a43d-700ad042b218_isrForCrosstalkSources_996993_23.stderr parsl.dataflow.dflow INFO: Task 105 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cbbab1d9fb35a1602592d332ec62ccc8 with result from task 105 parsl.dataflow.dflow DEBUG: Task 105 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a3fa0d90-e685-4bf4-a223-07737d8ea342_isrForCrosstalkSources_997011_59.stdout parsl.dataflow.dflow INFO: Standard error for task 105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a3fa0d90-e685-4bf4-a223-07737d8ea342_isrForCrosstalkSources_997011_59.stderr 2024-03-15 13:50:55 proc_lsst.multi:170 [INFO] [multi] job_id 17075057 proc_lsst.multi INFO: [multi] job_id 17075057 2024-03-15 13:50:55 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 21 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 21 2024-03-15 13:50:55 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:55 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075057 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075057 parsl.executors.status_handling DEBUG: Launched block 76 on executor multi with job ID 17075057 parsl.executors.status_handling INFO: Allocated block ID 77 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-77 2024-03-15 13:50:55 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-77 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-77 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 21 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 21 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:55 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=77 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:55 proc_lsst.multi:170 [INFO] [multi] job_id 17075058 proc_lsst.multi INFO: [multi] job_id 17075058 2024-03-15 13:50:55 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 22 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 22 2024-03-15 13:50:55 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:55 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075058 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075058 parsl.executors.status_handling DEBUG: Launched block 77 on executor multi with job ID 17075058 parsl.executors.status_handling INFO: Allocated block ID 78 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-78 2024-03-15 13:50:55 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-78 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-78 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:55 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 22 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 22 2024-03-15 13:50:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:55 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=78 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:50:56 proc_lsst.multi:170 [INFO] [multi] job_id 17075059 proc_lsst.multi INFO: [multi] job_id 17075059 2024-03-15 13:50:56 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 23 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 23 2024-03-15 13:50:56 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:56 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075059 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075059 parsl.executors.status_handling DEBUG: Launched block 78 on executor multi with job ID 17075059 parsl.executors.status_handling INFO: Allocated block ID 79 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-79 2024-03-15 13:50:56 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-79 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-79 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 23 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 23 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:56 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=79 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 132 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75ed1ca17c42010fc18caac007f934f1 with result from task 132 parsl.dataflow.dflow DEBUG: Task 132 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/08dbb5d2-b3c3-48cd-892b-79f7d83cbd2b_isrForCrosstalkSources_996967_6.stdout parsl.dataflow.dflow INFO: Standard error for task 132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/08dbb5d2-b3c3-48cd-892b-79f7d83cbd2b_isrForCrosstalkSources_996967_6.stderr parsl.dataflow.dflow INFO: Task 126 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry befc43c9a5dd9829750e73f3c0f47219 with result from task 126 parsl.dataflow.dflow DEBUG: Task 126 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/5a72cd91-3565-4858-bac6-f1eaf38c4097_isrForCrosstalkSources_997007_5.stdout parsl.dataflow.dflow INFO: Standard error for task 126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/5a72cd91-3565-4858-bac6-f1eaf38c4097_isrForCrosstalkSources_997007_5.stderr 2024-03-15 13:50:56 proc_lsst.multi:170 [INFO] [multi] job_id 17075060 proc_lsst.multi INFO: [multi] job_id 17075060 2024-03-15 13:50:56 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 24 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 24 2024-03-15 13:50:56 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:56 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075060 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075060 parsl.executors.status_handling DEBUG: Launched block 79 on executor multi with job ID 17075060 parsl.executors.status_handling INFO: Allocated block ID 80 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-80 2024-03-15 13:50:56 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-80 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-80 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 24 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 24 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:56 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=80 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 133 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c6f2ec66510fd6dcef1d4ffa8e016a5 with result from task 133 parsl.dataflow.dflow DEBUG: Task 133 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/29bdc3b1-9e77-4273-8118-1ecc82846ff8_isrForCrosstalkSources_997011_30.stdout parsl.dataflow.dflow INFO: Standard error for task 133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/29bdc3b1-9e77-4273-8118-1ecc82846ff8_isrForCrosstalkSources_997011_30.stderr 2024-03-15 13:50:56 proc_lsst.multi:170 [INFO] [multi] job_id 17075061 proc_lsst.multi INFO: [multi] job_id 17075061 2024-03-15 13:50:56 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 25 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 25 2024-03-15 13:50:56 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:56 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075061 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075061 parsl.executors.status_handling DEBUG: Launched block 80 on executor multi with job ID 17075061 parsl.executors.status_handling INFO: Allocated block ID 81 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-81 2024-03-15 13:50:56 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-81 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-81 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:56 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 25 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 25 2024-03-15 13:50:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:56 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=81 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 136 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 647ad5b9bce1e28d2af0f243e5cfd7b2 with result from task 136 parsl.dataflow.dflow DEBUG: Task 136 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e20e2a1f-bec9-428c-b83e-df1322496f06_isrForCrosstalkSources_997030_30.stdout parsl.dataflow.dflow INFO: Standard error for task 136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e20e2a1f-bec9-428c-b83e-df1322496f06_isrForCrosstalkSources_997030_30.stderr 2024-03-15 13:50:57 proc_lsst.multi:170 [INFO] [multi] job_id 17075062 proc_lsst.multi INFO: [multi] job_id 17075062 2024-03-15 13:50:57 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 26 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 26 2024-03-15 13:50:57 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:57 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075062 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075062 parsl.executors.status_handling DEBUG: Launched block 81 on executor multi with job ID 17075062 parsl.executors.status_handling INFO: Allocated block ID 82 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-82 2024-03-15 13:50:57 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-82 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-82 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 26 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 26 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:57 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=82 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 137 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fb5ad27a8c0238565f34d1a221bb56b with result from task 137 parsl.dataflow.dflow DEBUG: Task 137 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/a02fa3a0-c388-4203-adeb-93f60b127369_isrForCrosstalkSources_996986_45.stdout parsl.dataflow.dflow INFO: Standard error for task 137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/a02fa3a0-c388-4203-adeb-93f60b127369_isrForCrosstalkSources_996986_45.stderr parsl.dataflow.dflow INFO: Task 147 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62cadbb1cf5b7bdfb567faaaac9d4b28 with result from task 147 parsl.dataflow.dflow DEBUG: Task 147 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6d091807-6efc-47fa-bbee-03ee1f42c416_isrForCrosstalkSources_996954_24.stdout parsl.dataflow.dflow INFO: Standard error for task 147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6d091807-6efc-47fa-bbee-03ee1f42c416_isrForCrosstalkSources_996954_24.stderr 2024-03-15 13:50:57 proc_lsst.multi:170 [INFO] [multi] job_id 17075063 proc_lsst.multi INFO: [multi] job_id 17075063 2024-03-15 13:50:57 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 27 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 27 2024-03-15 13:50:57 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:57 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075063 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075063 parsl.executors.status_handling DEBUG: Launched block 82 on executor multi with job ID 17075063 parsl.executors.status_handling INFO: Allocated block ID 83 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-83 2024-03-15 13:50:57 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-83 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-83 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:57 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 27 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 27 2024-03-15 13:50:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:57 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=83 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 134 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16cb31d47eacd0a1e97daf08784bca80 with result from task 134 parsl.dataflow.dflow DEBUG: Task 134 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c645529b-1d87-4ea8-8b0a-e76be9278e2d_isrForCrosstalkSources_997028_41.stdout parsl.dataflow.dflow INFO: Standard error for task 134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c645529b-1d87-4ea8-8b0a-e76be9278e2d_isrForCrosstalkSources_997028_41.stderr parsl.dataflow.dflow INFO: Task 140 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b44001936dca3645f2a37ad37f5b19ca with result from task 140 parsl.dataflow.dflow DEBUG: Task 140 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5b4d8ad4-77a6-4b6c-8323-76761489a847_isrForCrosstalkSources_997004_21.stdout parsl.dataflow.dflow INFO: Standard error for task 140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5b4d8ad4-77a6-4b6c-8323-76761489a847_isrForCrosstalkSources_997004_21.stderr 2024-03-15 13:50:58 proc_lsst.multi:170 [INFO] [multi] job_id 17075064 proc_lsst.multi INFO: [multi] job_id 17075064 2024-03-15 13:50:58 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 28 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 28 2024-03-15 13:50:58 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:58 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075064 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075064 parsl.executors.status_handling DEBUG: Launched block 83 on executor multi with job ID 17075064 parsl.executors.status_handling INFO: Allocated block ID 84 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-84 2024-03-15 13:50:58 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-84 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-84 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 28 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 28 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:58 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=84 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 146 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff6400bcaa1b87766f9522cf1e9223df with result from task 146 parsl.dataflow.dflow DEBUG: Task 146 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d1e3a898-fa97-4af6-8eda-606fd9a4ac1e_isrForCrosstalkSources_996981_15.stdout parsl.dataflow.dflow INFO: Standard error for task 146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d1e3a898-fa97-4af6-8eda-606fd9a4ac1e_isrForCrosstalkSources_996981_15.stderr parsl.dataflow.dflow INFO: Task 135 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ea6580294160ce3f87a204dbf554f79 with result from task 135 parsl.dataflow.dflow DEBUG: Task 135 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6fb54150-36fb-4395-9006-250fae2766da_isrForCrosstalkSources_996970_7.stdout parsl.dataflow.dflow INFO: Standard error for task 135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6fb54150-36fb-4395-9006-250fae2766da_isrForCrosstalkSources_996970_7.stderr 2024-03-15 13:50:58 proc_lsst.multi:170 [INFO] [multi] job_id 17075065 proc_lsst.multi INFO: [multi] job_id 17075065 2024-03-15 13:50:58 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 29 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 29 2024-03-15 13:50:58 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:58 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075065 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075065 parsl.executors.status_handling DEBUG: Launched block 84 on executor multi with job ID 17075065 parsl.executors.status_handling INFO: Allocated block ID 85 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-85 2024-03-15 13:50:58 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-85 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-85 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:58 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 29 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 29 2024-03-15 13:50:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:58 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=85 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 131 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7690c3080b987e708e4dc4bea8fd87da with result from task 131 parsl.dataflow.dflow DEBUG: Task 131 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/8ae25595-3d99-40f6-948d-80483b43fbbe_isrForCrosstalkSources_997001_6.stdout parsl.dataflow.dflow INFO: Standard error for task 131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/8ae25595-3d99-40f6-948d-80483b43fbbe_isrForCrosstalkSources_997001_6.stderr parsl.dataflow.dflow INFO: Task 68 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32506fa13caa0c33b29a0a45f1964549 with result from task 68 parsl.dataflow.dflow DEBUG: Task 68 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 68 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/640e5fd2-32e1-4010-a436-5facde608175_isrForCrosstalkSources_996989_52.stdout parsl.dataflow.dflow INFO: Standard error for task 68 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/640e5fd2-32e1-4010-a436-5facde608175_isrForCrosstalkSources_996989_52.stderr parsl.dataflow.dflow INFO: Task 139 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91cdaf77891e8e19906c577571b6b246 with result from task 139 parsl.dataflow.dflow DEBUG: Task 139 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/6f559da1-75aa-4fbe-960a-ca09e2c005ad_isrForCrosstalkSources_997027_53.stdout parsl.dataflow.dflow INFO: Standard error for task 139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/6f559da1-75aa-4fbe-960a-ca09e2c005ad_isrForCrosstalkSources_997027_53.stderr parsl.dataflow.dflow INFO: Task 144 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4932da250504f251deac864007f5754 with result from task 144 parsl.dataflow.dflow DEBUG: Task 144 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6f089fc4-fed1-42d4-acac-62c995c2ed38_isrForCrosstalkSources_997030_42.stdout parsl.dataflow.dflow INFO: Standard error for task 144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6f089fc4-fed1-42d4-acac-62c995c2ed38_isrForCrosstalkSources_997030_42.stderr parsl.dataflow.dflow INFO: Task 155 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b1de88b01e92ed3914fd861185c5b41 with result from task 155 parsl.dataflow.dflow DEBUG: Task 155 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4dfecde3-bc23-4e54-89d0-808dbc6a9180_isrForCrosstalkSources_997014_25.stdout parsl.dataflow.dflow INFO: Standard error for task 155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4dfecde3-bc23-4e54-89d0-808dbc6a9180_isrForCrosstalkSources_997014_25.stderr 2024-03-15 13:50:59 proc_lsst.multi:170 [INFO] [multi] job_id 17075066 proc_lsst.multi INFO: [multi] job_id 17075066 2024-03-15 13:50:59 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:59 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:59 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075066 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075066 parsl.executors.status_handling DEBUG: Launched block 85 on executor multi with job ID 17075066 parsl.executors.status_handling INFO: Allocated block ID 86 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-86 2024-03-15 13:50:59 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-86 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-86 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:59 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=86 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Task 138 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2232c367642f242dc113a32ce04e07cf with result from task 138 parsl.dataflow.dflow DEBUG: Task 138 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/8842e99a-dc56-44ab-b4b0-d18e7bff5e3d_isrForCrosstalkSources_997026_28.stdout parsl.dataflow.dflow INFO: Standard error for task 138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/8842e99a-dc56-44ab-b4b0-d18e7bff5e3d_isrForCrosstalkSources_997026_28.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 141 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f54e356a3c8d3b30d873a716978c209 with result from task 141 parsl.dataflow.dflow DEBUG: Task 141 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2f63142-3bf6-4932-a1f6-8ddb9bf7b9f3_isrForCrosstalkSources_996996_30.stdout parsl.dataflow.dflow INFO: Standard error for task 141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2f63142-3bf6-4932-a1f6-8ddb9bf7b9f3_isrForCrosstalkSources_996996_30.stderr parsl.dataflow.dflow INFO: Task 160 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3947926d625e7a0d46324a17d2c2f252 with result from task 160 parsl.dataflow.dflow DEBUG: Task 160 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/72281006-32e4-4a47-afb2-8a1e2bd7b200_isrForCrosstalkSources_997023_25.stdout parsl.dataflow.dflow INFO: Standard error for task 160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/72281006-32e4-4a47-afb2-8a1e2bd7b200_isrForCrosstalkSources_997023_25.stderr parsl.dataflow.dflow INFO: Task 130 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae96883ada8d87d9a5eb1a763495e625 with result from task 130 parsl.dataflow.dflow DEBUG: Task 130 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/8f474223-1f82-4826-882d-b9a1159e4239_isrForCrosstalkSources_996988_11.stdout parsl.dataflow.dflow INFO: Standard error for task 130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/8f474223-1f82-4826-882d-b9a1159e4239_isrForCrosstalkSources_996988_11.stderr parsl.dataflow.dflow INFO: Task 153 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a80fcedb89f95b5d8551f02e3d32139a with result from task 153 parsl.dataflow.dflow DEBUG: Task 153 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/98cdd415-559d-427d-b2bf-7cccd22af008_isrForCrosstalkSources_996957_20.stdout parsl.dataflow.dflow INFO: Standard error for task 153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/98cdd415-559d-427d-b2bf-7cccd22af008_isrForCrosstalkSources_996957_20.stderr 2024-03-15 13:50:59 proc_lsst.multi:170 [INFO] [multi] job_id 17075068 proc_lsst.multi INFO: [multi] job_id 17075068 2024-03-15 13:50:59 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 31 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 31 2024-03-15 13:50:59 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:59 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075068 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075068 parsl.executors.status_handling DEBUG: Launched block 86 on executor multi with job ID 17075068 parsl.executors.status_handling INFO: Allocated block ID 87 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-87 2024-03-15 13:50:59 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-87 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-87 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 31 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 31 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:59 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=87 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 157 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a93608426730b2f2398031ac83dfcb4 with result from task 157 parsl.dataflow.dflow DEBUG: Task 157 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6c125359-a2ee-4c71-91a7-8f558060209a_isrForCrosstalkSources_996991_27.stdout parsl.dataflow.dflow INFO: Standard error for task 157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6c125359-a2ee-4c71-91a7-8f558060209a_isrForCrosstalkSources_996991_27.stderr 2024-03-15 13:50:59 proc_lsst.multi:170 [INFO] [multi] job_id 17075069 proc_lsst.multi INFO: [multi] job_id 17075069 2024-03-15 13:50:59 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 32 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 32 2024-03-15 13:50:59 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:59 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075069 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075069 parsl.executors.status_handling DEBUG: Launched block 87 on executor multi with job ID 17075069 parsl.executors.status_handling INFO: Allocated block ID 88 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-88 2024-03-15 13:50:59 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-88 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-88 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:50:59 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:50:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 32 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 32 2024-03-15 13:50:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:50:59 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=88 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 167 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 36d699eadf9dd43631e0d1d41a1ef365 with result from task 167 parsl.dataflow.dflow DEBUG: Task 167 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/b8492d18-1513-453a-8998-11171d6c5a77_isrForCrosstalkSources_997010_32.stdout parsl.dataflow.dflow INFO: Standard error for task 167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/b8492d18-1513-453a-8998-11171d6c5a77_isrForCrosstalkSources_997010_32.stderr 2024-03-15 13:51:00 proc_lsst.multi:170 [INFO] [multi] job_id 17075072 proc_lsst.multi INFO: [multi] job_id 17075072 2024-03-15 13:51:00 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 33 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 33 2024-03-15 13:51:00 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:00 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075072 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075072 parsl.executors.status_handling DEBUG: Launched block 88 on executor multi with job ID 17075072 parsl.executors.status_handling INFO: Allocated block ID 89 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-89 2024-03-15 13:51:00 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-89 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-89 2024-03-15 13:51:00 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:00 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:00 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:00 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:00 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 33 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 33 2024-03-15 13:51:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:00 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=89 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 152 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4175e819d1c4b1ac7a0cd4bb6aa75bcc with result from task 152 parsl.dataflow.dflow DEBUG: Task 152 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/691bd4be-0e78-44b0-8cd0-a404553b5e77_isrForCrosstalkSources_996979_50.stdout parsl.dataflow.dflow INFO: Standard error for task 152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/691bd4be-0e78-44b0-8cd0-a404553b5e77_isrForCrosstalkSources_996979_50.stderr parsl.dataflow.dflow INFO: Task 168 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f5f91b8238a5e5ffe2bc9d132b93c5c with result from task 168 parsl.dataflow.dflow DEBUG: Task 168 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ae288daa-28b0-45ed-92c2-200d38e254e2_isrForCrosstalkSources_997001_11.stdout parsl.dataflow.dflow INFO: Standard error for task 168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ae288daa-28b0-45ed-92c2-200d38e254e2_isrForCrosstalkSources_997001_11.stderr 2024-03-15 13:51:01 proc_lsst.multi:170 [INFO] [multi] job_id 17075073 proc_lsst.multi INFO: [multi] job_id 17075073 2024-03-15 13:51:01 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 34 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 34 2024-03-15 13:51:01 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:01 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075073 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075073 parsl.executors.status_handling DEBUG: Launched block 89 on executor multi with job ID 17075073 parsl.executors.status_handling INFO: Allocated block ID 90 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-90 2024-03-15 13:51:01 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-90 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-90 2024-03-15 13:51:01 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:01 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:01 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:01 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:01 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 34 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 34 2024-03-15 13:51:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:01 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=90 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 162 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 98848b0cf8b685837a51472cf168f4eb with result from task 162 parsl.dataflow.dflow DEBUG: Task 162 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/91194d3f-ab3a-4c7c-b15d-8ef43dd0a053_isrForCrosstalkSources_997007_47.stdout parsl.dataflow.dflow INFO: Standard error for task 162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/91194d3f-ab3a-4c7c-b15d-8ef43dd0a053_isrForCrosstalkSources_997007_47.stderr parsl.dataflow.dflow INFO: Task 165 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d76c3f7062eb12f5a7881b4b6368958 with result from task 165 parsl.dataflow.dflow DEBUG: Task 165 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/25cec907-7085-4326-bfdc-d3044413eb32_isrForCrosstalkSources_997001_44.stdout parsl.dataflow.dflow INFO: Standard error for task 165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/25cec907-7085-4326-bfdc-d3044413eb32_isrForCrosstalkSources_997001_44.stderr 2024-03-15 13:51:02 proc_lsst.multi:170 [INFO] [multi] job_id 17075074 proc_lsst.multi INFO: [multi] job_id 17075074 2024-03-15 13:51:02 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 35 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 35 2024-03-15 13:51:02 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:02 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075074 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075074 parsl.executors.status_handling DEBUG: Launched block 90 on executor multi with job ID 17075074 parsl.executors.status_handling INFO: Allocated block ID 91 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-91 2024-03-15 13:51:02 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-91 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-91 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 35 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 35 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:02 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=91 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 159 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf1839026f6ab4d26db4f541dc64b9ae with result from task 159 parsl.dataflow.dflow DEBUG: Task 159 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a42815b2-f9ef-4c90-98e8-89219df51fa5_isrForCrosstalkSources_996960_61.stdout parsl.dataflow.dflow INFO: Standard error for task 159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a42815b2-f9ef-4c90-98e8-89219df51fa5_isrForCrosstalkSources_996960_61.stderr 2024-03-15 13:51:02 proc_lsst.multi:170 [INFO] [multi] job_id 17075075 proc_lsst.multi INFO: [multi] job_id 17075075 2024-03-15 13:51:02 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 36 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 36 2024-03-15 13:51:02 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:02 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075075 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075075 parsl.executors.status_handling DEBUG: Launched block 91 on executor multi with job ID 17075075 parsl.executors.status_handling INFO: Allocated block ID 92 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-92 2024-03-15 13:51:02 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-92 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-92 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:02 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 36 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 36 2024-03-15 13:51:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:02 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=92 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:03 proc_lsst.multi:170 [INFO] [multi] job_id 17075076 proc_lsst.multi INFO: [multi] job_id 17075076 2024-03-15 13:51:03 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 37 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 37 2024-03-15 13:51:03 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:03 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075076 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075076 parsl.executors.status_handling DEBUG: Launched block 92 on executor multi with job ID 17075076 parsl.executors.status_handling INFO: Allocated block ID 93 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-93 2024-03-15 13:51:03 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-93 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-93 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 37 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 37 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:03 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=93 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:03 proc_lsst.multi:170 [INFO] [multi] job_id 17075078 proc_lsst.multi INFO: [multi] job_id 17075078 2024-03-15 13:51:03 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 38 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 38 2024-03-15 13:51:03 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:03 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075078 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075078 parsl.executors.status_handling DEBUG: Launched block 93 on executor multi with job ID 17075078 parsl.executors.status_handling INFO: Allocated block ID 94 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-94 2024-03-15 13:51:03 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-94 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-94 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:03 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 38 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 38 2024-03-15 13:51:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:03 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=94 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 182 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5aeabe9d252ed917fc60cfe81a84807 with result from task 182 parsl.dataflow.dflow DEBUG: Task 182 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/21d6cd6f-ca52-455d-8430-24406bb059f4_isrForCrosstalkSources_996979_13.stdout parsl.dataflow.dflow INFO: Standard error for task 182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/21d6cd6f-ca52-455d-8430-24406bb059f4_isrForCrosstalkSources_996979_13.stderr 2024-03-15 13:51:04 proc_lsst.multi:170 [INFO] [multi] job_id 17075080 proc_lsst.multi INFO: [multi] job_id 17075080 2024-03-15 13:51:04 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 39 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 39 2024-03-15 13:51:04 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:04 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075080 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075080 parsl.executors.status_handling DEBUG: Launched block 94 on executor multi with job ID 17075080 parsl.executors.status_handling INFO: Allocated block ID 95 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-95 2024-03-15 13:51:04 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-95 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-95 2024-03-15 13:51:04 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:04 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:04 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:04 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:04 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 39 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 39 2024-03-15 13:51:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:04 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=95 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 183 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9de42ceeec49018997bf3443797eab80 with result from task 183 parsl.dataflow.dflow DEBUG: Task 183 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/663042e3-d7a9-447a-93b0-9616d248e102_isrForCrosstalkSources_996971_18.stdout parsl.dataflow.dflow INFO: Standard error for task 183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/663042e3-d7a9-447a-93b0-9616d248e102_isrForCrosstalkSources_996971_18.stderr 2024-03-15 13:51:05 proc_lsst.multi:170 [INFO] [multi] job_id 17075081 proc_lsst.multi INFO: [multi] job_id 17075081 2024-03-15 13:51:05 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 40 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 40 2024-03-15 13:51:05 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:05 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075081 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075081 parsl.executors.status_handling DEBUG: Launched block 95 on executor multi with job ID 17075081 parsl.executors.status_handling INFO: Allocated block ID 96 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-96 2024-03-15 13:51:05 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-96 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-96 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 40 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 40 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:05 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=96 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 172 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b26fa66067e887989cca6efbd42d655c with result from task 172 parsl.dataflow.dflow DEBUG: Task 172 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ba629af3-22c5-41b1-8a52-7faa9e1a76cc_isrForCrosstalkSources_996989_24.stdout parsl.dataflow.dflow INFO: Standard error for task 172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ba629af3-22c5-41b1-8a52-7faa9e1a76cc_isrForCrosstalkSources_996989_24.stderr 2024-03-15 13:51:05 proc_lsst.multi:170 [INFO] [multi] job_id 17075082 proc_lsst.multi INFO: [multi] job_id 17075082 2024-03-15 13:51:05 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 41 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 41 2024-03-15 13:51:05 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:05 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075082 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075082 parsl.executors.status_handling DEBUG: Launched block 96 on executor multi with job ID 17075082 parsl.executors.status_handling INFO: Allocated block ID 97 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-97 2024-03-15 13:51:05 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-97 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-97 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:05 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 41 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 41 2024-03-15 13:51:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:05 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=97 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 190 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 453ada7ac66350dd46169a41a8da9e63 with result from task 190 parsl.dataflow.dflow DEBUG: Task 190 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/ff50f9a6-5a45-4588-8ddf-a7456ecfd891_isrForCrosstalkSources_996964_11.stdout parsl.dataflow.dflow INFO: Standard error for task 190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/ff50f9a6-5a45-4588-8ddf-a7456ecfd891_isrForCrosstalkSources_996964_11.stderr 2024-03-15 13:51:06 proc_lsst.multi:170 [INFO] [multi] job_id 17075083 proc_lsst.multi INFO: [multi] job_id 17075083 2024-03-15 13:51:06 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 42 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 42 2024-03-15 13:51:06 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:06 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075083 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075083 parsl.executors.status_handling DEBUG: Launched block 97 on executor multi with job ID 17075083 parsl.executors.status_handling INFO: Allocated block ID 98 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-98 2024-03-15 13:51:06 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-98 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-98 2024-03-15 13:51:06 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:06 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:06 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:06 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:06 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 42 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 42 2024-03-15 13:51:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:06 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=98 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 186 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a0e8a0d3c1b960efa0584d34f5bf6e9f with result from task 186 parsl.dataflow.dflow DEBUG: Task 186 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/98d002a0-ad9e-40a7-b405-f459b42fff9d_isrForCrosstalkSources_996972_42.stdout parsl.dataflow.dflow INFO: Standard error for task 186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/98d002a0-ad9e-40a7-b405-f459b42fff9d_isrForCrosstalkSources_996972_42.stderr parsl.dataflow.dflow INFO: Task 185 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb9c77c88a1cda83554bc0b8633e2f5a with result from task 185 parsl.dataflow.dflow DEBUG: Task 185 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c66e81af-6767-4964-a8a1-7e2198a539a5_isrForCrosstalkSources_997009_44.stdout parsl.dataflow.dflow INFO: Standard error for task 185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c66e81af-6767-4964-a8a1-7e2198a539a5_isrForCrosstalkSources_997009_44.stderr parsl.dataflow.dflow INFO: Task 192 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da0ec276c5a7e0fd2338f77b93e80166 with result from task 192 parsl.dataflow.dflow DEBUG: Task 192 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/824ec742-e9a7-482a-8837-6030989ac3ba_isrForCrosstalkSources_997009_53.stdout parsl.dataflow.dflow INFO: Standard error for task 192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/824ec742-e9a7-482a-8837-6030989ac3ba_isrForCrosstalkSources_997009_53.stderr 2024-03-15 13:51:07 proc_lsst.multi:170 [INFO] [multi] job_id 17075085 proc_lsst.multi INFO: [multi] job_id 17075085 2024-03-15 13:51:07 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 43 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 43 2024-03-15 13:51:07 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:07 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075085 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075085 parsl.executors.status_handling DEBUG: Launched block 98 on executor multi with job ID 17075085 parsl.executors.status_handling INFO: Allocated block ID 99 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-99 2024-03-15 13:51:07 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-99 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-99 2024-03-15 13:51:07 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:07 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:07 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:07 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:07 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 43 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 43 2024-03-15 13:51:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:07 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=99 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 199 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 28a2cbf5abc6d5fbc460adb0c96619e9 with result from task 199 parsl.dataflow.dflow DEBUG: Task 199 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/00ea58bd-c629-4c43-92ff-bc0b4cf9a346_isrForCrosstalkSources_996955_40.stdout parsl.dataflow.dflow INFO: Standard error for task 199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/00ea58bd-c629-4c43-92ff-bc0b4cf9a346_isrForCrosstalkSources_996955_40.stderr parsl.dataflow.dflow INFO: Task 206 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0399c2fae9a830a309dc1ae0e45141b with result from task 206 parsl.dataflow.dflow DEBUG: Task 206 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b122600e-48f2-4f40-89de-4dea222b8b69_isrForCrosstalkSources_997014_26.stdout parsl.dataflow.dflow INFO: Standard error for task 206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b122600e-48f2-4f40-89de-4dea222b8b69_isrForCrosstalkSources_997014_26.stderr parsl.dataflow.dflow INFO: Task 178 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 77a3bdc70711847a73b60d055674d111 with result from task 178 parsl.dataflow.dflow DEBUG: Task 178 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cbc9d2ac-f7cb-4e7f-bd88-cfc94e2c908a_isrForCrosstalkSources_996987_25.stdout parsl.dataflow.dflow INFO: Standard error for task 178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cbc9d2ac-f7cb-4e7f-bd88-cfc94e2c908a_isrForCrosstalkSources_996987_25.stderr parsl.dataflow.dflow INFO: Task 176 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da3adf4370de9d4d2043de243158dfcf with result from task 176 parsl.dataflow.dflow DEBUG: Task 176 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/36374751-9c82-44d4-bec2-a000b959ddee_isrForCrosstalkSources_997013_33.stdout parsl.dataflow.dflow INFO: Standard error for task 176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/36374751-9c82-44d4-bec2-a000b959ddee_isrForCrosstalkSources_997013_33.stderr parsl.dataflow.dflow INFO: Task 202 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7011f325e19ea4eaa6401b49a35a48e3 with result from task 202 parsl.dataflow.dflow DEBUG: Task 202 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5bf3606d-8cb9-4807-88e4-f1e83489c52d_isrForCrosstalkSources_996978_1.stdout parsl.dataflow.dflow INFO: Standard error for task 202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5bf3606d-8cb9-4807-88e4-f1e83489c52d_isrForCrosstalkSources_996978_1.stderr parsl.dataflow.dflow INFO: Task 193 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 615ebfae913549c19d3a579998ab8bb3 with result from task 193 parsl.dataflow.dflow DEBUG: Task 193 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/0fa42b75-b50a-4215-bbaf-88d67312f47e_isrForCrosstalkSources_996968_34.stdout parsl.dataflow.dflow INFO: Standard error for task 193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/0fa42b75-b50a-4215-bbaf-88d67312f47e_isrForCrosstalkSources_996968_34.stderr parsl.dataflow.dflow INFO: Task 158 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d261534fb99ddfd9698a1efe871f92a1 with result from task 158 parsl.dataflow.dflow DEBUG: Task 158 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a6b3769d-308e-4a89-89e8-c3c408d52fd7_isrForCrosstalkSources_997028_25.stdout parsl.dataflow.dflow INFO: Standard error for task 158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a6b3769d-308e-4a89-89e8-c3c408d52fd7_isrForCrosstalkSources_997028_25.stderr parsl.dataflow.dflow INFO: Task 154 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b61aec4126b43017be9ef39847cdb12 with result from task 154 parsl.dataflow.dflow DEBUG: Task 154 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/052b7fb1-2b82-4b49-b08b-3766999ea9dc_isrForCrosstalkSources_997010_4.stdout parsl.dataflow.dflow INFO: Standard error for task 154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/052b7fb1-2b82-4b49-b08b-3766999ea9dc_isrForCrosstalkSources_997010_4.stderr 2024-03-15 13:51:08 proc_lsst.multi:170 [INFO] [multi] job_id 17075086 proc_lsst.multi INFO: [multi] job_id 17075086 2024-03-15 13:51:08 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 44 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 44 2024-03-15 13:51:08 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:08 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075086 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075086 parsl.executors.status_handling DEBUG: Launched block 99 on executor multi with job ID 17075086 parsl.executors.status_handling INFO: Allocated block ID 100 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-100 2024-03-15 13:51:08 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-100 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-100 2024-03-15 13:51:08 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:08 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:08 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:08 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:08 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 44 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 44 2024-03-15 13:51:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:08 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=100 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 187 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ffa87f5b02062890f2d85a6c54f4951 with result from task 187 parsl.dataflow.dflow DEBUG: Task 187 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/1e909c76-4b95-40d0-83f4-2347c5aa7e78_isrForCrosstalkSources_996978_26.stdout parsl.dataflow.dflow INFO: Standard error for task 187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/1e909c76-4b95-40d0-83f4-2347c5aa7e78_isrForCrosstalkSources_996978_26.stderr parsl.dataflow.dflow INFO: Task 142 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b96a2e3fab51e7e1507ecdd35da20f58 with result from task 142 parsl.dataflow.dflow DEBUG: Task 142 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/4992e519-b6ad-4351-865f-2aaf8f246934_isrForCrosstalkSources_997011_19.stdout parsl.dataflow.dflow INFO: Standard error for task 142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/4992e519-b6ad-4351-865f-2aaf8f246934_isrForCrosstalkSources_997011_19.stderr parsl.dataflow.dflow INFO: Task 148 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 719e1e3d8e8f96354cb688a7a48ad157 with result from task 148 parsl.dataflow.dflow DEBUG: Task 148 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cd606719-65e8-4950-b601-2ce5de073951_isrForCrosstalkSources_996988_34.stdout parsl.dataflow.dflow INFO: Standard error for task 148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cd606719-65e8-4950-b601-2ce5de073951_isrForCrosstalkSources_996988_34.stderr parsl.dataflow.dflow INFO: Task 150 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e03e0cc7c660eaeb8a1cea44766a1f1 with result from task 150 parsl.dataflow.dflow DEBUG: Task 150 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0f10034e-739c-481a-bf0b-f8002daa4336_isrForCrosstalkSources_996966_19.stdout parsl.dataflow.dflow INFO: Standard error for task 150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0f10034e-739c-481a-bf0b-f8002daa4336_isrForCrosstalkSources_996966_19.stderr parsl.dataflow.dflow INFO: Task 170 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 343637131db0fbb3301c2f9518f8dbe6 with result from task 170 parsl.dataflow.dflow DEBUG: Task 170 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/6ff0f0a0-d431-4b71-8d4d-f9c9dfada8d9_isrForCrosstalkSources_997032_23.stdout parsl.dataflow.dflow INFO: Standard error for task 170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/6ff0f0a0-d431-4b71-8d4d-f9c9dfada8d9_isrForCrosstalkSources_997032_23.stderr parsl.dataflow.dflow INFO: Task 177 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6cf8a73b6e376a4995f5c2b097e6481e with result from task 177 parsl.dataflow.dflow DEBUG: Task 177 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/94b04152-18e3-43c1-90bc-228e5dfd667e_isrForCrosstalkSources_997000_4.stdout parsl.dataflow.dflow INFO: Standard error for task 177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/94b04152-18e3-43c1-90bc-228e5dfd667e_isrForCrosstalkSources_997000_4.stderr parsl.dataflow.dflow INFO: Task 198 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7071e299ad74ea1780de0524a1257664 with result from task 198 parsl.dataflow.dflow DEBUG: Task 198 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/38f5b633-aded-4239-aaf0-ef076e45f384_isrForCrosstalkSources_996977_17.stdout parsl.dataflow.dflow INFO: Standard error for task 198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/38f5b633-aded-4239-aaf0-ef076e45f384_isrForCrosstalkSources_996977_17.stderr parsl.dataflow.dflow INFO: Task 164 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57ae08323e82a36bcadf0741c10b3e8f with result from task 164 parsl.dataflow.dflow DEBUG: Task 164 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6c7c978c-9fb5-4d4e-a3ae-e5e8c23e99d4_isrForCrosstalkSources_997015_14.stdout parsl.dataflow.dflow INFO: Standard error for task 164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6c7c978c-9fb5-4d4e-a3ae-e5e8c23e99d4_isrForCrosstalkSources_997015_14.stderr parsl.dataflow.dflow INFO: Task 173 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 580765732597dcc6eb3106dca326611e with result from task 173 parsl.dataflow.dflow DEBUG: Task 173 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0fb71618-4ad0-43c2-8ab6-cb9281d8e677_isrForCrosstalkSources_996981_46.stdout parsl.dataflow.dflow INFO: Standard error for task 173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0fb71618-4ad0-43c2-8ab6-cb9281d8e677_isrForCrosstalkSources_996981_46.stderr 2024-03-15 13:51:09 proc_lsst.multi:170 [INFO] [multi] job_id 17075088 proc_lsst.multi INFO: [multi] job_id 17075088 2024-03-15 13:51:09 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 45 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 45 2024-03-15 13:51:09 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:09 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075088 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075088 parsl.executors.status_handling DEBUG: Launched block 100 on executor multi with job ID 17075088 parsl.executors.status_handling INFO: Allocated block ID 101 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-101 2024-03-15 13:51:09 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-101 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-101 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 45 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 45 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:09 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=101 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 161 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e9db247aca2dc3f7049e1ceef78f76f with result from task 161 parsl.dataflow.dflow DEBUG: Task 161 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a75d7578-4a6c-47f5-9d3a-567514e54484_isrForCrosstalkSources_997020_17.stdout parsl.dataflow.dflow INFO: Standard error for task 161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a75d7578-4a6c-47f5-9d3a-567514e54484_isrForCrosstalkSources_997020_17.stderr parsl.dataflow.dflow INFO: Task 201 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4cff44260fa0472e3fb4b27a966888eb with result from task 201 parsl.dataflow.dflow DEBUG: Task 201 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/a61d530b-1df7-4b3a-9b10-50027d302b76_isrForCrosstalkSources_996955_58.stdout parsl.dataflow.dflow INFO: Standard error for task 201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/a61d530b-1df7-4b3a-9b10-50027d302b76_isrForCrosstalkSources_996955_58.stderr parsl.dataflow.dflow INFO: Task 205 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca610664c20339218c0b176dcc00bc56 with result from task 205 parsl.dataflow.dflow DEBUG: Task 205 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d6923b37-1691-492a-8e83-14cb32163c0e_isrForCrosstalkSources_997010_2.stdout parsl.dataflow.dflow INFO: Standard error for task 205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d6923b37-1691-492a-8e83-14cb32163c0e_isrForCrosstalkSources_997010_2.stderr parsl.dataflow.dflow INFO: Task 163 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16e9e94051bc6a64ed2a2c88ffcd74d2 with result from task 163 parsl.dataflow.dflow DEBUG: Task 163 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c9466315-984f-4244-97e9-84a96678f209_isrForCrosstalkSources_997004_46.stdout parsl.dataflow.dflow INFO: Standard error for task 163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c9466315-984f-4244-97e9-84a96678f209_isrForCrosstalkSources_997004_46.stderr 2024-03-15 13:51:09 proc_lsst.multi:170 [INFO] [multi] job_id 17075090 proc_lsst.multi INFO: [multi] job_id 17075090 2024-03-15 13:51:09 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 46 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 46 2024-03-15 13:51:09 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:09 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075090 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075090 parsl.executors.status_handling DEBUG: Launched block 101 on executor multi with job ID 17075090 parsl.executors.status_handling INFO: Allocated block ID 102 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-102 2024-03-15 13:51:09 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-102 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-102 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:09 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 46 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 46 2024-03-15 13:51:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:09 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=102 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 189 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 675f24d4e504f58a42a11569e9034717 with result from task 189 parsl.dataflow.dflow DEBUG: Task 189 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8e23b2b4-0650-4fd9-a434-28c369740624_isrForCrosstalkSources_996992_61.stdout parsl.dataflow.dflow INFO: Standard error for task 189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8e23b2b4-0650-4fd9-a434-28c369740624_isrForCrosstalkSources_996992_61.stderr parsl.dataflow.dflow INFO: Task 143 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac351d43ca23f5603f8e21a0722837ae with result from task 143 parsl.dataflow.dflow DEBUG: Task 143 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/de93b5d7-484a-4ac0-ad41-3c65bdf9ada6_isrForCrosstalkSources_997010_12.stdout parsl.dataflow.dflow INFO: Standard error for task 143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/de93b5d7-484a-4ac0-ad41-3c65bdf9ada6_isrForCrosstalkSources_997010_12.stderr parsl.dataflow.dflow INFO: Task 194 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b19bb335261d3311069dce4eb5b1bea3 with result from task 194 parsl.dataflow.dflow DEBUG: Task 194 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b5c0b3ae-f330-430a-915f-08894805ecbc_isrForCrosstalkSources_996979_32.stdout parsl.dataflow.dflow INFO: Standard error for task 194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b5c0b3ae-f330-430a-915f-08894805ecbc_isrForCrosstalkSources_996979_32.stderr parsl.dataflow.dflow INFO: Task 191 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99e40de344335f66aa08fcfa0e18dcdd with result from task 191 parsl.dataflow.dflow DEBUG: Task 191 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/57d48a80-211b-4a83-aa5b-b2534effb0f8_isrForCrosstalkSources_996958_40.stdout parsl.dataflow.dflow INFO: Standard error for task 191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/57d48a80-211b-4a83-aa5b-b2534effb0f8_isrForCrosstalkSources_996958_40.stderr parsl.dataflow.dflow INFO: Task 188 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 174d3fadf13e9aa4edc30e6d6d985c2b with result from task 188 parsl.dataflow.dflow DEBUG: Task 188 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/907a24b9-edb6-48c2-800e-1a80edeb3ed8_isrForCrosstalkSources_997023_37.stdout parsl.dataflow.dflow INFO: Standard error for task 188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/907a24b9-edb6-48c2-800e-1a80edeb3ed8_isrForCrosstalkSources_997023_37.stderr 2024-03-15 13:51:10 proc_lsst.multi:170 [INFO] [multi] job_id 17075091 proc_lsst.multi INFO: [multi] job_id 17075091 2024-03-15 13:51:10 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 47 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 47 2024-03-15 13:51:10 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:10 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075091 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075091 parsl.executors.status_handling DEBUG: Launched block 102 on executor multi with job ID 17075091 parsl.executors.status_handling INFO: Allocated block ID 103 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-103 2024-03-15 13:51:10 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-103 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-103 2024-03-15 13:51:10 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:10 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:10 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:10 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:10 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 47 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 47 2024-03-15 13:51:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:10 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=103 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 208 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e06632dfc2263f146b2781953db204e with result from task 208 parsl.dataflow.dflow DEBUG: Task 208 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c02b27ec-960e-4d5d-a1c6-f7894f011a86_isrForCrosstalkSources_997009_27.stdout parsl.dataflow.dflow INFO: Standard error for task 208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c02b27ec-960e-4d5d-a1c6-f7894f011a86_isrForCrosstalkSources_997009_27.stderr parsl.dataflow.dflow INFO: Task 156 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9339e53f034261383ac5922e5ef18e85 with result from task 156 parsl.dataflow.dflow DEBUG: Task 156 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ffef8d4f-a8ef-4ef0-b3d3-a8a260f90c76_isrForCrosstalkSources_996992_12.stdout parsl.dataflow.dflow INFO: Standard error for task 156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ffef8d4f-a8ef-4ef0-b3d3-a8a260f90c76_isrForCrosstalkSources_996992_12.stderr 2024-03-15 13:51:11 proc_lsst.multi:170 [INFO] [multi] job_id 17075092 proc_lsst.multi INFO: [multi] job_id 17075092 2024-03-15 13:51:11 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 48 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 48 2024-03-15 13:51:11 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:11 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075092 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075092 parsl.executors.status_handling DEBUG: Launched block 103 on executor multi with job ID 17075092 parsl.executors.status_handling INFO: Allocated block ID 104 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-104 2024-03-15 13:51:11 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-104 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-104 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 48 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 48 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:11 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=104 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 149 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 089da2324b153f0141e2c2afef6f8aa0 with result from task 149 parsl.dataflow.dflow DEBUG: Task 149 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/06e39138-0364-4976-b42e-1cd1e3fdb857_isrForCrosstalkSources_997019_6.stdout parsl.dataflow.dflow INFO: Standard error for task 149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/06e39138-0364-4976-b42e-1cd1e3fdb857_isrForCrosstalkSources_997019_6.stderr parsl.dataflow.dflow INFO: Task 195 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf3978ec6b9bdc02ce1bc529a59428fb with result from task 195 parsl.dataflow.dflow DEBUG: Task 195 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/8222bb63-d060-434c-ae88-0fd195716ab9_isrForCrosstalkSources_997016_23.stdout parsl.dataflow.dflow INFO: Standard error for task 195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/8222bb63-d060-434c-ae88-0fd195716ab9_isrForCrosstalkSources_997016_23.stderr parsl.dataflow.dflow INFO: Task 145 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4896f1712bc9b730a48da4ac455bd601 with result from task 145 parsl.dataflow.dflow DEBUG: Task 145 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/df91b8a8-4823-4d77-a7f0-c1b884e4180c_isrForCrosstalkSources_996977_1.stdout parsl.dataflow.dflow INFO: Standard error for task 145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/df91b8a8-4823-4d77-a7f0-c1b884e4180c_isrForCrosstalkSources_996977_1.stderr parsl.dataflow.dflow INFO: Task 184 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59be94b79ac548efbaaee7e1f9630698 with result from task 184 parsl.dataflow.dflow DEBUG: Task 184 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/698c1247-3c49-4cc8-9185-20585f921e92_isrForCrosstalkSources_997011_29.stdout parsl.dataflow.dflow INFO: Standard error for task 184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/698c1247-3c49-4cc8-9185-20585f921e92_isrForCrosstalkSources_997011_29.stderr 2024-03-15 13:51:11 proc_lsst.multi:170 [INFO] [multi] job_id 17075093 proc_lsst.multi INFO: [multi] job_id 17075093 2024-03-15 13:51:11 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 49 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 49 2024-03-15 13:51:11 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:11 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075093 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075093 parsl.executors.status_handling DEBUG: Launched block 104 on executor multi with job ID 17075093 parsl.executors.status_handling INFO: Allocated block ID 105 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-105 2024-03-15 13:51:11 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-105 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-105 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:11 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 49 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 49 2024-03-15 13:51:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:11 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=105 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 174 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55b57fa48e7e84c2923b4213d9bb8807 with result from task 174 parsl.dataflow.dflow DEBUG: Task 174 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1f916f9f-f0d5-47d1-b081-1b5ce79545fa_isrForCrosstalkSources_996961_37.stdout parsl.dataflow.dflow INFO: Standard error for task 174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1f916f9f-f0d5-47d1-b081-1b5ce79545fa_isrForCrosstalkSources_996961_37.stderr parsl.dataflow.dflow INFO: Task 151 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 891f9dc6e928a38c9ed8c4d61499d03a with result from task 151 parsl.dataflow.dflow DEBUG: Task 151 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/68b8b0d5-3f3a-4a33-b3f5-02e4118e3259_isrForCrosstalkSources_997017_58.stdout parsl.dataflow.dflow INFO: Standard error for task 151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/68b8b0d5-3f3a-4a33-b3f5-02e4118e3259_isrForCrosstalkSources_997017_58.stderr 2024-03-15 13:51:12 proc_lsst.multi:170 [INFO] [multi] job_id 17075094 proc_lsst.multi INFO: [multi] job_id 17075094 2024-03-15 13:51:12 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 50 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 50 2024-03-15 13:51:12 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:12 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075094 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075094 parsl.executors.status_handling DEBUG: Launched block 105 on executor multi with job ID 17075094 parsl.executors.status_handling INFO: Allocated block ID 106 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-106 2024-03-15 13:51:12 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-106 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-106 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 50 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 50 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:12 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=106 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 196 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a58a24be9478df45b649ee9f74308dfc with result from task 196 parsl.dataflow.dflow DEBUG: Task 196 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/861eed98-0eda-4f4e-bcd5-fbb28ca90ab9_isrForCrosstalkSources_997001_41.stdout parsl.dataflow.dflow INFO: Standard error for task 196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/861eed98-0eda-4f4e-bcd5-fbb28ca90ab9_isrForCrosstalkSources_997001_41.stderr parsl.dataflow.dflow INFO: Task 166 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dc81ef507d756b2185fd0bc0efb10b1f with result from task 166 parsl.dataflow.dflow DEBUG: Task 166 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4f6f9ac1-71f7-4a2e-8ec8-4e2cedcb32ee_isrForCrosstalkSources_996957_41.stdout parsl.dataflow.dflow INFO: Standard error for task 166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4f6f9ac1-71f7-4a2e-8ec8-4e2cedcb32ee_isrForCrosstalkSources_996957_41.stderr parsl.dataflow.dflow INFO: Task 210 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d29d0aff63cd9712beb3c5e7629e4de6 with result from task 210 parsl.dataflow.dflow DEBUG: Task 210 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9baccfe5-a714-4d2c-ad41-e673fcc31c24_isrForCrosstalkSources_997030_15.stdout parsl.dataflow.dflow INFO: Standard error for task 210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9baccfe5-a714-4d2c-ad41-e673fcc31c24_isrForCrosstalkSources_997030_15.stderr parsl.dataflow.dflow INFO: Task 171 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4226ca2739d84620f9c72cb7e20ee11e with result from task 171 parsl.dataflow.dflow DEBUG: Task 171 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a92e28f5-8e86-4eab-afe5-9b14c5788efc_isrForCrosstalkSources_996991_36.stdout parsl.dataflow.dflow INFO: Standard error for task 171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a92e28f5-8e86-4eab-afe5-9b14c5788efc_isrForCrosstalkSources_996991_36.stderr parsl.dataflow.dflow INFO: Task 169 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af63338d9d54838c0e4dede95f98cdf7 with result from task 169 parsl.dataflow.dflow DEBUG: Task 169 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/72e3cc6b-a7f0-4391-b3d4-18ed5436e7d3_isrForCrosstalkSources_996975_3.stdout parsl.dataflow.dflow INFO: Standard error for task 169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/72e3cc6b-a7f0-4391-b3d4-18ed5436e7d3_isrForCrosstalkSources_996975_3.stderr parsl.dataflow.dflow INFO: Task 197 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6067ec93f11024e6eb0a6a1db9d97cf0 with result from task 197 parsl.dataflow.dflow DEBUG: Task 197 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/38126090-ee38-44cc-a1d5-2fc112bd5ee2_isrForCrosstalkSources_996959_29.stdout parsl.dataflow.dflow INFO: Standard error for task 197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/38126090-ee38-44cc-a1d5-2fc112bd5ee2_isrForCrosstalkSources_996959_29.stderr parsl.dataflow.dflow INFO: Task 200 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53aa750a2d0510725b672a6f34149ce6 with result from task 200 parsl.dataflow.dflow DEBUG: Task 200 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/e4d662a7-2b07-477f-a180-6ca1f734492e_isrForCrosstalkSources_996953_53.stdout parsl.dataflow.dflow INFO: Standard error for task 200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/e4d662a7-2b07-477f-a180-6ca1f734492e_isrForCrosstalkSources_996953_53.stderr parsl.dataflow.dflow INFO: Task 203 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a75121d40e10ecfa88ace7822883bfc with result from task 203 parsl.dataflow.dflow DEBUG: Task 203 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/bcf53adb-bcf6-4309-a5c2-9a9ba74f6d11_isrForCrosstalkSources_997018_3.stdout parsl.dataflow.dflow INFO: Standard error for task 203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/bcf53adb-bcf6-4309-a5c2-9a9ba74f6d11_isrForCrosstalkSources_997018_3.stderr 2024-03-15 13:51:12 proc_lsst.multi:170 [INFO] [multi] job_id 17075095 proc_lsst.multi INFO: [multi] job_id 17075095 2024-03-15 13:51:12 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 51 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 51 2024-03-15 13:51:12 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:12 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075095 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075095 parsl.executors.status_handling DEBUG: Launched block 106 on executor multi with job ID 17075095 parsl.executors.status_handling INFO: Allocated block ID 107 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-107 2024-03-15 13:51:12 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-107 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-107 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:12 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 51 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 51 2024-03-15 13:51:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:12 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=107 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 211 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f868324bf8f200bf6e66353046931804 with result from task 211 parsl.dataflow.dflow DEBUG: Task 211 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ebce626d-fe78-4726-a4fe-876083e84c42_isrForCrosstalkSources_996984_8.stdout parsl.dataflow.dflow INFO: Standard error for task 211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ebce626d-fe78-4726-a4fe-876083e84c42_isrForCrosstalkSources_996984_8.stderr parsl.dataflow.dflow INFO: Task 175 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7655762cf5455708f64744eace6f0446 with result from task 175 parsl.dataflow.dflow DEBUG: Task 175 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7e8da971-fe03-4c86-9ac6-d6fa3cc42049_isrForCrosstalkSources_997025_41.stdout parsl.dataflow.dflow INFO: Standard error for task 175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7e8da971-fe03-4c86-9ac6-d6fa3cc42049_isrForCrosstalkSources_997025_41.stderr 2024-03-15 13:51:13 proc_lsst.multi:170 [INFO] [multi] job_id 17075096 proc_lsst.multi INFO: [multi] job_id 17075096 2024-03-15 13:51:13 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 52 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 52 2024-03-15 13:51:13 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:13 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075096 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075096 parsl.executors.status_handling DEBUG: Launched block 107 on executor multi with job ID 17075096 parsl.executors.status_handling INFO: Allocated block ID 108 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-108 2024-03-15 13:51:13 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-108 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-108 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 52 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 52 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:13 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=108 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:13 proc_lsst.multi:170 [INFO] [multi] job_id 17075097 proc_lsst.multi INFO: [multi] job_id 17075097 2024-03-15 13:51:13 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 53 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 53 2024-03-15 13:51:13 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:13 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075097 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075097 parsl.executors.status_handling DEBUG: Launched block 108 on executor multi with job ID 17075097 parsl.executors.status_handling INFO: Allocated block ID 109 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-109 2024-03-15 13:51:13 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-109 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-109 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:13 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 53 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 53 2024-03-15 13:51:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:13 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=109 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 207 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b70329c9703f18d84652c3714a403425 with result from task 207 parsl.dataflow.dflow DEBUG: Task 207 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/a7f428f2-d86c-42ba-9ac4-dc7a5bb4ef9d_isrForCrosstalkSources_997012_29.stdout parsl.dataflow.dflow INFO: Standard error for task 207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/a7f428f2-d86c-42ba-9ac4-dc7a5bb4ef9d_isrForCrosstalkSources_997012_29.stderr 2024-03-15 13:51:14 proc_lsst.multi:170 [INFO] [multi] job_id 17075098 proc_lsst.multi INFO: [multi] job_id 17075098 2024-03-15 13:51:14 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 54 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 54 2024-03-15 13:51:14 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:14 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075098 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075098 parsl.executors.status_handling DEBUG: Launched block 109 on executor multi with job ID 17075098 parsl.executors.status_handling INFO: Allocated block ID 110 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-110 2024-03-15 13:51:14 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-110 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-110 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 54 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 54 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:14 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=110 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 209 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6aa24f646ad22b15a717de88bda04c7 with result from task 209 parsl.dataflow.dflow DEBUG: Task 209 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/26bf42e5-790c-437b-a627-cffacd6469c6_isrForCrosstalkSources_996974_59.stdout parsl.dataflow.dflow INFO: Standard error for task 209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/26bf42e5-790c-437b-a627-cffacd6469c6_isrForCrosstalkSources_996974_59.stderr 2024-03-15 13:51:14 proc_lsst.multi:170 [INFO] [multi] job_id 17075099 proc_lsst.multi INFO: [multi] job_id 17075099 2024-03-15 13:51:14 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 55 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 55 2024-03-15 13:51:14 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:14 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075099 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075099 parsl.executors.status_handling DEBUG: Launched block 110 on executor multi with job ID 17075099 parsl.executors.status_handling INFO: Allocated block ID 111 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-111 2024-03-15 13:51:14 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-111 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-111 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:14 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 55 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 55 2024-03-15 13:51:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:14 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=111 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:15 proc_lsst.multi:170 [INFO] [multi] job_id 17075100 proc_lsst.multi INFO: [multi] job_id 17075100 2024-03-15 13:51:15 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 56 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 56 2024-03-15 13:51:15 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:15 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075100 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075100 parsl.executors.status_handling DEBUG: Launched block 111 on executor multi with job ID 17075100 parsl.executors.status_handling INFO: Allocated block ID 112 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-112 2024-03-15 13:51:15 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-112 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-112 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 56 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 56 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:15 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=112 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 204 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f796598e7df0116186703165ae7dabd2 with result from task 204 parsl.dataflow.dflow DEBUG: Task 204 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/32f1524a-d913-4245-8937-f328edcaf359_isrForCrosstalkSources_996996_62.stdout parsl.dataflow.dflow INFO: Standard error for task 204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/32f1524a-d913-4245-8937-f328edcaf359_isrForCrosstalkSources_996996_62.stderr 2024-03-15 13:51:15 proc_lsst.multi:170 [INFO] [multi] job_id 17075101 proc_lsst.multi INFO: [multi] job_id 17075101 2024-03-15 13:51:15 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 57 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 57 2024-03-15 13:51:15 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:15 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075101 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075101 parsl.executors.status_handling DEBUG: Launched block 112 on executor multi with job ID 17075101 parsl.executors.status_handling INFO: Allocated block ID 113 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-113 2024-03-15 13:51:15 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-113 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-113 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:15 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 57 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 57 2024-03-15 13:51:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:15 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=113 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 213 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 66d88b43e0802936a1077a0dc9030b74 with result from task 213 parsl.dataflow.dflow DEBUG: Task 213 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d280ab18-ca0d-4559-b6c1-8f09257bdac5_isrForCrosstalkSources_996989_32.stdout parsl.dataflow.dflow INFO: Standard error for task 213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d280ab18-ca0d-4559-b6c1-8f09257bdac5_isrForCrosstalkSources_996989_32.stderr 2024-03-15 13:51:16 proc_lsst.multi:170 [INFO] [multi] job_id 17075102 proc_lsst.multi INFO: [multi] job_id 17075102 2024-03-15 13:51:16 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 58 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 58 2024-03-15 13:51:16 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:16 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075102 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075102 parsl.executors.status_handling DEBUG: Launched block 113 on executor multi with job ID 17075102 parsl.executors.status_handling INFO: Allocated block ID 114 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-114 2024-03-15 13:51:16 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-114 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-114 2024-03-15 13:51:16 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:16 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:16 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:16 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:16 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 58 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 58 2024-03-15 13:51:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:16 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=114 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 216 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03ca2db348e8a9ff52c086f0041e61ae with result from task 216 parsl.dataflow.dflow DEBUG: Task 216 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a12d938f-09ce-493e-b783-3e86db9b3bbe_isrForCrosstalkSources_997002_4.stdout parsl.dataflow.dflow INFO: Standard error for task 216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a12d938f-09ce-493e-b783-3e86db9b3bbe_isrForCrosstalkSources_997002_4.stderr parsl.dataflow.dflow INFO: Task 220 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e452580b6ac95732f3c5a534f3a7d123 with result from task 220 parsl.dataflow.dflow DEBUG: Task 220 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/3ef2c260-4344-4df1-ae18-989d41eb165a_isrForCrosstalkSources_996960_33.stdout parsl.dataflow.dflow INFO: Standard error for task 220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/3ef2c260-4344-4df1-ae18-989d41eb165a_isrForCrosstalkSources_996960_33.stderr parsl.dataflow.dflow INFO: Task 214 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d700f23c9b37e67950ef9165a1077f4e with result from task 214 parsl.dataflow.dflow DEBUG: Task 214 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/8bc3e6f8-b54a-4995-8b14-46cfb9bf3907_isrForCrosstalkSources_996979_22.stdout parsl.dataflow.dflow INFO: Standard error for task 214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/8bc3e6f8-b54a-4995-8b14-46cfb9bf3907_isrForCrosstalkSources_996979_22.stderr parsl.dataflow.dflow INFO: Task 223 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b4b6200289e0032f6b8c580e0597751 with result from task 223 parsl.dataflow.dflow DEBUG: Task 223 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/149f058e-94f3-400a-82b5-c7398c9d85a7_isrForCrosstalkSources_996955_16.stdout parsl.dataflow.dflow INFO: Standard error for task 223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/149f058e-94f3-400a-82b5-c7398c9d85a7_isrForCrosstalkSources_996955_16.stderr parsl.dataflow.dflow INFO: Task 219 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04126ecbf6096815ac350be09b286f91 with result from task 219 parsl.dataflow.dflow DEBUG: Task 219 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/aebd8aa1-ee70-4aa4-af2d-3b9cd7ce30a5_isrForCrosstalkSources_996988_2.stdout parsl.dataflow.dflow INFO: Standard error for task 219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/aebd8aa1-ee70-4aa4-af2d-3b9cd7ce30a5_isrForCrosstalkSources_996988_2.stderr parsl.dataflow.dflow INFO: Task 212 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19f7e3d4abcb761f4f114ca5f58ebf69 with result from task 212 parsl.dataflow.dflow DEBUG: Task 212 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9e0b4ce5-0157-4d59-88c3-0e45a4679ac8_isrForCrosstalkSources_997000_40.stdout parsl.dataflow.dflow INFO: Standard error for task 212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9e0b4ce5-0157-4d59-88c3-0e45a4679ac8_isrForCrosstalkSources_997000_40.stderr parsl.dataflow.dflow INFO: Task 218 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa080f4f687e708178877df345f08c94 with result from task 218 parsl.dataflow.dflow DEBUG: Task 218 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/3d5fcbe0-a7d1-4cd1-8bb7-973ec8ee9f4b_isrForCrosstalkSources_996983_56.stdout parsl.dataflow.dflow INFO: Standard error for task 218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/3d5fcbe0-a7d1-4cd1-8bb7-973ec8ee9f4b_isrForCrosstalkSources_996983_56.stderr parsl.dataflow.dflow INFO: Task 217 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70149189fb63353c16c97aa994fa261d with result from task 217 parsl.dataflow.dflow DEBUG: Task 217 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9faecc48-548e-4c20-ab9e-b97d8aafa4b0_isrForCrosstalkSources_996979_42.stdout parsl.dataflow.dflow INFO: Standard error for task 217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9faecc48-548e-4c20-ab9e-b97d8aafa4b0_isrForCrosstalkSources_996979_42.stderr parsl.dataflow.dflow INFO: Task 224 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 629d184a9664d2ac5f1072c11a6af1ae with result from task 224 parsl.dataflow.dflow DEBUG: Task 224 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/748b5c14-a716-450e-8a82-0e7cdd64dd71_isrForCrosstalkSources_996981_42.stdout parsl.dataflow.dflow INFO: Standard error for task 224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/748b5c14-a716-450e-8a82-0e7cdd64dd71_isrForCrosstalkSources_996981_42.stderr 2024-03-15 13:51:19 proc_lsst.multi:170 [INFO] [multi] job_id 17075103 proc_lsst.multi INFO: [multi] job_id 17075103 2024-03-15 13:51:19 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 59 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 59 2024-03-15 13:51:19 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:19 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075103 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075103 parsl.executors.status_handling DEBUG: Launched block 114 on executor multi with job ID 17075103 parsl.executors.status_handling INFO: Allocated block ID 115 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-115 2024-03-15 13:51:19 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-115 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-115 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 59 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 59 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:19 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=115 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:19 proc_lsst.multi:170 [INFO] [multi] job_id 17075109 proc_lsst.multi INFO: [multi] job_id 17075109 2024-03-15 13:51:19 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 60 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 60 2024-03-15 13:51:19 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:19 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075109 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075109 parsl.executors.status_handling DEBUG: Launched block 115 on executor multi with job ID 17075109 parsl.executors.status_handling INFO: Allocated block ID 116 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-116 2024-03-15 13:51:19 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-116 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-116 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:19 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 60 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 60 2024-03-15 13:51:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:19 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=116 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:20 proc_lsst.multi:170 [INFO] [multi] job_id 17075110 proc_lsst.multi INFO: [multi] job_id 17075110 2024-03-15 13:51:20 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 61 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 61 2024-03-15 13:51:20 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:20 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075110 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075110 parsl.executors.status_handling DEBUG: Launched block 116 on executor multi with job ID 17075110 parsl.executors.status_handling INFO: Allocated block ID 117 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-117 2024-03-15 13:51:20 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-117 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-117 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 61 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 61 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:20 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=117 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 215 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d28b8b0a243ac02fae9e192501bf8451 with result from task 215 parsl.dataflow.dflow DEBUG: Task 215 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/6521fa8f-688a-48e8-8788-2c3efd5cbe7d_isrForCrosstalkSources_997009_54.stdout parsl.dataflow.dflow INFO: Standard error for task 215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/6521fa8f-688a-48e8-8788-2c3efd5cbe7d_isrForCrosstalkSources_997009_54.stderr 2024-03-15 13:51:20 proc_lsst.multi:170 [INFO] [multi] job_id 17075111 proc_lsst.multi INFO: [multi] job_id 17075111 2024-03-15 13:51:20 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 62 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 62 2024-03-15 13:51:20 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:20 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075111 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075111 parsl.executors.status_handling DEBUG: Launched block 117 on executor multi with job ID 17075111 parsl.executors.status_handling INFO: Allocated block ID 118 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-118 2024-03-15 13:51:20 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-118 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-118 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:20 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 62 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 62 2024-03-15 13:51:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:20 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=118 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 227 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e17caeea55401d35ea005a66fc0f9377 with result from task 227 parsl.dataflow.dflow DEBUG: Task 227 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/08db3d1b-60b4-42bd-9caa-91fcb641702b_isrForCrosstalkSources_997022_39.stdout parsl.dataflow.dflow INFO: Standard error for task 227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/08db3d1b-60b4-42bd-9caa-91fcb641702b_isrForCrosstalkSources_997022_39.stderr parsl.dataflow.dflow INFO: Task 245 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 79debdb013333d6e166ea51ac138d558 with result from task 245 parsl.dataflow.dflow DEBUG: Task 245 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8bddbbf3-f769-4bf8-ac9f-dde7886b26a2_isrForCrosstalkSources_996997_13.stdout parsl.dataflow.dflow INFO: Standard error for task 245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8bddbbf3-f769-4bf8-ac9f-dde7886b26a2_isrForCrosstalkSources_996997_13.stderr 2024-03-15 13:51:21 proc_lsst.multi:170 [INFO] [multi] job_id 17075112 proc_lsst.multi INFO: [multi] job_id 17075112 2024-03-15 13:51:21 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 63 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 63 2024-03-15 13:51:21 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:21 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075112 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075112 parsl.executors.status_handling DEBUG: Launched block 118 on executor multi with job ID 17075112 parsl.executors.status_handling INFO: Allocated block ID 119 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-119 2024-03-15 13:51:21 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-119 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-119 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 63 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 63 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:21 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=119 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 237 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33596125a2388c43d4eb1b271db48b0d with result from task 237 parsl.dataflow.dflow DEBUG: Task 237 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/175059bf-3ac9-4819-a41d-785d359cc57a_isrForCrosstalkSources_996977_36.stdout parsl.dataflow.dflow INFO: Standard error for task 237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/175059bf-3ac9-4819-a41d-785d359cc57a_isrForCrosstalkSources_996977_36.stderr parsl.dataflow.dflow INFO: Task 233 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 95fd83785dddae727e4b24301c0f4a7c with result from task 233 parsl.dataflow.dflow DEBUG: Task 233 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/b6e245f6-8783-4616-b5fa-4ddb8629554e_isrForCrosstalkSources_996984_12.stdout parsl.dataflow.dflow INFO: Standard error for task 233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/b6e245f6-8783-4616-b5fa-4ddb8629554e_isrForCrosstalkSources_996984_12.stderr parsl.dataflow.dflow INFO: Task 240 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0b4ce257fbe133a5f78bc1377aecb4a6 with result from task 240 parsl.dataflow.dflow DEBUG: Task 240 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/26dfde53-6b63-4331-bd29-de377a156cc7_isrForCrosstalkSources_996989_17.stdout parsl.dataflow.dflow INFO: Standard error for task 240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/26dfde53-6b63-4331-bd29-de377a156cc7_isrForCrosstalkSources_996989_17.stderr 2024-03-15 13:51:21 proc_lsst.multi:170 [INFO] [multi] job_id 17075113 proc_lsst.multi INFO: [multi] job_id 17075113 2024-03-15 13:51:21 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 64 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 64 2024-03-15 13:51:21 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:21 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075113 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075113 parsl.executors.status_handling DEBUG: Launched block 119 on executor multi with job ID 17075113 parsl.executors.status_handling INFO: Allocated block ID 120 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-120 2024-03-15 13:51:21 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-120 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-120 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:21 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 64 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 64 2024-03-15 13:51:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:21 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=120 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:22 proc_lsst.multi:170 [INFO] [multi] job_id 17075114 proc_lsst.multi INFO: [multi] job_id 17075114 2024-03-15 13:51:22 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 65 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 65 2024-03-15 13:51:22 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:22 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075114 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075114 parsl.executors.status_handling DEBUG: Launched block 120 on executor multi with job ID 17075114 parsl.executors.status_handling INFO: Allocated block ID 121 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-121 2024-03-15 13:51:22 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-121 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-121 2024-03-15 13:51:22 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:22 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:22 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:22 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:22 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 65 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 65 2024-03-15 13:51:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:22 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=121 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 258 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a02f8b3e9fa909034c6177e88193852 with result from task 258 parsl.dataflow.dflow DEBUG: Task 258 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/26726060-6e46-4252-9cbf-4de8cd15743e_isrForCrosstalkSources_996968_19.stdout parsl.dataflow.dflow INFO: Standard error for task 258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/26726060-6e46-4252-9cbf-4de8cd15743e_isrForCrosstalkSources_996968_19.stderr parsl.dataflow.dflow INFO: Task 261 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 359f7e4f79f659a0bcfa635c3cb5c3b8 with result from task 261 parsl.dataflow.dflow DEBUG: Task 261 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/cdfdd9db-4267-4129-8345-2dd6730b7d66_isrForCrosstalkSources_996974_14.stdout parsl.dataflow.dflow INFO: Standard error for task 261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/cdfdd9db-4267-4129-8345-2dd6730b7d66_isrForCrosstalkSources_996974_14.stderr parsl.dataflow.dflow INFO: Task 242 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 372bc94cec64d3b676ef64c95ceb88e6 with result from task 242 2024-03-15 13:51:23 proc_lsst.multi:170 [INFO] [multi] job_id 17075115 proc_lsst.multi INFO: [multi] job_id 17075115 2024-03-15 13:51:23 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 66 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 66 2024-03-15 13:51:23 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:23 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075115 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075115 parsl.executors.status_handling DEBUG: Launched block 121 on executor multi with job ID 17075115 parsl.executors.status_handling INFO: Allocated block ID 122 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-122 2024-03-15 13:51:23 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-122 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-122 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 66 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 66 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:23 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=122 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 242 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/6f091bc4-e4ca-4b55-9547-cdb1c9762bb1_isrForCrosstalkSources_997013_44.stdout parsl.dataflow.dflow INFO: Standard error for task 242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/6f091bc4-e4ca-4b55-9547-cdb1c9762bb1_isrForCrosstalkSources_997013_44.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 243 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8943f4d3a2bd1a2dc35ede559c4247f0 with result from task 243 parsl.dataflow.dflow DEBUG: Task 243 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/805116a2-5f8f-46c1-ba05-1ee417c9d055_isrForCrosstalkSources_996980_3.stdout parsl.dataflow.dflow INFO: Standard error for task 243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/805116a2-5f8f-46c1-ba05-1ee417c9d055_isrForCrosstalkSources_996980_3.stderr parsl.dataflow.dflow INFO: Task 244 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c63e96bc6c5bb8c30d50ed5b574606e with result from task 244 parsl.dataflow.dflow DEBUG: Task 244 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6b58120-be70-46ad-8a8f-524f652ad0dc_isrForCrosstalkSources_997026_37.stdout parsl.dataflow.dflow INFO: Standard error for task 244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6b58120-be70-46ad-8a8f-524f652ad0dc_isrForCrosstalkSources_997026_37.stderr 2024-03-15 13:51:23 proc_lsst.multi:170 [INFO] [multi] job_id 17075116 proc_lsst.multi INFO: [multi] job_id 17075116 2024-03-15 13:51:23 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 67 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 67 2024-03-15 13:51:23 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:23 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075116 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075116 parsl.executors.status_handling DEBUG: Launched block 122 on executor multi with job ID 17075116 parsl.executors.status_handling INFO: Allocated block ID 123 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-123 2024-03-15 13:51:23 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-123 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-123 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:23 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 67 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 67 2024-03-15 13:51:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:23 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=123 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 260 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e9a6f86fc2f1f7d234e654d8f955e27 with result from task 260 parsl.dataflow.dflow DEBUG: Task 260 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/809e2f5c-3505-4776-a3a3-d6c1121c4b0f_isrForCrosstalkSources_997031_33.stdout parsl.dataflow.dflow INFO: Standard error for task 260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/809e2f5c-3505-4776-a3a3-d6c1121c4b0f_isrForCrosstalkSources_997031_33.stderr 2024-03-15 13:51:24 proc_lsst.multi:170 [INFO] [multi] job_id 17075117 proc_lsst.multi INFO: [multi] job_id 17075117 2024-03-15 13:51:24 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 68 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 68 2024-03-15 13:51:24 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:24 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075117 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075117 parsl.executors.status_handling DEBUG: Launched block 123 on executor multi with job ID 17075117 parsl.executors.status_handling INFO: Allocated block ID 124 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-124 2024-03-15 13:51:24 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-124 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-124 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 68 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 68 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:24 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=124 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 263 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 186ba9a8c465002b63b9cfa35da3189e with result from task 263 parsl.dataflow.dflow DEBUG: Task 263 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/f6ca5d89-39d4-4dfb-a98c-ee098e3516dd_isrForCrosstalkSources_997025_14.stdout parsl.dataflow.dflow INFO: Standard error for task 263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/f6ca5d89-39d4-4dfb-a98c-ee098e3516dd_isrForCrosstalkSources_997025_14.stderr parsl.dataflow.dflow INFO: Task 249 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f51a39c4c5a91d513f4982a4b23343c with result from task 249 parsl.dataflow.dflow DEBUG: Task 249 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/42cb7ee4-7a50-43a0-9cdf-d450652b698b_isrForCrosstalkSources_997018_13.stdout parsl.dataflow.dflow INFO: Standard error for task 249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/42cb7ee4-7a50-43a0-9cdf-d450652b698b_isrForCrosstalkSources_997018_13.stderr parsl.dataflow.dflow INFO: Task 229 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa14f99580db2f4709d3a144c98fa006 with result from task 229 parsl.dataflow.dflow DEBUG: Task 229 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d18e2f1-c5f7-441a-a410-9afd64c56533_isrForCrosstalkSources_996995_13.stdout parsl.dataflow.dflow INFO: Standard error for task 229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d18e2f1-c5f7-441a-a410-9afd64c56533_isrForCrosstalkSources_996995_13.stderr parsl.dataflow.dflow INFO: Task 248 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b345b3727df70b0ad69e6b3cb4f426a1 with result from task 248 parsl.dataflow.dflow DEBUG: Task 248 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d5941b02-cead-42d9-9978-d1d7b14a6f94_isrForCrosstalkSources_996964_31.stdout parsl.dataflow.dflow INFO: Standard error for task 248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d5941b02-cead-42d9-9978-d1d7b14a6f94_isrForCrosstalkSources_996964_31.stderr 2024-03-15 13:51:24 proc_lsst.multi:170 [INFO] [multi] job_id 17075118 proc_lsst.multi INFO: [multi] job_id 17075118 2024-03-15 13:51:24 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 69 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 69 2024-03-15 13:51:24 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:24 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075118 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075118 parsl.executors.status_handling DEBUG: Launched block 124 on executor multi with job ID 17075118 parsl.executors.status_handling INFO: Allocated block ID 125 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-125 2024-03-15 13:51:24 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-125 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-125 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:24 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 69 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 69 2024-03-15 13:51:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:24 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=125 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 259 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ab0c6a14501bebc7446fb693fbb67a9 with result from task 259 parsl.dataflow.dflow DEBUG: Task 259 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e337ee50-4942-473e-b0ff-594148b84cb1_isrForCrosstalkSources_996964_60.stdout parsl.dataflow.dflow INFO: Standard error for task 259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e337ee50-4942-473e-b0ff-594148b84cb1_isrForCrosstalkSources_996964_60.stderr parsl.dataflow.dflow INFO: Task 225 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6f8b7c3debebc63d70c9c05e8c3fd65 with result from task 225 parsl.dataflow.dflow DEBUG: Task 225 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/381fc8f5-6dcb-4aa7-b750-dfbec75730be_isrForCrosstalkSources_996962_46.stdout parsl.dataflow.dflow INFO: Standard error for task 225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/381fc8f5-6dcb-4aa7-b750-dfbec75730be_isrForCrosstalkSources_996962_46.stderr parsl.dataflow.dflow INFO: Task 271 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a704ce1130d1340ba8e62724323c2b53 with result from task 271 parsl.dataflow.dflow DEBUG: Task 271 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/148988df-1d3b-425c-98f9-68e5802b745e_isrForCrosstalkSources_997019_45.stdout parsl.dataflow.dflow INFO: Standard error for task 271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/148988df-1d3b-425c-98f9-68e5802b745e_isrForCrosstalkSources_997019_45.stderr 2024-03-15 13:51:25 proc_lsst.multi:170 [INFO] [multi] job_id 17075121 proc_lsst.multi INFO: [multi] job_id 17075121 2024-03-15 13:51:25 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 70 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 70 2024-03-15 13:51:25 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:25 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075121 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075121 parsl.executors.status_handling DEBUG: Launched block 125 on executor multi with job ID 17075121 parsl.executors.status_handling INFO: Allocated block ID 126 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-126 2024-03-15 13:51:25 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-126 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-126 2024-03-15 13:51:25 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:25 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:25 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:25 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:25 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 70 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 70 2024-03-15 13:51:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:25 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=126 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 253 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 67754eb06f86511b45927aed61fc2719 with result from task 253 parsl.dataflow.dflow DEBUG: Task 253 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9cd0b273-d235-484c-89b6-4f66a963c42c_isrForCrosstalkSources_997018_23.stdout parsl.dataflow.dflow INFO: Standard error for task 253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9cd0b273-d235-484c-89b6-4f66a963c42c_isrForCrosstalkSources_997018_23.stderr parsl.dataflow.dflow INFO: Task 270 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58b6bfe98c836554d90f8abb0a292145 with result from task 270 parsl.dataflow.dflow DEBUG: Task 270 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f02032fb-8646-4bc6-bcb1-22941638a8c6_isrForCrosstalkSources_996967_5.stdout parsl.dataflow.dflow INFO: Standard error for task 270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f02032fb-8646-4bc6-bcb1-22941638a8c6_isrForCrosstalkSources_996967_5.stderr 2024-03-15 13:51:26 proc_lsst.multi:170 [INFO] [multi] job_id 17075122 proc_lsst.multi INFO: [multi] job_id 17075122 2024-03-15 13:51:26 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 71 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 71 2024-03-15 13:51:26 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:26 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075122 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075122 parsl.executors.status_handling DEBUG: Launched block 126 on executor multi with job ID 17075122 parsl.executors.status_handling INFO: Allocated block ID 127 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-127 2024-03-15 13:51:26 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-127 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-127 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 71 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 71 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:26 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=127 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 268 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f8bfacc356b85f197175ecdc5ba0cb1 with result from task 268 parsl.dataflow.dflow DEBUG: Task 268 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/c9fe3b9a-9a7d-4ada-a26f-2872e1133012_isrForCrosstalkSources_997024_50.stdout parsl.dataflow.dflow INFO: Standard error for task 268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/c9fe3b9a-9a7d-4ada-a26f-2872e1133012_isrForCrosstalkSources_997024_50.stderr parsl.dataflow.dflow INFO: Task 228 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a56e0a492c0789c1917b33bfa3ea46d with result from task 228 parsl.dataflow.dflow DEBUG: Task 228 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ba909b15-9026-4149-8204-3f75d927fba1_isrForCrosstalkSources_997028_51.stdout parsl.dataflow.dflow INFO: Standard error for task 228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ba909b15-9026-4149-8204-3f75d927fba1_isrForCrosstalkSources_997028_51.stderr 2024-03-15 13:51:26 proc_lsst.multi:170 [INFO] [multi] job_id 17075124 proc_lsst.multi INFO: [multi] job_id 17075124 2024-03-15 13:51:26 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 72 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 72 2024-03-15 13:51:26 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:26 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075124 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075124 parsl.executors.status_handling DEBUG: Launched block 127 on executor multi with job ID 17075124 parsl.executors.status_handling INFO: Allocated block ID 128 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-128 2024-03-15 13:51:26 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-128 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-128 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:26 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 72 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 72 2024-03-15 13:51:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:26 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=128 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 241 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fbed5d1a9d3d1b4a448672d4250f4992 with result from task 241 parsl.dataflow.dflow DEBUG: Task 241 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5cc8ab78-194d-427a-8342-ff11bd79f27b_isrForCrosstalkSources_997009_23.stdout parsl.dataflow.dflow INFO: Standard error for task 241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5cc8ab78-194d-427a-8342-ff11bd79f27b_isrForCrosstalkSources_997009_23.stderr parsl.dataflow.dflow INFO: Task 247 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd4829bf741d360f680d9b1405cb6a52 with result from task 247 parsl.dataflow.dflow DEBUG: Task 247 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1470b61e-fbcc-4bc3-b26f-c8502fb1a874_isrForCrosstalkSources_996953_16.stdout parsl.dataflow.dflow INFO: Standard error for task 247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1470b61e-fbcc-4bc3-b26f-c8502fb1a874_isrForCrosstalkSources_996953_16.stderr parsl.dataflow.dflow INFO: Task 251 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0d55b4c0c528959940214b6a035a06ea with result from task 251 parsl.dataflow.dflow DEBUG: Task 251 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5d898322-acdc-4504-9773-827f1686de6b_isrForCrosstalkSources_997015_20.stdout parsl.dataflow.dflow INFO: Standard error for task 251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5d898322-acdc-4504-9773-827f1686de6b_isrForCrosstalkSources_997015_20.stderr 2024-03-15 13:51:27 proc_lsst.multi:170 [INFO] [multi] job_id 17075127 proc_lsst.multi INFO: [multi] job_id 17075127 2024-03-15 13:51:27 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 73 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 73 2024-03-15 13:51:27 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:27 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075127 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075127 parsl.executors.status_handling DEBUG: Launched block 128 on executor multi with job ID 17075127 parsl.executors.status_handling INFO: Allocated block ID 129 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-129 2024-03-15 13:51:27 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-129 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-129 2024-03-15 13:51:27 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:27 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:27 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:27 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:27 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 73 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 73 2024-03-15 13:51:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:27 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=129 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:28 proc_lsst.multi:170 [INFO] [multi] job_id 17075128 proc_lsst.multi INFO: [multi] job_id 17075128 2024-03-15 13:51:28 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 74 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 74 2024-03-15 13:51:28 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:28 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075128 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075128 parsl.executors.status_handling DEBUG: Launched block 129 on executor multi with job ID 17075128 parsl.executors.status_handling INFO: Allocated block ID 130 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-130 2024-03-15 13:51:28 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-130 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-130 2024-03-15 13:51:28 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:28 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:28 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:28 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:28 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 74 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 74 2024-03-15 13:51:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:28 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=130 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 257 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ca440b4819bdffacd5ad05b1bb76c11 with result from task 257 parsl.dataflow.dflow DEBUG: Task 257 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/0e092664-2ad7-4a09-a7c0-6658d160b688_isrForCrosstalkSources_997006_23.stdout parsl.dataflow.dflow INFO: Standard error for task 257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/0e092664-2ad7-4a09-a7c0-6658d160b688_isrForCrosstalkSources_997006_23.stderr parsl.dataflow.dflow INFO: Task 252 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ba7d329ef5546ae9dff44f8a9e2f762 with result from task 252 parsl.dataflow.dflow DEBUG: Task 252 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/be5ce00b-7fcc-496b-95f2-cc7afd54bb2a_isrForCrosstalkSources_997016_52.stdout parsl.dataflow.dflow INFO: Standard error for task 252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/be5ce00b-7fcc-496b-95f2-cc7afd54bb2a_isrForCrosstalkSources_997016_52.stderr parsl.dataflow.dflow INFO: Task 256 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 222b82b6dcd4ef2eb2f88d9eaad52119 with result from task 256 parsl.dataflow.dflow DEBUG: Task 256 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/f47c0440-05fc-4688-8b6f-387b9a6ed24e_isrForCrosstalkSources_997027_56.stdout parsl.dataflow.dflow INFO: Standard error for task 256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/f47c0440-05fc-4688-8b6f-387b9a6ed24e_isrForCrosstalkSources_997027_56.stderr parsl.dataflow.dflow INFO: Task 267 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17fed749da56c1a8b2ec009f52c35a7a with result from task 267 parsl.dataflow.dflow DEBUG: Task 267 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/42f37f7d-1380-4cc2-a37e-4a67a95720df_isrForCrosstalkSources_996974_2.stdout parsl.dataflow.dflow INFO: Standard error for task 267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/42f37f7d-1380-4cc2-a37e-4a67a95720df_isrForCrosstalkSources_996974_2.stderr parsl.dataflow.dflow INFO: Task 269 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4503b09844d870624cbd4ffe474a56a5 with result from task 269 parsl.dataflow.dflow DEBUG: Task 269 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8db7c8d6-52cd-428e-90ce-4614923a728d_isrForCrosstalkSources_996956_1.stdout parsl.dataflow.dflow INFO: Standard error for task 269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8db7c8d6-52cd-428e-90ce-4614923a728d_isrForCrosstalkSources_996956_1.stderr parsl.dataflow.dflow INFO: Task 235 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 79ddf84cc8530898c2f80cf358ef59d6 with result from task 235 parsl.dataflow.dflow DEBUG: Task 235 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/25c06052-4d50-44ad-8d86-a27b2d1e60d3_isrForCrosstalkSources_996952_11.stdout parsl.dataflow.dflow INFO: Standard error for task 235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/25c06052-4d50-44ad-8d86-a27b2d1e60d3_isrForCrosstalkSources_996952_11.stderr parsl.dataflow.dflow INFO: Task 226 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a6dbd126123c016fdcc7b230e99249c with result from task 226 parsl.dataflow.dflow DEBUG: Task 226 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/52bb9479-8fca-48b8-8ea8-a15c8d674041_isrForCrosstalkSources_996978_48.stdout parsl.dataflow.dflow INFO: Standard error for task 226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/52bb9479-8fca-48b8-8ea8-a15c8d674041_isrForCrosstalkSources_996978_48.stderr parsl.dataflow.dflow INFO: Task 264 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c0c44537fee4c489efaec0a89cc503b7 with result from task 264 2024-03-15 13:51:29 proc_lsst.multi:170 [INFO] [multi] job_id 17075129 proc_lsst.multi INFO: [multi] job_id 17075129 2024-03-15 13:51:29 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 75 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 75 2024-03-15 13:51:29 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:29 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075129 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075129 parsl.executors.status_handling DEBUG: Launched block 130 on executor multi with job ID 17075129 parsl.executors.status_handling INFO: Allocated block ID 131 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-131 2024-03-15 13:51:29 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-131 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-131 2024-03-15 13:51:29 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:29 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:29 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:29 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:29 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 75 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 75 2024-03-15 13:51:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:29 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=131 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 264 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/9a68c496-3ddc-45da-a795-0cdacff15800_isrForCrosstalkSources_996993_12.stdout parsl.dataflow.dflow INFO: Standard error for task 264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/9a68c496-3ddc-45da-a795-0cdacff15800_isrForCrosstalkSources_996993_12.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 230 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59525b57241fcbfab13957ef3427ca0d with result from task 230 parsl.dataflow.dflow DEBUG: Task 230 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2047f377-5696-4cc0-a960-e62106da2012_isrForCrosstalkSources_996963_56.stdout parsl.dataflow.dflow INFO: Standard error for task 230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2047f377-5696-4cc0-a960-e62106da2012_isrForCrosstalkSources_996963_56.stderr parsl.dataflow.dflow INFO: Task 234 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 384e9c2a284cdf852df48d0a4e314169 with result from task 234 parsl.dataflow.dflow DEBUG: Task 234 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7b5d6531-eb25-4098-8d4f-f76df8e7815b_isrForCrosstalkSources_996994_55.stdout parsl.dataflow.dflow INFO: Standard error for task 234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7b5d6531-eb25-4098-8d4f-f76df8e7815b_isrForCrosstalkSources_996994_55.stderr parsl.dataflow.dflow INFO: Task 273 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c5dfc0aa5a088ee0514fbe670713f75b with result from task 273 parsl.dataflow.dflow DEBUG: Task 273 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/2947c406-6abf-47a8-90c7-808be10d49f9_isrForCrosstalkSources_996964_48.stdout parsl.dataflow.dflow INFO: Standard error for task 273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/2947c406-6abf-47a8-90c7-808be10d49f9_isrForCrosstalkSources_996964_48.stderr 2024-03-15 13:51:30 proc_lsst.multi:170 [INFO] [multi] job_id 17075131 proc_lsst.multi INFO: [multi] job_id 17075131 2024-03-15 13:51:30 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 76 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 76 2024-03-15 13:51:30 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:30 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075131 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075131 parsl.executors.status_handling DEBUG: Launched block 131 on executor multi with job ID 17075131 parsl.executors.status_handling INFO: Allocated block ID 132 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-132 2024-03-15 13:51:30 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-132 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-132 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 76 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 76 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:30 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=132 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 239 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32ddf94374ae06c0e2320c70848e223a with result from task 239 parsl.dataflow.dflow DEBUG: Task 239 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8f9e39b3-f675-4ef0-9ca6-04db25318ea9_isrForCrosstalkSources_996987_16.stdout parsl.dataflow.dflow INFO: Standard error for task 239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8f9e39b3-f675-4ef0-9ca6-04db25318ea9_isrForCrosstalkSources_996987_16.stderr 2024-03-15 13:51:30 proc_lsst.multi:170 [INFO] [multi] job_id 17075132 proc_lsst.multi INFO: [multi] job_id 17075132 2024-03-15 13:51:30 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 77 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 77 2024-03-15 13:51:30 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:30 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075132 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075132 parsl.executors.status_handling DEBUG: Launched block 132 on executor multi with job ID 17075132 parsl.executors.status_handling INFO: Allocated block ID 133 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-133 2024-03-15 13:51:30 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-133 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-133 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:30 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 77 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 77 2024-03-15 13:51:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:30 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=133 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 274 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 257b09a1bc456b581f9481118616633d with result from task 274 parsl.dataflow.dflow DEBUG: Task 274 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/43a80d84-5b6c-48dc-971b-11a2be08c058_isrForCrosstalkSources_996982_24.stdout parsl.dataflow.dflow INFO: Standard error for task 274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/43a80d84-5b6c-48dc-971b-11a2be08c058_isrForCrosstalkSources_996982_24.stderr 2024-03-15 13:51:31 proc_lsst.multi:170 [INFO] [multi] job_id 17075133 proc_lsst.multi INFO: [multi] job_id 17075133 2024-03-15 13:51:31 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 78 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 78 2024-03-15 13:51:31 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:31 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075133 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075133 parsl.executors.status_handling DEBUG: Launched block 133 on executor multi with job ID 17075133 parsl.executors.status_handling INFO: Allocated block ID 134 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-134 2024-03-15 13:51:31 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-134 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-134 2024-03-15 13:51:31 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:31 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:31 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:31 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:31 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 78 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 78 2024-03-15 13:51:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:31 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=134 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 236 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ec9f0095d47e0069b16f88adc3a92ad with result from task 236 parsl.dataflow.dflow DEBUG: Task 236 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6fea21bd-ca55-45b9-b0a7-df4d312fd860_isrForCrosstalkSources_996973_55.stdout parsl.dataflow.dflow INFO: Standard error for task 236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6fea21bd-ca55-45b9-b0a7-df4d312fd860_isrForCrosstalkSources_996973_55.stderr parsl.dataflow.dflow INFO: Task 221 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1adf07e8060934e2539e1ea0cfa08b21 with result from task 221 parsl.dataflow.dflow DEBUG: Task 221 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/eed6f060-1708-4240-a19b-57f7b7be73b8_isrForCrosstalkSources_997018_58.stdout parsl.dataflow.dflow INFO: Standard error for task 221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/eed6f060-1708-4240-a19b-57f7b7be73b8_isrForCrosstalkSources_997018_58.stderr parsl.dataflow.dflow INFO: Task 232 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fda67be4134b5527d38faf1e8d90a9a8 with result from task 232 parsl.dataflow.dflow DEBUG: Task 232 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/f9148d69-7064-46f7-bb6c-cc6934f84e30_isrForCrosstalkSources_996972_47.stdout parsl.dataflow.dflow INFO: Standard error for task 232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/f9148d69-7064-46f7-bb6c-cc6934f84e30_isrForCrosstalkSources_996972_47.stderr 2024-03-15 13:51:32 proc_lsst.multi:170 [INFO] [multi] job_id 17075134 proc_lsst.multi INFO: [multi] job_id 17075134 2024-03-15 13:51:32 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 79 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 79 2024-03-15 13:51:32 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:32 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075134 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075134 parsl.executors.status_handling DEBUG: Launched block 134 on executor multi with job ID 17075134 parsl.executors.status_handling INFO: Allocated block ID 135 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-135 2024-03-15 13:51:32 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-135 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-135 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 79 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 79 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 parsl.dataflow.dflow INFO: Task 266 completed (launched -> exec_done) proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:32 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.dataflow.memoization DEBUG: Storing app cache entry ddab5ba099ec90b611e50d7ae67e1db4 with result from task 266 proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=135 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 266 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/41dad024-76b7-40b3-9b1f-6b780acc8155_isrForCrosstalkSources_997024_2.stdout parsl.dataflow.dflow INFO: Standard error for task 266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/41dad024-76b7-40b3-9b1f-6b780acc8155_isrForCrosstalkSources_997024_2.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 246 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c826b85f9ad8b9132348ee2a5a364d47 with result from task 246 parsl.dataflow.dflow DEBUG: Task 246 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/25d00663-629f-4429-b6b9-6f63891b6535_isrForCrosstalkSources_996971_58.stdout parsl.dataflow.dflow INFO: Standard error for task 246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/25d00663-629f-4429-b6b9-6f63891b6535_isrForCrosstalkSources_996971_58.stderr parsl.dataflow.dflow INFO: Task 231 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8cb2109cad5b055accd79fe5705ce04d with result from task 231 parsl.dataflow.dflow DEBUG: Task 231 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/01750578-37b3-4ee0-886b-660673030f26_isrForCrosstalkSources_996999_41.stdout parsl.dataflow.dflow INFO: Standard error for task 231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/01750578-37b3-4ee0-886b-660673030f26_isrForCrosstalkSources_996999_41.stderr parsl.dataflow.dflow INFO: Task 254 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c319615f95ca25d662ea5b0f324103c with result from task 254 parsl.dataflow.dflow DEBUG: Task 254 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7f40e489-6b40-486d-9b70-fc619769c339_isrForCrosstalkSources_997003_42.stdout parsl.dataflow.dflow INFO: Standard error for task 254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7f40e489-6b40-486d-9b70-fc619769c339_isrForCrosstalkSources_997003_42.stderr parsl.dataflow.dflow INFO: Task 250 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e2b8d96ba4acc2fc1e6e9b01b030ee7 with result from task 250 parsl.dataflow.dflow DEBUG: Task 250 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b59a07f0-93cb-4e49-b00e-612509551427_isrForCrosstalkSources_996964_50.stdout parsl.dataflow.dflow INFO: Standard error for task 250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b59a07f0-93cb-4e49-b00e-612509551427_isrForCrosstalkSources_996964_50.stderr parsl.dataflow.dflow INFO: Task 262 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a06c27d38217e8daad2ac9e680daff2f with result from task 262 parsl.dataflow.dflow DEBUG: Task 262 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ede64a10-8ac0-4604-a5c2-ce362df85691_isrForCrosstalkSources_996984_60.stdout parsl.dataflow.dflow INFO: Standard error for task 262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ede64a10-8ac0-4604-a5c2-ce362df85691_isrForCrosstalkSources_996984_60.stderr 2024-03-15 13:51:32 proc_lsst.multi:170 [INFO] [multi] job_id 17075135 proc_lsst.multi INFO: [multi] job_id 17075135 2024-03-15 13:51:32 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 80 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 80 2024-03-15 13:51:32 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:32 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075135 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075135 parsl.executors.status_handling DEBUG: Launched block 135 on executor multi with job ID 17075135 parsl.executors.status_handling INFO: Allocated block ID 136 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-136 2024-03-15 13:51:32 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-136 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-136 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:32 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 80 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 80 2024-03-15 13:51:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:32 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=136 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 222 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2102614bb2dfc1931daedd0bd701a17f with result from task 222 parsl.dataflow.dflow DEBUG: Task 222 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/60315d7c-2c16-42be-a7ef-ba4381bb6fee_isrForCrosstalkSources_997001_50.stdout parsl.dataflow.dflow INFO: Standard error for task 222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/60315d7c-2c16-42be-a7ef-ba4381bb6fee_isrForCrosstalkSources_997001_50.stderr 2024-03-15 13:51:33 proc_lsst.multi:170 [INFO] [multi] job_id 17075136 proc_lsst.multi INFO: [multi] job_id 17075136 2024-03-15 13:51:33 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 81 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 81 2024-03-15 13:51:33 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:33 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075136 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075136 parsl.executors.status_handling DEBUG: Launched block 136 on executor multi with job ID 17075136 parsl.executors.status_handling INFO: Allocated block ID 137 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-137 2024-03-15 13:51:33 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-137 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-137 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 81 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 81 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:33 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=137 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 255 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 617733def177ee5a2c34d99cb2e675df with result from task 255 parsl.dataflow.dflow DEBUG: Task 255 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7abcfe4a-23cc-46e9-8f8e-ce3eb3644f61_isrForCrosstalkSources_996963_44.stdout parsl.dataflow.dflow INFO: Standard error for task 255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7abcfe4a-23cc-46e9-8f8e-ce3eb3644f61_isrForCrosstalkSources_996963_44.stderr parsl.dataflow.dflow INFO: Task 277 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 79578570f3ea4c44a4fe9f3d77fc585f with result from task 277 parsl.dataflow.dflow DEBUG: Task 277 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/60357f1f-262c-4927-9d30-55d5f63a3dfd_isrForCrosstalkSources_997016_21.stdout parsl.dataflow.dflow INFO: Standard error for task 277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/60357f1f-262c-4927-9d30-55d5f63a3dfd_isrForCrosstalkSources_997016_21.stderr 2024-03-15 13:51:33 proc_lsst.multi:170 [INFO] [multi] job_id 17075137 proc_lsst.multi INFO: [multi] job_id 17075137 2024-03-15 13:51:33 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 82 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 82 2024-03-15 13:51:33 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:33 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075137 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075137 parsl.executors.status_handling DEBUG: Launched block 137 on executor multi with job ID 17075137 parsl.executors.status_handling INFO: Allocated block ID 138 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-138 2024-03-15 13:51:33 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-138 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-138 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:33 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 82 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 82 2024-03-15 13:51:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:33 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=138 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 281 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b794c74685f0a3f95d0fcfa2bafee76d with result from task 281 parsl.dataflow.dflow DEBUG: Task 281 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7bb51dc0-b331-42c1-b725-5be15f17f12c_isrForCrosstalkSources_996953_20.stdout parsl.dataflow.dflow INFO: Standard error for task 281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7bb51dc0-b331-42c1-b725-5be15f17f12c_isrForCrosstalkSources_996953_20.stderr 2024-03-15 13:51:34 proc_lsst.multi:170 [INFO] [multi] job_id 17075138 proc_lsst.multi INFO: [multi] job_id 17075138 2024-03-15 13:51:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 83 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 83 2024-03-15 13:51:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:34 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075138 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075138 parsl.executors.status_handling DEBUG: Launched block 138 on executor multi with job ID 17075138 parsl.executors.status_handling INFO: Allocated block ID 139 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-139 2024-03-15 13:51:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-139 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-139 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 83 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 83 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=139 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 238 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea8e14c9873ec7fd15ea2678558960c3 with result from task 238 parsl.dataflow.dflow DEBUG: Task 238 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a5002ea6-e10a-479f-9adf-1c2e402fc927_isrForCrosstalkSources_997022_58.stdout parsl.dataflow.dflow INFO: Standard error for task 238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a5002ea6-e10a-479f-9adf-1c2e402fc927_isrForCrosstalkSources_997022_58.stderr 2024-03-15 13:51:34 proc_lsst.multi:170 [INFO] [multi] job_id 17075139 proc_lsst.multi INFO: [multi] job_id 17075139 2024-03-15 13:51:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 84 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 84 2024-03-15 13:51:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:34 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075139 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075139 parsl.executors.status_handling DEBUG: Launched block 139 on executor multi with job ID 17075139 parsl.executors.status_handling INFO: Allocated block ID 140 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-140 2024-03-15 13:51:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-140 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-140 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 84 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 84 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=140 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 294 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd8fdcde4acb54fb4185315a769ab8e3 with result from task 294 parsl.dataflow.dflow DEBUG: Task 294 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/10a0aa23-cec9-42e8-9fd3-27340c34706d_isrForCrosstalkSources_997009_39.stdout parsl.dataflow.dflow INFO: Standard error for task 294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/10a0aa23-cec9-42e8-9fd3-27340c34706d_isrForCrosstalkSources_997009_39.stderr 2024-03-15 13:51:34 proc_lsst.multi:170 [INFO] [multi] job_id 17075140 proc_lsst.multi INFO: [multi] job_id 17075140 2024-03-15 13:51:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 85 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 85 2024-03-15 13:51:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:34 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075140 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075140 parsl.executors.status_handling DEBUG: Launched block 140 on executor multi with job ID 17075140 parsl.executors.status_handling INFO: Allocated block ID 141 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-141 2024-03-15 13:51:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-141 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-141 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:34 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 85 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 85 2024-03-15 13:51:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=141 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:35 proc_lsst.multi:170 [INFO] [multi] job_id 17075141 proc_lsst.multi INFO: [multi] job_id 17075141 2024-03-15 13:51:35 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 86 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 86 2024-03-15 13:51:35 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:35 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075141 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075141 parsl.executors.status_handling DEBUG: Launched block 141 on executor multi with job ID 17075141 parsl.executors.status_handling INFO: Allocated block ID 142 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-142 2024-03-15 13:51:35 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-142 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-142 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 86 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 86 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:35 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=142 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 285 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a513d9d7e9a597ca02cd6b891f7f297 with result from task 285 parsl.dataflow.dflow DEBUG: Task 285 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/8608d81f-e6ec-41b8-bb86-f56b4aae798e_isrForCrosstalkSources_996955_35.stdout parsl.dataflow.dflow INFO: Standard error for task 285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/8608d81f-e6ec-41b8-bb86-f56b4aae798e_isrForCrosstalkSources_996955_35.stderr 2024-03-15 13:51:35 proc_lsst.multi:170 [INFO] [multi] job_id 17075142 proc_lsst.multi INFO: [multi] job_id 17075142 2024-03-15 13:51:35 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 87 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 87 2024-03-15 13:51:35 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:35 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075142 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075142 parsl.executors.status_handling DEBUG: Launched block 142 on executor multi with job ID 17075142 parsl.executors.status_handling INFO: Allocated block ID 143 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-143 2024-03-15 13:51:35 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-143 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-143 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 87 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 87 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:35 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=143 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 284 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb05a796998b21a70773a3d2244470db with result from task 284 parsl.dataflow.dflow DEBUG: Task 284 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/97c880b6-6aab-4dba-a1ce-7983aca71977_isrForCrosstalkSources_997032_52.stdout parsl.dataflow.dflow INFO: Standard error for task 284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/97c880b6-6aab-4dba-a1ce-7983aca71977_isrForCrosstalkSources_997032_52.stderr parsl.dataflow.dflow INFO: Task 297 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf36490ce00c06a8ee61ab17dbb5c9ef with result from task 297 parsl.dataflow.dflow DEBUG: Task 297 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/15e16b83-35db-4eba-b530-5d2ac4fa91e6_isrForCrosstalkSources_997033_7.stdout parsl.dataflow.dflow INFO: Standard error for task 297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/15e16b83-35db-4eba-b530-5d2ac4fa91e6_isrForCrosstalkSources_997033_7.stderr 2024-03-15 13:51:35 proc_lsst.multi:170 [INFO] [multi] job_id 17075143 proc_lsst.multi INFO: [multi] job_id 17075143 2024-03-15 13:51:35 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 88 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 88 2024-03-15 13:51:35 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:35 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075143 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075143 parsl.executors.status_handling DEBUG: Launched block 143 on executor multi with job ID 17075143 parsl.dataflow.dflow INFO: Task 276 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3ac323996d56c9891513285f26068483 with result from task 276 parsl.executors.status_handling INFO: Allocated block ID 144 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-144 2024-03-15 13:51:35 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-144 parsl.dataflow.dflow DEBUG: Task 276 checkpointed proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-144 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 parsl.dataflow.dflow INFO: Done checkpointing 1 tasks proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn parsl.dataflow.dflow INFO: Standard output for task 276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/42da37ee-6d16-4181-b2da-8b1b6a0afb31_isrForCrosstalkSources_996999_40.stdout proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:35 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 88 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 88 2024-03-15 13:51:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:35 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=144 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.dataflow.dflow INFO: Standard error for task 276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/42da37ee-6d16-4181-b2da-8b1b6a0afb31_isrForCrosstalkSources_996999_40.stderr parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 296 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e4b56eb8d98cc46bc135b7c508a531b with result from task 296 parsl.dataflow.dflow DEBUG: Task 296 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b5cbfa28-8d3f-4388-9eb8-2a071f9a6b4b_isrForCrosstalkSources_997030_37.stdout parsl.dataflow.dflow INFO: Standard error for task 296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b5cbfa28-8d3f-4388-9eb8-2a071f9a6b4b_isrForCrosstalkSources_997030_37.stderr parsl.dataflow.dflow INFO: Task 286 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 647d68b5881b58318541b09214d64858 with result from task 286 parsl.dataflow.dflow DEBUG: Task 286 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/7ec61a31-b64d-49b8-a7e4-7565ad6d3993_isrForCrosstalkSources_996969_40.stdout parsl.dataflow.dflow INFO: Standard error for task 286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/7ec61a31-b64d-49b8-a7e4-7565ad6d3993_isrForCrosstalkSources_996969_40.stderr parsl.dataflow.dflow INFO: Task 278 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5a4e539995d1b6a1b7efb8b2e0e2068 with result from task 278 parsl.dataflow.dflow DEBUG: Task 278 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4a82e811-0542-476e-8b03-e136f26c4bcc_isrForCrosstalkSources_996958_19.stdout parsl.dataflow.dflow INFO: Standard error for task 278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4a82e811-0542-476e-8b03-e136f26c4bcc_isrForCrosstalkSources_996958_19.stderr parsl.dataflow.dflow INFO: Task 180 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b08beb40bb82ab9e77c9fea7fc0cd72 with result from task 180 parsl.dataflow.dflow DEBUG: Task 180 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/edea1a4c-a085-48f4-b1c7-f2ff3ac53ce9_isrForCrosstalkSources_996992_59.stdout parsl.dataflow.dflow INFO: Standard error for task 180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/edea1a4c-a085-48f4-b1c7-f2ff3ac53ce9_isrForCrosstalkSources_996992_59.stderr parsl.dataflow.dflow INFO: Task 179 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aca2244c169b3a93408a7fd80e80fd57 with result from task 179 parsl.dataflow.dflow DEBUG: Task 179 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6326830d-48df-4e3f-a29f-d10e3bc692b2_isrForCrosstalkSources_997005_40.stdout parsl.dataflow.dflow INFO: Standard error for task 179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6326830d-48df-4e3f-a29f-d10e3bc692b2_isrForCrosstalkSources_997005_40.stderr parsl.dataflow.dflow INFO: Task 181 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 162516838ef924313cb81b2df49dd734 with result from task 181 parsl.dataflow.dflow DEBUG: Task 181 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/85becbf1-4997-432a-a255-3106f2ceeb59_isrForCrosstalkSources_997008_10.stdout parsl.dataflow.dflow INFO: Standard error for task 181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/85becbf1-4997-432a-a255-3106f2ceeb59_isrForCrosstalkSources_997008_10.stderr parsl.dataflow.dflow INFO: Task 275 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e54cb63b52c8772ddc4caebe88926f19 with result from task 275 parsl.dataflow.dflow DEBUG: Task 275 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/15d3d7b1-c8b3-40b2-9a1f-ac2367fc81b4_isrForCrosstalkSources_996981_50.stdout parsl.dataflow.dflow INFO: Standard error for task 275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/15d3d7b1-c8b3-40b2-9a1f-ac2367fc81b4_isrForCrosstalkSources_996981_50.stderr 2024-03-15 13:51:36 proc_lsst.multi:170 [INFO] [multi] job_id 17075144 proc_lsst.multi INFO: [multi] job_id 17075144 2024-03-15 13:51:36 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 89 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 89 2024-03-15 13:51:36 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:36 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075144 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075144 parsl.executors.status_handling DEBUG: Launched block 144 on executor multi with job ID 17075144 parsl.executors.status_handling INFO: Allocated block ID 145 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-145 2024-03-15 13:51:36 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-145 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-145 2024-03-15 13:51:36 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:36 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:36 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:36 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:36 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 89 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 89 2024-03-15 13:51:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:36 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=145 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 283 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 813cfad2acdb8b942c9bbf72973bfb89 with result from task 283 parsl.dataflow.dflow DEBUG: Task 283 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/33e8cb2b-c64b-4355-9bc5-6bf51b736caa_isrForCrosstalkSources_997024_22.stdout parsl.dataflow.dflow INFO: Standard error for task 283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/33e8cb2b-c64b-4355-9bc5-6bf51b736caa_isrForCrosstalkSources_997024_22.stderr parsl.dataflow.dflow INFO: Task 272 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d6dc7d1d4fe33171a22eed4d0e1f1fab with result from task 272 parsl.dataflow.dflow DEBUG: Task 272 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/eabc891b-6eb4-4fb1-a73d-ea51cfba0980_isrForCrosstalkSources_996985_28.stdout parsl.dataflow.dflow INFO: Standard error for task 272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/eabc891b-6eb4-4fb1-a73d-ea51cfba0980_isrForCrosstalkSources_996985_28.stderr parsl.dataflow.dflow INFO: Task 265 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c57fb4c01450148c6ae9210eca5b4b70 with result from task 265 parsl.dataflow.dflow DEBUG: Task 265 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bd753d5c-9054-4cbb-9739-38e8fd85a0f0_isrForCrosstalkSources_997026_50.stdout parsl.dataflow.dflow INFO: Standard error for task 265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bd753d5c-9054-4cbb-9739-38e8fd85a0f0_isrForCrosstalkSources_997026_50.stderr parsl.dataflow.dflow INFO: Task 287 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e0a68973eac4023f3367bc3205ee6b9 with result from task 287 parsl.dataflow.dflow DEBUG: Task 287 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/cf160bab-2be0-4346-9d4d-7a0796f76d19_isrForCrosstalkSources_996996_3.stdout parsl.dataflow.dflow INFO: Standard error for task 287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/cf160bab-2be0-4346-9d4d-7a0796f76d19_isrForCrosstalkSources_996996_3.stderr parsl.dataflow.dflow INFO: Task 291 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8dac4cb474610faaa734db80fde8f15a with result from task 291 parsl.dataflow.dflow DEBUG: Task 291 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/1455e419-9c40-43e6-9398-c9732750e85f_isrForCrosstalkSources_996996_44.stdout parsl.dataflow.dflow INFO: Standard error for task 291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/1455e419-9c40-43e6-9398-c9732750e85f_isrForCrosstalkSources_996996_44.stderr parsl.dataflow.dflow INFO: Task 292 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a6f4379975340f0e3e48eb45c9044ba with result from task 292 parsl.dataflow.dflow DEBUG: Task 292 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/8b1f04c0-5783-400b-ac13-0ccc4cf0871b_isrForCrosstalkSources_997023_7.stdout parsl.dataflow.dflow INFO: Standard error for task 292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/8b1f04c0-5783-400b-ac13-0ccc4cf0871b_isrForCrosstalkSources_997023_7.stderr parsl.dataflow.dflow INFO: Task 282 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9aee59e323c5265e79cc4bb5d186afdf with result from task 282 parsl.dataflow.dflow DEBUG: Task 282 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/f4a7ec81-9649-401e-95d3-475724e9862f_isrForCrosstalkSources_996992_9.stdout parsl.dataflow.dflow INFO: Standard error for task 282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/f4a7ec81-9649-401e-95d3-475724e9862f_isrForCrosstalkSources_996992_9.stderr parsl.dataflow.dflow INFO: Task 295 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65fae1543b33042fc7b5b71e898f1035 with result from task 295 2024-03-15 13:51:38 proc_lsst.multi:170 [INFO] [multi] job_id 17075145 proc_lsst.multi INFO: [multi] job_id 17075145 2024-03-15 13:51:38 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 90 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 90 2024-03-15 13:51:38 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:38 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075145 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075145 parsl.executors.status_handling DEBUG: Launched block 145 on executor multi with job ID 17075145 parsl.executors.status_handling INFO: Allocated block ID 146 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-146 2024-03-15 13:51:38 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-146 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-146 2024-03-15 13:51:38 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:38 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:38 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:38 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:38 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 90 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 90 2024-03-15 13:51:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:38 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=146 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 295 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/67a4415e-e372-439e-9a84-8da03b166aae_isrForCrosstalkSources_997002_56.stdout parsl.dataflow.dflow INFO: Standard error for task 295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/67a4415e-e372-439e-9a84-8da03b166aae_isrForCrosstalkSources_997002_56.stderr parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:39 proc_lsst.multi:170 [INFO] [multi] job_id 17075146 proc_lsst.multi INFO: [multi] job_id 17075146 2024-03-15 13:51:39 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 91 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 91 2024-03-15 13:51:39 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:39 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075146 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075146 parsl.executors.status_handling DEBUG: Launched block 146 on executor multi with job ID 17075146 parsl.executors.status_handling INFO: Allocated block ID 147 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-147 2024-03-15 13:51:39 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-147 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-147 2024-03-15 13:51:39 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:39 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:39 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:39 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:39 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 91 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 91 2024-03-15 13:51:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:39 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=147 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 312 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 256b8220249676098857eba0e6969bac with result from task 312 parsl.dataflow.dflow DEBUG: Task 312 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/49cf2ac5-1e07-4e87-afd7-3a93be4513bd_isrForCrosstalkSources_996993_9.stdout parsl.dataflow.dflow INFO: Standard error for task 312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/49cf2ac5-1e07-4e87-afd7-3a93be4513bd_isrForCrosstalkSources_996993_9.stderr 2024-03-15 13:51:40 proc_lsst.multi:170 [INFO] [multi] job_id 17075147 proc_lsst.multi INFO: [multi] job_id 17075147 2024-03-15 13:51:40 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 92 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 92 2024-03-15 13:51:40 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:40 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075147 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075147 parsl.executors.status_handling DEBUG: Launched block 147 on executor multi with job ID 17075147 parsl.executors.status_handling INFO: Allocated block ID 148 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-148 2024-03-15 13:51:40 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-148 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-148 2024-03-15 13:51:40 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:40 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:40 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:40 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:40 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:40 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 92 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 92 2024-03-15 13:51:40 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:40 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=148 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 306 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c83a51311bb1a8773265708a6fcc3630 with result from task 306 parsl.dataflow.dflow DEBUG: Task 306 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/502e7b26-9649-44d4-a942-1cabda176125_isrForCrosstalkSources_997010_37.stdout parsl.dataflow.dflow INFO: Standard error for task 306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/502e7b26-9649-44d4-a942-1cabda176125_isrForCrosstalkSources_997010_37.stderr parsl.dataflow.dflow INFO: Task 279 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed23756b42d32ff04d9d4eefd9a821e0 with result from task 279 parsl.dataflow.dflow DEBUG: Task 279 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/5229ad77-8198-4398-bb20-e2a6d383ea7b_isrForCrosstalkSources_997001_47.stdout parsl.dataflow.dflow INFO: Standard error for task 279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/5229ad77-8198-4398-bb20-e2a6d383ea7b_isrForCrosstalkSources_997001_47.stderr 2024-03-15 13:51:41 proc_lsst.multi:170 [INFO] [multi] job_id 17075148 proc_lsst.multi INFO: [multi] job_id 17075148 2024-03-15 13:51:41 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 93 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 93 2024-03-15 13:51:41 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:41 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075148 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075148 parsl.executors.status_handling DEBUG: Launched block 148 on executor multi with job ID 17075148 parsl.executors.status_handling INFO: Allocated block ID 149 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-149 2024-03-15 13:51:41 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-149 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-149 2024-03-15 13:51:41 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:41 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:41 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:41 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:41 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 93 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 93 2024-03-15 13:51:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:41 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=149 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 288 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd9858d95cb255c2a1e7ea747fad3373 with result from task 288 parsl.dataflow.dflow DEBUG: Task 288 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/92a506a5-f9a6-434f-abc5-eca1ded66ce3_isrForCrosstalkSources_996980_16.stdout parsl.dataflow.dflow INFO: Standard error for task 288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/92a506a5-f9a6-434f-abc5-eca1ded66ce3_isrForCrosstalkSources_996980_16.stderr 2024-03-15 13:51:42 proc_lsst.multi:170 [INFO] [multi] job_id 17075149 proc_lsst.multi INFO: [multi] job_id 17075149 2024-03-15 13:51:42 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 94 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 94 2024-03-15 13:51:42 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:42 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075149 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075149 parsl.executors.status_handling DEBUG: Launched block 149 on executor multi with job ID 17075149 parsl.executors.status_handling INFO: Allocated block ID 150 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-150 2024-03-15 13:51:42 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-150 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-150 2024-03-15 13:51:42 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:42 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:42 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:42 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:42 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 94 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 94 2024-03-15 13:51:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:42 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=150 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 309 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 812c18b65396aa5948fc66e8db6ee90d with result from task 309 parsl.dataflow.dflow DEBUG: Task 309 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/971d351f-baf1-4211-901a-4234aba57741_isrForCrosstalkSources_997013_37.stdout parsl.dataflow.dflow INFO: Standard error for task 309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/971d351f-baf1-4211-901a-4234aba57741_isrForCrosstalkSources_997013_37.stderr parsl.dataflow.dflow INFO: Task 305 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a985c76fdbeca7189839c2e771f46016 with result from task 305 parsl.dataflow.dflow DEBUG: Task 305 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b19cde89-c54f-4ec7-9bed-264a079e9174_isrForCrosstalkSources_996959_37.stdout parsl.dataflow.dflow INFO: Standard error for task 305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b19cde89-c54f-4ec7-9bed-264a079e9174_isrForCrosstalkSources_996959_37.stderr parsl.dataflow.dflow INFO: Task 314 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6bcfc38c518841b9f5e5f327a8c77e43 with result from task 314 parsl.dataflow.dflow DEBUG: Task 314 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/6d520c28-a4cb-4900-a665-c27b0ce63ff5_isrForCrosstalkSources_996965_7.stdout parsl.dataflow.dflow INFO: Standard error for task 314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/6d520c28-a4cb-4900-a665-c27b0ce63ff5_isrForCrosstalkSources_996965_7.stderr 2024-03-15 13:51:43 proc_lsst.multi:170 [INFO] [multi] job_id 17075150 proc_lsst.multi INFO: [multi] job_id 17075150 2024-03-15 13:51:43 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 95 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 95 2024-03-15 13:51:43 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:43 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075150 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075150 parsl.executors.status_handling DEBUG: Launched block 150 on executor multi with job ID 17075150 parsl.executors.status_handling INFO: Allocated block ID 151 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-151 2024-03-15 13:51:43 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-151 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-151 2024-03-15 13:51:43 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:43 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:43 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:43 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:43 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 95 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 95 2024-03-15 13:51:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:43 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=151 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 321 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c697db1f7b7fd7f66997205663b46a5b with result from task 321 parsl.dataflow.dflow DEBUG: Task 321 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/18db146c-a3fa-4cad-aa43-37d914da22ef_isrForCrosstalkSources_996964_27.stdout parsl.dataflow.dflow INFO: Standard error for task 321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/18db146c-a3fa-4cad-aa43-37d914da22ef_isrForCrosstalkSources_996964_27.stderr parsl.dataflow.dflow INFO: Task 332 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f3decd4136131fe8d76d189a714600d with result from task 332 parsl.dataflow.dflow DEBUG: Task 332 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d43c7b07-1bb2-4c15-b36e-bc6d5955a7c0_isrForCrosstalkSources_997004_34.stdout parsl.dataflow.dflow INFO: Standard error for task 332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d43c7b07-1bb2-4c15-b36e-bc6d5955a7c0_isrForCrosstalkSources_997004_34.stderr parsl.dataflow.dflow INFO: Task 328 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 085a3996af57262ee0623c2f4398876d with result from task 328 parsl.dataflow.dflow DEBUG: Task 328 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/3d773dd3-7824-4498-89cd-1c1570ed63b9_isrForCrosstalkSources_996979_21.stdout parsl.dataflow.dflow INFO: Standard error for task 328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/3d773dd3-7824-4498-89cd-1c1570ed63b9_isrForCrosstalkSources_996979_21.stderr parsl.dataflow.dflow INFO: Task 310 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 649d19308603b58084ff4fc82ac1de17 with result from task 310 parsl.dataflow.dflow DEBUG: Task 310 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/5a5a0e4f-bd4b-42e0-ae70-1b62035ff0d9_isrForCrosstalkSources_996973_33.stdout parsl.dataflow.dflow INFO: Standard error for task 310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/5a5a0e4f-bd4b-42e0-ae70-1b62035ff0d9_isrForCrosstalkSources_996973_33.stderr parsl.dataflow.dflow INFO: Task 302 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9743ca0c78f6ee03ac294da821510239 with result from task 302 parsl.dataflow.dflow DEBUG: Task 302 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5b772235-96e5-48bc-a150-4f1a1f7b41dc_isrForCrosstalkSources_996988_33.stdout parsl.dataflow.dflow INFO: Standard error for task 302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5b772235-96e5-48bc-a150-4f1a1f7b41dc_isrForCrosstalkSources_996988_33.stderr parsl.dataflow.dflow INFO: Task 299 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6fce926caebc5b49a1fdb5b431fe562a with result from task 299 parsl.dataflow.dflow DEBUG: Task 299 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/25a91652-7212-4dc7-9072-068886b8998a_isrForCrosstalkSources_996978_14.stdout parsl.dataflow.dflow INFO: Standard error for task 299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/25a91652-7212-4dc7-9072-068886b8998a_isrForCrosstalkSources_996978_14.stderr parsl.dataflow.dflow INFO: Task 316 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d731e1164f6f4929e877b2b2cf935082 with result from task 316 parsl.dataflow.dflow DEBUG: Task 316 checkpointed 2024-03-15 13:51:44 proc_lsst.multi:170 [INFO] [multi] job_id 17075151 proc_lsst.multi INFO: [multi] job_id 17075151 2024-03-15 13:51:44 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 96 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 96 2024-03-15 13:51:44 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:44 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075151 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075151 parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4a868a22-8428-4bb3-b7a1-cc014f182a8d_isrForCrosstalkSources_996959_21.stdout parsl.dataflow.dflow INFO: Standard error for task 316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4a868a22-8428-4bb3-b7a1-cc014f182a8d_isrForCrosstalkSources_996959_21.stderr parsl.executors.status_handling DEBUG: Launched block 151 on executor multi with job ID 17075151 parsl.executors.status_handling INFO: Allocated block ID 152 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-152 2024-03-15 13:51:44 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-152 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-152 2024-03-15 13:51:44 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:44 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:44 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:44 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:44 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 96 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 96 2024-03-15 13:51:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:44 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=152 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 308 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d31eed44970efd10ec961bc58b95baec with result from task 308 parsl.dataflow.dflow DEBUG: Task 308 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/24221d94-3b36-4570-9bf1-b14b41c25c62_isrForCrosstalkSources_997000_26.stdout parsl.dataflow.dflow INFO: Standard error for task 308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/24221d94-3b36-4570-9bf1-b14b41c25c62_isrForCrosstalkSources_997000_26.stderr parsl.dataflow.dflow INFO: Task 311 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c9f7f0c033fe21f0f7cc969203b3b8f with result from task 311 parsl.dataflow.dflow DEBUG: Task 311 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ee691298-c8cf-45fe-9744-d108ffdfd7ce_isrForCrosstalkSources_996990_19.stdout parsl.dataflow.dflow INFO: Standard error for task 311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ee691298-c8cf-45fe-9744-d108ffdfd7ce_isrForCrosstalkSources_996990_19.stderr parsl.dataflow.dflow INFO: Task 289 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fad3ecdd6ef8b2dd4cfd52dbb60e649 with result from task 289 parsl.dataflow.dflow DEBUG: Task 289 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/26fd3b20-7453-4c84-b14e-0d119a50a364_isrForCrosstalkSources_996961_19.stdout parsl.dataflow.dflow INFO: Standard error for task 289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/26fd3b20-7453-4c84-b14e-0d119a50a364_isrForCrosstalkSources_996961_19.stderr parsl.dataflow.dflow INFO: Task 319 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f33cfc68f32e548955ff8daad3104a41 with result from task 319 parsl.dataflow.dflow DEBUG: Task 319 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/92bfe988-877c-4db7-bd77-997b56741a6e_isrForCrosstalkSources_996980_14.stdout parsl.dataflow.dflow INFO: Standard error for task 319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/92bfe988-877c-4db7-bd77-997b56741a6e_isrForCrosstalkSources_996980_14.stderr parsl.dataflow.dflow INFO: Task 307 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be60a12af860dc69f96ab9bd81939f6b with result from task 307 parsl.dataflow.dflow DEBUG: Task 307 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/6b0ec8a7-4460-4545-8076-02e38f06dd9c_isrForCrosstalkSources_996995_45.stdout parsl.dataflow.dflow INFO: Standard error for task 307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/6b0ec8a7-4460-4545-8076-02e38f06dd9c_isrForCrosstalkSources_996995_45.stderr parsl.dataflow.dflow INFO: Task 325 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 692d45bba5169341c1c7d0bb0e8bfc2e with result from task 325 parsl.dataflow.dflow DEBUG: Task 325 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/156726e6-f63e-4090-bb28-f5731baa03f6_isrForCrosstalkSources_997004_1.stdout parsl.dataflow.dflow INFO: Standard error for task 325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/156726e6-f63e-4090-bb28-f5731baa03f6_isrForCrosstalkSources_997004_1.stderr parsl.dataflow.dflow INFO: Task 320 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b8cb7b7b0ee6e593b3a1dc2e4c7d3ae with result from task 320 parsl.dataflow.dflow DEBUG: Task 320 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/55a154e0-7a41-4f67-b3e1-69f61df98cff_isrForCrosstalkSources_997026_13.stdout parsl.dataflow.dflow INFO: Standard error for task 320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/55a154e0-7a41-4f67-b3e1-69f61df98cff_isrForCrosstalkSources_997026_13.stderr 2024-03-15 13:51:45 proc_lsst.multi:170 [INFO] [multi] job_id 17075152 proc_lsst.multi INFO: [multi] job_id 17075152 2024-03-15 13:51:45 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 97 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 97 2024-03-15 13:51:45 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:45 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075152 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075152 parsl.executors.status_handling DEBUG: Launched block 152 on executor multi with job ID 17075152 parsl.executors.status_handling INFO: Allocated block ID 153 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-153 2024-03-15 13:51:45 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-153 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-153 2024-03-15 13:51:45 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:45 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:45 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:45 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:45 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 97 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 97 2024-03-15 13:51:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:45 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=153 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Task 329 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4bc214ab529dee10668e37d85c6c88c with result from task 329 parsl.dataflow.dflow DEBUG: Task 329 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/7a8ed3ab-a07a-40a8-8770-d7bfb0ea0330_isrForCrosstalkSources_996998_62.stdout parsl.dataflow.dflow INFO: Standard error for task 329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/7a8ed3ab-a07a-40a8-8770-d7bfb0ea0330_isrForCrosstalkSources_996998_62.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 326 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6612c521f0fd1dcd31e815849ad1ec8e with result from task 326 parsl.dataflow.dflow DEBUG: Task 326 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/431248ba-297d-49dd-b723-e4790b36b983_isrForCrosstalkSources_997012_19.stdout parsl.dataflow.dflow INFO: Standard error for task 326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/431248ba-297d-49dd-b723-e4790b36b983_isrForCrosstalkSources_997012_19.stderr parsl.dataflow.dflow INFO: Task 280 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a99a57fad86ae4723401231e8e2dbf49 with result from task 280 parsl.dataflow.dflow DEBUG: Task 280 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/e33f2f9f-50df-4b20-bcaf-48b2ac89ae17_isrForCrosstalkSources_996982_58.stdout parsl.dataflow.dflow INFO: Standard error for task 280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/e33f2f9f-50df-4b20-bcaf-48b2ac89ae17_isrForCrosstalkSources_996982_58.stderr parsl.dataflow.dflow INFO: Task 334 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d3909ed4ad20076db1715f4576e25cb with result from task 334 parsl.dataflow.dflow DEBUG: Task 334 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c6e55c69-c054-46d0-a6ba-ca4027cad1f1_isrForCrosstalkSources_997000_37.stdout parsl.dataflow.dflow INFO: Standard error for task 334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c6e55c69-c054-46d0-a6ba-ca4027cad1f1_isrForCrosstalkSources_997000_37.stderr parsl.dataflow.dflow INFO: Task 300 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02dba0f375c97d57e62141e9fdd1e67f with result from task 300 parsl.dataflow.dflow DEBUG: Task 300 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/06abc97e-f9ec-4ac0-8337-0217ccac31ce_isrForCrosstalkSources_997013_50.stdout parsl.dataflow.dflow INFO: Standard error for task 300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/06abc97e-f9ec-4ac0-8337-0217ccac31ce_isrForCrosstalkSources_997013_50.stderr parsl.dataflow.dflow INFO: Task 322 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6de8ce80192e5f687e6564721094f00e with result from task 322 parsl.dataflow.dflow DEBUG: Task 322 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/2ac8b7e5-9b5d-4147-a0a1-c5dee24afe1e_isrForCrosstalkSources_997024_9.stdout parsl.dataflow.dflow INFO: Standard error for task 322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/2ac8b7e5-9b5d-4147-a0a1-c5dee24afe1e_isrForCrosstalkSources_997024_9.stderr parsl.dataflow.dflow INFO: Task 318 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd8c5950835aa0c067071828d9128a71 with result from task 318 parsl.dataflow.dflow DEBUG: Task 318 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ad04b079-2c07-4108-8ecc-56988dd77fd8_isrForCrosstalkSources_997021_41.stdout parsl.dataflow.dflow INFO: Standard error for task 318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ad04b079-2c07-4108-8ecc-56988dd77fd8_isrForCrosstalkSources_997021_41.stderr 2024-03-15 13:51:46 proc_lsst.multi:170 [INFO] [multi] job_id 17075153 proc_lsst.multi INFO: [multi] job_id 17075153 2024-03-15 13:51:46 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 98 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 98 2024-03-15 13:51:46 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:46 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075153 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075153 parsl.executors.status_handling DEBUG: Launched block 153 on executor multi with job ID 17075153 parsl.executors.status_handling INFO: Allocated block ID 154 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-154 2024-03-15 13:51:46 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-154 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-154 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 98 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 98 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:46 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=154 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 317 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 452f5583b5c63bad998274b3f5af776e with result from task 317 parsl.dataflow.dflow DEBUG: Task 317 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/75d9d059-cfe1-4014-a507-14b029c2877c_isrForCrosstalkSources_996971_51.stdout parsl.dataflow.dflow INFO: Standard error for task 317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/75d9d059-cfe1-4014-a507-14b029c2877c_isrForCrosstalkSources_996971_51.stderr parsl.dataflow.dflow INFO: Task 298 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa944bfb773fc83b0612c3a91ee27c79 with result from task 298 parsl.dataflow.dflow DEBUG: Task 298 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/806ef374-d179-4a99-8858-d6ed95a132dc_isrForCrosstalkSources_997009_2.stdout parsl.dataflow.dflow INFO: Standard error for task 298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/806ef374-d179-4a99-8858-d6ed95a132dc_isrForCrosstalkSources_997009_2.stderr parsl.dataflow.dflow INFO: Task 343 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0814f1d4263ab4834653f570fec3fd4c with result from task 343 parsl.dataflow.dflow DEBUG: Task 343 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e02b4e3b-7bee-464c-aced-86df7821d630_isrForCrosstalkSources_996965_13.stdout parsl.dataflow.dflow INFO: Standard error for task 343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e02b4e3b-7bee-464c-aced-86df7821d630_isrForCrosstalkSources_996965_13.stderr parsl.dataflow.dflow INFO: Task 315 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86eda3650cbf752d473cf883cb96bdf9 with result from task 315 parsl.dataflow.dflow DEBUG: Task 315 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d52954e8-d53e-4546-9111-fab797915dc2_isrForCrosstalkSources_997024_44.stdout parsl.dataflow.dflow INFO: Standard error for task 315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d52954e8-d53e-4546-9111-fab797915dc2_isrForCrosstalkSources_997024_44.stderr parsl.dataflow.dflow INFO: Task 324 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a183924b70827f8672d624135c3b5ff2 with result from task 324 parsl.dataflow.dflow DEBUG: Task 324 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f848d1c7-6460-49b2-884c-0656f7213aa5_isrForCrosstalkSources_997020_23.stdout parsl.dataflow.dflow INFO: Standard error for task 324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f848d1c7-6460-49b2-884c-0656f7213aa5_isrForCrosstalkSources_997020_23.stderr parsl.dataflow.dflow INFO: Task 323 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da3f4c431b19871b5de706653ec2899b with result from task 323 parsl.dataflow.dflow DEBUG: Task 323 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/3e7182ca-683e-42d5-8ff4-040b078571b9_isrForCrosstalkSources_997006_43.stdout parsl.dataflow.dflow INFO: Standard error for task 323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/3e7182ca-683e-42d5-8ff4-040b078571b9_isrForCrosstalkSources_997006_43.stderr 2024-03-15 13:51:46 proc_lsst.multi:170 [INFO] [multi] job_id 17075154 proc_lsst.multi INFO: [multi] job_id 17075154 2024-03-15 13:51:46 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 99 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 99 2024-03-15 13:51:46 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:46 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075154 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075154 parsl.executors.status_handling DEBUG: Launched block 154 on executor multi with job ID 17075154 parsl.executors.status_handling INFO: Allocated block ID 155 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-155 2024-03-15 13:51:46 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-155 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-155 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:46 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 99 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 99 2024-03-15 13:51:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:46 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=155 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 303 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0440ed9ddca6de162479a70d5d8fcf1 with result from task 303 parsl.dataflow.dflow DEBUG: Task 303 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f894cbe2-5a66-4572-9fad-1ab6da42dc05_isrForCrosstalkSources_996996_61.stdout parsl.dataflow.dflow INFO: Standard error for task 303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f894cbe2-5a66-4572-9fad-1ab6da42dc05_isrForCrosstalkSources_996996_61.stderr parsl.dataflow.dflow INFO: Task 336 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry beee3b6f7f7ac419880d28fc965b93b8 with result from task 336 parsl.dataflow.dflow DEBUG: Task 336 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2ae830b2-8627-4675-ae6e-3af9b2725501_isrForCrosstalkSources_996980_8.stdout parsl.dataflow.dflow INFO: Standard error for task 336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2ae830b2-8627-4675-ae6e-3af9b2725501_isrForCrosstalkSources_996980_8.stderr parsl.dataflow.dflow INFO: Task 327 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c10e125fd5eded6fd98afd3eb6dc0af with result from task 327 parsl.dataflow.dflow DEBUG: Task 327 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/1dc0914c-8f9d-485b-a290-827b9c7145a5_isrForCrosstalkSources_996969_54.stdout parsl.dataflow.dflow INFO: Standard error for task 327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/1dc0914c-8f9d-485b-a290-827b9c7145a5_isrForCrosstalkSources_996969_54.stderr parsl.dataflow.dflow INFO: Task 290 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ce6f66b38b676417ca16780db959460 with result from task 290 parsl.dataflow.dflow DEBUG: Task 290 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5d955431-4aa5-45ea-98ad-39fdb0e4582b_isrForCrosstalkSources_997021_6.stdout parsl.dataflow.dflow INFO: Standard error for task 290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5d955431-4aa5-45ea-98ad-39fdb0e4582b_isrForCrosstalkSources_997021_6.stderr parsl.dataflow.dflow INFO: Task 293 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4bf04bc74f230ad0df9771f428e8eee with result from task 293 parsl.dataflow.dflow DEBUG: Task 293 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/f1bd9f09-a515-47e4-bdb8-c6a9f809c847_isrForCrosstalkSources_996998_35.stdout parsl.dataflow.dflow INFO: Standard error for task 293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/f1bd9f09-a515-47e4-bdb8-c6a9f809c847_isrForCrosstalkSources_996998_35.stderr 2024-03-15 13:51:47 proc_lsst.multi:170 [INFO] [multi] job_id 17075155 proc_lsst.multi INFO: [multi] job_id 17075155 2024-03-15 13:51:47 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 100 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 100 2024-03-15 13:51:47 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:47 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075155 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075155 parsl.executors.status_handling DEBUG: Launched block 155 on executor multi with job ID 17075155 parsl.executors.status_handling INFO: Allocated block ID 156 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-156 2024-03-15 13:51:47 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-156 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-156 2024-03-15 13:51:47 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:47 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:47 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:47 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:47 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 100 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 100 2024-03-15 13:51:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:47 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=156 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 313 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 529cf4fe7d12b31bf12d43e948f8e56c with result from task 313 parsl.dataflow.dflow DEBUG: Task 313 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b9c4b36d-d690-4fa1-88fc-236cd7f58ad9_isrForCrosstalkSources_996966_2.stdout parsl.dataflow.dflow INFO: Standard error for task 313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b9c4b36d-d690-4fa1-88fc-236cd7f58ad9_isrForCrosstalkSources_996966_2.stderr parsl.dataflow.dflow INFO: Task 347 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7d3b781ad5992eaac6b0848a9f74be2 with result from task 347 parsl.dataflow.dflow DEBUG: Task 347 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/427b348e-8c29-4262-9824-ee22283ef5b9_isrForCrosstalkSources_996958_14.stdout parsl.dataflow.dflow INFO: Standard error for task 347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/427b348e-8c29-4262-9824-ee22283ef5b9_isrForCrosstalkSources_996958_14.stderr parsl.dataflow.dflow INFO: Task 330 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4952c9cca1fdc9b168794f2a39f583f with result from task 330 parsl.dataflow.dflow DEBUG: Task 330 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a7c854dd-f037-4b8c-a63e-1177f5a02c08_isrForCrosstalkSources_996978_61.stdout parsl.dataflow.dflow INFO: Standard error for task 330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a7c854dd-f037-4b8c-a63e-1177f5a02c08_isrForCrosstalkSources_996978_61.stderr parsl.dataflow.dflow INFO: Task 335 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6cf0fd709e9c431a7cc74a5011f3508 with result from task 335 parsl.dataflow.dflow DEBUG: Task 335 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/c936e6ea-b16e-4e1a-8a45-2ce3fd7297aa_isrForCrosstalkSources_996978_31.stdout parsl.dataflow.dflow INFO: Standard error for task 335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/c936e6ea-b16e-4e1a-8a45-2ce3fd7297aa_isrForCrosstalkSources_996978_31.stderr parsl.dataflow.dflow INFO: Task 333 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 74c0dd7177fdaa47d74d57a743c81335 with result from task 333 parsl.dataflow.dflow DEBUG: Task 333 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/18953af5-9b64-4037-b488-69c23e5486e1_isrForCrosstalkSources_997028_24.stdout parsl.dataflow.dflow INFO: Standard error for task 333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/18953af5-9b64-4037-b488-69c23e5486e1_isrForCrosstalkSources_997028_24.stderr parsl.dataflow.dflow INFO: Task 301 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e045d044b4d2c9b5a5c421eafc7c2e2 with result from task 301 parsl.dataflow.dflow DEBUG: Task 301 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/2750800c-8105-418d-b7ce-7b84c35f7b5e_isrForCrosstalkSources_996958_42.stdout parsl.dataflow.dflow INFO: Standard error for task 301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/2750800c-8105-418d-b7ce-7b84c35f7b5e_isrForCrosstalkSources_996958_42.stderr parsl.dataflow.dflow INFO: Task 331 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry faddcc7faae2607eca5c36febbf56933 with result from task 331 parsl.dataflow.dflow DEBUG: Task 331 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/763d1a83-5316-4e41-9392-9e20fb73966e_isrForCrosstalkSources_997005_7.stdout parsl.dataflow.dflow INFO: Standard error for task 331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/763d1a83-5316-4e41-9392-9e20fb73966e_isrForCrosstalkSources_997005_7.stderr parsl.dataflow.dflow INFO: Task 304 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd687c82e1cb1af40089135613cce284 with result from task 304 parsl.dataflow.dflow DEBUG: Task 304 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/95793988-dc05-44ee-b9b5-25efdcf76041_isrForCrosstalkSources_997033_28.stdout parsl.dataflow.dflow INFO: Standard error for task 304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/95793988-dc05-44ee-b9b5-25efdcf76041_isrForCrosstalkSources_997033_28.stderr 2024-03-15 13:51:48 proc_lsst.multi:170 [INFO] [multi] job_id 17075157 proc_lsst.multi INFO: [multi] job_id 17075157 2024-03-15 13:51:48 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 101 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 101 2024-03-15 13:51:48 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:48 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075157 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075157 parsl.executors.status_handling DEBUG: Launched block 156 on executor multi with job ID 17075157 parsl.executors.status_handling INFO: Allocated block ID 157 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-157 2024-03-15 13:51:48 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-157 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-157 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 101 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 101 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:48 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=157 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:48 proc_lsst.multi:170 [INFO] [multi] job_id 17075159 proc_lsst.multi INFO: [multi] job_id 17075159 2024-03-15 13:51:48 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 102 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 102 2024-03-15 13:51:48 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:48 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075159 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075159 parsl.executors.status_handling DEBUG: Launched block 157 on executor multi with job ID 17075159 parsl.executors.status_handling INFO: Allocated block ID 158 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-158 2024-03-15 13:51:48 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-158 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-158 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:48 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 102 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 102 2024-03-15 13:51:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:48 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=158 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:51:49 proc_lsst.multi:170 [INFO] [multi] job_id 17075160 proc_lsst.multi INFO: [multi] job_id 17075160 2024-03-15 13:51:49 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 103 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 103 2024-03-15 13:51:49 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:49 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075160 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075160 parsl.executors.status_handling DEBUG: Launched block 158 on executor multi with job ID 17075160 parsl.executors.status_handling INFO: Allocated block ID 159 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-159 2024-03-15 13:51:49 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-159 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-159 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 103 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 103 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:49 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=159 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 345 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d4b94ec3212263a1997fc6dd491fdeb with result from task 345 parsl.dataflow.dflow DEBUG: Task 345 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/da1de5af-941f-42df-99ba-3ed268c4f19e_isrForCrosstalkSources_996996_45.stdout parsl.dataflow.dflow INFO: Standard error for task 345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/da1de5af-941f-42df-99ba-3ed268c4f19e_isrForCrosstalkSources_996996_45.stderr 2024-03-15 13:51:49 proc_lsst.multi:170 [INFO] [multi] job_id 17075161 proc_lsst.multi INFO: [multi] job_id 17075161 2024-03-15 13:51:49 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 104 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 104 2024-03-15 13:51:49 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:49 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075161 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075161 parsl.executors.status_handling DEBUG: Launched block 159 on executor multi with job ID 17075161 parsl.executors.status_handling INFO: Allocated block ID 160 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-160 2024-03-15 13:51:49 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-160 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-160 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:49 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 104 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 104 2024-03-15 13:51:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:49 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=160 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 346 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38b8fbcbc8a35a6b2026032065c6dce7 with result from task 346 parsl.dataflow.dflow DEBUG: Task 346 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/14f81efe-6d05-4054-a2d0-686d364c8633_isrForCrosstalkSources_997029_36.stdout parsl.dataflow.dflow INFO: Standard error for task 346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/14f81efe-6d05-4054-a2d0-686d364c8633_isrForCrosstalkSources_997029_36.stderr 2024-03-15 13:51:50 proc_lsst.multi:170 [INFO] [multi] job_id 17075162 proc_lsst.multi INFO: [multi] job_id 17075162 2024-03-15 13:51:50 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 105 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 105 2024-03-15 13:51:50 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:50 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075162 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075162 parsl.executors.status_handling DEBUG: Launched block 160 on executor multi with job ID 17075162 parsl.executors.status_handling INFO: Allocated block ID 161 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-161 2024-03-15 13:51:50 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-161 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-161 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 105 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 105 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:50 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=161 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 344 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a4632fc283eab22f161007446d44e00 with result from task 344 parsl.dataflow.dflow DEBUG: Task 344 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/00a89453-6805-48e8-955f-40e987b6c18c_isrForCrosstalkSources_997030_53.stdout parsl.dataflow.dflow INFO: Standard error for task 344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/00a89453-6805-48e8-955f-40e987b6c18c_isrForCrosstalkSources_997030_53.stderr parsl.dataflow.dflow INFO: Task 341 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1cea99f427da98b9991fb3ece98066dc with result from task 341 parsl.dataflow.dflow DEBUG: Task 341 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/1db2cd6b-03ff-4912-a01e-2d31bb7e402f_isrForCrosstalkSources_997006_41.stdout parsl.dataflow.dflow INFO: Standard error for task 341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/1db2cd6b-03ff-4912-a01e-2d31bb7e402f_isrForCrosstalkSources_997006_41.stderr 2024-03-15 13:51:50 proc_lsst.multi:170 [INFO] [multi] job_id 17075163 proc_lsst.multi INFO: [multi] job_id 17075163 2024-03-15 13:51:50 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 106 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 106 2024-03-15 13:51:50 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:50 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075163 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075163 parsl.executors.status_handling DEBUG: Launched block 161 on executor multi with job ID 17075163 parsl.executors.status_handling INFO: Allocated block ID 162 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-162 2024-03-15 13:51:50 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-162 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-162 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:50 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 106 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 106 2024-03-15 13:51:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:50 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=162 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 348 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec9858d33efea5f3a7425f0917ff61dc with result from task 348 parsl.dataflow.dflow DEBUG: Task 348 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/93c27a30-f3d7-4eba-8dfa-d2630a7e12e6_isrForCrosstalkSources_997028_26.stdout parsl.dataflow.dflow INFO: Standard error for task 348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/93c27a30-f3d7-4eba-8dfa-d2630a7e12e6_isrForCrosstalkSources_997028_26.stderr parsl.dataflow.dflow INFO: Task 351 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 139ce654e78d11789a7c521b7f363830 with result from task 351 parsl.dataflow.dflow DEBUG: Task 351 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/7e19d65e-1294-4bee-90ed-28ffb8c4295d_isrForCrosstalkSources_996979_27.stdout parsl.dataflow.dflow INFO: Standard error for task 351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/7e19d65e-1294-4bee-90ed-28ffb8c4295d_isrForCrosstalkSources_996979_27.stderr parsl.dataflow.dflow INFO: Task 342 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1246d912ba18b32fe40766dbd3311a21 with result from task 342 parsl.dataflow.dflow DEBUG: Task 342 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/53fc4f8e-57dd-4304-ae4c-7b3b95adf092_isrForCrosstalkSources_996983_44.stdout parsl.dataflow.dflow INFO: Standard error for task 342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/53fc4f8e-57dd-4304-ae4c-7b3b95adf092_isrForCrosstalkSources_996983_44.stderr 2024-03-15 13:51:51 proc_lsst.multi:170 [INFO] [multi] job_id 17075166 proc_lsst.multi INFO: [multi] job_id 17075166 2024-03-15 13:51:51 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 107 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 107 2024-03-15 13:51:51 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:51 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075166 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075166 parsl.executors.status_handling DEBUG: Launched block 162 on executor multi with job ID 17075166 parsl.executors.status_handling INFO: Allocated block ID 163 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-163 2024-03-15 13:51:51 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-163 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-163 2024-03-15 13:51:51 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:51 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:51 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:51 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:51 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 107 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 107 2024-03-15 13:51:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:51 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=163 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 350 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f170ee67c9ed6b81962ba28e94a6aa65 with result from task 350 parsl.dataflow.dflow DEBUG: Task 350 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f88678b7-44c3-4cf6-8d9a-36ee13c420fa_isrForCrosstalkSources_996974_40.stdout parsl.dataflow.dflow INFO: Standard error for task 350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f88678b7-44c3-4cf6-8d9a-36ee13c420fa_isrForCrosstalkSources_996974_40.stderr parsl.dataflow.dflow INFO: Task 349 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4da613a5a19a449880da36e042bb7469 with result from task 349 parsl.dataflow.dflow DEBUG: Task 349 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5ea8ebce-7771-462d-93af-ac613467575d_isrForCrosstalkSources_996964_33.stdout parsl.dataflow.dflow INFO: Standard error for task 349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5ea8ebce-7771-462d-93af-ac613467575d_isrForCrosstalkSources_996964_33.stderr parsl.dataflow.dflow INFO: Task 352 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e852add73bc342a26f5b499de398f5c with result from task 352 parsl.dataflow.dflow DEBUG: Task 352 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/cdf68f38-7cee-4af2-bbbf-afdc4c953136_isrForCrosstalkSources_997000_13.stdout parsl.dataflow.dflow INFO: Standard error for task 352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/cdf68f38-7cee-4af2-bbbf-afdc4c953136_isrForCrosstalkSources_997000_13.stderr parsl.dataflow.dflow INFO: Task 357 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 342c2c18ab4d70d053c6afea80a92f3b with result from task 357 parsl.dataflow.dflow DEBUG: Task 357 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/98686114-5397-477b-bc63-4aee57c7cd1d_isrForCrosstalkSources_996990_4.stdout parsl.dataflow.dflow INFO: Standard error for task 357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/98686114-5397-477b-bc63-4aee57c7cd1d_isrForCrosstalkSources_996990_4.stderr parsl.dataflow.dflow INFO: Task 358 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00b3ddc3e2c73704eee7be86a79eefec with result from task 358 parsl.dataflow.dflow DEBUG: Task 358 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ccb8a67a-1906-475c-bfaa-73528e8dce19_isrForCrosstalkSources_997006_4.stdout parsl.dataflow.dflow INFO: Standard error for task 358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ccb8a67a-1906-475c-bfaa-73528e8dce19_isrForCrosstalkSources_997006_4.stderr 2024-03-15 13:51:53 proc_lsst.multi:170 [INFO] [multi] job_id 17075167 proc_lsst.multi INFO: [multi] job_id 17075167 2024-03-15 13:51:53 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 108 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 108 2024-03-15 13:51:53 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:53 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075167 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075167 parsl.executors.status_handling DEBUG: Launched block 163 on executor multi with job ID 17075167 parsl.executors.status_handling INFO: Allocated block ID 164 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-164 2024-03-15 13:51:53 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-164 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-164 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 108 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 108 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:53 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=164 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 354 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f69ab3438ee785ff34a505c1ff44e510 with result from task 354 parsl.dataflow.dflow DEBUG: Task 354 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7d55a850-ad4b-4e26-b003-d92d02bd97b4_isrForCrosstalkSources_996975_46.stdout parsl.dataflow.dflow INFO: Standard error for task 354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7d55a850-ad4b-4e26-b003-d92d02bd97b4_isrForCrosstalkSources_996975_46.stderr 2024-03-15 13:51:53 proc_lsst.multi:170 [INFO] [multi] job_id 17075168 proc_lsst.multi INFO: [multi] job_id 17075168 2024-03-15 13:51:53 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 109 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 109 2024-03-15 13:51:53 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:53 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075168 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075168 parsl.executors.status_handling DEBUG: Launched block 164 on executor multi with job ID 17075168 parsl.executors.status_handling INFO: Allocated block ID 165 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-165 2024-03-15 13:51:53 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-165 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-165 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:53 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 109 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 109 2024-03-15 13:51:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:53 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=165 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 381 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3bf57cc8f1f4b253fcd24273d9e40f49 with result from task 381 parsl.dataflow.dflow DEBUG: Task 381 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9d1c647e-537e-4dfb-900e-8464de1c1999_isrForCrosstalkSources_996986_46.stdout parsl.dataflow.dflow INFO: Standard error for task 381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9d1c647e-537e-4dfb-900e-8464de1c1999_isrForCrosstalkSources_996986_46.stderr parsl.dataflow.dflow INFO: Task 340 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d142f69abbfad850d4c8e70c35734427 with result from task 340 parsl.dataflow.dflow DEBUG: Task 340 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/862b2700-ce51-42f2-acc5-24c286107113_isrForCrosstalkSources_997031_55.stdout parsl.dataflow.dflow INFO: Standard error for task 340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/862b2700-ce51-42f2-acc5-24c286107113_isrForCrosstalkSources_997031_55.stderr parsl.dataflow.dflow INFO: Task 368 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b17e9caa6cfc6094238ea6111e869ea9 with result from task 368 parsl.dataflow.dflow DEBUG: Task 368 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b900ab46-53ef-4b75-8ac8-693ec175ad86_isrForCrosstalkSources_996986_51.stdout parsl.dataflow.dflow INFO: Standard error for task 368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b900ab46-53ef-4b75-8ac8-693ec175ad86_isrForCrosstalkSources_996986_51.stderr 2024-03-15 13:51:54 proc_lsst.multi:170 [INFO] [multi] job_id 17075171 proc_lsst.multi INFO: [multi] job_id 17075171 2024-03-15 13:51:54 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 110 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 110 2024-03-15 13:51:54 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:54 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075171 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075171 parsl.executors.status_handling DEBUG: Launched block 165 on executor multi with job ID 17075171 parsl.executors.status_handling INFO: Allocated block ID 166 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-166 2024-03-15 13:51:54 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-166 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-166 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 110 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 110 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:54 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=166 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 370 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 49c90d5c8a89675415bd9336358dc774 with result from task 370 parsl.dataflow.dflow DEBUG: Task 370 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c5080bfe-5c5a-48b7-bb4d-d1fdcb27437d_isrForCrosstalkSources_997015_49.stdout parsl.dataflow.dflow INFO: Standard error for task 370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c5080bfe-5c5a-48b7-bb4d-d1fdcb27437d_isrForCrosstalkSources_997015_49.stderr 2024-03-15 13:51:54 proc_lsst.multi:170 [INFO] [multi] job_id 17075172 proc_lsst.multi INFO: [multi] job_id 17075172 2024-03-15 13:51:54 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 111 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 111 2024-03-15 13:51:54 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:54 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075172 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075172 parsl.executors.status_handling DEBUG: Launched block 166 on executor multi with job ID 17075172 parsl.executors.status_handling INFO: Allocated block ID 167 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-167 2024-03-15 13:51:54 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-167 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-167 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:54 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 111 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 111 2024-03-15 13:51:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:54 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=167 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 353 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 49e0043c0cf379f5085ca48c053f64b6 with result from task 353 parsl.dataflow.dflow DEBUG: Task 353 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a23ba9fb-ab82-49ca-b1da-6d811ca5ba89_isrForCrosstalkSources_997027_35.stdout parsl.dataflow.dflow INFO: Standard error for task 353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a23ba9fb-ab82-49ca-b1da-6d811ca5ba89_isrForCrosstalkSources_997027_35.stderr 2024-03-15 13:51:55 proc_lsst.multi:170 [INFO] [multi] job_id 17075173 proc_lsst.multi INFO: [multi] job_id 17075173 2024-03-15 13:51:55 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 112 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 112 2024-03-15 13:51:55 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:55 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075173 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075173 parsl.executors.status_handling DEBUG: Launched block 167 on executor multi with job ID 17075173 parsl.executors.status_handling INFO: Allocated block ID 168 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-168 2024-03-15 13:51:55 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-168 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-168 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 112 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 112 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:55 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=168 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 387 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73be8686466734aaf43463373d794824 with result from task 387 parsl.dataflow.dflow DEBUG: Task 387 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/1848fb79-937f-4091-b602-e29cfdb08520_isrForCrosstalkSources_996989_39.stdout parsl.dataflow.dflow INFO: Standard error for task 387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/1848fb79-937f-4091-b602-e29cfdb08520_isrForCrosstalkSources_996989_39.stderr 2024-03-15 13:51:55 proc_lsst.multi:170 [INFO] [multi] job_id 17075179 proc_lsst.multi INFO: [multi] job_id 17075179 2024-03-15 13:51:55 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 113 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 113 2024-03-15 13:51:55 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:55 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075179 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075179 parsl.executors.status_handling DEBUG: Launched block 168 on executor multi with job ID 17075179 parsl.executors.status_handling INFO: Allocated block ID 169 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-169 2024-03-15 13:51:55 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-169 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-169 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:55 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 113 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 113 2024-03-15 13:51:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:55 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=169 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 372 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 190dab8374da269c46e346a6e9d0aa2e with result from task 372 parsl.dataflow.dflow DEBUG: Task 372 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5ca73b03-10c7-4dbf-a5b6-52fd51df1b62_isrForCrosstalkSources_996954_37.stdout parsl.dataflow.dflow INFO: Standard error for task 372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5ca73b03-10c7-4dbf-a5b6-52fd51df1b62_isrForCrosstalkSources_996954_37.stderr parsl.dataflow.dflow INFO: Task 337 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4c355b8b03cd6f215dcdc573290e8a7 with result from task 337 parsl.dataflow.dflow DEBUG: Task 337 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/02c6bedf-36e4-4c8d-85ef-41939d7aa451_isrForCrosstalkSources_997010_27.stdout parsl.dataflow.dflow INFO: Standard error for task 337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/02c6bedf-36e4-4c8d-85ef-41939d7aa451_isrForCrosstalkSources_997010_27.stderr parsl.dataflow.dflow INFO: Task 355 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86cb0bb80369ce8109f180ebf653cb88 with result from task 355 parsl.dataflow.dflow DEBUG: Task 355 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/41a96e08-d826-4473-85aa-f669be62655e_isrForCrosstalkSources_997001_55.stdout parsl.dataflow.dflow INFO: Standard error for task 355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/41a96e08-d826-4473-85aa-f669be62655e_isrForCrosstalkSources_997001_55.stderr 2024-03-15 13:51:56 proc_lsst.multi:170 [INFO] [multi] job_id 17075180 proc_lsst.multi INFO: [multi] job_id 17075180 2024-03-15 13:51:56 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 114 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 114 2024-03-15 13:51:56 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:56 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075180 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075180 parsl.executors.status_handling DEBUG: Launched block 169 on executor multi with job ID 17075180 parsl.executors.status_handling INFO: Allocated block ID 170 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-170 2024-03-15 13:51:56 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-170 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-170 2024-03-15 13:51:56 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:56 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:56 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:56 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:56 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 114 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 114 2024-03-15 13:51:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:56 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=170 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 493 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07a98a81ce644841114265aa662f8d1b with result from task 493 parsl.dataflow.dflow DEBUG: Task 493 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/00695c9e-94b6-4f6e-b4da-1949cb179150_isrForCrosstalkSources_997018_33.stdout parsl.dataflow.dflow INFO: Standard error for task 493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/00695c9e-94b6-4f6e-b4da-1949cb179150_isrForCrosstalkSources_997018_33.stderr parsl.dataflow.dflow INFO: Task 338 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f42cada4f323915025d7988d59480859 with result from task 338 parsl.dataflow.dflow DEBUG: Task 338 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/de64a921-098e-4d5d-8c78-b98efdb21da1_isrForCrosstalkSources_997027_21.stdout parsl.dataflow.dflow INFO: Standard error for task 338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/de64a921-098e-4d5d-8c78-b98efdb21da1_isrForCrosstalkSources_997027_21.stderr parsl.dataflow.dflow INFO: Task 384 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 607a10a99a763eab444dee318a7be5fb with result from task 384 parsl.dataflow.dflow DEBUG: Task 384 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/81db0716-d415-4105-8e8f-f15c0e390155_isrForCrosstalkSources_996982_43.stdout parsl.dataflow.dflow INFO: Standard error for task 384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/81db0716-d415-4105-8e8f-f15c0e390155_isrForCrosstalkSources_996982_43.stderr 2024-03-15 13:51:57 proc_lsst.multi:170 [INFO] [multi] job_id 17075181 proc_lsst.multi INFO: [multi] job_id 17075181 2024-03-15 13:51:57 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 115 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 115 2024-03-15 13:51:57 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:57 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075181 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075181 parsl.executors.status_handling DEBUG: Launched block 170 on executor multi with job ID 17075181 parsl.executors.status_handling INFO: Allocated block ID 171 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-171 2024-03-15 13:51:57 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-171 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-171 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 115 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 115 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:57 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=171 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 371 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b96e51ca187965183e22c9fb83b051d8 with result from task 371 parsl.dataflow.dflow DEBUG: Task 371 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/a94e9bdd-d543-47d1-9163-c2bbeb0ecab8_isrForCrosstalkSources_996974_38.stdout parsl.dataflow.dflow INFO: Standard error for task 371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/a94e9bdd-d543-47d1-9163-c2bbeb0ecab8_isrForCrosstalkSources_996974_38.stderr parsl.dataflow.dflow INFO: Task 432 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 861dc4dd88d5521d477ee5487d735b1f with result from task 432 parsl.dataflow.dflow DEBUG: Task 432 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c3e04b10-3653-4912-9fbf-ae46c97a4229_isrForCrosstalkSources_996980_30.stdout parsl.dataflow.dflow INFO: Standard error for task 432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c3e04b10-3653-4912-9fbf-ae46c97a4229_isrForCrosstalkSources_996980_30.stderr 2024-03-15 13:51:57 proc_lsst.multi:170 [INFO] [multi] job_id 17075182 proc_lsst.multi INFO: [multi] job_id 17075182 2024-03-15 13:51:57 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 116 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 116 2024-03-15 13:51:57 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:57 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075182 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075182 parsl.executors.status_handling DEBUG: Launched block 171 on executor multi with job ID 17075182 parsl.executors.status_handling INFO: Allocated block ID 172 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-172 2024-03-15 13:51:57 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-172 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-172 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:57 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 116 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 116 2024-03-15 13:51:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:57 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=172 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Task 339 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba83912d5f5400220d904b54eec20a55 with result from task 339 parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Task 339 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4d4bb956-f0dc-4b3d-9490-2f847703aec8_isrForCrosstalkSources_997020_2.stdout parsl.dataflow.dflow INFO: Standard error for task 339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4d4bb956-f0dc-4b3d-9490-2f847703aec8_isrForCrosstalkSources_997020_2.stderr 2024-03-15 13:51:58 proc_lsst.multi:170 [INFO] [multi] job_id 17075183 proc_lsst.multi INFO: [multi] job_id 17075183 2024-03-15 13:51:58 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 117 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 117 2024-03-15 13:51:58 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:58 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075183 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075183 parsl.executors.status_handling DEBUG: Launched block 172 on executor multi with job ID 17075183 parsl.executors.status_handling INFO: Allocated block ID 173 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-173 2024-03-15 13:51:58 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-173 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-173 2024-03-15 13:51:58 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:58 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:58 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:58 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:58 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 117 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 117 2024-03-15 13:51:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:58 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=173 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 356 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a988378ee20c76337d03016d299e698d with result from task 356 parsl.dataflow.dflow DEBUG: Task 356 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3214c2a6-2d96-4309-8e01-6f9ee887f0bb_isrForCrosstalkSources_996967_45.stdout parsl.dataflow.dflow INFO: Standard error for task 356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3214c2a6-2d96-4309-8e01-6f9ee887f0bb_isrForCrosstalkSources_996967_45.stderr parsl.dataflow.dflow INFO: Task 496 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d369be0734cf95ee8bd875616328aac8 with result from task 496 parsl.dataflow.dflow DEBUG: Task 496 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/1355b93a-5850-428c-9912-c8cc273cf883_isrForCrosstalkSources_997029_47.stdout parsl.dataflow.dflow INFO: Standard error for task 496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/1355b93a-5850-428c-9912-c8cc273cf883_isrForCrosstalkSources_997029_47.stderr parsl.dataflow.dflow INFO: Task 499 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75f2766c4d7c9ce55d23c3d083d16565 with result from task 499 parsl.dataflow.dflow DEBUG: Task 499 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cb5ec149-45c4-4c3e-be3c-3b472a9e2acc_isrForCrosstalkSources_997015_39.stdout parsl.dataflow.dflow INFO: Standard error for task 499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cb5ec149-45c4-4c3e-be3c-3b472a9e2acc_isrForCrosstalkSources_997015_39.stderr parsl.dataflow.dflow INFO: Task 449 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1347d6fec49b7c6ba191e9b95285e01e with result from task 449 parsl.dataflow.dflow DEBUG: Task 449 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/9b2ad1c3-53e6-4963-b215-23f97f1d4010_isrForCrosstalkSources_997031_50.stdout parsl.dataflow.dflow INFO: Standard error for task 449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/9b2ad1c3-53e6-4963-b215-23f97f1d4010_isrForCrosstalkSources_997031_50.stderr parsl.dataflow.dflow INFO: Task 366 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6106e30cf8233facf2694ea35cfaf9a9 with result from task 366 parsl.dataflow.dflow DEBUG: Task 366 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5edb0088-5009-4954-aa76-8a4f023c3ea2_isrForCrosstalkSources_997015_27.stdout parsl.dataflow.dflow INFO: Standard error for task 366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5edb0088-5009-4954-aa76-8a4f023c3ea2_isrForCrosstalkSources_997015_27.stderr 2024-03-15 13:51:59 proc_lsst.multi:170 [INFO] [multi] job_id 17075185 proc_lsst.multi INFO: [multi] job_id 17075185 2024-03-15 13:51:59 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 118 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 118 2024-03-15 13:51:59 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:59 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075185 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075185 parsl.executors.status_handling DEBUG: Launched block 173 on executor multi with job ID 17075185 parsl.executors.status_handling INFO: Allocated block ID 174 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-174 2024-03-15 13:51:59 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-174 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-174 2024-03-15 13:51:59 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:51:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:51:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:51:59 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:51:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:51:59 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:51:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:51:59 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:51:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:51:59 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:51:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 118 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 118 2024-03-15 13:51:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:51:59 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=174 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 407 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 43322b718f09b9f0201e90c543c4e811 with result from task 407 parsl.dataflow.dflow DEBUG: Task 407 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3d2d3d57-e959-4b96-a2d8-d8ed442bad6c_isrForCrosstalkSources_997014_33.stdout parsl.dataflow.dflow INFO: Standard error for task 407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3d2d3d57-e959-4b96-a2d8-d8ed442bad6c_isrForCrosstalkSources_997014_33.stderr parsl.dataflow.dflow INFO: Task 383 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2fbb6699ff41e9a4d8ab3e42901c383d with result from task 383 parsl.dataflow.dflow DEBUG: Task 383 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/139216da-99ad-441e-9013-c3ee7f4168a8_isrForCrosstalkSources_996985_4.stdout parsl.dataflow.dflow INFO: Standard error for task 383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/139216da-99ad-441e-9013-c3ee7f4168a8_isrForCrosstalkSources_996985_4.stderr parsl.dataflow.dflow INFO: Task 498 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86f9de6538c4cf3bb04f017da965cecf with result from task 498 parsl.dataflow.dflow DEBUG: Task 498 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/dc5a53ce-b503-47a2-b017-0feb4e994fa3_isrForCrosstalkSources_996988_13.stdout parsl.dataflow.dflow INFO: Standard error for task 498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/dc5a53ce-b503-47a2-b017-0feb4e994fa3_isrForCrosstalkSources_996988_13.stderr parsl.dataflow.dflow INFO: Task 386 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4219856e7be1cf0e650920420defbe97 with result from task 386 parsl.dataflow.dflow DEBUG: Task 386 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4c6dc2dc-4686-4533-94fe-a6f415c7e004_isrForCrosstalkSources_996963_20.stdout parsl.dataflow.dflow INFO: Standard error for task 386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4c6dc2dc-4686-4533-94fe-a6f415c7e004_isrForCrosstalkSources_996963_20.stderr parsl.dataflow.dflow INFO: Task 495 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a985184d2e7a6ea0ede46237d5efc85a with result from task 495 parsl.dataflow.dflow DEBUG: Task 495 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d9d04fbf-fbfe-425f-bcfa-141d07faf9ac_isrForCrosstalkSources_996972_62.stdout parsl.dataflow.dflow INFO: Standard error for task 495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d9d04fbf-fbfe-425f-bcfa-141d07faf9ac_isrForCrosstalkSources_996972_62.stderr parsl.dataflow.dflow INFO: Task 508 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c8ec764f24b6bc852953f08ccb88c81 with result from task 508 2024-03-15 13:52:01 proc_lsst.multi:170 [INFO] [multi] job_id 17075187 proc_lsst.multi INFO: [multi] job_id 17075187 2024-03-15 13:52:01 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 119 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 119 2024-03-15 13:52:01 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:01 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075187 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075187 parsl.executors.status_handling DEBUG: Launched block 174 on executor multi with job ID 17075187 parsl.executors.status_handling INFO: Allocated block ID 175 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-175 2024-03-15 13:52:01 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-175 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-175 2024-03-15 13:52:01 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:01 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:01 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:01 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:01 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 119 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 119 2024-03-15 13:52:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:01 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=175 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 508 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/4bf746d9-4fdb-46f8-8064-3001f965555f_isrForCrosstalkSources_996977_25.stdout parsl.dataflow.dflow INFO: Standard error for task 508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/4bf746d9-4fdb-46f8-8064-3001f965555f_isrForCrosstalkSources_996977_25.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 390 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13035f0819202abf550e5c432d4439b1 with result from task 390 parsl.dataflow.dflow DEBUG: Task 390 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/87d17837-2a1e-4ff1-9fe7-cc9fa76f8a0e_isrForCrosstalkSources_996962_56.stdout parsl.dataflow.dflow INFO: Standard error for task 390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/87d17837-2a1e-4ff1-9fe7-cc9fa76f8a0e_isrForCrosstalkSources_996962_56.stderr parsl.dataflow.dflow INFO: Task 494 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99f9a0e829af8d4c06fa8bd54f4454e5 with result from task 494 parsl.dataflow.dflow DEBUG: Task 494 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/be56c2b4-4ffd-4167-8481-affb41ad8160_isrForCrosstalkSources_996990_48.stdout parsl.dataflow.dflow INFO: Standard error for task 494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/be56c2b4-4ffd-4167-8481-affb41ad8160_isrForCrosstalkSources_996990_48.stderr parsl.dataflow.dflow INFO: Task 369 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 14f48a5c75db8b997cb081618ae0f7ed with result from task 369 parsl.dataflow.dflow DEBUG: Task 369 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/b4c71dd2-cd89-464c-9ddf-4315a57fd6d1_isrForCrosstalkSources_996976_15.stdout parsl.dataflow.dflow INFO: Standard error for task 369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/b4c71dd2-cd89-464c-9ddf-4315a57fd6d1_isrForCrosstalkSources_996976_15.stderr parsl.dataflow.dflow INFO: Task 360 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9fad27054875687b613e3a6e7fe2edd with result from task 360 2024-03-15 13:52:02 proc_lsst.multi:170 [INFO] [multi] job_id 17075188 proc_lsst.multi INFO: [multi] job_id 17075188 2024-03-15 13:52:02 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 120 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 120 2024-03-15 13:52:02 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:02 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075188 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075188 parsl.executors.status_handling DEBUG: Launched block 175 on executor multi with job ID 17075188 parsl.executors.status_handling INFO: Allocated block ID 176 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-176 2024-03-15 13:52:02 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-176 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-176 2024-03-15 13:52:02 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:02 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:02 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:02 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:02 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 120 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 120 2024-03-15 13:52:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:02 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=176 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 360 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/ff1114c1-134c-4c97-8191-c4433b60a442_isrForCrosstalkSources_996999_42.stdout parsl.dataflow.dflow INFO: Standard error for task 360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/ff1114c1-134c-4c97-8191-c4433b60a442_isrForCrosstalkSources_996999_42.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 450 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c2f4931784ac67e31a45482c7d2a838 with result from task 450 parsl.dataflow.dflow DEBUG: Task 450 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5fce2064-4a2d-4563-9c08-cd5fdcace021_isrForCrosstalkSources_997009_36.stdout parsl.dataflow.dflow INFO: Standard error for task 450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5fce2064-4a2d-4563-9c08-cd5fdcace021_isrForCrosstalkSources_997009_36.stderr parsl.dataflow.dflow INFO: Task 364 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d5dd7c0d1a2accbd69fbd74ce720455d with result from task 364 parsl.dataflow.dflow DEBUG: Task 364 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f8cb6052-6200-421b-a1e9-a05c634bf83a_isrForCrosstalkSources_996957_16.stdout parsl.dataflow.dflow INFO: Standard error for task 364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f8cb6052-6200-421b-a1e9-a05c634bf83a_isrForCrosstalkSources_996957_16.stderr parsl.dataflow.dflow INFO: Task 385 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a416872033b5f405e42cf5eecb87e4f9 with result from task 385 parsl.dataflow.dflow DEBUG: Task 385 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/295a0b95-fe7f-425e-b198-99444d66f6ac_isrForCrosstalkSources_996993_15.stdout parsl.dataflow.dflow INFO: Standard error for task 385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/295a0b95-fe7f-425e-b198-99444d66f6ac_isrForCrosstalkSources_996993_15.stderr parsl.dataflow.dflow INFO: Task 388 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef8f22c304041896e7a0320195e0a173 with result from task 388 parsl.dataflow.dflow DEBUG: Task 388 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/138dde35-d0c5-4619-ba67-e42b613258e5_isrForCrosstalkSources_997016_6.stdout parsl.dataflow.dflow INFO: Standard error for task 388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/138dde35-d0c5-4619-ba67-e42b613258e5_isrForCrosstalkSources_997016_6.stderr parsl.dataflow.dflow INFO: Task 363 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fa1df46f7f4f2905405a6c0c4dbfc25 with result from task 363 parsl.dataflow.dflow DEBUG: Task 363 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6ed0b3ab-34b3-4b1a-acbc-9b54d1933054_isrForCrosstalkSources_997017_35.stdout parsl.dataflow.dflow INFO: Standard error for task 363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6ed0b3ab-34b3-4b1a-acbc-9b54d1933054_isrForCrosstalkSources_997017_35.stderr parsl.dataflow.dflow INFO: Task 359 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b1b5f309a45ca6bca48b4a8f1570605c with result from task 359 parsl.dataflow.dflow DEBUG: Task 359 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7f2a4993-825a-49c2-b114-194438753b02_isrForCrosstalkSources_997005_3.stdout parsl.dataflow.dflow INFO: Standard error for task 359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7f2a4993-825a-49c2-b114-194438753b02_isrForCrosstalkSources_997005_3.stderr 2024-03-15 13:52:03 proc_lsst.multi:170 [INFO] [multi] job_id 17075190 proc_lsst.multi INFO: [multi] job_id 17075190 2024-03-15 13:52:03 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 121 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 121 2024-03-15 13:52:03 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:03 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075190 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075190 parsl.executors.status_handling DEBUG: Launched block 176 on executor multi with job ID 17075190 parsl.executors.status_handling INFO: Allocated block ID 177 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-177 2024-03-15 13:52:03 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-177 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-177 2024-03-15 13:52:03 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:03 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:03 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:03 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:03 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 121 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 121 2024-03-15 13:52:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:03 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=177 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 367 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ad1c9acd3938eba01dbf601e26b6b5b with result from task 367 parsl.dataflow.dflow DEBUG: Task 367 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/93cea72d-386c-463b-839d-eff3a878d75f_isrForCrosstalkSources_997025_31.stdout parsl.dataflow.dflow INFO: Standard error for task 367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/93cea72d-386c-463b-839d-eff3a878d75f_isrForCrosstalkSources_997025_31.stderr parsl.dataflow.dflow INFO: Task 509 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7435ade850134c4952ce71c7b5304b0 with result from task 509 parsl.dataflow.dflow DEBUG: Task 509 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/875dd3d6-02e5-4f82-a045-c5fa26781101_isrForCrosstalkSources_996965_59.stdout parsl.dataflow.dflow INFO: Standard error for task 509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/875dd3d6-02e5-4f82-a045-c5fa26781101_isrForCrosstalkSources_996965_59.stderr parsl.dataflow.dflow INFO: Task 532 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc6110aba7ed531d012375118ae456eb with result from task 532 parsl.dataflow.dflow DEBUG: Task 532 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/21a99695-0273-4c89-a15c-69cda77fa959_isrForCrosstalkSources_997017_42.stdout parsl.dataflow.dflow INFO: Standard error for task 532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/21a99695-0273-4c89-a15c-69cda77fa959_isrForCrosstalkSources_997017_42.stderr parsl.dataflow.dflow INFO: Task 515 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed789d5570b00d5c951f64c0ce305b38 with result from task 515 parsl.dataflow.dflow DEBUG: Task 515 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/74fc3d8b-f00e-4591-be65-e0c4a7b3e094_isrForCrosstalkSources_997004_6.stdout parsl.dataflow.dflow INFO: Standard error for task 515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/74fc3d8b-f00e-4591-be65-e0c4a7b3e094_isrForCrosstalkSources_997004_6.stderr parsl.dataflow.dflow INFO: Task 361 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b203f4bc1dd820a741ac2b5b3a21d5b9 with result from task 361 parsl.dataflow.dflow DEBUG: Task 361 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a7e86d17-b1bb-4454-aec6-f0d0b01f666f_isrForCrosstalkSources_996995_59.stdout parsl.dataflow.dflow INFO: Standard error for task 361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a7e86d17-b1bb-4454-aec6-f0d0b01f666f_isrForCrosstalkSources_996995_59.stderr parsl.dataflow.dflow INFO: Task 513 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a94a67289271124ae2a27ad7072acc30 with result from task 513 parsl.dataflow.dflow DEBUG: Task 513 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/95c94cae-11d6-4a16-8509-61799c443c67_isrForCrosstalkSources_997024_19.stdout parsl.dataflow.dflow INFO: Standard error for task 513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/95c94cae-11d6-4a16-8509-61799c443c67_isrForCrosstalkSources_997024_19.stderr parsl.dataflow.dflow INFO: Task 530 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ed85edc54c973c45bb16fb659c091e0 with result from task 530 parsl.dataflow.dflow DEBUG: Task 530 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0c226aae-2ffc-4ea2-899d-73775c98187d_isrForCrosstalkSources_996997_14.stdout parsl.dataflow.dflow INFO: Standard error for task 530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0c226aae-2ffc-4ea2-899d-73775c98187d_isrForCrosstalkSources_996997_14.stderr parsl.dataflow.dflow INFO: Task 510 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83f99fb640bf1b459549fa8f4cd89abe with result from task 510 parsl.dataflow.dflow DEBUG: Task 510 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/d06d2740-de19-4c7a-860a-3d989a442933_isrForCrosstalkSources_997006_30.stdout parsl.dataflow.dflow INFO: Standard error for task 510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/d06d2740-de19-4c7a-860a-3d989a442933_isrForCrosstalkSources_997006_30.stderr parsl.dataflow.dflow INFO: Task 518 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91319e594445adde9aa69c46c16a98b9 with result from task 518 parsl.dataflow.dflow DEBUG: Task 518 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/c550c26b-e42b-4a91-9244-ea8040753899_isrForCrosstalkSources_996996_42.stdout parsl.dataflow.dflow INFO: Standard error for task 518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/c550c26b-e42b-4a91-9244-ea8040753899_isrForCrosstalkSources_996996_42.stderr parsl.dataflow.dflow INFO: Task 529 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea52587f2590c6ebfe57e994127a2133 with result from task 529 parsl.dataflow.dflow DEBUG: Task 529 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/28cf44e2-9228-4699-99e5-d7e7c35758ba_isrForCrosstalkSources_996956_29.stdout parsl.dataflow.dflow INFO: Standard error for task 529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/28cf44e2-9228-4699-99e5-d7e7c35758ba_isrForCrosstalkSources_996956_29.stderr parsl.dataflow.dflow INFO: Task 497 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f93e5de04383feec0068a2272c21a907 with result from task 497 parsl.dataflow.dflow DEBUG: Task 497 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b021ec37-388f-407a-b296-2f4ec8c088ec_isrForCrosstalkSources_996969_50.stdout parsl.dataflow.dflow INFO: Standard error for task 497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b021ec37-388f-407a-b296-2f4ec8c088ec_isrForCrosstalkSources_996969_50.stderr 2024-03-15 13:52:04 proc_lsst.multi:170 [INFO] [multi] job_id 17075191 proc_lsst.multi INFO: [multi] job_id 17075191 2024-03-15 13:52:04 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 122 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 122 2024-03-15 13:52:04 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:04 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075191 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075191 parsl.executors.status_handling DEBUG: Launched block 177 on executor multi with job ID 17075191 parsl.executors.status_handling INFO: Allocated block ID 178 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-178 2024-03-15 13:52:04 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-178 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-178 2024-03-15 13:52:04 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:04 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:04 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:04 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:04 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 122 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 122 2024-03-15 13:52:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:04 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=178 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 382 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f8fbcd7e93cab00167a9228b97b914e3 with result from task 382 parsl.dataflow.dflow DEBUG: Task 382 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f8d7a9f4-d4ee-4868-af98-45551c2ac61a_isrForCrosstalkSources_996989_58.stdout parsl.dataflow.dflow INFO: Standard error for task 382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f8d7a9f4-d4ee-4868-af98-45551c2ac61a_isrForCrosstalkSources_996989_58.stderr parsl.dataflow.dflow INFO: Task 365 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d0313878f53db3426d0d971194b56c41 with result from task 365 parsl.dataflow.dflow DEBUG: Task 365 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/9602c128-92e1-47f6-a22b-be46fa03497d_isrForCrosstalkSources_997012_47.stdout parsl.dataflow.dflow INFO: Standard error for task 365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/9602c128-92e1-47f6-a22b-be46fa03497d_isrForCrosstalkSources_997012_47.stderr parsl.dataflow.dflow INFO: Task 512 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3cda27b83b8ee9ff28d56ad70bb1cc4 with result from task 512 parsl.dataflow.dflow DEBUG: Task 512 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/73801838-25ce-4e21-95fb-c38065359c16_isrForCrosstalkSources_997021_29.stdout parsl.dataflow.dflow INFO: Standard error for task 512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/73801838-25ce-4e21-95fb-c38065359c16_isrForCrosstalkSources_997021_29.stderr parsl.dataflow.dflow INFO: Task 389 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d92c31c4033826930c243f0cd9e8dea8 with result from task 389 parsl.dataflow.dflow DEBUG: Task 389 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b40c1e6b-8887-4cd2-80e3-cc75a37222f7_isrForCrosstalkSources_996953_54.stdout parsl.dataflow.dflow INFO: Standard error for task 389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b40c1e6b-8887-4cd2-80e3-cc75a37222f7_isrForCrosstalkSources_996953_54.stderr parsl.dataflow.dflow INFO: Task 362 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3795ee2fe1255f5fc27dae55773497c6 with result from task 362 parsl.dataflow.dflow DEBUG: Task 362 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9a584c27-ef90-40ca-ae7a-2dfacb4282c3_isrForCrosstalkSources_996997_54.stdout parsl.dataflow.dflow INFO: Standard error for task 362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9a584c27-ef90-40ca-ae7a-2dfacb4282c3_isrForCrosstalkSources_996997_54.stderr parsl.dataflow.dflow INFO: Task 492 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a8de389ab704d5d2281548511bc3c0a with result from task 492 parsl.dataflow.dflow DEBUG: Task 492 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/019b1046-a190-44bd-b865-73beebff66d2_isrForCrosstalkSources_996971_57.stdout parsl.dataflow.dflow INFO: Standard error for task 492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/019b1046-a190-44bd-b865-73beebff66d2_isrForCrosstalkSources_996971_57.stderr 2024-03-15 13:52:05 proc_lsst.multi:170 [INFO] [multi] job_id 17075197 proc_lsst.multi INFO: [multi] job_id 17075197 2024-03-15 13:52:05 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 123 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 123 2024-03-15 13:52:05 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:05 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075197 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075197 parsl.executors.status_handling DEBUG: Launched block 178 on executor multi with job ID 17075197 parsl.executors.status_handling INFO: Allocated block ID 179 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-179 2024-03-15 13:52:05 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-179 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-179 2024-03-15 13:52:05 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:05 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:05 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:05 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:05 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 123 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 123 2024-03-15 13:52:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:05 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=179 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 475 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3ad34a98b99acbc09968189ddf88a766 with result from task 475 parsl.dataflow.dflow DEBUG: Task 475 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f61b22b7-4cce-47f9-8bcf-da9d5d9ecc44_isrForCrosstalkSources_996985_59.stdout parsl.dataflow.dflow INFO: Standard error for task 475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f61b22b7-4cce-47f9-8bcf-da9d5d9ecc44_isrForCrosstalkSources_996985_59.stderr parsl.dataflow.dflow INFO: Task 540 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da235e03d989673b8bcd1afdcefab589 with result from task 540 parsl.dataflow.dflow DEBUG: Task 540 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/6c89c288-5942-452a-85fe-55749065e483_isrForCrosstalkSources_996968_33.stdout parsl.dataflow.dflow INFO: Standard error for task 540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/6c89c288-5942-452a-85fe-55749065e483_isrForCrosstalkSources_996968_33.stderr parsl.dataflow.dflow INFO: Task 538 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 570d117b156384535e00ed761dc45c3a with result from task 538 parsl.dataflow.dflow DEBUG: Task 538 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/93a69597-2253-465e-99f1-aa6fbc20dabe_isrForCrosstalkSources_996956_21.stdout parsl.dataflow.dflow INFO: Standard error for task 538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/93a69597-2253-465e-99f1-aa6fbc20dabe_isrForCrosstalkSources_996956_21.stderr 2024-03-15 13:52:06 proc_lsst.multi:170 [INFO] [multi] job_id 17075200 proc_lsst.multi INFO: [multi] job_id 17075200 2024-03-15 13:52:06 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 124 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 124 2024-03-15 13:52:06 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:06 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075200 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075200 parsl.executors.status_handling DEBUG: Launched block 179 on executor multi with job ID 17075200 parsl.executors.status_handling INFO: Allocated block ID 180 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-180 2024-03-15 13:52:06 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-180 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-180 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 124 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 124 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:06 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=180 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 542 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c002c633d818670615228a3c118a8a9 with result from task 542 parsl.dataflow.dflow DEBUG: Task 542 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f8928018-f5c3-4436-bb34-3473ca9c6c01_isrForCrosstalkSources_996990_33.stdout parsl.dataflow.dflow INFO: Standard error for task 542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f8928018-f5c3-4436-bb34-3473ca9c6c01_isrForCrosstalkSources_996990_33.stderr 2024-03-15 13:52:06 proc_lsst.multi:170 [INFO] [multi] job_id 17075201 proc_lsst.multi INFO: [multi] job_id 17075201 2024-03-15 13:52:06 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 125 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 125 2024-03-15 13:52:06 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:06 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075201 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075201 parsl.executors.status_handling DEBUG: Launched block 180 on executor multi with job ID 17075201 parsl.executors.status_handling INFO: Allocated block ID 181 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-181 2024-03-15 13:52:06 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-181 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-181 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:06 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 125 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 125 2024-03-15 13:52:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:06 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=181 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 517 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 849da78cd238eedbc1bc7ee4075c73e7 with result from task 517 parsl.dataflow.dflow DEBUG: Task 517 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/390f7aaa-237e-49ed-912b-6113e7da3dd9_isrForCrosstalkSources_996992_23.stdout parsl.dataflow.dflow INFO: Standard error for task 517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/390f7aaa-237e-49ed-912b-6113e7da3dd9_isrForCrosstalkSources_996992_23.stderr 2024-03-15 13:52:07 proc_lsst.multi:170 [INFO] [multi] job_id 17075202 proc_lsst.multi INFO: [multi] job_id 17075202 2024-03-15 13:52:07 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 126 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 126 2024-03-15 13:52:07 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:07 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075202 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075202 parsl.executors.status_handling DEBUG: Launched block 181 on executor multi with job ID 17075202 parsl.executors.status_handling INFO: Allocated block ID 182 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-182 2024-03-15 13:52:07 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-182 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-182 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 126 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 126 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:07 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=182 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 528 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad7ab804fb39eaf82600c89abf0473c0 with result from task 528 parsl.dataflow.dflow DEBUG: Task 528 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/979007e6-5ea3-4910-bdba-985158e344e6_isrForCrosstalkSources_997031_35.stdout parsl.dataflow.dflow INFO: Standard error for task 528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/979007e6-5ea3-4910-bdba-985158e344e6_isrForCrosstalkSources_997031_35.stderr parsl.dataflow.dflow INFO: Task 514 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62b578f385c168edadd0de36772b4ff9 with result from task 514 parsl.dataflow.dflow DEBUG: Task 514 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9657cfa1-17a6-4e10-b023-54df38e61c95_isrForCrosstalkSources_997000_12.stdout parsl.dataflow.dflow INFO: Standard error for task 514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9657cfa1-17a6-4e10-b023-54df38e61c95_isrForCrosstalkSources_997000_12.stderr 2024-03-15 13:52:07 proc_lsst.multi:170 [INFO] [multi] job_id 17075203 proc_lsst.multi INFO: [multi] job_id 17075203 2024-03-15 13:52:07 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 127 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 127 2024-03-15 13:52:07 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:07 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075203 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075203 parsl.executors.status_handling DEBUG: Launched block 182 on executor multi with job ID 17075203 parsl.executors.status_handling INFO: Allocated block ID 183 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-183 2024-03-15 13:52:07 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-183 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-183 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:07 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 127 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 127 2024-03-15 13:52:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:07 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=183 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 511 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0b79bf7de4d79e342c97014ac0a2f189 with result from task 511 parsl.dataflow.dflow DEBUG: Task 511 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/073f2c7e-6fff-48ff-abca-f5b438dbc53c_isrForCrosstalkSources_996956_42.stdout parsl.dataflow.dflow INFO: Standard error for task 511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/073f2c7e-6fff-48ff-abca-f5b438dbc53c_isrForCrosstalkSources_996956_42.stderr parsl.dataflow.dflow INFO: Task 537 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 016d767a74271b21145ec804e490db46 with result from task 537 parsl.dataflow.dflow DEBUG: Task 537 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d2235bd6-4b3a-4e8c-b602-fa092f96192e_isrForCrosstalkSources_997008_5.stdout parsl.dataflow.dflow INFO: Standard error for task 537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d2235bd6-4b3a-4e8c-b602-fa092f96192e_isrForCrosstalkSources_997008_5.stderr 2024-03-15 13:52:08 proc_lsst.multi:170 [INFO] [multi] job_id 17075204 proc_lsst.multi INFO: [multi] job_id 17075204 2024-03-15 13:52:08 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 128 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 128 2024-03-15 13:52:08 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:08 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075204 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075204 parsl.executors.status_handling DEBUG: Launched block 183 on executor multi with job ID 17075204 parsl.executors.status_handling INFO: Allocated block ID 184 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-184 2024-03-15 13:52:08 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-184 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-184 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 128 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 128 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:08 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=184 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:08 proc_lsst.multi:170 [INFO] [multi] job_id 17075205 proc_lsst.multi INFO: [multi] job_id 17075205 2024-03-15 13:52:08 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 129 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 129 2024-03-15 13:52:08 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:08 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075205 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075205 parsl.executors.status_handling DEBUG: Launched block 184 on executor multi with job ID 17075205 parsl.executors.status_handling INFO: Allocated block ID 185 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-185 2024-03-15 13:52:08 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-185 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-185 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:08 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 129 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 129 2024-03-15 13:52:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:08 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=185 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 531 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de5f6bb49c0320cb2e6eb2bef20ca90c with result from task 531 parsl.dataflow.dflow DEBUG: Task 531 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7f2ca16a-c7d8-495e-ac5c-7cd019fb4285_isrForCrosstalkSources_997024_47.stdout parsl.dataflow.dflow INFO: Standard error for task 531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7f2ca16a-c7d8-495e-ac5c-7cd019fb4285_isrForCrosstalkSources_997024_47.stderr 2024-03-15 13:52:09 proc_lsst.multi:170 [INFO] [multi] job_id 17075206 proc_lsst.multi INFO: [multi] job_id 17075206 2024-03-15 13:52:09 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 130 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 130 2024-03-15 13:52:09 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:09 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075206 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075206 parsl.executors.status_handling DEBUG: Launched block 185 on executor multi with job ID 17075206 parsl.executors.status_handling INFO: Allocated block ID 186 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-186 2024-03-15 13:52:09 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-186 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-186 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 130 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 130 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:09 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=186 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 541 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b35e8f24f0c97d31fdd29c5e3c3f988 with result from task 541 parsl.dataflow.dflow DEBUG: Task 541 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d6139823-10fc-4821-986e-a8abe1b1ccb1_isrForCrosstalkSources_997027_39.stdout parsl.dataflow.dflow INFO: Standard error for task 541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d6139823-10fc-4821-986e-a8abe1b1ccb1_isrForCrosstalkSources_997027_39.stderr 2024-03-15 13:52:09 proc_lsst.multi:170 [INFO] [multi] job_id 17075207 proc_lsst.multi INFO: [multi] job_id 17075207 2024-03-15 13:52:09 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 131 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 131 2024-03-15 13:52:09 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:09 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075207 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075207 parsl.executors.status_handling DEBUG: Launched block 186 on executor multi with job ID 17075207 parsl.executors.status_handling INFO: Allocated block ID 187 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-187 2024-03-15 13:52:09 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-187 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-187 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:09 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 131 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 131 2024-03-15 13:52:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:09 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=187 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 516 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6e4080f4cdd1deed6dc2f4a36ec6aa1a with result from task 516 parsl.dataflow.dflow DEBUG: Task 516 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/bb71f467-6ca0-47cd-adb1-81419f974eab_isrForCrosstalkSources_997015_44.stdout parsl.dataflow.dflow INFO: Standard error for task 516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/bb71f467-6ca0-47cd-adb1-81419f974eab_isrForCrosstalkSources_997015_44.stderr parsl.dataflow.dflow INFO: Task 534 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21b8f89995e21fd168095ea7910117ac with result from task 534 parsl.dataflow.dflow DEBUG: Task 534 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/d5152f07-3fe1-4281-b795-0c073ff5f981_isrForCrosstalkSources_997014_38.stdout parsl.dataflow.dflow INFO: Standard error for task 534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/d5152f07-3fe1-4281-b795-0c073ff5f981_isrForCrosstalkSources_997014_38.stderr 2024-03-15 13:52:10 proc_lsst.multi:170 [INFO] [multi] job_id 17075208 proc_lsst.multi INFO: [multi] job_id 17075208 2024-03-15 13:52:10 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 132 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 132 2024-03-15 13:52:10 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:10 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075208 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075208 parsl.executors.status_handling DEBUG: Launched block 187 on executor multi with job ID 17075208 parsl.executors.status_handling INFO: Allocated block ID 188 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-188 2024-03-15 13:52:10 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-188 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-188 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 132 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 132 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:10 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=188 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:10 proc_lsst.multi:170 [INFO] [multi] job_id 17075209 proc_lsst.multi INFO: [multi] job_id 17075209 2024-03-15 13:52:10 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 133 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 133 2024-03-15 13:52:10 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:10 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075209 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075209 parsl.executors.status_handling DEBUG: Launched block 188 on executor multi with job ID 17075209 parsl.executors.status_handling INFO: Allocated block ID 189 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-189 2024-03-15 13:52:10 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-189 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-189 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:10 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 133 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 133 2024-03-15 13:52:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:10 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=189 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:11 proc_lsst.multi:170 [INFO] [multi] job_id 17075210 proc_lsst.multi INFO: [multi] job_id 17075210 2024-03-15 13:52:11 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 134 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 134 2024-03-15 13:52:11 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:11 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075210 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075210 parsl.executors.status_handling DEBUG: Launched block 189 on executor multi with job ID 17075210 parsl.executors.status_handling INFO: Allocated block ID 190 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-190 2024-03-15 13:52:11 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-190 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-190 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 134 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 134 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:11 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=190 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:11 proc_lsst.multi:170 [INFO] [multi] job_id 17075211 proc_lsst.multi INFO: [multi] job_id 17075211 2024-03-15 13:52:11 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 135 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 135 2024-03-15 13:52:11 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:11 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075211 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075211 parsl.executors.status_handling DEBUG: Launched block 190 on executor multi with job ID 17075211 parsl.executors.status_handling INFO: Allocated block ID 191 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-191 2024-03-15 13:52:11 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-191 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-191 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:11 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 135 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 135 2024-03-15 13:52:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:11 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=191 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 547 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b2f31ee3480f5678c7f18d2ddf1b98b with result from task 547 parsl.dataflow.dflow DEBUG: Task 547 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/9558db22-35dc-487d-ad84-17073c5e49ed_isrForCrosstalkSources_996989_12.stdout parsl.dataflow.dflow INFO: Standard error for task 547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/9558db22-35dc-487d-ad84-17073c5e49ed_isrForCrosstalkSources_996989_12.stderr 2024-03-15 13:52:12 proc_lsst.multi:170 [INFO] [multi] job_id 17075212 proc_lsst.multi INFO: [multi] job_id 17075212 2024-03-15 13:52:12 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 136 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 136 2024-03-15 13:52:12 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:12 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075212 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075212 parsl.executors.status_handling DEBUG: Launched block 191 on executor multi with job ID 17075212 parsl.executors.status_handling INFO: Allocated block ID 192 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-192 2024-03-15 13:52:12 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-192 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-192 2024-03-15 13:52:12 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:12 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:12 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:12 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:12 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 136 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 136 2024-03-15 13:52:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:12 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=192 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 519 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e08af2aa882f28295cee1f558890af9a with result from task 519 parsl.dataflow.dflow DEBUG: Task 519 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4d603963-953a-4cc7-a7e4-14dee912270b_isrForCrosstalkSources_996958_48.stdout parsl.dataflow.dflow INFO: Standard error for task 519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4d603963-953a-4cc7-a7e4-14dee912270b_isrForCrosstalkSources_996958_48.stderr parsl.dataflow.dflow INFO: Task 567 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 64ff9bf7aa2670b0353a8dc007d530d0 with result from task 567 parsl.dataflow.dflow DEBUG: Task 567 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/21942f4b-d6a3-4257-b5b3-cd4efa287b85_isrForCrosstalkSources_997013_13.stdout parsl.dataflow.dflow INFO: Standard error for task 567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/21942f4b-d6a3-4257-b5b3-cd4efa287b85_isrForCrosstalkSources_997013_13.stderr parsl.dataflow.dflow INFO: Task 553 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fb6890947e0cebe342ddaac6864b2db with result from task 553 parsl.dataflow.dflow DEBUG: Task 553 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/d089db6c-dd9b-4968-a681-7de617ae429e_isrForCrosstalkSources_997019_14.stdout parsl.dataflow.dflow INFO: Standard error for task 553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/d089db6c-dd9b-4968-a681-7de617ae429e_isrForCrosstalkSources_997019_14.stderr parsl.dataflow.dflow INFO: Task 573 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8abee0aeede9d5d7958063e3b772bdd with result from task 573 parsl.dataflow.dflow DEBUG: Task 573 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/deb15880-50d2-4d8f-8add-936bfa8b21c2_isrForCrosstalkSources_996993_40.stdout parsl.dataflow.dflow INFO: Standard error for task 573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/deb15880-50d2-4d8f-8add-936bfa8b21c2_isrForCrosstalkSources_996993_40.stderr parsl.dataflow.dflow INFO: Task 551 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb88391a6f1e4fd261d41da5e66f287d with result from task 551 parsl.dataflow.dflow DEBUG: Task 551 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/746fe48c-80d8-4c9e-b4d1-6221668928f8_isrForCrosstalkSources_996966_22.stdout parsl.dataflow.dflow INFO: Standard error for task 551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/746fe48c-80d8-4c9e-b4d1-6221668928f8_isrForCrosstalkSources_996966_22.stderr 2024-03-15 13:52:13 proc_lsst.multi:170 [INFO] [multi] job_id 17075213 proc_lsst.multi INFO: [multi] job_id 17075213 2024-03-15 13:52:13 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 137 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 137 2024-03-15 13:52:13 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:13 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075213 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075213 parsl.executors.status_handling DEBUG: Launched block 192 on executor multi with job ID 17075213 parsl.executors.status_handling INFO: Allocated block ID 193 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-193 2024-03-15 13:52:13 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-193 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-193 2024-03-15 13:52:13 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:13 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:13 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:13 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:13 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 137 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 137 2024-03-15 13:52:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:13 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=193 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 574 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a801ed85672d51987bd1efaa53f47378 with result from task 574 parsl.dataflow.dflow DEBUG: Task 574 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/aee1c00b-2ab0-4ac9-8271-b303f2cf3803_isrForCrosstalkSources_997029_5.stdout parsl.dataflow.dflow INFO: Standard error for task 574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/aee1c00b-2ab0-4ac9-8271-b303f2cf3803_isrForCrosstalkSources_997029_5.stderr parsl.dataflow.dflow INFO: Task 556 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a9eb76452c4027d026b466859670598 with result from task 556 parsl.dataflow.dflow DEBUG: Task 556 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/da55b1d4-4362-4571-b113-b884537fe25f_isrForCrosstalkSources_996958_57.stdout parsl.dataflow.dflow INFO: Standard error for task 556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/da55b1d4-4362-4571-b113-b884537fe25f_isrForCrosstalkSources_996958_57.stderr parsl.dataflow.dflow INFO: Task 554 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 735fb01643a17504f779b229654b7f26 with result from task 554 parsl.dataflow.dflow DEBUG: Task 554 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/7bfb47b0-e610-428d-a0ab-492ed70d798d_isrForCrosstalkSources_997002_54.stdout parsl.dataflow.dflow INFO: Standard error for task 554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/7bfb47b0-e610-428d-a0ab-492ed70d798d_isrForCrosstalkSources_997002_54.stderr 2024-03-15 13:52:14 proc_lsst.multi:170 [INFO] [multi] job_id 17075214 proc_lsst.multi INFO: [multi] job_id 17075214 2024-03-15 13:52:14 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 138 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 138 2024-03-15 13:52:14 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:14 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075214 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075214 parsl.executors.status_handling DEBUG: Launched block 193 on executor multi with job ID 17075214 parsl.executors.status_handling INFO: Allocated block ID 194 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-194 2024-03-15 13:52:14 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-194 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-194 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 138 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 138 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:14 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=194 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 536 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5d6268c373881fbc6de1e05eb6a68fb with result from task 536 parsl.dataflow.dflow DEBUG: Task 536 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bb561524-4366-4364-8315-387fa3a4f275_isrForCrosstalkSources_996959_20.stdout parsl.dataflow.dflow INFO: Standard error for task 536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bb561524-4366-4364-8315-387fa3a4f275_isrForCrosstalkSources_996959_20.stderr parsl.dataflow.dflow INFO: Task 533 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 22c45d77cec4c69e1a204fcc8d85c8ce with result from task 533 parsl.dataflow.dflow DEBUG: Task 533 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d1a0c4bd-66c7-4357-b161-40cd7c2872d1_isrForCrosstalkSources_996964_61.stdout parsl.dataflow.dflow INFO: Standard error for task 533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d1a0c4bd-66c7-4357-b161-40cd7c2872d1_isrForCrosstalkSources_996964_61.stderr 2024-03-15 13:52:14 proc_lsst.multi:170 [INFO] [multi] job_id 17075216 proc_lsst.multi INFO: [multi] job_id 17075216 2024-03-15 13:52:14 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 139 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 139 2024-03-15 13:52:14 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:14 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075216 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075216 parsl.executors.status_handling DEBUG: Launched block 194 on executor multi with job ID 17075216 parsl.executors.status_handling INFO: Allocated block ID 195 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-195 2024-03-15 13:52:14 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-195 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-195 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:14 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:14 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 139 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 139 2024-03-15 13:52:14 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:14 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=195 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:15 proc_lsst.multi:170 [INFO] [multi] job_id 17075217 proc_lsst.multi INFO: [multi] job_id 17075217 2024-03-15 13:52:15 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 140 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 140 2024-03-15 13:52:15 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:15 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075217 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075217 parsl.executors.status_handling DEBUG: Launched block 195 on executor multi with job ID 17075217 parsl.executors.status_handling INFO: Allocated block ID 196 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-196 2024-03-15 13:52:15 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-196 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-196 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 140 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 140 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:15 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=196 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:15 proc_lsst.multi:170 [INFO] [multi] job_id 17075218 proc_lsst.multi INFO: [multi] job_id 17075218 2024-03-15 13:52:15 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 141 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 141 2024-03-15 13:52:15 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:15 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075218 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075218 parsl.executors.status_handling DEBUG: Launched block 196 on executor multi with job ID 17075218 parsl.executors.status_handling INFO: Allocated block ID 197 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-197 2024-03-15 13:52:15 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-197 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-197 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:15 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 141 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 141 2024-03-15 13:52:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:15 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=197 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 568 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2938d475be02f825d3bc70e146979c81 with result from task 568 parsl.dataflow.dflow DEBUG: Task 568 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/d4220be6-50b2-44d4-bce0-f98f1feee772_isrForCrosstalkSources_997002_38.stdout parsl.dataflow.dflow INFO: Standard error for task 568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/d4220be6-50b2-44d4-bce0-f98f1feee772_isrForCrosstalkSources_997002_38.stderr parsl.dataflow.dflow INFO: Task 375 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7246597ef2e1695c9585a780d4f82e1c with result from task 375 parsl.dataflow.dflow DEBUG: Task 375 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/3a199a8f-002d-4815-b55f-55b3a846c8cd_isrForCrosstalkSources_997009_37.stdout parsl.dataflow.dflow INFO: Standard error for task 375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/3a199a8f-002d-4815-b55f-55b3a846c8cd_isrForCrosstalkSources_997009_37.stderr parsl.dataflow.dflow INFO: Task 378 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73f6ccc86b5dac0a3d3cf41e424b1ee5 with result from task 378 parsl.dataflow.dflow DEBUG: Task 378 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/d2bc552a-a2e3-4392-84d4-d829947b6cc6_isrForCrosstalkSources_996970_44.stdout parsl.dataflow.dflow INFO: Standard error for task 378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/d2bc552a-a2e3-4392-84d4-d829947b6cc6_isrForCrosstalkSources_996970_44.stderr parsl.dataflow.dflow INFO: Task 373 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29af126b42696e614217862e6709b417 with result from task 373 parsl.dataflow.dflow DEBUG: Task 373 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cb7b7737-fb9d-4e08-88f8-c3ae5c691902_isrForCrosstalkSources_996956_20.stdout parsl.dataflow.dflow INFO: Standard error for task 373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cb7b7737-fb9d-4e08-88f8-c3ae5c691902_isrForCrosstalkSources_996956_20.stderr parsl.dataflow.dflow INFO: Task 374 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0caa8baa2b79914e6f686835f5718945 with result from task 374 parsl.dataflow.dflow DEBUG: Task 374 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/5424dd0c-a806-4c03-8069-5f3103612956_isrForCrosstalkSources_996960_2.stdout parsl.dataflow.dflow INFO: Standard error for task 374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/5424dd0c-a806-4c03-8069-5f3103612956_isrForCrosstalkSources_996960_2.stderr parsl.dataflow.dflow INFO: Task 376 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 41b1e6c05781edefa7b1f5a91d87ef8a with result from task 376 parsl.dataflow.dflow DEBUG: Task 376 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/8253f94b-f87b-4f8f-8c70-12217e130cee_isrForCrosstalkSources_997009_9.stdout parsl.dataflow.dflow INFO: Standard error for task 376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/8253f94b-f87b-4f8f-8c70-12217e130cee_isrForCrosstalkSources_997009_9.stderr parsl.dataflow.dflow INFO: Task 380 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f1ab3f268bf60cf6cc1d33f2fdc91344 with result from task 380 parsl.dataflow.dflow DEBUG: Task 380 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/6eb3d3f4-d163-4d92-b542-a803b7c89ce7_isrForCrosstalkSources_997011_58.stdout parsl.dataflow.dflow INFO: Standard error for task 380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/6eb3d3f4-d163-4d92-b542-a803b7c89ce7_isrForCrosstalkSources_997011_58.stderr parsl.dataflow.dflow INFO: Task 379 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8708f228d5c463fd017aee236ede5918 with result from task 379 parsl.dataflow.dflow DEBUG: Task 379 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/086bcd9d-8998-4682-a358-215ea498d581_isrForCrosstalkSources_997028_27.stdout parsl.dataflow.dflow INFO: Standard error for task 379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/086bcd9d-8998-4682-a358-215ea498d581_isrForCrosstalkSources_997028_27.stderr parsl.dataflow.dflow INFO: Task 377 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 14739be2fc107f7dbe24777ef826b698 with result from task 377 parsl.dataflow.dflow DEBUG: Task 377 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9cf20c12-5098-47f8-baf7-4dc28e47499f_isrForCrosstalkSources_997015_31.stdout parsl.dataflow.dflow INFO: Standard error for task 377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9cf20c12-5098-47f8-baf7-4dc28e47499f_isrForCrosstalkSources_997015_31.stderr 2024-03-15 13:52:16 proc_lsst.multi:170 [INFO] [multi] job_id 17075219 proc_lsst.multi INFO: [multi] job_id 17075219 2024-03-15 13:52:16 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 142 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 142 2024-03-15 13:52:16 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:16 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075219 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075219 parsl.executors.status_handling DEBUG: Launched block 197 on executor multi with job ID 17075219 parsl.executors.status_handling INFO: Allocated block ID 198 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-198 2024-03-15 13:52:16 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-198 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-198 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 142 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 142 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:16 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=198 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 566 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ba2d1aa6210d80d1e91c3166b707a85 with result from task 566 parsl.dataflow.dflow DEBUG: Task 566 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/b71cd5eb-4f49-4be0-b464-bc1409da2f10_isrForCrosstalkSources_996961_52.stdout parsl.dataflow.dflow INFO: Standard error for task 566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/b71cd5eb-4f49-4be0-b464-bc1409da2f10_isrForCrosstalkSources_996961_52.stderr parsl.dataflow.dflow INFO: Task 572 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91307d14425539429d9faff2fd6af21b with result from task 572 parsl.dataflow.dflow DEBUG: Task 572 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/4fcc4eed-9826-4809-bcc2-5be73e0bce07_isrForCrosstalkSources_997000_6.stdout parsl.dataflow.dflow INFO: Standard error for task 572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/4fcc4eed-9826-4809-bcc2-5be73e0bce07_isrForCrosstalkSources_997000_6.stderr 2024-03-15 13:52:16 proc_lsst.multi:170 [INFO] [multi] job_id 17075220 proc_lsst.multi INFO: [multi] job_id 17075220 2024-03-15 13:52:16 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 143 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 143 2024-03-15 13:52:16 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:16 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075220 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075220 parsl.executors.status_handling DEBUG: Launched block 198 on executor multi with job ID 17075220 parsl.executors.status_handling INFO: Allocated block ID 199 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-199 2024-03-15 13:52:16 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-199 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-199 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:16 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 143 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 143 2024-03-15 13:52:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:16 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=199 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:17 proc_lsst.multi:170 [INFO] [multi] job_id 17075221 proc_lsst.multi INFO: [multi] job_id 17075221 2024-03-15 13:52:17 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 144 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 144 2024-03-15 13:52:17 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:17 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075221 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075221 parsl.executors.status_handling DEBUG: Launched block 199 on executor multi with job ID 17075221 parsl.executors.status_handling INFO: Allocated block ID 200 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-200 2024-03-15 13:52:17 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-200 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-200 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 144 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 144 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:17 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=200 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:17 proc_lsst.multi:170 [INFO] [multi] job_id 17075222 proc_lsst.multi INFO: [multi] job_id 17075222 2024-03-15 13:52:17 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 145 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 145 2024-03-15 13:52:17 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:17 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075222 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075222 parsl.executors.status_handling DEBUG: Launched block 200 on executor multi with job ID 17075222 parsl.executors.status_handling INFO: Allocated block ID 201 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-201 2024-03-15 13:52:17 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-201 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-201 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:17 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 145 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 145 2024-03-15 13:52:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:17 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=201 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:18 proc_lsst.multi:170 [INFO] [multi] job_id 17075223 proc_lsst.multi INFO: [multi] job_id 17075223 2024-03-15 13:52:18 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 146 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 146 2024-03-15 13:52:18 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:18 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075223 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075223 parsl.executors.status_handling DEBUG: Launched block 201 on executor multi with job ID 17075223 parsl.executors.status_handling INFO: Allocated block ID 202 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-202 2024-03-15 13:52:18 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-202 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-202 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 146 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 146 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:18 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=202 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:18 proc_lsst.multi:170 [INFO] [multi] job_id 17075224 proc_lsst.multi INFO: [multi] job_id 17075224 2024-03-15 13:52:18 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 147 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 147 2024-03-15 13:52:18 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:18 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075224 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075224 parsl.executors.status_handling DEBUG: Launched block 202 on executor multi with job ID 17075224 parsl.executors.status_handling INFO: Allocated block ID 203 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-203 2024-03-15 13:52:18 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-203 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-203 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:18 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 147 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 147 2024-03-15 13:52:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:18 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=203 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:19 proc_lsst.multi:170 [INFO] [multi] job_id 17075225 proc_lsst.multi INFO: [multi] job_id 17075225 2024-03-15 13:52:19 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 148 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 148 2024-03-15 13:52:19 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:19 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075225 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075225 parsl.executors.status_handling DEBUG: Launched block 203 on executor multi with job ID 17075225 parsl.executors.status_handling INFO: Allocated block ID 204 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-204 2024-03-15 13:52:19 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-204 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-204 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 148 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 148 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:19 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=204 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 578 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 123eacf08d68b8da6b362d5814d55cfc with result from task 578 parsl.dataflow.dflow DEBUG: Task 578 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5af276a1-5cf7-474a-aada-a83787325421_isrForCrosstalkSources_996968_7.stdout parsl.dataflow.dflow INFO: Standard error for task 578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5af276a1-5cf7-474a-aada-a83787325421_isrForCrosstalkSources_996968_7.stderr 2024-03-15 13:52:19 proc_lsst.multi:170 [INFO] [multi] job_id 17075226 proc_lsst.multi INFO: [multi] job_id 17075226 2024-03-15 13:52:19 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 149 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 149 2024-03-15 13:52:19 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:19 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075226 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075226 parsl.executors.status_handling DEBUG: Launched block 204 on executor multi with job ID 17075226 parsl.executors.status_handling INFO: Allocated block ID 205 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-205 2024-03-15 13:52:19 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-205 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-205 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:19 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 149 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 149 2024-03-15 13:52:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:19 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=205 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 545 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1adf9e94315929cf989d6264c1fad7b with result from task 545 parsl.dataflow.dflow DEBUG: Task 545 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a6db1b68-de6f-4e28-b2a5-2a82f86c1c8d_isrForCrosstalkSources_997013_61.stdout parsl.dataflow.dflow INFO: Standard error for task 545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a6db1b68-de6f-4e28-b2a5-2a82f86c1c8d_isrForCrosstalkSources_997013_61.stderr parsl.dataflow.dflow INFO: Task 543 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65c69d101bfaf872c5199748fc02046f with result from task 543 parsl.dataflow.dflow DEBUG: Task 543 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/58c021ce-2b10-43f4-97ab-911354956e40_isrForCrosstalkSources_997027_30.stdout parsl.dataflow.dflow INFO: Standard error for task 543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/58c021ce-2b10-43f4-97ab-911354956e40_isrForCrosstalkSources_997027_30.stderr 2024-03-15 13:52:20 proc_lsst.multi:170 [INFO] [multi] job_id 17075227 proc_lsst.multi INFO: [multi] job_id 17075227 2024-03-15 13:52:20 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 150 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 150 2024-03-15 13:52:20 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:20 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075227 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075227 parsl.executors.status_handling DEBUG: Launched block 205 on executor multi with job ID 17075227 parsl.executors.status_handling INFO: Allocated block ID 206 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-206 2024-03-15 13:52:20 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-206 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-206 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 150 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 150 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:20 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=206 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:20 proc_lsst.multi:170 [INFO] [multi] job_id 17075228 proc_lsst.multi INFO: [multi] job_id 17075228 2024-03-15 13:52:20 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 151 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 151 2024-03-15 13:52:20 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:20 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075228 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075228 parsl.executors.status_handling DEBUG: Launched block 206 on executor multi with job ID 17075228 parsl.executors.status_handling INFO: Allocated block ID 207 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-207 2024-03-15 13:52:20 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-207 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-207 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 151 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 151 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:20 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=207 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:20 proc_lsst.multi:170 [INFO] [multi] job_id 17075229 proc_lsst.multi INFO: [multi] job_id 17075229 2024-03-15 13:52:20 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 152 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 152 2024-03-15 13:52:20 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:20 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075229 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075229 parsl.executors.status_handling DEBUG: Launched block 207 on executor multi with job ID 17075229 parsl.executors.status_handling INFO: Allocated block ID 208 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-208 2024-03-15 13:52:20 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-208 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-208 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:20 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 152 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 152 2024-03-15 13:52:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:20 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=208 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 535 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78e0026ad34b8177673c8ceadc2abe6a with result from task 535 parsl.dataflow.dflow DEBUG: Task 535 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6c3670ab-b3b2-4b03-b781-a8f63f3cb58e_isrForCrosstalkSources_996952_60.stdout parsl.dataflow.dflow INFO: Standard error for task 535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6c3670ab-b3b2-4b03-b781-a8f63f3cb58e_isrForCrosstalkSources_996952_60.stderr 2024-03-15 13:52:21 proc_lsst.multi:170 [INFO] [multi] job_id 17075230 proc_lsst.multi INFO: [multi] job_id 17075230 2024-03-15 13:52:21 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 153 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 153 2024-03-15 13:52:21 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:21 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075230 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075230 parsl.executors.status_handling DEBUG: Launched block 208 on executor multi with job ID 17075230 parsl.executors.status_handling INFO: Allocated block ID 209 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-209 2024-03-15 13:52:21 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-209 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-209 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 153 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 153 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:21 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=209 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:21 proc_lsst.multi:170 [INFO] [multi] job_id 17075231 proc_lsst.multi INFO: [multi] job_id 17075231 2024-03-15 13:52:21 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 154 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 154 2024-03-15 13:52:21 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:21 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075231 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075231 parsl.executors.status_handling DEBUG: Launched block 209 on executor multi with job ID 17075231 parsl.executors.status_handling INFO: Allocated block ID 210 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-210 2024-03-15 13:52:21 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-210 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-210 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:21 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 154 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 154 2024-03-15 13:52:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:21 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=210 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:22 proc_lsst.multi:170 [INFO] [multi] job_id 17075232 proc_lsst.multi INFO: [multi] job_id 17075232 2024-03-15 13:52:22 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 155 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 155 2024-03-15 13:52:22 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:22 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075232 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075232 parsl.executors.status_handling DEBUG: Launched block 210 on executor multi with job ID 17075232 parsl.executors.status_handling INFO: Allocated block ID 211 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-211 2024-03-15 13:52:22 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-211 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-211 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 155 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 155 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:22 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=211 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:22 proc_lsst.multi:170 [INFO] [multi] job_id 17075233 proc_lsst.multi INFO: [multi] job_id 17075233 2024-03-15 13:52:22 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 156 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 156 2024-03-15 13:52:22 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:22 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075233 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075233 parsl.executors.status_handling DEBUG: Launched block 211 on executor multi with job ID 17075233 parsl.executors.status_handling INFO: Allocated block ID 212 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-212 2024-03-15 13:52:22 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-212 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-212 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:22 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 156 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 156 2024-03-15 13:52:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:22 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=212 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:23 proc_lsst.multi:170 [INFO] [multi] job_id 17075234 proc_lsst.multi INFO: [multi] job_id 17075234 2024-03-15 13:52:23 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 157 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 157 2024-03-15 13:52:23 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:23 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075234 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075234 parsl.executors.status_handling DEBUG: Launched block 212 on executor multi with job ID 17075234 parsl.executors.status_handling INFO: Allocated block ID 213 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-213 2024-03-15 13:52:23 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-213 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-213 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 157 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 157 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:23 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=213 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 539 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31b8d4a1beaa7cc87fa5cab358585a72 with result from task 539 parsl.dataflow.dflow DEBUG: Task 539 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e659f389-ad93-4fdf-be69-b02800800ead_isrForCrosstalkSources_996993_54.stdout parsl.dataflow.dflow INFO: Standard error for task 539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e659f389-ad93-4fdf-be69-b02800800ead_isrForCrosstalkSources_996993_54.stderr 2024-03-15 13:52:23 proc_lsst.multi:170 [INFO] [multi] job_id 17075238 proc_lsst.multi INFO: [multi] job_id 17075238 2024-03-15 13:52:23 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 158 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 158 2024-03-15 13:52:23 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:23 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075238 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075238 parsl.executors.status_handling DEBUG: Launched block 213 on executor multi with job ID 17075238 parsl.executors.status_handling INFO: Allocated block ID 214 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-214 2024-03-15 13:52:23 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-214 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-214 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:23 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 158 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 158 2024-03-15 13:52:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:23 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=214 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 592 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 257fb7187d640fd7c42ea40a2a3b786e with result from task 592 parsl.dataflow.dflow DEBUG: Task 592 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/27c016e3-bb8a-4391-8fb7-6976a7267245_isrForCrosstalkSources_996981_16.stdout parsl.dataflow.dflow INFO: Standard error for task 592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/27c016e3-bb8a-4391-8fb7-6976a7267245_isrForCrosstalkSources_996981_16.stderr 2024-03-15 13:52:24 proc_lsst.multi:170 [INFO] [multi] job_id 17075242 proc_lsst.multi INFO: [multi] job_id 17075242 2024-03-15 13:52:24 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 159 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 159 2024-03-15 13:52:24 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:24 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075242 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075242 parsl.executors.status_handling DEBUG: Launched block 214 on executor multi with job ID 17075242 parsl.executors.status_handling INFO: Allocated block ID 215 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-215 2024-03-15 13:52:24 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-215 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-215 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 159 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 159 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:24 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=215 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:24 proc_lsst.multi:170 [INFO] [multi] job_id 17075245 proc_lsst.multi INFO: [multi] job_id 17075245 2024-03-15 13:52:24 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 160 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 160 2024-03-15 13:52:24 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:24 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075245 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075245 parsl.executors.status_handling DEBUG: Launched block 215 on executor multi with job ID 17075245 parsl.executors.status_handling INFO: Allocated block ID 216 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-216 2024-03-15 13:52:24 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-216 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-216 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:24 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:24 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 160 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 160 2024-03-15 13:52:24 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:24 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=216 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:25 proc_lsst.multi:170 [INFO] [multi] job_id 17075246 proc_lsst.multi INFO: [multi] job_id 17075246 2024-03-15 13:52:25 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 161 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 161 2024-03-15 13:52:25 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:25 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075246 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075246 parsl.executors.status_handling DEBUG: Launched block 216 on executor multi with job ID 17075246 parsl.executors.status_handling INFO: Allocated block ID 217 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-217 2024-03-15 13:52:25 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-217 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-217 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 161 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 161 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:25 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=217 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:25 proc_lsst.multi:170 [INFO] [multi] job_id 17075247 proc_lsst.multi INFO: [multi] job_id 17075247 2024-03-15 13:52:25 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 162 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 162 2024-03-15 13:52:25 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:25 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075247 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075247 parsl.executors.status_handling DEBUG: Launched block 217 on executor multi with job ID 17075247 parsl.executors.status_handling INFO: Allocated block ID 218 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-218 2024-03-15 13:52:25 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-218 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-218 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:25 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:25 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 162 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 162 2024-03-15 13:52:25 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:25 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=218 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:26 proc_lsst.multi:170 [INFO] [multi] job_id 17075250 proc_lsst.multi INFO: [multi] job_id 17075250 2024-03-15 13:52:26 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 163 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 163 2024-03-15 13:52:26 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:26 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075250 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075250 parsl.executors.status_handling DEBUG: Launched block 218 on executor multi with job ID 17075250 parsl.executors.status_handling INFO: Allocated block ID 219 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-219 2024-03-15 13:52:26 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-219 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-219 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 163 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 163 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:26 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=219 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:26 proc_lsst.multi:170 [INFO] [multi] job_id 17075254 proc_lsst.multi INFO: [multi] job_id 17075254 2024-03-15 13:52:26 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 164 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 164 2024-03-15 13:52:26 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:26 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075254 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075254 parsl.executors.status_handling DEBUG: Launched block 219 on executor multi with job ID 17075254 parsl.executors.status_handling INFO: Allocated block ID 220 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-220 2024-03-15 13:52:26 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-220 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-220 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 164 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 164 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:26 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=220 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:26 proc_lsst.multi:170 [INFO] [multi] job_id 17075255 proc_lsst.multi INFO: [multi] job_id 17075255 2024-03-15 13:52:26 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 165 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 165 2024-03-15 13:52:26 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:26 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075255 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075255 parsl.executors.status_handling DEBUG: Launched block 220 on executor multi with job ID 17075255 parsl.executors.status_handling INFO: Allocated block ID 221 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-221 2024-03-15 13:52:26 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-221 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-221 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:26 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:26 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 165 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 165 2024-03-15 13:52:26 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:26 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=221 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 589 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 22ae72f1fb8de00c38e7dab94108a152 with result from task 589 parsl.dataflow.dflow DEBUG: Task 589 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/068c6bb9-bc64-4ed3-9608-a0891c521836_isrForCrosstalkSources_997013_22.stdout parsl.dataflow.dflow INFO: Standard error for task 589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/068c6bb9-bc64-4ed3-9608-a0891c521836_isrForCrosstalkSources_997013_22.stderr parsl.dataflow.dflow INFO: Task 595 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b05ad3b70086ddefd2c18088a12ff56 with result from task 595 parsl.dataflow.dflow DEBUG: Task 595 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e03298d3-3f2b-4f6f-bfb9-2dc0289b1331_isrForCrosstalkSources_996992_43.stdout parsl.dataflow.dflow INFO: Standard error for task 595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e03298d3-3f2b-4f6f-bfb9-2dc0289b1331_isrForCrosstalkSources_996992_43.stderr parsl.dataflow.dflow INFO: Task 584 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7c8f2c95e125c48966764f4700a8d74 with result from task 584 parsl.dataflow.dflow DEBUG: Task 584 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/82d9b2e5-6d66-47da-823b-b6aa7769cfd0_isrForCrosstalkSources_997009_25.stdout parsl.dataflow.dflow INFO: Standard error for task 584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/82d9b2e5-6d66-47da-823b-b6aa7769cfd0_isrForCrosstalkSources_997009_25.stderr 2024-03-15 13:52:27 proc_lsst.multi:170 [INFO] [multi] job_id 17075256 proc_lsst.multi INFO: [multi] job_id 17075256 2024-03-15 13:52:27 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 166 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 166 2024-03-15 13:52:27 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:27 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075256 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075256 parsl.executors.status_handling DEBUG: Launched block 221 on executor multi with job ID 17075256 parsl.executors.status_handling INFO: Allocated block ID 222 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-222 2024-03-15 13:52:27 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-222 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-222 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 166 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 166 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:27 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=222 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:27 proc_lsst.multi:170 [INFO] [multi] job_id 17075257 proc_lsst.multi INFO: [multi] job_id 17075257 2024-03-15 13:52:27 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 167 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 167 2024-03-15 13:52:27 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:27 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075257 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075257 parsl.executors.status_handling DEBUG: Launched block 222 on executor multi with job ID 17075257 parsl.executors.status_handling INFO: Allocated block ID 223 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-223 2024-03-15 13:52:27 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-223 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-223 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:27 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:27 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 167 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 167 2024-03-15 13:52:27 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:27 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=223 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 598 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd9c56ff417c098dd21fd712e027c111 with result from task 598 parsl.dataflow.dflow DEBUG: Task 598 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/07e65e51-4cef-4b82-8ed4-7dceb76532df_isrForCrosstalkSources_996994_23.stdout parsl.dataflow.dflow INFO: Standard error for task 598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/07e65e51-4cef-4b82-8ed4-7dceb76532df_isrForCrosstalkSources_996994_23.stderr parsl.dataflow.dflow INFO: Task 602 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b5fe67bc54ec108c668b4fe01f276eb with result from task 602 parsl.dataflow.dflow DEBUG: Task 602 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d7f21927-6118-4504-871d-b9efb09366b2_isrForCrosstalkSources_997027_52.stdout parsl.dataflow.dflow INFO: Standard error for task 602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d7f21927-6118-4504-871d-b9efb09366b2_isrForCrosstalkSources_997027_52.stderr 2024-03-15 13:52:28 proc_lsst.multi:170 [INFO] [multi] job_id 17075258 proc_lsst.multi INFO: [multi] job_id 17075258 2024-03-15 13:52:28 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 168 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 168 2024-03-15 13:52:28 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:28 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075258 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075258 parsl.executors.status_handling DEBUG: Launched block 223 on executor multi with job ID 17075258 parsl.executors.status_handling INFO: Allocated block ID 224 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-224 2024-03-15 13:52:28 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-224 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-224 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 168 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 168 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:28 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=224 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 585 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e618f0c1a7a589b4099494b7c5135d8e with result from task 585 parsl.dataflow.dflow DEBUG: Task 585 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b8babf2c-b147-4532-82c3-8742c4886503_isrForCrosstalkSources_997025_34.stdout parsl.dataflow.dflow INFO: Standard error for task 585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b8babf2c-b147-4532-82c3-8742c4886503_isrForCrosstalkSources_997025_34.stderr 2024-03-15 13:52:28 proc_lsst.multi:170 [INFO] [multi] job_id 17075264 proc_lsst.multi INFO: [multi] job_id 17075264 2024-03-15 13:52:28 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 169 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 169 2024-03-15 13:52:28 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:28 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075264 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075264 parsl.executors.status_handling DEBUG: Launched block 224 on executor multi with job ID 17075264 parsl.executors.status_handling INFO: Allocated block ID 225 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-225 2024-03-15 13:52:28 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-225 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-225 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:28 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:28 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 169 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 169 2024-03-15 13:52:28 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:28 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=225 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:29 proc_lsst.multi:170 [INFO] [multi] job_id 17075265 proc_lsst.multi INFO: [multi] job_id 17075265 2024-03-15 13:52:29 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 170 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 170 2024-03-15 13:52:29 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:29 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075265 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075265 parsl.executors.status_handling DEBUG: Launched block 225 on executor multi with job ID 17075265 parsl.executors.status_handling INFO: Allocated block ID 226 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-226 2024-03-15 13:52:29 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-226 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-226 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 170 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 170 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:29 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=226 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 590 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e593ab4fb9dab4352efabd8072073b9 with result from task 590 parsl.dataflow.dflow DEBUG: Task 590 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d0130a21-235d-42f0-b2c9-8fa99782771c_isrForCrosstalkSources_996958_46.stdout parsl.dataflow.dflow INFO: Standard error for task 590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d0130a21-235d-42f0-b2c9-8fa99782771c_isrForCrosstalkSources_996958_46.stderr parsl.dataflow.dflow INFO: Task 596 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3cfd8701575cbb6815adab12064ca8be with result from task 596 parsl.dataflow.dflow DEBUG: Task 596 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/cb27cffe-83fe-46ac-811b-eb23a9e3d951_isrForCrosstalkSources_997011_13.stdout parsl.dataflow.dflow INFO: Standard error for task 596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/cb27cffe-83fe-46ac-811b-eb23a9e3d951_isrForCrosstalkSources_997011_13.stderr 2024-03-15 13:52:29 proc_lsst.multi:170 [INFO] [multi] job_id 17075273 proc_lsst.multi INFO: [multi] job_id 17075273 2024-03-15 13:52:29 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 171 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 171 2024-03-15 13:52:29 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:29 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075273 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075273 parsl.executors.status_handling DEBUG: Launched block 226 on executor multi with job ID 17075273 parsl.executors.status_handling INFO: Allocated block ID 227 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-227 2024-03-15 13:52:29 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-227 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-227 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:29 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:29 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 171 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 171 2024-03-15 13:52:29 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:29 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=227 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:30 proc_lsst.multi:170 [INFO] [multi] job_id 17075274 proc_lsst.multi INFO: [multi] job_id 17075274 2024-03-15 13:52:30 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 172 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 172 2024-03-15 13:52:30 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:30 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075274 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075274 parsl.executors.status_handling DEBUG: Launched block 227 on executor multi with job ID 17075274 parsl.executors.status_handling INFO: Allocated block ID 228 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-228 2024-03-15 13:52:30 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-228 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-228 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 172 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 172 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:30 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=228 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:30 proc_lsst.multi:170 [INFO] [multi] job_id 17075275 proc_lsst.multi INFO: [multi] job_id 17075275 2024-03-15 13:52:30 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 173 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 173 2024-03-15 13:52:30 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:30 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075275 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075275 parsl.executors.status_handling DEBUG: Launched block 228 on executor multi with job ID 17075275 parsl.executors.status_handling INFO: Allocated block ID 229 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-229 2024-03-15 13:52:30 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-229 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-229 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 173 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 173 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:30 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=229 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 606 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6d411e91e0afcf0dbb4172c97f8a91b with result from task 606 parsl.dataflow.dflow DEBUG: Task 606 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/2ccd3756-9005-4145-a9a1-78faaf6904e5_isrForCrosstalkSources_996979_40.stdout parsl.dataflow.dflow INFO: Standard error for task 606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/2ccd3756-9005-4145-a9a1-78faaf6904e5_isrForCrosstalkSources_996979_40.stderr 2024-03-15 13:52:30 proc_lsst.multi:170 [INFO] [multi] job_id 17075276 proc_lsst.multi INFO: [multi] job_id 17075276 2024-03-15 13:52:30 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 174 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 174 2024-03-15 13:52:30 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:30 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075276 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075276 parsl.executors.status_handling DEBUG: Launched block 229 on executor multi with job ID 17075276 parsl.executors.status_handling INFO: Allocated block ID 230 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-230 2024-03-15 13:52:30 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-230 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-230 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:30 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:30 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 174 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 174 2024-03-15 13:52:30 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:30 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=230 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 593 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ecb6c9fad05eaa1ab82fd87aed49333 with result from task 593 parsl.dataflow.dflow DEBUG: Task 593 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6f353c39-5b14-408c-aeff-282843ef8a5c_isrForCrosstalkSources_996991_13.stdout parsl.dataflow.dflow INFO: Standard error for task 593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6f353c39-5b14-408c-aeff-282843ef8a5c_isrForCrosstalkSources_996991_13.stderr parsl.dataflow.dflow INFO: Task 580 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 543142a9a78de75897970847ff89f162 with result from task 580 parsl.dataflow.dflow DEBUG: Task 580 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f1df6ac4-3db4-4b50-ac40-28696bccb3d9_isrForCrosstalkSources_996956_19.stdout parsl.dataflow.dflow INFO: Standard error for task 580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f1df6ac4-3db4-4b50-ac40-28696bccb3d9_isrForCrosstalkSources_996956_19.stderr 2024-03-15 13:52:31 proc_lsst.multi:170 [INFO] [multi] job_id 17075278 proc_lsst.multi INFO: [multi] job_id 17075278 2024-03-15 13:52:31 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 175 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 175 2024-03-15 13:52:31 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:31 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075278 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075278 parsl.executors.status_handling DEBUG: Launched block 230 on executor multi with job ID 17075278 parsl.executors.status_handling INFO: Allocated block ID 231 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-231 2024-03-15 13:52:31 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-231 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-231 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 175 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 175 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:31 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=231 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:31 proc_lsst.multi:170 [INFO] [multi] job_id 17075279 proc_lsst.multi INFO: [multi] job_id 17075279 2024-03-15 13:52:31 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 176 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 176 2024-03-15 13:52:31 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:31 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075279 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075279 parsl.executors.status_handling DEBUG: Launched block 231 on executor multi with job ID 17075279 parsl.executors.status_handling INFO: Allocated block ID 232 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-232 2024-03-15 13:52:31 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-232 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-232 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:31 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:31 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 176 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 176 2024-03-15 13:52:31 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:31 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=232 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:32 proc_lsst.multi:170 [INFO] [multi] job_id 17075280 proc_lsst.multi INFO: [multi] job_id 17075280 2024-03-15 13:52:32 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 177 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 177 2024-03-15 13:52:32 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:32 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075280 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075280 parsl.executors.status_handling DEBUG: Launched block 232 on executor multi with job ID 17075280 parsl.executors.status_handling INFO: Allocated block ID 233 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-233 2024-03-15 13:52:32 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-233 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-233 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 177 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 177 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:32 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=233 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:32 proc_lsst.multi:170 [INFO] [multi] job_id 17075281 proc_lsst.multi INFO: [multi] job_id 17075281 2024-03-15 13:52:32 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 178 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 178 2024-03-15 13:52:32 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:32 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075281 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075281 parsl.executors.status_handling DEBUG: Launched block 233 on executor multi with job ID 17075281 parsl.executors.status_handling INFO: Allocated block ID 234 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-234 2024-03-15 13:52:32 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-234 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-234 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:32 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:32 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 178 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 178 2024-03-15 13:52:32 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:32 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=234 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 616 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8025965d9c8375ee560b2f94ee5bd17 with result from task 616 parsl.dataflow.dflow DEBUG: Task 616 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/494d4d83-224b-4f31-8abb-00b79c7115c7_isrForCrosstalkSources_996969_27.stdout parsl.dataflow.dflow INFO: Standard error for task 616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/494d4d83-224b-4f31-8abb-00b79c7115c7_isrForCrosstalkSources_996969_27.stderr parsl.dataflow.dflow INFO: Task 599 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73aa30cabb2c3112656c087ce49d6687 with result from task 599 parsl.dataflow.dflow DEBUG: Task 599 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/2f5e9fb7-7982-41cc-adb4-31a97019808a_isrForCrosstalkSources_997010_43.stdout parsl.dataflow.dflow INFO: Standard error for task 599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/2f5e9fb7-7982-41cc-adb4-31a97019808a_isrForCrosstalkSources_997010_43.stderr 2024-03-15 13:52:33 proc_lsst.multi:170 [INFO] [multi] job_id 17075282 proc_lsst.multi INFO: [multi] job_id 17075282 2024-03-15 13:52:33 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 179 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 179 2024-03-15 13:52:33 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:33 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075282 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075282 parsl.executors.status_handling DEBUG: Launched block 234 on executor multi with job ID 17075282 parsl.executors.status_handling INFO: Allocated block ID 235 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-235 2024-03-15 13:52:33 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-235 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-235 2024-03-15 13:52:33 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:33 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:33 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:33 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:33 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:33 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 179 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 179 2024-03-15 13:52:33 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:33 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=235 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 620 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c7a3ffbc967b0996ec4f251372ab791 with result from task 620 parsl.dataflow.dflow DEBUG: Task 620 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3ce02e0b-a885-4de4-85e7-71f768e02043_isrForCrosstalkSources_997031_45.stdout parsl.dataflow.dflow INFO: Standard error for task 620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3ce02e0b-a885-4de4-85e7-71f768e02043_isrForCrosstalkSources_997031_45.stderr parsl.dataflow.dflow INFO: Task 601 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4be9adb0b7f14127624b35b3ee21219a with result from task 601 parsl.dataflow.dflow DEBUG: Task 601 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7c124250-e9e1-4e7a-9b65-4b45af6e9b18_isrForCrosstalkSources_997003_36.stdout parsl.dataflow.dflow INFO: Standard error for task 601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7c124250-e9e1-4e7a-9b65-4b45af6e9b18_isrForCrosstalkSources_997003_36.stderr 2024-03-15 13:52:34 proc_lsst.multi:170 [INFO] [multi] job_id 17075283 proc_lsst.multi INFO: [multi] job_id 17075283 2024-03-15 13:52:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 180 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 180 2024-03-15 13:52:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:34 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075283 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075283 parsl.executors.status_handling DEBUG: Launched block 235 on executor multi with job ID 17075283 parsl.executors.status_handling INFO: Allocated block ID 236 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-236 2024-03-15 13:52:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-236 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-236 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 180 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 180 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=236 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:34 proc_lsst.multi:170 [INFO] [multi] job_id 17075284 proc_lsst.multi INFO: [multi] job_id 17075284 2024-03-15 13:52:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 181 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 181 2024-03-15 13:52:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:34 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075284 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075284 parsl.executors.status_handling DEBUG: Launched block 236 on executor multi with job ID 17075284 parsl.executors.status_handling INFO: Allocated block ID 237 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-237 2024-03-15 13:52:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-237 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-237 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 181 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 181 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=237 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Task 597 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 887f4c4e115af61681867c5456aed996 with result from task 597 parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Task 597 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9904f1bf-0a72-411d-acac-f658456e85d6_isrForCrosstalkSources_996958_52.stdout parsl.dataflow.dflow INFO: Standard error for task 597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9904f1bf-0a72-411d-acac-f658456e85d6_isrForCrosstalkSources_996958_52.stderr 2024-03-15 13:52:34 proc_lsst.multi:170 [INFO] [multi] job_id 17075285 proc_lsst.multi INFO: [multi] job_id 17075285 2024-03-15 13:52:34 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 182 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 182 2024-03-15 13:52:34 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:34 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075285 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075285 parsl.executors.status_handling DEBUG: Launched block 237 on executor multi with job ID 17075285 parsl.executors.status_handling INFO: Allocated block ID 238 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-238 2024-03-15 13:52:34 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-238 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-238 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:34 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:34 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 182 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 182 2024-03-15 13:52:34 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:34 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=238 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 605 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d7ee43f256568934eeb11b7fff5d8ba with result from task 605 parsl.dataflow.dflow DEBUG: Task 605 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/4476cd38-7168-4ddf-b5ca-91554ae02007_isrForCrosstalkSources_997019_17.stdout parsl.dataflow.dflow INFO: Standard error for task 605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/4476cd38-7168-4ddf-b5ca-91554ae02007_isrForCrosstalkSources_997019_17.stderr parsl.dataflow.dflow INFO: Task 549 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4da8d7bb2db767c6861f87ee6b2a96a with result from task 549 parsl.dataflow.dflow DEBUG: Task 549 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/380d4714-6a22-4189-b63a-91151a0601a7_isrForCrosstalkSources_997022_21.stdout parsl.dataflow.dflow INFO: Standard error for task 549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/380d4714-6a22-4189-b63a-91151a0601a7_isrForCrosstalkSources_997022_21.stderr parsl.dataflow.dflow INFO: Task 615 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19b33d10e40122270249a0b60ed3ceb7 with result from task 615 parsl.dataflow.dflow DEBUG: Task 615 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d3190d70-9bd3-4db6-a3e2-27e569e12170_isrForCrosstalkSources_997027_27.stdout parsl.dataflow.dflow INFO: Standard error for task 615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d3190d70-9bd3-4db6-a3e2-27e569e12170_isrForCrosstalkSources_997027_27.stderr 2024-03-15 13:52:35 proc_lsst.multi:170 [INFO] [multi] job_id 17075286 proc_lsst.multi INFO: [multi] job_id 17075286 2024-03-15 13:52:35 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 183 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 183 2024-03-15 13:52:35 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:35 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075286 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075286 parsl.executors.status_handling DEBUG: Launched block 238 on executor multi with job ID 17075286 parsl.executors.status_handling INFO: Allocated block ID 239 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-239 2024-03-15 13:52:35 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-239 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-239 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 183 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 183 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:35 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=239 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:35 proc_lsst.multi:170 [INFO] [multi] job_id 17075287 proc_lsst.multi INFO: [multi] job_id 17075287 2024-03-15 13:52:35 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 184 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 184 2024-03-15 13:52:35 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:35 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075287 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075287 parsl.executors.status_handling DEBUG: Launched block 239 on executor multi with job ID 17075287 parsl.executors.status_handling INFO: Allocated block ID 240 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-240 2024-03-15 13:52:35 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-240 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-240 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:35 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:35 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 184 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 184 2024-03-15 13:52:35 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:35 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=240 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 452 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e00a6003664f37e5dd64bd320d52c0f3 with result from task 452 parsl.dataflow.dflow DEBUG: Task 452 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7b74e025-b9ae-4d53-bc7b-63a14ce3209e_isrForCrosstalkSources_996968_39.stdout parsl.dataflow.dflow INFO: Standard error for task 452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7b74e025-b9ae-4d53-bc7b-63a14ce3209e_isrForCrosstalkSources_996968_39.stderr parsl.dataflow.dflow INFO: Task 456 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed61091b39c346ca4c4883412038a22c with result from task 456 parsl.dataflow.dflow DEBUG: Task 456 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ff4dfac3-906b-4064-8597-72d37eca882c_isrForCrosstalkSources_996959_42.stdout parsl.dataflow.dflow INFO: Standard error for task 456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ff4dfac3-906b-4064-8597-72d37eca882c_isrForCrosstalkSources_996959_42.stderr parsl.dataflow.dflow INFO: Task 451 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a07735057b04752142984fcb04b9df15 with result from task 451 parsl.dataflow.dflow DEBUG: Task 451 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/65b2979d-83d6-47db-b504-3495a5114c60_isrForCrosstalkSources_997022_51.stdout parsl.dataflow.dflow INFO: Standard error for task 451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/65b2979d-83d6-47db-b504-3495a5114c60_isrForCrosstalkSources_997022_51.stderr parsl.dataflow.dflow INFO: Task 458 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53b8e84fd8a689c5880d0c13355f7a5e with result from task 458 parsl.dataflow.dflow DEBUG: Task 458 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7757b19c-a127-4288-89ae-1ec480447739_isrForCrosstalkSources_997026_19.stdout parsl.dataflow.dflow INFO: Standard error for task 458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7757b19c-a127-4288-89ae-1ec480447739_isrForCrosstalkSources_997026_19.stderr parsl.dataflow.dflow INFO: Task 453 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6bcfebd8e6868660be4712ba50a12bd4 with result from task 453 parsl.dataflow.dflow DEBUG: Task 453 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/02988f69-6519-43a0-b3a9-f563965aee5c_isrForCrosstalkSources_996974_52.stdout parsl.dataflow.dflow INFO: Standard error for task 453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/02988f69-6519-43a0-b3a9-f563965aee5c_isrForCrosstalkSources_996974_52.stderr parsl.dataflow.dflow INFO: Task 439 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d1a351a3e83f303e666a55ad74ac85e with result from task 439 parsl.dataflow.dflow DEBUG: Task 439 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/8916bcc2-541b-4060-9089-cbfcade307f3_isrForCrosstalkSources_997029_38.stdout parsl.dataflow.dflow INFO: Standard error for task 439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/8916bcc2-541b-4060-9089-cbfcade307f3_isrForCrosstalkSources_997029_38.stderr parsl.dataflow.dflow INFO: Task 438 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 243265f0298e9dcc1cea7080f6dfbc1e with result from task 438 parsl.dataflow.dflow DEBUG: Task 438 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/cc49b279-2ad7-4a89-8097-9f4738d8b697_isrForCrosstalkSources_996977_3.stdout parsl.dataflow.dflow INFO: Standard error for task 438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/cc49b279-2ad7-4a89-8097-9f4738d8b697_isrForCrosstalkSources_996977_3.stderr 2024-03-15 13:52:36 proc_lsst.multi:170 [INFO] [multi] job_id 17075288 proc_lsst.multi INFO: [multi] job_id 17075288 2024-03-15 13:52:36 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 185 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 185 2024-03-15 13:52:36 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:36 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075288 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075288 parsl.executors.status_handling DEBUG: Launched block 240 on executor multi with job ID 17075288 parsl.executors.status_handling INFO: Allocated block ID 241 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-241 2024-03-15 13:52:36 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-241 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-241 2024-03-15 13:52:36 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:36 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:36 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:36 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:36 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:36 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 185 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 185 2024-03-15 13:52:36 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:36 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=241 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 444 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2bfa2ba89b5b22e887146aa1fc2d9060 with result from task 444 parsl.dataflow.dflow DEBUG: Task 444 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/fbf25cf2-728f-438c-a6a9-f3493160ee90_isrForCrosstalkSources_997024_17.stdout parsl.dataflow.dflow INFO: Standard error for task 444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/fbf25cf2-728f-438c-a6a9-f3493160ee90_isrForCrosstalkSources_997024_17.stderr parsl.dataflow.dflow INFO: Task 442 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 392403ebe05483418eac9f879defdc10 with result from task 442 parsl.dataflow.dflow DEBUG: Task 442 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2f04a853-e565-4e31-8626-be5a77461991_isrForCrosstalkSources_996990_17.stdout parsl.dataflow.dflow INFO: Standard error for task 442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2f04a853-e565-4e31-8626-be5a77461991_isrForCrosstalkSources_996990_17.stderr parsl.dataflow.dflow INFO: Task 445 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0dc84dc7e6712e987878bfb38a824217 with result from task 445 parsl.dataflow.dflow DEBUG: Task 445 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f95bede1-ba37-43ba-bd0f-e036d5c76d2b_isrForCrosstalkSources_996973_45.stdout parsl.dataflow.dflow INFO: Standard error for task 445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f95bede1-ba37-43ba-bd0f-e036d5c76d2b_isrForCrosstalkSources_996973_45.stderr parsl.dataflow.dflow INFO: Task 434 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ee15ac2938c8ad2a0073acf8748f982 with result from task 434 parsl.dataflow.dflow DEBUG: Task 434 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0c5e499c-22c8-47f0-9f95-0ede9ccc4b6b_isrForCrosstalkSources_996985_45.stdout parsl.dataflow.dflow INFO: Standard error for task 434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0c5e499c-22c8-47f0-9f95-0ede9ccc4b6b_isrForCrosstalkSources_996985_45.stderr parsl.dataflow.dflow INFO: Task 448 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab0ce5053521ca4a2f9814a976f433a4 with result from task 448 parsl.dataflow.dflow DEBUG: Task 448 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/ba7cebf9-9e09-4ee1-9a23-6a3540c31411_isrForCrosstalkSources_996970_47.stdout parsl.dataflow.dflow INFO: Standard error for task 448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/ba7cebf9-9e09-4ee1-9a23-6a3540c31411_isrForCrosstalkSources_996970_47.stderr parsl.dataflow.dflow INFO: Task 437 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6096affa12cc9bd540d8c524fe15a8f1 with result from task 437 parsl.dataflow.dflow DEBUG: Task 437 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/8ffcef67-eb65-41b1-bec4-124a5e887d62_isrForCrosstalkSources_996957_24.stdout parsl.dataflow.dflow INFO: Standard error for task 437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/8ffcef67-eb65-41b1-bec4-124a5e887d62_isrForCrosstalkSources_996957_24.stderr parsl.dataflow.dflow INFO: Task 440 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ba93b23b955de5be838f28581509629 with result from task 440 parsl.dataflow.dflow DEBUG: Task 440 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7148144a-392f-4a80-a265-417b57333701_isrForCrosstalkSources_997021_34.stdout parsl.dataflow.dflow INFO: Standard error for task 440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7148144a-392f-4a80-a265-417b57333701_isrForCrosstalkSources_997021_34.stderr parsl.dataflow.dflow INFO: Task 447 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 77b0663ec1baeba707fb9c52a2934c6a with result from task 447 parsl.dataflow.dflow DEBUG: Task 447 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/006c7fb8-0360-467f-b7fa-b9ebc239574a_isrForCrosstalkSources_996958_18.stdout parsl.dataflow.dflow INFO: Standard error for task 447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/006c7fb8-0360-467f-b7fa-b9ebc239574a_isrForCrosstalkSources_996958_18.stderr parsl.dataflow.dflow INFO: Task 441 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b60db28fb80f631c377120afddfc6296 with result from task 441 parsl.dataflow.dflow DEBUG: Task 441 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/64679921-e376-49c7-9d8b-71b5a677bccd_isrForCrosstalkSources_997011_39.stdout parsl.dataflow.dflow INFO: Standard error for task 441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/64679921-e376-49c7-9d8b-71b5a677bccd_isrForCrosstalkSources_997011_39.stderr 2024-03-15 13:52:37 proc_lsst.multi:170 [INFO] [multi] job_id 17075289 proc_lsst.multi INFO: [multi] job_id 17075289 2024-03-15 13:52:37 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 186 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 186 2024-03-15 13:52:37 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:37 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075289 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075289 parsl.executors.status_handling DEBUG: Launched block 241 on executor multi with job ID 17075289 parsl.executors.status_handling INFO: Allocated block ID 242 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-242 2024-03-15 13:52:37 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-242 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-242 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 186 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 186 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:37 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=242 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 446 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 042b15b7dd7ee9e6c53f7ac8f6365074 with result from task 446 parsl.dataflow.dflow DEBUG: Task 446 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d628f2d0-73a6-4105-9fa9-d6262a6102e1_isrForCrosstalkSources_996979_38.stdout parsl.dataflow.dflow INFO: Standard error for task 446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d628f2d0-73a6-4105-9fa9-d6262a6102e1_isrForCrosstalkSources_996979_38.stderr parsl.dataflow.dflow INFO: Task 576 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25f49186da55eb63e091b35ea509d4c4 with result from task 576 parsl.dataflow.dflow DEBUG: Task 576 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7ae1b603-6e65-45e4-b2b9-b88c694cdf3a_isrForCrosstalkSources_997024_24.stdout parsl.dataflow.dflow INFO: Standard error for task 576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7ae1b603-6e65-45e4-b2b9-b88c694cdf3a_isrForCrosstalkSources_997024_24.stderr parsl.dataflow.dflow INFO: Task 457 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7513b3cf404804243afc1db8126c0f25 with result from task 457 parsl.dataflow.dflow DEBUG: Task 457 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/40025d54-18ff-4297-aa32-b86f076876c4_isrForCrosstalkSources_996990_49.stdout parsl.dataflow.dflow INFO: Standard error for task 457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/40025d54-18ff-4297-aa32-b86f076876c4_isrForCrosstalkSources_996990_49.stderr parsl.dataflow.dflow INFO: Task 557 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3056959c6fbf3fbe131f68813401b26e with result from task 557 parsl.dataflow.dflow DEBUG: Task 557 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/f5f3b95b-ff68-48a8-b06c-5190b61848a1_isrForCrosstalkSources_996968_51.stdout parsl.dataflow.dflow INFO: Standard error for task 557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/f5f3b95b-ff68-48a8-b06c-5190b61848a1_isrForCrosstalkSources_996968_51.stderr parsl.dataflow.dflow INFO: Task 600 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0b59c8c61cf7b3ab12f9172e38d95c3c with result from task 600 parsl.dataflow.dflow DEBUG: Task 600 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/4f65d7e4-69bc-42e1-a1d3-f4cf0d07f7dd_isrForCrosstalkSources_997001_54.stdout parsl.dataflow.dflow INFO: Standard error for task 600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/4f65d7e4-69bc-42e1-a1d3-f4cf0d07f7dd_isrForCrosstalkSources_997001_54.stderr 2024-03-15 13:52:37 proc_lsst.multi:170 [INFO] [multi] job_id 17075290 proc_lsst.multi INFO: [multi] job_id 17075290 2024-03-15 13:52:37 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 187 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 187 2024-03-15 13:52:37 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:37 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075290 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075290 parsl.executors.status_handling DEBUG: Launched block 242 on executor multi with job ID 17075290 parsl.executors.status_handling INFO: Allocated block ID 243 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-243 2024-03-15 13:52:37 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-243 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-243 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:37 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:37 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 187 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 187 2024-03-15 13:52:37 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:37 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=243 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 622 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a78e1275f198cee109e9a9e2b6493d2 with result from task 622 parsl.dataflow.dflow DEBUG: Task 622 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f5de12e8-cc6b-4334-b2a2-6a3a2993f597_isrForCrosstalkSources_996967_22.stdout parsl.dataflow.dflow INFO: Standard error for task 622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f5de12e8-cc6b-4334-b2a2-6a3a2993f597_isrForCrosstalkSources_996967_22.stderr parsl.dataflow.dflow INFO: Task 555 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04924206670c83a03d2ed26542fcd58f with result from task 555 parsl.dataflow.dflow DEBUG: Task 555 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2009a79d-3d6f-470a-af2e-96a1374155be_isrForCrosstalkSources_996995_5.stdout parsl.dataflow.dflow INFO: Standard error for task 555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2009a79d-3d6f-470a-af2e-96a1374155be_isrForCrosstalkSources_996995_5.stderr parsl.dataflow.dflow INFO: Task 603 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 288530e68aa25cb005fdfaaa90f2f26d with result from task 603 parsl.dataflow.dflow DEBUG: Task 603 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d4b009d7-0672-40e9-99c6-4a92c6af878f_isrForCrosstalkSources_996958_53.stdout parsl.dataflow.dflow INFO: Standard error for task 603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d4b009d7-0672-40e9-99c6-4a92c6af878f_isrForCrosstalkSources_996958_53.stderr 2024-03-15 13:52:38 proc_lsst.multi:170 [INFO] [multi] job_id 17075293 proc_lsst.multi INFO: [multi] job_id 17075293 2024-03-15 13:52:38 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 188 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 188 2024-03-15 13:52:38 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:38 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075293 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075293 parsl.executors.status_handling DEBUG: Launched block 243 on executor multi with job ID 17075293 parsl.executors.status_handling INFO: Allocated block ID 244 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-244 2024-03-15 13:52:38 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-244 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-244 2024-03-15 13:52:38 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:38 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:38 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:38 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:38 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:38 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 188 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 188 2024-03-15 13:52:38 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:38 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=244 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Task 579 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e1b0bd51ffd3620c3211c78e77878ff8 with result from task 579 parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Task 579 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a824a98d-a972-4d01-a618-7b65db98a65b_isrForCrosstalkSources_997015_12.stdout parsl.dataflow.dflow INFO: Standard error for task 579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a824a98d-a972-4d01-a618-7b65db98a65b_isrForCrosstalkSources_997015_12.stderr parsl.dataflow.dflow INFO: Task 582 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 923e5e57b3675fa0cdf5b333c8908832 with result from task 582 parsl.dataflow.dflow DEBUG: Task 582 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4660ce9b-d261-4a87-a033-08f9977a3a67_isrForCrosstalkSources_996972_5.stdout parsl.dataflow.dflow INFO: Standard error for task 582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4660ce9b-d261-4a87-a033-08f9977a3a67_isrForCrosstalkSources_996972_5.stderr parsl.dataflow.dflow INFO: Task 624 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d8ccad5648e381315937ad1d0a19cdb with result from task 624 parsl.dataflow.dflow DEBUG: Task 624 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/9f5974ba-8178-477d-9fd5-827f712a5371_isrForCrosstalkSources_997002_48.stdout parsl.dataflow.dflow INFO: Standard error for task 624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/9f5974ba-8178-477d-9fd5-827f712a5371_isrForCrosstalkSources_997002_48.stderr parsl.dataflow.dflow INFO: Task 435 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3cd004ca21811571ac87c88e42383d6f with result from task 435 parsl.dataflow.dflow DEBUG: Task 435 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/544da854-c8ca-423d-840f-08d67be6f8fe_isrForCrosstalkSources_997030_36.stdout parsl.dataflow.dflow INFO: Standard error for task 435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/544da854-c8ca-423d-840f-08d67be6f8fe_isrForCrosstalkSources_997030_36.stderr parsl.dataflow.dflow INFO: Task 477 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4528ee1deb316d61855e1b799fde387 with result from task 477 parsl.dataflow.dflow DEBUG: Task 477 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/0460b0c5-d728-4a72-9040-cd0752c6b8c2_isrForCrosstalkSources_997024_43.stdout parsl.dataflow.dflow INFO: Standard error for task 477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/0460b0c5-d728-4a72-9040-cd0752c6b8c2_isrForCrosstalkSources_997024_43.stderr parsl.dataflow.dflow INFO: Task 482 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 971664c386c1e26a1b66e8145e6c2688 with result from task 482 parsl.dataflow.dflow DEBUG: Task 482 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/3898d01d-c7a5-487e-9ab6-ad276719c87b_isrForCrosstalkSources_997024_41.stdout parsl.dataflow.dflow INFO: Standard error for task 482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/3898d01d-c7a5-487e-9ab6-ad276719c87b_isrForCrosstalkSources_997024_41.stderr parsl.dataflow.dflow INFO: Task 481 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d023ef54fcbdd578066cd919360cd9cc with result from task 481 parsl.dataflow.dflow DEBUG: Task 481 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f901a167-79e9-41ad-b404-51b1d376fec8_isrForCrosstalkSources_996989_56.stdout parsl.dataflow.dflow INFO: Standard error for task 481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f901a167-79e9-41ad-b404-51b1d376fec8_isrForCrosstalkSources_996989_56.stderr parsl.dataflow.dflow INFO: Task 489 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e729917e987463cba33889b18e293352 with result from task 489 parsl.dataflow.dflow DEBUG: Task 489 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/78cf5ac9-dd04-44a5-8cc8-451b59596766_isrForCrosstalkSources_997015_37.stdout parsl.dataflow.dflow INFO: Standard error for task 489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/78cf5ac9-dd04-44a5-8cc8-451b59596766_isrForCrosstalkSources_997015_37.stderr parsl.dataflow.dflow INFO: Task 476 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae7fdd7195f1beb4fb166ed9056f2f52 with result from task 476 parsl.dataflow.dflow DEBUG: Task 476 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/44ff053c-9678-496b-bde5-6e541637c0e5_isrForCrosstalkSources_996984_34.stdout parsl.dataflow.dflow INFO: Standard error for task 476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/44ff053c-9678-496b-bde5-6e541637c0e5_isrForCrosstalkSources_996984_34.stderr parsl.dataflow.dflow INFO: Task 479 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81ebf205e1f05265c2a2ba9738cb9b69 with result from task 479 parsl.dataflow.dflow DEBUG: Task 479 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/24a783dc-0a93-4c8b-b630-fab9ee382804_isrForCrosstalkSources_996994_39.stdout parsl.dataflow.dflow INFO: Standard error for task 479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/24a783dc-0a93-4c8b-b630-fab9ee382804_isrForCrosstalkSources_996994_39.stderr parsl.dataflow.dflow INFO: Task 486 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c96c94c80b7b2f164648dfe3d87f1a3 with result from task 486 parsl.dataflow.dflow DEBUG: Task 486 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a55c83ea-b392-4bf1-80bc-3f7a8b322ced_isrForCrosstalkSources_996994_27.stdout parsl.dataflow.dflow INFO: Standard error for task 486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a55c83ea-b392-4bf1-80bc-3f7a8b322ced_isrForCrosstalkSources_996994_27.stderr parsl.dataflow.dflow INFO: Task 485 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a9c1a01ceeb78cfd45e0cbdbebee4dfe with result from task 485 parsl.dataflow.dflow DEBUG: Task 485 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c8a82da2-c1bb-487d-bf21-c23768edc87d_isrForCrosstalkSources_997025_33.stdout parsl.dataflow.dflow INFO: Standard error for task 485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c8a82da2-c1bb-487d-bf21-c23768edc87d_isrForCrosstalkSources_997025_33.stderr parsl.dataflow.dflow INFO: Task 491 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b63fe5a6bb8d23798e65c910d3076cf5 with result from task 491 parsl.dataflow.dflow DEBUG: Task 491 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/67a56488-d264-4e7c-a5aa-736b33d318b1_isrForCrosstalkSources_997011_62.stdout parsl.dataflow.dflow INFO: Standard error for task 491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/67a56488-d264-4e7c-a5aa-736b33d318b1_isrForCrosstalkSources_997011_62.stderr parsl.dataflow.dflow INFO: Task 484 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a822a58ef95096bda6357518763146b with result from task 484 parsl.dataflow.dflow DEBUG: Task 484 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/040c5088-c92c-4f97-bf35-0cb492250fcb_isrForCrosstalkSources_997011_54.stdout parsl.dataflow.dflow INFO: Standard error for task 484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/040c5088-c92c-4f97-bf35-0cb492250fcb_isrForCrosstalkSources_997011_54.stderr parsl.dataflow.dflow INFO: Task 488 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d4486059fa40ff57d74f1f11a1ef268 with result from task 488 parsl.dataflow.dflow DEBUG: Task 488 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/9dd7ee75-e024-46a8-93e3-cea188335743_isrForCrosstalkSources_997028_30.stdout parsl.dataflow.dflow INFO: Standard error for task 488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/9dd7ee75-e024-46a8-93e3-cea188335743_isrForCrosstalkSources_997028_30.stderr parsl.dataflow.dflow INFO: Task 478 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 324b8168d5cc3c61fb375a5ffd005291 with result from task 478 parsl.dataflow.dflow DEBUG: Task 478 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5867e8fc-2655-4600-b665-ba9e0afe5c3d_isrForCrosstalkSources_996999_19.stdout parsl.dataflow.dflow INFO: Standard error for task 478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5867e8fc-2655-4600-b665-ba9e0afe5c3d_isrForCrosstalkSources_996999_19.stderr parsl.dataflow.dflow INFO: Task 480 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 734385025ac7ae0ab1fcc80ffe0920c8 with result from task 480 parsl.dataflow.dflow DEBUG: Task 480 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/c9306167-ee3b-4b04-94d7-2b6f6b65b41a_isrForCrosstalkSources_997005_28.stdout parsl.dataflow.dflow INFO: Standard error for task 480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/c9306167-ee3b-4b04-94d7-2b6f6b65b41a_isrForCrosstalkSources_997005_28.stderr parsl.dataflow.dflow INFO: Task 454 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cebc0e9dd0064475d4e59c71e916a1ef with result from task 454 parsl.dataflow.dflow DEBUG: Task 454 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8e8cb852-60d7-4d30-b355-4511345021cb_isrForCrosstalkSources_996976_61.stdout parsl.dataflow.dflow INFO: Standard error for task 454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8e8cb852-60d7-4d30-b355-4511345021cb_isrForCrosstalkSources_996976_61.stderr parsl.dataflow.dflow INFO: Task 571 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a16c046599e09a3cfbbc508c3eba3955 with result from task 571 parsl.dataflow.dflow DEBUG: Task 571 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/195b6d9b-d264-4421-82bb-b3b91b49ed53_isrForCrosstalkSources_997026_53.stdout parsl.dataflow.dflow INFO: Standard error for task 571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/195b6d9b-d264-4421-82bb-b3b91b49ed53_isrForCrosstalkSources_997026_53.stderr parsl.dataflow.dflow INFO: Task 487 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73066490685a8787a6cc4f7790e909cb with result from task 487 parsl.dataflow.dflow DEBUG: Task 487 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/7d12f4d0-e8b6-4e35-a081-becdfbe59ad7_isrForCrosstalkSources_996965_47.stdout parsl.dataflow.dflow INFO: Standard error for task 487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/7d12f4d0-e8b6-4e35-a081-becdfbe59ad7_isrForCrosstalkSources_996965_47.stderr parsl.dataflow.dflow INFO: Task 483 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f19722b755bf71a3b65b9abcbbb10e04 with result from task 483 parsl.dataflow.dflow DEBUG: Task 483 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/eb0863b4-f094-440c-b37c-08fd44d26be1_isrForCrosstalkSources_996980_21.stdout parsl.dataflow.dflow INFO: Standard error for task 483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/eb0863b4-f094-440c-b37c-08fd44d26be1_isrForCrosstalkSources_996980_21.stderr parsl.dataflow.dflow INFO: Task 395 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 996296723a7e63ea33f6bcbb479d0f79 with result from task 395 parsl.dataflow.dflow DEBUG: Task 395 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/90aece87-522d-4d20-9f11-3896ffeca0c4_isrForCrosstalkSources_997008_37.stdout parsl.dataflow.dflow INFO: Standard error for task 395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/90aece87-522d-4d20-9f11-3896ffeca0c4_isrForCrosstalkSources_997008_37.stderr parsl.dataflow.dflow INFO: Task 394 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21fa5901467dd679cc0bd5302fe6381f with result from task 394 parsl.dataflow.dflow DEBUG: Task 394 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/255c352a-2b28-4045-a08c-2ad06a7707ae_isrForCrosstalkSources_996996_1.stdout parsl.dataflow.dflow INFO: Standard error for task 394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/255c352a-2b28-4045-a08c-2ad06a7707ae_isrForCrosstalkSources_996996_1.stderr parsl.dataflow.dflow INFO: Task 398 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 706550b039fe747f2e558f2dc4a85982 with result from task 398 parsl.dataflow.dflow DEBUG: Task 398 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/2ea902a6-4abf-4984-ae58-bc9dabbf3dcc_isrForCrosstalkSources_997004_3.stdout parsl.dataflow.dflow INFO: Standard error for task 398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/2ea902a6-4abf-4984-ae58-bc9dabbf3dcc_isrForCrosstalkSources_997004_3.stderr parsl.dataflow.dflow INFO: Task 397 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a84f9a84048a0df48678c0dca5ad0119 with result from task 397 parsl.dataflow.dflow DEBUG: Task 397 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/4b1bf8c7-4eac-4c1d-ad4d-d6cfe64f28f7_isrForCrosstalkSources_997002_34.stdout parsl.dataflow.dflow INFO: Standard error for task 397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/4b1bf8c7-4eac-4c1d-ad4d-d6cfe64f28f7_isrForCrosstalkSources_997002_34.stderr parsl.dataflow.dflow INFO: Task 396 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 68b4e636ca9dd1dbea89c87524f6992d with result from task 396 parsl.dataflow.dflow DEBUG: Task 396 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/12578100-1175-4848-be84-7895207b47a6_isrForCrosstalkSources_996993_7.stdout parsl.dataflow.dflow INFO: Standard error for task 396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/12578100-1175-4848-be84-7895207b47a6_isrForCrosstalkSources_996993_7.stderr parsl.dataflow.dflow INFO: Task 393 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 68c54f743b57454bc178c53185cc32c7 with result from task 393 parsl.dataflow.dflow DEBUG: Task 393 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/62075e10-8559-4741-91fc-cd380c814fa9_isrForCrosstalkSources_996953_39.stdout parsl.dataflow.dflow INFO: Standard error for task 393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/62075e10-8559-4741-91fc-cd380c814fa9_isrForCrosstalkSources_996953_39.stderr parsl.dataflow.dflow INFO: Task 611 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bceed166e2e7cb0a560f1d056f20df0a with result from task 611 parsl.dataflow.dflow DEBUG: Task 611 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/9614842d-eb11-48ec-8b4f-dbeeef5fc7b3_isrForCrosstalkSources_996967_14.stdout parsl.dataflow.dflow INFO: Standard error for task 611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/9614842d-eb11-48ec-8b4f-dbeeef5fc7b3_isrForCrosstalkSources_996967_14.stderr parsl.dataflow.dflow INFO: Task 455 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ca25523502d7b346c1cf0118b3034f3 with result from task 455 parsl.dataflow.dflow DEBUG: Task 455 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/31a1a6ef-47bf-4da2-a562-4f06aaae9e6d_isrForCrosstalkSources_996975_54.stdout parsl.dataflow.dflow INFO: Standard error for task 455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/31a1a6ef-47bf-4da2-a562-4f06aaae9e6d_isrForCrosstalkSources_996975_54.stderr parsl.dataflow.dflow INFO: Task 490 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70f3abf4e4f9e27078eafb45a6dd1679 with result from task 490 parsl.dataflow.dflow DEBUG: Task 490 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/20d6a9eb-e9e6-4c14-8bd6-ca086e8e2c03_isrForCrosstalkSources_996955_49.stdout parsl.dataflow.dflow INFO: Standard error for task 490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/20d6a9eb-e9e6-4c14-8bd6-ca086e8e2c03_isrForCrosstalkSources_996955_49.stderr parsl.dataflow.dflow INFO: Task 474 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d8b69fc2441252a5ae7a8c057f55c45 with result from task 474 parsl.dataflow.dflow DEBUG: Task 474 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/bb80437c-5930-4c0f-a10b-8ffdefba2b8c_isrForCrosstalkSources_997003_32.stdout parsl.dataflow.dflow INFO: Standard error for task 474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/bb80437c-5930-4c0f-a10b-8ffdefba2b8c_isrForCrosstalkSources_997003_32.stderr parsl.dataflow.dflow INFO: Task 469 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9399cf39f34a9fe952c0c94d94e4fb6 with result from task 469 parsl.dataflow.dflow DEBUG: Task 469 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/af5a69b8-069d-4174-aca1-916f54bb8025_isrForCrosstalkSources_996983_3.stdout parsl.dataflow.dflow INFO: Standard error for task 469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/af5a69b8-069d-4174-aca1-916f54bb8025_isrForCrosstalkSources_996983_3.stderr parsl.dataflow.dflow INFO: Task 468 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a821a67eb847f65a60d0db172e157882 with result from task 468 parsl.dataflow.dflow DEBUG: Task 468 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/92622fba-760c-4bf3-a059-99f6320aac34_isrForCrosstalkSources_996958_47.stdout parsl.dataflow.dflow INFO: Standard error for task 468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/92622fba-760c-4bf3-a059-99f6320aac34_isrForCrosstalkSources_996958_47.stderr parsl.dataflow.dflow INFO: Task 613 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f35d57039026e6c58815968f211e8138 with result from task 613 parsl.dataflow.dflow DEBUG: Task 613 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6a685536-63a1-4950-a1f0-c86493fdc49f_isrForCrosstalkSources_996971_4.stdout parsl.dataflow.dflow INFO: Standard error for task 613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6a685536-63a1-4950-a1f0-c86493fdc49f_isrForCrosstalkSources_996971_4.stderr parsl.dataflow.dflow INFO: Task 471 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a14d8e3f8534ec40f4a3fc56e30fb21 with result from task 471 parsl.dataflow.dflow DEBUG: Task 471 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a464dc41-67ab-4047-bca1-54cbe3ed09b8_isrForCrosstalkSources_996972_41.stdout parsl.dataflow.dflow INFO: Standard error for task 471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a464dc41-67ab-4047-bca1-54cbe3ed09b8_isrForCrosstalkSources_996972_41.stderr parsl.dataflow.dflow INFO: Task 507 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c5662459870561f2d08985009901212b with result from task 507 parsl.dataflow.dflow DEBUG: Task 507 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/462160e7-95a0-4a20-afdd-41f511e1bc1a_isrForCrosstalkSources_997023_61.stdout parsl.dataflow.dflow INFO: Standard error for task 507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/462160e7-95a0-4a20-afdd-41f511e1bc1a_isrForCrosstalkSources_997023_61.stderr parsl.dataflow.dflow INFO: Task 502 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 20240316b674dd835ccfe205db5754b5 with result from task 502 parsl.dataflow.dflow DEBUG: Task 502 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/662b3ce9-45fa-4333-8b9e-7d9e5da3f360_isrForCrosstalkSources_996978_47.stdout parsl.dataflow.dflow INFO: Standard error for task 502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/662b3ce9-45fa-4333-8b9e-7d9e5da3f360_isrForCrosstalkSources_996978_47.stderr parsl.dataflow.dflow INFO: Task 506 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d1cf61b7668a020369404a042c4f24b with result from task 506 parsl.dataflow.dflow DEBUG: Task 506 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/49c93bae-dc85-42af-aaf1-9fe6416d77a1_isrForCrosstalkSources_996997_62.stdout parsl.dataflow.dflow INFO: Standard error for task 506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/49c93bae-dc85-42af-aaf1-9fe6416d77a1_isrForCrosstalkSources_996997_62.stderr parsl.dataflow.dflow INFO: Task 462 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3eefa632ec9917b9b1ed19e03ae104ed with result from task 462 parsl.dataflow.dflow DEBUG: Task 462 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/aecbb434-7315-4de4-8d48-b93d7fa61c64_isrForCrosstalkSources_997012_20.stdout parsl.dataflow.dflow INFO: Standard error for task 462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/aecbb434-7315-4de4-8d48-b93d7fa61c64_isrForCrosstalkSources_997012_20.stderr parsl.dataflow.dflow INFO: Task 459 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b83a07ef4a2a9e38caee0c9b8306b190 with result from task 459 parsl.dataflow.dflow DEBUG: Task 459 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e21126aa-c55e-45f9-b528-ee494050db96_isrForCrosstalkSources_996961_1.stdout parsl.dataflow.dflow INFO: Standard error for task 459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e21126aa-c55e-45f9-b528-ee494050db96_isrForCrosstalkSources_996961_1.stderr parsl.dataflow.dflow INFO: Task 465 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d90c23785b5742bea3cc7beb0390487 with result from task 465 parsl.dataflow.dflow DEBUG: Task 465 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c2e33e09-942d-4668-8c36-5717abf75860_isrForCrosstalkSources_997003_59.stdout parsl.dataflow.dflow INFO: Standard error for task 465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c2e33e09-942d-4668-8c36-5717abf75860_isrForCrosstalkSources_997003_59.stderr parsl.dataflow.dflow INFO: Task 470 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d464d1e7d538b6b6a724f3ed9eaf03e7 with result from task 470 parsl.dataflow.dflow DEBUG: Task 470 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/53b67780-93bc-46a0-b5f7-af7c83e456e4_isrForCrosstalkSources_997002_26.stdout parsl.dataflow.dflow INFO: Standard error for task 470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/53b67780-93bc-46a0-b5f7-af7c83e456e4_isrForCrosstalkSources_997002_26.stderr parsl.dataflow.dflow INFO: Task 505 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3716f9279834d3def44496de333c3093 with result from task 505 parsl.dataflow.dflow DEBUG: Task 505 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c66d5de3-28f8-4e10-9b65-24919f0c1c74_isrForCrosstalkSources_996993_60.stdout parsl.dataflow.dflow INFO: Standard error for task 505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c66d5de3-28f8-4e10-9b65-24919f0c1c74_isrForCrosstalkSources_996993_60.stderr parsl.dataflow.dflow INFO: Task 466 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7dd001da6d4578030d91d2022f003a82 with result from task 466 parsl.dataflow.dflow DEBUG: Task 466 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/05ab7ec8-9660-4f5e-85c0-8de773f410ef_isrForCrosstalkSources_996973_56.stdout parsl.dataflow.dflow INFO: Standard error for task 466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/05ab7ec8-9660-4f5e-85c0-8de773f410ef_isrForCrosstalkSources_996973_56.stderr parsl.dataflow.dflow INFO: Task 460 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 422a6096b1a49274b80214da6a157e59 with result from task 460 parsl.dataflow.dflow DEBUG: Task 460 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/be14dd3c-cd69-464d-9fa6-0cb3a23758ff_isrForCrosstalkSources_996973_30.stdout parsl.dataflow.dflow INFO: Standard error for task 460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/be14dd3c-cd69-464d-9fa6-0cb3a23758ff_isrForCrosstalkSources_996973_30.stderr parsl.dataflow.dflow INFO: Task 473 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4f30cd2a96f3492713edbf5d2e24e04 with result from task 473 parsl.dataflow.dflow DEBUG: Task 473 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/c9c15df6-012b-44be-87b7-231f9ae7e60e_isrForCrosstalkSources_996987_10.stdout parsl.dataflow.dflow INFO: Standard error for task 473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/c9c15df6-012b-44be-87b7-231f9ae7e60e_isrForCrosstalkSources_996987_10.stderr parsl.dataflow.dflow INFO: Task 467 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96052534ddaaf17fbe8f979f790331ce with result from task 467 parsl.dataflow.dflow DEBUG: Task 467 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f1b1afc3-c011-405d-97c8-e0bdc0b508c3_isrForCrosstalkSources_996989_9.stdout parsl.dataflow.dflow INFO: Standard error for task 467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f1b1afc3-c011-405d-97c8-e0bdc0b508c3_isrForCrosstalkSources_996989_9.stderr parsl.dataflow.dflow INFO: Task 504 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29f1928794b0a0b242bc1cb0f88f41f3 with result from task 504 parsl.dataflow.dflow DEBUG: Task 504 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/3e036e08-c0dd-4e1e-99cc-c9baafda1c21_isrForCrosstalkSources_996966_5.stdout parsl.dataflow.dflow INFO: Standard error for task 504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/3e036e08-c0dd-4e1e-99cc-c9baafda1c21_isrForCrosstalkSources_996966_5.stderr parsl.dataflow.dflow INFO: Task 501 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e469d2ce1edc15cdad6103ddc67a1497 with result from task 501 parsl.dataflow.dflow DEBUG: Task 501 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ba5cf8ea-755a-4805-b1eb-cedfc31f4cb9_isrForCrosstalkSources_996959_3.stdout parsl.dataflow.dflow INFO: Standard error for task 501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ba5cf8ea-755a-4805-b1eb-cedfc31f4cb9_isrForCrosstalkSources_996959_3.stderr parsl.dataflow.dflow INFO: Task 500 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56f8d018da60f10f758c9421a358173f with result from task 500 parsl.dataflow.dflow DEBUG: Task 500 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/80ec2437-2afc-4cee-a284-433ec9b2a57b_isrForCrosstalkSources_996994_21.stdout parsl.dataflow.dflow INFO: Standard error for task 500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/80ec2437-2afc-4cee-a284-433ec9b2a57b_isrForCrosstalkSources_996994_21.stderr parsl.dataflow.dflow INFO: Task 503 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6bfb97465ae568bb259395dc952b4367 with result from task 503 parsl.dataflow.dflow DEBUG: Task 503 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/77eca3c1-8ace-46b8-baca-038255a25741_isrForCrosstalkSources_996979_60.stdout parsl.dataflow.dflow INFO: Standard error for task 503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/77eca3c1-8ace-46b8-baca-038255a25741_isrForCrosstalkSources_996979_60.stderr parsl.dataflow.dflow INFO: Task 463 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 838789c700b7eab6c3cd6bef63ace59a with result from task 463 parsl.dataflow.dflow DEBUG: Task 463 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/bb1c328a-fa0c-4335-9070-4e13408671e5_isrForCrosstalkSources_997019_28.stdout parsl.dataflow.dflow INFO: Standard error for task 463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/bb1c328a-fa0c-4335-9070-4e13408671e5_isrForCrosstalkSources_997019_28.stderr parsl.dataflow.dflow INFO: Task 461 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4d7118453d804a0a3409c418976d65f3 with result from task 461 parsl.dataflow.dflow DEBUG: Task 461 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f64e152e-3531-4463-b776-12622db2e852_isrForCrosstalkSources_997026_14.stdout parsl.dataflow.dflow INFO: Standard error for task 461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f64e152e-3531-4463-b776-12622db2e852_isrForCrosstalkSources_997026_14.stderr parsl.dataflow.dflow INFO: Task 472 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e2526c1445bc2be1e3ebfc342afd254 with result from task 472 parsl.dataflow.dflow DEBUG: Task 472 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7bda589e-a469-499b-8439-e2ce651f3f2d_isrForCrosstalkSources_997013_7.stdout parsl.dataflow.dflow INFO: Standard error for task 472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7bda589e-a469-499b-8439-e2ce651f3f2d_isrForCrosstalkSources_997013_7.stderr parsl.dataflow.dflow INFO: Task 406 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17446a552ac61f565b7a06db30effa12 with result from task 406 parsl.dataflow.dflow DEBUG: Task 406 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/49723508-8c4e-43c7-a9a7-5af1071bebda_isrForCrosstalkSources_996966_50.stdout parsl.dataflow.dflow INFO: Standard error for task 406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/49723508-8c4e-43c7-a9a7-5af1071bebda_isrForCrosstalkSources_996966_50.stderr parsl.dataflow.dflow INFO: Task 400 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf57cfabd435d383160b2a7b0205de28 with result from task 400 parsl.dataflow.dflow DEBUG: Task 400 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e11427b0-ff32-4ac2-983a-8a99def333c2_isrForCrosstalkSources_997016_47.stdout parsl.dataflow.dflow INFO: Standard error for task 400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e11427b0-ff32-4ac2-983a-8a99def333c2_isrForCrosstalkSources_997016_47.stderr parsl.dataflow.dflow INFO: Task 401 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4995552f850c0d76350f5e86ed851a11 with result from task 401 2024-03-15 13:52:39 proc_lsst.multi:170 [INFO] [multi] job_id 17075298 proc_lsst.multi INFO: [multi] job_id 17075298 2024-03-15 13:52:39 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 189 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 189 2024-03-15 13:52:39 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:39 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075298 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075298 parsl.executors.status_handling DEBUG: Launched block 244 on executor multi with job ID 17075298 parsl.executors.status_handling INFO: Allocated block ID 245 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-245 2024-03-15 13:52:39 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-245 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-245 2024-03-15 13:52:39 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:39 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:39 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:39 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:39 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:39 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 189 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 189 2024-03-15 13:52:39 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:39 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=245 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 401 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/ca5d2855-207d-44e6-ac7e-13f39a79e2b1_isrForCrosstalkSources_996993_8.stdout parsl.dataflow.dflow INFO: Standard error for task 401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/ca5d2855-207d-44e6-ac7e-13f39a79e2b1_isrForCrosstalkSources_996993_8.stderr parsl.dataflow.dflow INFO: Task 402 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd4d836737fc191adf94f59e9511dc7b with result from task 402 parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Task 402 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/19122548-755b-47f0-b2e3-f6beb6dbca90_isrForCrosstalkSources_996971_48.stdout parsl.dataflow.dflow INFO: Standard error for task 402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/19122548-755b-47f0-b2e3-f6beb6dbca90_isrForCrosstalkSources_996971_48.stderr parsl.dataflow.dflow INFO: Task 405 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d59679ee2c532bfa60a03c8a4db5cfef with result from task 405 parsl.dataflow.dflow DEBUG: Task 405 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5db3e851-b800-49e5-b65a-564c3bb83ae3_isrForCrosstalkSources_996988_44.stdout parsl.dataflow.dflow INFO: Standard error for task 405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5db3e851-b800-49e5-b65a-564c3bb83ae3_isrForCrosstalkSources_996988_44.stderr parsl.dataflow.dflow INFO: Task 403 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 876c35efbf5caff5e28e6a21b343bf57 with result from task 403 parsl.dataflow.dflow DEBUG: Task 403 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/eee1f0d9-c023-4992-8a2d-30df552cbd56_isrForCrosstalkSources_996969_20.stdout parsl.dataflow.dflow INFO: Standard error for task 403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/eee1f0d9-c023-4992-8a2d-30df552cbd56_isrForCrosstalkSources_996969_20.stderr parsl.dataflow.dflow INFO: Task 421 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dc195c1b957e04eaee931d337ec1f315 with result from task 421 parsl.dataflow.dflow DEBUG: Task 421 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f28b7acf-b031-41ad-8313-597d9435fd0b_isrForCrosstalkSources_997015_17.stdout parsl.dataflow.dflow INFO: Standard error for task 421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f28b7acf-b031-41ad-8313-597d9435fd0b_isrForCrosstalkSources_997015_17.stderr parsl.dataflow.dflow INFO: Task 420 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab8fcf4f2a0e108fd8d2323812f622d6 with result from task 420 parsl.dataflow.dflow DEBUG: Task 420 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6b1b3eac-ac77-40f3-886a-a1a1d0b982f2_isrForCrosstalkSources_996954_5.stdout parsl.dataflow.dflow INFO: Standard error for task 420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6b1b3eac-ac77-40f3-886a-a1a1d0b982f2_isrForCrosstalkSources_996954_5.stderr parsl.dataflow.dflow INFO: Task 423 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 916eb831cb60f1c9cf4193c7363ed1f5 with result from task 423 parsl.dataflow.dflow DEBUG: Task 423 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/f7e49f93-89d9-44f8-bf82-9c817fa262b8_isrForCrosstalkSources_996970_48.stdout parsl.dataflow.dflow INFO: Standard error for task 423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/f7e49f93-89d9-44f8-bf82-9c817fa262b8_isrForCrosstalkSources_996970_48.stderr parsl.dataflow.dflow INFO: Task 404 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb4fab36a36e35bd8c818f40d8093cbc with result from task 404 parsl.dataflow.dflow DEBUG: Task 404 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/79e78b6c-4c09-4813-a3f7-13d2ef5f496b_isrForCrosstalkSources_996982_28.stdout parsl.dataflow.dflow INFO: Standard error for task 404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/79e78b6c-4c09-4813-a3f7-13d2ef5f496b_isrForCrosstalkSources_996982_28.stderr parsl.dataflow.dflow INFO: Task 399 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0869e8ba57a975788c3b3914c94c9933 with result from task 399 parsl.dataflow.dflow DEBUG: Task 399 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/aeaa9001-549c-4e30-907f-0c55d5929858_isrForCrosstalkSources_997031_31.stdout parsl.dataflow.dflow INFO: Standard error for task 399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/aeaa9001-549c-4e30-907f-0c55d5929858_isrForCrosstalkSources_997031_31.stderr parsl.dataflow.dflow INFO: Task 419 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fe5be9cfebcc883486f940e05597a4c with result from task 419 parsl.dataflow.dflow DEBUG: Task 419 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6568113f-62e5-4f62-b1dd-cd50e81c8f76_isrForCrosstalkSources_996971_36.stdout parsl.dataflow.dflow INFO: Standard error for task 419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6568113f-62e5-4f62-b1dd-cd50e81c8f76_isrForCrosstalkSources_996971_36.stderr parsl.dataflow.dflow INFO: Task 416 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1eec9fa68dd9d8f6a9f73f3506cb0b29 with result from task 416 parsl.dataflow.dflow DEBUG: Task 416 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/67bdda8a-a3b1-4bb0-8518-928c201c2cf5_isrForCrosstalkSources_997019_7.stdout parsl.dataflow.dflow INFO: Standard error for task 416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/67bdda8a-a3b1-4bb0-8518-928c201c2cf5_isrForCrosstalkSources_997019_7.stderr parsl.dataflow.dflow INFO: Task 418 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d73c458e558c2c8022bba37572dbba75 with result from task 418 parsl.dataflow.dflow DEBUG: Task 418 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/f293f229-929b-4450-9236-c993e74267e3_isrForCrosstalkSources_996970_1.stdout parsl.dataflow.dflow INFO: Standard error for task 418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/f293f229-929b-4450-9236-c993e74267e3_isrForCrosstalkSources_996970_1.stderr parsl.dataflow.dflow INFO: Task 422 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39eaccfa068a63ed0e39d694b3b50799 with result from task 422 parsl.dataflow.dflow DEBUG: Task 422 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1ef549f7-cc1a-46d6-a08e-6519b49d53fc_isrForCrosstalkSources_996953_58.stdout parsl.dataflow.dflow INFO: Standard error for task 422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1ef549f7-cc1a-46d6-a08e-6519b49d53fc_isrForCrosstalkSources_996953_58.stderr parsl.dataflow.dflow INFO: Task 417 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d37d10c5103c2df9d5b3b6eddfe61be7 with result from task 417 parsl.dataflow.dflow DEBUG: Task 417 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/be2ee2f1-8d87-409f-b6d4-24f4fd890576_isrForCrosstalkSources_996962_54.stdout parsl.dataflow.dflow INFO: Standard error for task 417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/be2ee2f1-8d87-409f-b6d4-24f4fd890576_isrForCrosstalkSources_996962_54.stderr parsl.dataflow.dflow INFO: Task 614 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a4ba17fd0559e0f98aaa805443be554 with result from task 614 parsl.dataflow.dflow DEBUG: Task 614 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/aafe6966-14a9-4bf2-88d7-c326a36a2f88_isrForCrosstalkSources_997009_8.stdout parsl.dataflow.dflow INFO: Standard error for task 614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/aafe6966-14a9-4bf2-88d7-c326a36a2f88_isrForCrosstalkSources_997009_8.stderr parsl.dataflow.dflow INFO: Task 464 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 06fef5004bbce98db45cc9226144a6d5 with result from task 464 parsl.dataflow.dflow DEBUG: Task 464 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/cbb77257-88c4-4740-b60f-3e59201d5191_isrForCrosstalkSources_996982_49.stdout parsl.dataflow.dflow INFO: Standard error for task 464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/cbb77257-88c4-4740-b60f-3e59201d5191_isrForCrosstalkSources_996982_49.stderr parsl.dataflow.dflow INFO: Task 617 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9906211041894db959757489654ec683 with result from task 617 parsl.dataflow.dflow DEBUG: Task 617 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/26cdff44-2d85-46c4-b115-d788e3d4d93a_isrForCrosstalkSources_996990_61.stdout parsl.dataflow.dflow INFO: Standard error for task 617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/26cdff44-2d85-46c4-b115-d788e3d4d93a_isrForCrosstalkSources_996990_61.stderr parsl.dataflow.dflow INFO: Task 552 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a727b1a3e96a9bf504b88e80ecb88ee with result from task 552 parsl.dataflow.dflow DEBUG: Task 552 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a65c2fa8-8ec6-43ff-8b09-8f79991fedec_isrForCrosstalkSources_997002_2.stdout parsl.dataflow.dflow INFO: Standard error for task 552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a65c2fa8-8ec6-43ff-8b09-8f79991fedec_isrForCrosstalkSources_997002_2.stderr parsl.dataflow.dflow INFO: Task 564 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81daa58bd08baac5409db71625cc9c71 with result from task 564 parsl.dataflow.dflow DEBUG: Task 564 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/7b4994e1-a2ff-4dfd-adf1-f979159e9e29_isrForCrosstalkSources_997018_16.stdout parsl.dataflow.dflow INFO: Standard error for task 564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/7b4994e1-a2ff-4dfd-adf1-f979159e9e29_isrForCrosstalkSources_997018_16.stderr parsl.dataflow.dflow INFO: Task 581 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ca26f5e9f706871d2abe995ee19242e with result from task 581 parsl.dataflow.dflow DEBUG: Task 581 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f735536d-e876-4960-af6e-8b5eb448c36b_isrForCrosstalkSources_996961_3.stdout parsl.dataflow.dflow INFO: Standard error for task 581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f735536d-e876-4960-af6e-8b5eb448c36b_isrForCrosstalkSources_996961_3.stderr parsl.dataflow.dflow INFO: Task 569 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04e5c98defd8788fb3429c4461511f45 with result from task 569 parsl.dataflow.dflow DEBUG: Task 569 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/853627a3-3d21-4cc6-a688-3bfd19ada7f9_isrForCrosstalkSources_996989_5.stdout parsl.dataflow.dflow INFO: Standard error for task 569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/853627a3-3d21-4cc6-a688-3bfd19ada7f9_isrForCrosstalkSources_996989_5.stderr parsl.dataflow.dflow INFO: Task 575 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 475239db976f6496db82d11417350270 with result from task 575 parsl.dataflow.dflow DEBUG: Task 575 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/355bb696-9c28-4160-bd86-1eb2fd2f036b_isrForCrosstalkSources_996984_31.stdout parsl.dataflow.dflow INFO: Standard error for task 575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/355bb696-9c28-4160-bd86-1eb2fd2f036b_isrForCrosstalkSources_996984_31.stderr parsl.dataflow.dflow INFO: Task 577 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62e72055d7ef8d41fd6c5dd8a119cedd with result from task 577 parsl.dataflow.dflow DEBUG: Task 577 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b9b6d412-ad60-4a35-8501-006f9f013ca6_isrForCrosstalkSources_997015_57.stdout parsl.dataflow.dflow INFO: Standard error for task 577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b9b6d412-ad60-4a35-8501-006f9f013ca6_isrForCrosstalkSources_997015_57.stderr parsl.dataflow.dflow INFO: Task 562 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b39ea0adee7a8f5f81dcbff1ac33365b with result from task 562 parsl.dataflow.dflow DEBUG: Task 562 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/4241c2a3-8620-4e93-963c-627ce8a49987_isrForCrosstalkSources_997007_17.stdout parsl.dataflow.dflow INFO: Standard error for task 562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/4241c2a3-8620-4e93-963c-627ce8a49987_isrForCrosstalkSources_997007_17.stderr parsl.dataflow.dflow INFO: Task 559 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8095446c3af33ef588f31d8d1bcd4ffc with result from task 559 parsl.dataflow.dflow DEBUG: Task 559 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/91cff648-37fc-424f-9940-97e44c107b86_isrForCrosstalkSources_997020_3.stdout parsl.dataflow.dflow INFO: Standard error for task 559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/91cff648-37fc-424f-9940-97e44c107b86_isrForCrosstalkSources_997020_3.stderr parsl.dataflow.dflow INFO: Task 607 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fdc95a3b7893cc57a55366e29bae2c2 with result from task 607 parsl.dataflow.dflow DEBUG: Task 607 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/de248182-8559-460d-aa1e-d3a8bbff5e82_isrForCrosstalkSources_997032_17.stdout parsl.dataflow.dflow INFO: Standard error for task 607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/de248182-8559-460d-aa1e-d3a8bbff5e82_isrForCrosstalkSources_997032_17.stderr parsl.dataflow.dflow INFO: Task 391 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17d69e65795b972c0161df0e15d7f409 with result from task 391 parsl.dataflow.dflow DEBUG: Task 391 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/915567b0-e432-4cf8-8a17-ea81bee3a1dd_isrForCrosstalkSources_996973_61.stdout parsl.dataflow.dflow INFO: Standard error for task 391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/915567b0-e432-4cf8-8a17-ea81bee3a1dd_isrForCrosstalkSources_996973_61.stderr 2024-03-15 13:52:41 proc_lsst.multi:170 [INFO] [multi] job_id 17075299 proc_lsst.multi INFO: [multi] job_id 17075299 2024-03-15 13:52:41 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 190 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 190 2024-03-15 13:52:41 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:41 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075299 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075299 parsl.executors.status_handling DEBUG: Launched block 245 on executor multi with job ID 17075299 parsl.executors.status_handling INFO: Allocated block ID 246 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-246 2024-03-15 13:52:41 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-246 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-246 2024-03-15 13:52:41 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:41 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:41 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:41 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:41 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:41 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 190 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 190 2024-03-15 13:52:41 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:41 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=246 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 612 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d55b612f5e127fae0378784287db20fb with result from task 612 parsl.dataflow.dflow DEBUG: Task 612 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8bc66dec-96f2-43a0-a246-f9cad3777214_isrForCrosstalkSources_996952_45.stdout parsl.dataflow.dflow INFO: Standard error for task 612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8bc66dec-96f2-43a0-a246-f9cad3777214_isrForCrosstalkSources_996952_45.stderr parsl.dataflow.dflow INFO: Task 431 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85da958fa0d213aed1e8aa83f24bdbda with result from task 431 parsl.dataflow.dflow DEBUG: Task 431 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1dc7fb8-9eaa-4904-a448-36265344c293_isrForCrosstalkSources_996975_41.stdout parsl.dataflow.dflow INFO: Standard error for task 431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1dc7fb8-9eaa-4904-a448-36265344c293_isrForCrosstalkSources_996975_41.stderr parsl.dataflow.dflow INFO: Task 409 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fac137baf39836c98bd5c8553b383ab with result from task 409 parsl.dataflow.dflow DEBUG: Task 409 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/48fd3b63-6e94-4aa6-a2e8-c16173953525_isrForCrosstalkSources_996994_40.stdout parsl.dataflow.dflow INFO: Standard error for task 409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/48fd3b63-6e94-4aa6-a2e8-c16173953525_isrForCrosstalkSources_996994_40.stderr parsl.dataflow.dflow INFO: Task 426 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04c8a741637c858bec899805f7e3056f with result from task 426 parsl.dataflow.dflow DEBUG: Task 426 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bcc2cac7-583d-4586-b178-b7de50485e66_isrForCrosstalkSources_997030_14.stdout parsl.dataflow.dflow INFO: Standard error for task 426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bcc2cac7-583d-4586-b178-b7de50485e66_isrForCrosstalkSources_997030_14.stderr parsl.dataflow.dflow INFO: Task 429 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d773a43be4caece6d493f418842a02f0 with result from task 429 parsl.dataflow.dflow DEBUG: Task 429 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1821a837-9fa4-448d-abb3-b411c8e3e669_isrForCrosstalkSources_997000_50.stdout parsl.dataflow.dflow INFO: Standard error for task 429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1821a837-9fa4-448d-abb3-b411c8e3e669_isrForCrosstalkSources_997000_50.stderr parsl.dataflow.dflow INFO: Task 425 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2993cc9a88801b42486a1aea5ce6ed40 with result from task 425 parsl.dataflow.dflow DEBUG: Task 425 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/04732ca5-fc1c-4efb-b596-092f425c12e4_isrForCrosstalkSources_996984_1.stdout parsl.dataflow.dflow INFO: Standard error for task 425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/04732ca5-fc1c-4efb-b596-092f425c12e4_isrForCrosstalkSources_996984_1.stderr parsl.dataflow.dflow INFO: Task 424 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd513f36f3b46f5aa65cd268589e2774 with result from task 424 parsl.dataflow.dflow DEBUG: Task 424 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ae61f202-06d7-42d5-a338-1c3ee05c73ea_isrForCrosstalkSources_997009_7.stdout parsl.dataflow.dflow INFO: Standard error for task 424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ae61f202-06d7-42d5-a338-1c3ee05c73ea_isrForCrosstalkSources_997009_7.stderr parsl.dataflow.dflow INFO: Task 413 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry afa18dcf0ab7b7620d78ea003ae198bc with result from task 413 parsl.dataflow.dflow DEBUG: Task 413 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/332e8064-b22f-45af-b977-0da0c58e5d04_isrForCrosstalkSources_997027_47.stdout parsl.dataflow.dflow INFO: Standard error for task 413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/332e8064-b22f-45af-b977-0da0c58e5d04_isrForCrosstalkSources_997027_47.stderr parsl.dataflow.dflow INFO: Task 410 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad3fc185422c2d977995e9ebc9a8fa56 with result from task 410 parsl.dataflow.dflow DEBUG: Task 410 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/39f24c1d-b87c-4452-87b3-f9add6ede7cd_isrForCrosstalkSources_996983_37.stdout parsl.dataflow.dflow INFO: Standard error for task 410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/39f24c1d-b87c-4452-87b3-f9add6ede7cd_isrForCrosstalkSources_996983_37.stderr parsl.dataflow.dflow INFO: Task 430 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 874b9a5e772c9ce7bb3ec4a09dbe58ca with result from task 430 parsl.dataflow.dflow DEBUG: Task 430 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d5d767b4-2c8c-4d64-afeb-2abe5be11f86_isrForCrosstalkSources_996965_16.stdout parsl.dataflow.dflow INFO: Standard error for task 430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d5d767b4-2c8c-4d64-afeb-2abe5be11f86_isrForCrosstalkSources_996965_16.stderr parsl.dataflow.dflow INFO: Task 427 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bde7db16c69d5e7e2f6a3e17315a6bb5 with result from task 427 parsl.dataflow.dflow DEBUG: Task 427 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c132cd6b-e85c-492b-8cc6-a04eba5b4140_isrForCrosstalkSources_997033_36.stdout parsl.dataflow.dflow INFO: Standard error for task 427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c132cd6b-e85c-492b-8cc6-a04eba5b4140_isrForCrosstalkSources_997033_36.stderr parsl.dataflow.dflow INFO: Task 428 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89daa0b95ae442cf09a8b2c7c1a58e32 with result from task 428 parsl.dataflow.dflow DEBUG: Task 428 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/36897564-02c4-4abd-b29f-f06b530d34db_isrForCrosstalkSources_996952_41.stdout parsl.dataflow.dflow INFO: Standard error for task 428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/36897564-02c4-4abd-b29f-f06b530d34db_isrForCrosstalkSources_996952_41.stderr parsl.dataflow.dflow INFO: Task 412 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93e2dc1a9c910a0cb5507b44f6c5bc28 with result from task 412 parsl.dataflow.dflow DEBUG: Task 412 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0e1523cf-e13c-4c50-bd69-a3f3261227f4_isrForCrosstalkSources_996982_37.stdout parsl.dataflow.dflow INFO: Standard error for task 412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0e1523cf-e13c-4c50-bd69-a3f3261227f4_isrForCrosstalkSources_996982_37.stderr parsl.dataflow.dflow INFO: Task 408 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b5f772317f42fd0b31949ed57a46ab0 with result from task 408 parsl.dataflow.dflow DEBUG: Task 408 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/49cb4143-7ce9-41d3-b006-ad48067cfb94_isrForCrosstalkSources_996982_2.stdout parsl.dataflow.dflow INFO: Standard error for task 408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/49cb4143-7ce9-41d3-b006-ad48067cfb94_isrForCrosstalkSources_996982_2.stderr parsl.dataflow.dflow INFO: Task 415 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fcaa08f6276e1b719c35707f6928b39 with result from task 415 parsl.dataflow.dflow DEBUG: Task 415 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/90a52a52-facf-41a9-8559-13b679cdfd22_isrForCrosstalkSources_996982_9.stdout parsl.dataflow.dflow INFO: Standard error for task 415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/90a52a52-facf-41a9-8559-13b679cdfd22_isrForCrosstalkSources_996982_9.stderr parsl.dataflow.dflow INFO: Task 411 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 953132cf488e1093bd06fc35b8b06c4a with result from task 411 parsl.dataflow.dflow DEBUG: Task 411 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/314d9662-5014-4a4e-8368-5571bb4b37fa_isrForCrosstalkSources_997019_60.stdout parsl.dataflow.dflow INFO: Standard error for task 411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/314d9662-5014-4a4e-8368-5571bb4b37fa_isrForCrosstalkSources_997019_60.stderr parsl.dataflow.dflow INFO: Task 560 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d72044236427c5cc4ff95648635f5d95 with result from task 560 parsl.dataflow.dflow DEBUG: Task 560 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/3070675a-7c38-4fd7-bdd0-aff4ab9a8f60_isrForCrosstalkSources_997017_54.stdout parsl.dataflow.dflow INFO: Standard error for task 560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/3070675a-7c38-4fd7-bdd0-aff4ab9a8f60_isrForCrosstalkSources_997017_54.stderr parsl.dataflow.dflow INFO: Task 548 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ebfaeb2ad0da464b25ee32eae85ba90 with result from task 548 parsl.dataflow.dflow DEBUG: Task 548 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/282313ed-b4f0-4a12-b195-ae7c9d34bd61_isrForCrosstalkSources_996954_61.stdout parsl.dataflow.dflow INFO: Standard error for task 548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/282313ed-b4f0-4a12-b195-ae7c9d34bd61_isrForCrosstalkSources_996954_61.stderr parsl.dataflow.dflow INFO: Task 546 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry adbb97bd346ff9a4d725ebc764304a86 with result from task 546 parsl.dataflow.dflow DEBUG: Task 546 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/2704a84f-08e3-4724-b0ae-a16c7a4d54cf_isrForCrosstalkSources_997023_29.stdout parsl.dataflow.dflow INFO: Standard error for task 546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/2704a84f-08e3-4724-b0ae-a16c7a4d54cf_isrForCrosstalkSources_997023_29.stderr parsl.dataflow.dflow INFO: Task 436 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7493e9666506f7940b2aea3c155ea1c4 with result from task 436 parsl.dataflow.dflow DEBUG: Task 436 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/5ce7879d-ac23-45de-a9fa-54051b095bfe_isrForCrosstalkSources_996974_44.stdout parsl.dataflow.dflow INFO: Standard error for task 436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/5ce7879d-ac23-45de-a9fa-54051b095bfe_isrForCrosstalkSources_996974_44.stderr 2024-03-15 13:52:42 proc_lsst.multi:170 [INFO] [multi] job_id 17075301 proc_lsst.multi INFO: [multi] job_id 17075301 2024-03-15 13:52:42 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 191 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 191 2024-03-15 13:52:42 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:42 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075301 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075301 parsl.executors.status_handling DEBUG: Launched block 246 on executor multi with job ID 17075301 parsl.executors.status_handling INFO: Allocated block ID 247 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-247 2024-03-15 13:52:42 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-247 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-247 2024-03-15 13:52:42 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:42 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:42 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:42 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:42 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:42 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 191 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 191 2024-03-15 13:52:42 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:42 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=247 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 626 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f116f8cff250542e9ef5ce09dd112da2 with result from task 626 parsl.dataflow.dflow DEBUG: Task 626 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/183483ab-232a-4776-ac13-6f58774453d9_isrForCrosstalkSources_996968_40.stdout parsl.dataflow.dflow INFO: Standard error for task 626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/183483ab-232a-4776-ac13-6f58774453d9_isrForCrosstalkSources_996968_40.stderr parsl.dataflow.dflow INFO: Task 565 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c937a785ce07f09dfcce459375727118 with result from task 565 parsl.dataflow.dflow DEBUG: Task 565 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/81ee3942-0235-4656-891b-c3fbb18f5b68_isrForCrosstalkSources_996994_30.stdout parsl.dataflow.dflow INFO: Standard error for task 565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/81ee3942-0235-4656-891b-c3fbb18f5b68_isrForCrosstalkSources_996994_30.stderr parsl.dataflow.dflow INFO: Task 610 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d15fe4e00dacafef5ded14e82243dc98 with result from task 610 parsl.dataflow.dflow DEBUG: Task 610 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/3db517de-4d1d-4260-ae5e-e85c13e87c27_isrForCrosstalkSources_997026_11.stdout parsl.dataflow.dflow INFO: Standard error for task 610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/3db517de-4d1d-4260-ae5e-e85c13e87c27_isrForCrosstalkSources_997026_11.stderr parsl.dataflow.dflow INFO: Task 414 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 968228c4293965b1ed1d300f4291aa77 with result from task 414 parsl.dataflow.dflow DEBUG: Task 414 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e745762c-840b-4549-a325-9645bfaa6008_isrForCrosstalkSources_996998_54.stdout parsl.dataflow.dflow INFO: Standard error for task 414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e745762c-840b-4549-a325-9645bfaa6008_isrForCrosstalkSources_996998_54.stderr parsl.dataflow.dflow INFO: Task 558 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 680881e13b18637044a07b3be826bb89 with result from task 558 parsl.dataflow.dflow DEBUG: Task 558 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/4821582d-f328-49d5-9ca7-925ebb9d3f68_isrForCrosstalkSources_997012_36.stdout parsl.dataflow.dflow INFO: Standard error for task 558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/4821582d-f328-49d5-9ca7-925ebb9d3f68_isrForCrosstalkSources_997012_36.stderr parsl.dataflow.dflow INFO: Task 608 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2a2819be5bea3314b816e8cc9cb9d8d8 with result from task 608 parsl.dataflow.dflow DEBUG: Task 608 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c97cebe8-ae08-4309-b6c7-b63a545af6a6_isrForCrosstalkSources_996965_55.stdout parsl.dataflow.dflow INFO: Standard error for task 608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c97cebe8-ae08-4309-b6c7-b63a545af6a6_isrForCrosstalkSources_996965_55.stderr parsl.dataflow.dflow INFO: Task 609 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6252e36bf5b68102384068cc1572c42b with result from task 609 parsl.dataflow.dflow DEBUG: Task 609 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/92a78bbb-1057-4a22-816b-c6916ae03e01_isrForCrosstalkSources_996973_62.stdout parsl.dataflow.dflow INFO: Standard error for task 609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/92a78bbb-1057-4a22-816b-c6916ae03e01_isrForCrosstalkSources_996973_62.stderr 2024-03-15 13:52:43 proc_lsst.multi:170 [INFO] [multi] job_id 17075306 proc_lsst.multi INFO: [multi] job_id 17075306 2024-03-15 13:52:43 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 192 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 192 2024-03-15 13:52:43 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:43 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075306 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075306 parsl.executors.status_handling DEBUG: Launched block 247 on executor multi with job ID 17075306 parsl.executors.status_handling INFO: Allocated block ID 248 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-248 2024-03-15 13:52:43 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-248 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-248 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 192 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 192 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:43 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=248 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 563 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dfada8d95b91d8337c863e68319ca7e2 with result from task 563 parsl.dataflow.dflow DEBUG: Task 563 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/92166d78-b5ad-4fd0-bfcb-17a4c7ac0f8b_isrForCrosstalkSources_996986_42.stdout parsl.dataflow.dflow INFO: Standard error for task 563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/92166d78-b5ad-4fd0-bfcb-17a4c7ac0f8b_isrForCrosstalkSources_996986_42.stderr parsl.dataflow.dflow INFO: Task 631 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 239f89c604ffc68ef8f24236e4e37635 with result from task 631 parsl.dataflow.dflow DEBUG: Task 631 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/16257e6e-f69b-4d1b-af32-4cc5351926ca_isrForCrosstalkSources_996987_51.stdout parsl.dataflow.dflow INFO: Standard error for task 631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/16257e6e-f69b-4d1b-af32-4cc5351926ca_isrForCrosstalkSources_996987_51.stderr parsl.dataflow.dflow INFO: Task 627 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5256f40c993bab4da6a23ad6f5ba1d10 with result from task 627 parsl.dataflow.dflow DEBUG: Task 627 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/13c4f091-244f-4856-9eda-07bee3914cdf_isrForCrosstalkSources_997020_42.stdout parsl.dataflow.dflow INFO: Standard error for task 627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/13c4f091-244f-4856-9eda-07bee3914cdf_isrForCrosstalkSources_997020_42.stderr parsl.dataflow.dflow INFO: Task 433 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a02f0920ad2ab254a4758073d24c29f with result from task 433 parsl.dataflow.dflow DEBUG: Task 433 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/eb2f8e1c-2fca-4ce9-b8a5-71c22c252368_isrForCrosstalkSources_997002_62.stdout parsl.dataflow.dflow INFO: Standard error for task 433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/eb2f8e1c-2fca-4ce9-b8a5-71c22c252368_isrForCrosstalkSources_997002_62.stderr 2024-03-15 13:52:43 proc_lsst.multi:170 [INFO] [multi] job_id 17075307 proc_lsst.multi INFO: [multi] job_id 17075307 2024-03-15 13:52:43 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 193 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 193 2024-03-15 13:52:43 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:43 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075307 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075307 parsl.executors.status_handling DEBUG: Launched block 248 on executor multi with job ID 17075307 parsl.executors.status_handling INFO: Allocated block ID 249 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-249 2024-03-15 13:52:43 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-249 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-249 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:43 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:43 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 193 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 193 2024-03-15 13:52:43 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:43 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=249 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 550 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89cc250c351e3f5e0ea77a0d702fba88 with result from task 550 parsl.dataflow.dflow DEBUG: Task 550 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/30291cbf-6b74-4c61-a978-a3857fb12aef_isrForCrosstalkSources_997028_43.stdout parsl.dataflow.dflow INFO: Standard error for task 550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/30291cbf-6b74-4c61-a978-a3857fb12aef_isrForCrosstalkSources_997028_43.stderr parsl.dataflow.dflow INFO: Task 544 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd9c0bd9e0f79ea79c003b1d4abbdd09 with result from task 544 parsl.dataflow.dflow DEBUG: Task 544 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/1abdd64a-6055-4838-8924-b7b944014c30_isrForCrosstalkSources_996955_48.stdout parsl.dataflow.dflow INFO: Standard error for task 544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/1abdd64a-6055-4838-8924-b7b944014c30_isrForCrosstalkSources_996955_48.stderr parsl.dataflow.dflow INFO: Task 561 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f2e3c11fafc27a0b73e399097ca17c4 with result from task 561 parsl.dataflow.dflow DEBUG: Task 561 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/0f9d5b2c-b222-4d1d-9594-94b03b33a53b_isrForCrosstalkSources_996963_61.stdout parsl.dataflow.dflow INFO: Standard error for task 561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/0f9d5b2c-b222-4d1d-9594-94b03b33a53b_isrForCrosstalkSources_996963_61.stderr parsl.dataflow.dflow INFO: Task 443 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 61f223e1fff7c98a9990b7b07aac9363 with result from task 443 2024-03-15 13:52:44 proc_lsst.multi:170 [INFO] [multi] job_id 17075308 proc_lsst.multi INFO: [multi] job_id 17075308 2024-03-15 13:52:44 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 194 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 194 2024-03-15 13:52:44 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:44 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075308 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075308 parsl.executors.status_handling DEBUG: Launched block 249 on executor multi with job ID 17075308 parsl.executors.status_handling INFO: Allocated block ID 250 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-250 2024-03-15 13:52:44 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-250 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-250 2024-03-15 13:52:44 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:44 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:44 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:44 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:44 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:44 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 194 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 194 2024-03-15 13:52:44 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:44 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=250 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 443 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ff8baf52-48e6-4cae-9e7a-827a773ca579_isrForCrosstalkSources_996978_62.stdout parsl.dataflow.dflow INFO: Standard error for task 443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ff8baf52-48e6-4cae-9e7a-827a773ca579_isrForCrosstalkSources_996978_62.stderr parsl.dataflow.dflow INFO: Task 588 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 36a22970544b08bb4939e85a0a905b29 with result from task 588 parsl.dataflow.dflow DEBUG: Task 588 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/64138df4-d433-45b3-9c28-6da31e4ee11b_isrForCrosstalkSources_996970_30.stdout parsl.dataflow.dflow INFO: Standard error for task 588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/64138df4-d433-45b3-9c28-6da31e4ee11b_isrForCrosstalkSources_996970_30.stderr parsl.dataflow.dflow INFO: Task 618 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e29b406136fe8b8e682763890f543f7 with result from task 618 parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Task 618 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4f64564f-a322-4095-ae4f-63c2975d0ab0_isrForCrosstalkSources_997005_60.stdout parsl.dataflow.dflow INFO: Standard error for task 618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4f64564f-a322-4095-ae4f-63c2975d0ab0_isrForCrosstalkSources_997005_60.stderr parsl.dataflow.dflow INFO: Task 587 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3a9a15e9ecbb4530a3ca82f4030bc73 with result from task 587 parsl.dataflow.dflow DEBUG: Task 587 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/287f4451-1425-48c6-8c4d-31a598ea488e_isrForCrosstalkSources_997005_44.stdout parsl.dataflow.dflow INFO: Standard error for task 587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/287f4451-1425-48c6-8c4d-31a598ea488e_isrForCrosstalkSources_997005_44.stderr parsl.dataflow.dflow INFO: Task 586 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9112904ffa1bac6458e7b243bbaf06d7 with result from task 586 parsl.dataflow.dflow DEBUG: Task 586 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0ea418c6-a986-47ef-bf26-b5274a164f48_isrForCrosstalkSources_997021_43.stdout parsl.dataflow.dflow INFO: Standard error for task 586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0ea418c6-a986-47ef-bf26-b5274a164f48_isrForCrosstalkSources_997021_43.stderr 2024-03-15 13:52:45 proc_lsst.multi:170 [INFO] [multi] job_id 17075309 proc_lsst.multi INFO: [multi] job_id 17075309 2024-03-15 13:52:45 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 195 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 195 2024-03-15 13:52:45 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:45 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075309 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075309 parsl.executors.status_handling DEBUG: Launched block 250 on executor multi with job ID 17075309 parsl.executors.status_handling INFO: Allocated block ID 251 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-251 2024-03-15 13:52:45 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-251 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-251 2024-03-15 13:52:45 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:45 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:45 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:45 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:45 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:45 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 195 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 195 2024-03-15 13:52:45 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:45 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=251 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 392 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 015a804ece89641b5274b0b7da808acc with result from task 392 parsl.dataflow.dflow DEBUG: Task 392 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c73a5aa7-32a9-45d5-84e3-dc4296555aeb_isrForCrosstalkSources_997020_61.stdout parsl.dataflow.dflow INFO: Standard error for task 392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c73a5aa7-32a9-45d5-84e3-dc4296555aeb_isrForCrosstalkSources_997020_61.stderr parsl.dataflow.dflow INFO: Task 594 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da45ad4c455d1b88b39f07bd459cb9c8 with result from task 594 parsl.dataflow.dflow DEBUG: Task 594 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/06d34a27-26b0-4faf-bf1e-a478f0db1161_isrForCrosstalkSources_997029_43.stdout parsl.dataflow.dflow INFO: Standard error for task 594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/06d34a27-26b0-4faf-bf1e-a478f0db1161_isrForCrosstalkSources_997029_43.stderr 2024-03-15 13:52:46 proc_lsst.multi:170 [INFO] [multi] job_id 17075310 proc_lsst.multi INFO: [multi] job_id 17075310 2024-03-15 13:52:46 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 196 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 196 2024-03-15 13:52:46 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:46 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075310 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075310 parsl.executors.status_handling DEBUG: Launched block 251 on executor multi with job ID 17075310 parsl.executors.status_handling INFO: Allocated block ID 252 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-252 2024-03-15 13:52:46 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-252 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-252 2024-03-15 13:52:46 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:46 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:46 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:46 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:46 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:46 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 196 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 196 2024-03-15 13:52:46 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:46 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=252 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow INFO: Task 638 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e4e76d69b15998f672375c02370512c with result from task 638 parsl.dataflow.dflow DEBUG: Task 638 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/f1f8482b-40b4-4453-872c-7383ae60ed04_isrForCrosstalkSources_997033_56.stdout parsl.dataflow.dflow INFO: Standard error for task 638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/f1f8482b-40b4-4453-872c-7383ae60ed04_isrForCrosstalkSources_997033_56.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 637 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02c835afacc492f48e30e16f6812dd11 with result from task 637 parsl.dataflow.dflow DEBUG: Task 637 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5c4b0430-fe37-4a00-b579-9b03cdbbbfb9_isrForCrosstalkSources_997012_21.stdout parsl.dataflow.dflow INFO: Standard error for task 637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5c4b0430-fe37-4a00-b579-9b03cdbbbfb9_isrForCrosstalkSources_997012_21.stderr 2024-03-15 13:52:47 proc_lsst.multi:170 [INFO] [multi] job_id 17075311 proc_lsst.multi INFO: [multi] job_id 17075311 2024-03-15 13:52:47 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 197 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 197 2024-03-15 13:52:47 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:47 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075311 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075311 parsl.executors.status_handling DEBUG: Launched block 252 on executor multi with job ID 17075311 parsl.executors.status_handling INFO: Allocated block ID 253 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-253 2024-03-15 13:52:47 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-253 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-253 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 197 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 197 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:47 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=253 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 591 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7910cafe5c3f2e1ed48048aed9667f8a with result from task 591 parsl.dataflow.dflow DEBUG: Task 591 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/764db181-1bac-4bde-a5da-fae6523d2fc9_isrForCrosstalkSources_997014_54.stdout parsl.dataflow.dflow INFO: Standard error for task 591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/764db181-1bac-4bde-a5da-fae6523d2fc9_isrForCrosstalkSources_997014_54.stderr parsl.dataflow.dflow INFO: Task 635 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 346451b65f05439980a214abcc55abb1 with result from task 635 parsl.dataflow.dflow DEBUG: Task 635 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b941c94c-9fb6-459d-9a86-12ead70062ad_isrForCrosstalkSources_996996_38.stdout parsl.dataflow.dflow INFO: Standard error for task 635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b941c94c-9fb6-459d-9a86-12ead70062ad_isrForCrosstalkSources_996996_38.stderr 2024-03-15 13:52:47 proc_lsst.multi:170 [INFO] [multi] job_id 17075312 proc_lsst.multi INFO: [multi] job_id 17075312 2024-03-15 13:52:47 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 198 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 198 2024-03-15 13:52:47 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:47 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075312 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075312 parsl.executors.status_handling DEBUG: Launched block 253 on executor multi with job ID 17075312 parsl.executors.status_handling INFO: Allocated block ID 254 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-254 2024-03-15 13:52:47 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-254 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-254 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:47 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:47 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 198 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 198 2024-03-15 13:52:47 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:47 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=254 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 570 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00f30cfaafcf942dfe2efec5193c92e4 with result from task 570 parsl.dataflow.dflow DEBUG: Task 570 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/d925ace7-2817-4005-859c-25b81e1a0e03_isrForCrosstalkSources_996969_52.stdout parsl.dataflow.dflow INFO: Standard error for task 570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/d925ace7-2817-4005-859c-25b81e1a0e03_isrForCrosstalkSources_996969_52.stderr 2024-03-15 13:52:48 proc_lsst.multi:170 [INFO] [multi] job_id 17075315 proc_lsst.multi INFO: [multi] job_id 17075315 2024-03-15 13:52:48 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 199 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 199 2024-03-15 13:52:48 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:48 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075315 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075315 parsl.executors.status_handling DEBUG: Launched block 254 on executor multi with job ID 17075315 parsl.executors.status_handling INFO: Allocated block ID 255 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-255 2024-03-15 13:52:48 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-255 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-255 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 199 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 199 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:48 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=255 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 583 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3805655fdac5a676513b91febfe2aa62 with result from task 583 parsl.dataflow.dflow DEBUG: Task 583 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/33b59da4-8b09-4d6c-a8af-4bf3ae4b2351_isrForCrosstalkSources_997003_60.stdout parsl.dataflow.dflow INFO: Standard error for task 583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/33b59da4-8b09-4d6c-a8af-4bf3ae4b2351_isrForCrosstalkSources_997003_60.stderr 2024-03-15 13:52:48 proc_lsst.multi:170 [INFO] [multi] job_id 17075318 proc_lsst.multi INFO: [multi] job_id 17075318 2024-03-15 13:52:48 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 200 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 200 2024-03-15 13:52:48 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:48 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075318 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075318 parsl.executors.status_handling DEBUG: Launched block 255 on executor multi with job ID 17075318 parsl.executors.status_handling INFO: Allocated block ID 256 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-256 2024-03-15 13:52:48 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-256 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-256 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:48 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:48 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 200 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 200 2024-03-15 13:52:48 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:48 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=256 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 619 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89791decc03f63f74d370def5e7d5f95 with result from task 619 parsl.dataflow.dflow DEBUG: Task 619 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1e5c7299-ea5a-47e6-8b1c-73837f18d03a_isrForCrosstalkSources_997014_24.stdout parsl.dataflow.dflow INFO: Standard error for task 619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1e5c7299-ea5a-47e6-8b1c-73837f18d03a_isrForCrosstalkSources_997014_24.stderr 2024-03-15 13:52:49 proc_lsst.multi:170 [INFO] [multi] job_id 17075319 proc_lsst.multi INFO: [multi] job_id 17075319 2024-03-15 13:52:49 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 201 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 201 2024-03-15 13:52:49 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:49 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075319 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075319 parsl.executors.status_handling DEBUG: Launched block 256 on executor multi with job ID 17075319 parsl.executors.status_handling INFO: Allocated block ID 257 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-257 2024-03-15 13:52:49 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-257 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-257 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 201 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 201 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:49 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=257 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 621 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fa9cb50720869aaaad03717d8ed13b2 with result from task 621 parsl.dataflow.dflow DEBUG: Task 621 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b85389b9-8593-4db5-b767-b212a214a19e_isrForCrosstalkSources_996979_9.stdout parsl.dataflow.dflow INFO: Standard error for task 621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b85389b9-8593-4db5-b767-b212a214a19e_isrForCrosstalkSources_996979_9.stderr parsl.dataflow.dflow INFO: Task 604 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0691244c51e8cbafb3db4c92d1f8809a with result from task 604 parsl.dataflow.dflow DEBUG: Task 604 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7a1964e5-b3f6-43ef-ad4b-2f294f5f6f78_isrForCrosstalkSources_996993_3.stdout parsl.dataflow.dflow INFO: Standard error for task 604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7a1964e5-b3f6-43ef-ad4b-2f294f5f6f78_isrForCrosstalkSources_996993_3.stderr parsl.dataflow.dflow INFO: Task 663 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32e6c66dfde3cf5cf8223c7efe6f3cd6 with result from task 663 parsl.dataflow.dflow DEBUG: Task 663 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/2c8cff96-b806-47ac-8fc3-04596c6152e6_isrForCrosstalkSources_996983_25.stdout parsl.dataflow.dflow INFO: Standard error for task 663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/2c8cff96-b806-47ac-8fc3-04596c6152e6_isrForCrosstalkSources_996983_25.stderr 2024-03-15 13:52:49 proc_lsst.multi:170 [INFO] [multi] job_id 17075320 proc_lsst.multi INFO: [multi] job_id 17075320 2024-03-15 13:52:49 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 202 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 202 2024-03-15 13:52:49 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:49 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075320 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075320 parsl.executors.status_handling DEBUG: Launched block 257 on executor multi with job ID 17075320 parsl.executors.status_handling INFO: Allocated block ID 258 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-258 2024-03-15 13:52:49 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-258 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-258 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:49 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:49 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 202 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 202 2024-03-15 13:52:49 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:49 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=258 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:50 proc_lsst.multi:170 [INFO] [multi] job_id 17075321 proc_lsst.multi INFO: [multi] job_id 17075321 2024-03-15 13:52:50 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 203 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 203 2024-03-15 13:52:50 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:50 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075321 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075321 parsl.executors.status_handling DEBUG: Launched block 258 on executor multi with job ID 17075321 parsl.executors.status_handling INFO: Allocated block ID 259 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-259 2024-03-15 13:52:50 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-259 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-259 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 203 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 203 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:50 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=259 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 749 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e19aa3896229fc86790115a05518a020 with result from task 749 parsl.dataflow.dflow DEBUG: Task 749 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/3ffea72a-ca29-445e-abb6-68a652e7efff_isrForCrosstalkSources_996974_34.stdout parsl.dataflow.dflow INFO: Standard error for task 749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/3ffea72a-ca29-445e-abb6-68a652e7efff_isrForCrosstalkSources_996974_34.stderr parsl.dataflow.dflow INFO: Task 629 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b1a96400962371c252beb63c331327bd with result from task 629 parsl.dataflow.dflow DEBUG: Task 629 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/28d8777b-2392-48a8-81f7-317c5079ca54_isrForCrosstalkSources_997016_33.stdout parsl.dataflow.dflow INFO: Standard error for task 629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/28d8777b-2392-48a8-81f7-317c5079ca54_isrForCrosstalkSources_997016_33.stderr parsl.dataflow.dflow INFO: Task 632 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c0ddd2cdb7b8b47ea23424abe3207d42 with result from task 632 parsl.dataflow.dflow DEBUG: Task 632 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/3188d077-cd2e-41e7-972a-0d912d1f8066_isrForCrosstalkSources_997003_40.stdout parsl.dataflow.dflow INFO: Standard error for task 632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/3188d077-cd2e-41e7-972a-0d912d1f8066_isrForCrosstalkSources_997003_40.stderr parsl.dataflow.dflow INFO: Task 639 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b15a3419641a71cffd951968909ce38a with result from task 639 parsl.dataflow.dflow DEBUG: Task 639 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/ef6d33f5-4002-4cd0-a2bd-e850f7f5291c_isrForCrosstalkSources_996983_27.stdout parsl.dataflow.dflow INFO: Standard error for task 639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/ef6d33f5-4002-4cd0-a2bd-e850f7f5291c_isrForCrosstalkSources_996983_27.stderr 2024-03-15 13:52:50 proc_lsst.multi:170 [INFO] [multi] job_id 17075326 proc_lsst.multi INFO: [multi] job_id 17075326 2024-03-15 13:52:50 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 204 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 204 2024-03-15 13:52:50 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:50 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075326 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075326 parsl.executors.status_handling DEBUG: Launched block 259 on executor multi with job ID 17075326 parsl.executors.status_handling INFO: Allocated block ID 260 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-260 2024-03-15 13:52:50 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-260 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-260 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:50 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:50 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 204 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 204 2024-03-15 13:52:50 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:50 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=260 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 771 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 80c4af5b5e98dd4cf14b16265d39c03f with result from task 771 parsl.dataflow.dflow DEBUG: Task 771 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/674a2622-e384-4125-afbb-460bc97c0a3f_isrForCrosstalkSources_996970_33.stdout parsl.dataflow.dflow INFO: Standard error for task 771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/674a2622-e384-4125-afbb-460bc97c0a3f_isrForCrosstalkSources_996970_33.stderr parsl.dataflow.dflow INFO: Task 650 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c0baf43f2812aa3d3977d77ad257af6f with result from task 650 parsl.dataflow.dflow DEBUG: Task 650 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9adeae47-579b-4b9f-9853-5be5fa0a32e0_isrForCrosstalkSources_996984_25.stdout parsl.dataflow.dflow INFO: Standard error for task 650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9adeae47-579b-4b9f-9853-5be5fa0a32e0_isrForCrosstalkSources_996984_25.stderr parsl.dataflow.dflow INFO: Task 687 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed8085fa4b981dfa53a5fb1ae2cdb727 with result from task 687 parsl.dataflow.dflow DEBUG: Task 687 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c9ec783e-9c1d-4923-8149-72b0afe0c7b4_isrForCrosstalkSources_997007_26.stdout parsl.dataflow.dflow INFO: Standard error for task 687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c9ec783e-9c1d-4923-8149-72b0afe0c7b4_isrForCrosstalkSources_997007_26.stderr parsl.dataflow.dflow INFO: Task 721 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a8927363430d74e86387edf26b40273 with result from task 721 parsl.dataflow.dflow DEBUG: Task 721 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d1250c3f-a0ee-427f-8267-ad545d9f7e00_isrForCrosstalkSources_996980_26.stdout parsl.dataflow.dflow INFO: Standard error for task 721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d1250c3f-a0ee-427f-8267-ad545d9f7e00_isrForCrosstalkSources_996980_26.stderr parsl.dataflow.dflow INFO: Task 741 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b36d1d04669f81fbfcabd1d66e72283 with result from task 741 parsl.dataflow.dflow DEBUG: Task 741 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/d0ca6ff1-adb2-4818-a641-6c975b13d1e3_isrForCrosstalkSources_997021_26.stdout parsl.dataflow.dflow INFO: Standard error for task 741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/d0ca6ff1-adb2-4818-a641-6c975b13d1e3_isrForCrosstalkSources_997021_26.stderr parsl.dataflow.dflow INFO: Task 655 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d0d9e7e908c060b8b65e4b1872955a5 with result from task 655 parsl.dataflow.dflow DEBUG: Task 655 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/5c6dbff9-9042-4229-a66d-7c7c2748fecb_isrForCrosstalkSources_996963_14.stdout parsl.dataflow.dflow INFO: Standard error for task 655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/5c6dbff9-9042-4229-a66d-7c7c2748fecb_isrForCrosstalkSources_996963_14.stderr parsl.dataflow.dflow INFO: Task 668 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f58d780b385420b4f64d0661c68cee1 with result from task 668 parsl.dataflow.dflow DEBUG: Task 668 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3d560042-bfcb-401c-942c-754db5b50f82_isrForCrosstalkSources_996984_13.stdout parsl.dataflow.dflow INFO: Standard error for task 668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3d560042-bfcb-401c-942c-754db5b50f82_isrForCrosstalkSources_996984_13.stderr 2024-03-15 13:52:51 proc_lsst.multi:170 [INFO] [multi] job_id 17075329 proc_lsst.multi INFO: [multi] job_id 17075329 2024-03-15 13:52:51 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 205 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 205 2024-03-15 13:52:51 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:51 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075329 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075329 parsl.executors.status_handling DEBUG: Launched block 260 on executor multi with job ID 17075329 parsl.executors.status_handling INFO: Allocated block ID 261 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-261 2024-03-15 13:52:51 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-261 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-261 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 205 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 205 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:51 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=261 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 634 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 197c879eff5e0bd3241b3660d0c6fd4c with result from task 634 parsl.dataflow.dflow DEBUG: Task 634 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/377c2b77-8177-403d-a99d-db8e9ad18dbc_isrForCrosstalkSources_996997_48.stdout parsl.dataflow.dflow INFO: Standard error for task 634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/377c2b77-8177-403d-a99d-db8e9ad18dbc_isrForCrosstalkSources_996997_48.stderr parsl.dataflow.dflow INFO: Task 641 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b47410fcae86510fffa3c68062d280c9 with result from task 641 parsl.dataflow.dflow DEBUG: Task 641 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6b18b9ad-599a-41ab-a4c5-0ce252568268_isrForCrosstalkSources_997006_45.stdout parsl.dataflow.dflow INFO: Standard error for task 641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6b18b9ad-599a-41ab-a4c5-0ce252568268_isrForCrosstalkSources_997006_45.stderr parsl.dataflow.dflow INFO: Task 630 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90bf3f06518baa4b66501a02d2621306 with result from task 630 parsl.dataflow.dflow DEBUG: Task 630 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e7abe875-9172-4daa-8e16-20ba83742217_isrForCrosstalkSources_996995_51.stdout parsl.dataflow.dflow INFO: Standard error for task 630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e7abe875-9172-4daa-8e16-20ba83742217_isrForCrosstalkSources_996995_51.stderr parsl.dataflow.dflow INFO: Task 688 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cae26caf78b3386b452c95991b913e50 with result from task 688 parsl.dataflow.dflow DEBUG: Task 688 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/948df5a6-0cca-4e69-aa99-633e928a1da2_isrForCrosstalkSources_997031_26.stdout parsl.dataflow.dflow INFO: Standard error for task 688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/948df5a6-0cca-4e69-aa99-633e928a1da2_isrForCrosstalkSources_997031_26.stderr 2024-03-15 13:52:51 proc_lsst.multi:170 [INFO] [multi] job_id 17075330 proc_lsst.multi INFO: [multi] job_id 17075330 2024-03-15 13:52:51 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 206 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 206 2024-03-15 13:52:51 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:51 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075330 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075330 parsl.executors.status_handling DEBUG: Launched block 261 on executor multi with job ID 17075330 parsl.executors.status_handling INFO: Allocated block ID 262 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-262 2024-03-15 13:52:51 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-262 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-262 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:51 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:51 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 206 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 206 2024-03-15 13:52:51 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:51 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=262 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 737 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d16b0978efca7854548dfdd7de19c41c with result from task 737 parsl.dataflow.dflow DEBUG: Task 737 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f52d235d-8fdd-4833-90e2-24b6844950dd_isrForCrosstalkSources_996960_26.stdout parsl.dataflow.dflow INFO: Standard error for task 737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f52d235d-8fdd-4833-90e2-24b6844950dd_isrForCrosstalkSources_996960_26.stderr parsl.dataflow.dflow INFO: Task 665 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89e9e44f5b55f9d2dea80c3c11e67800 with result from task 665 parsl.dataflow.dflow DEBUG: Task 665 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/b60f1914-3796-4374-beb3-21f087627491_isrForCrosstalkSources_996995_60.stdout parsl.dataflow.dflow INFO: Standard error for task 665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/b60f1914-3796-4374-beb3-21f087627491_isrForCrosstalkSources_996995_60.stderr parsl.dataflow.dflow INFO: Task 662 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d588a9082f5cc71fe07e9ccd2fc45947 with result from task 662 parsl.dataflow.dflow DEBUG: Task 662 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/af36ff8e-6438-4994-8814-e8bb87c34380_isrForCrosstalkSources_996957_60.stdout parsl.dataflow.dflow INFO: Standard error for task 662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/af36ff8e-6438-4994-8814-e8bb87c34380_isrForCrosstalkSources_996957_60.stderr 2024-03-15 13:52:52 proc_lsst.multi:170 [INFO] [multi] job_id 17075332 proc_lsst.multi INFO: [multi] job_id 17075332 2024-03-15 13:52:52 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 207 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 207 2024-03-15 13:52:52 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:52 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075332 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075332 parsl.executors.status_handling DEBUG: Launched block 262 on executor multi with job ID 17075332 parsl.executors.status_handling INFO: Allocated block ID 263 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-263 2024-03-15 13:52:52 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-263 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-263 2024-03-15 13:52:52 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:52 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:52 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:52 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:52 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:52 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 207 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 207 2024-03-15 13:52:52 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:52 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=263 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 666 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5ceadf29437a9f0f10905ab79db4920 with result from task 666 parsl.dataflow.dflow DEBUG: Task 666 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/a42947e2-9441-4af0-b45a-4201636ab8c1_isrForCrosstalkSources_997030_20.stdout parsl.dataflow.dflow INFO: Standard error for task 666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/a42947e2-9441-4af0-b45a-4201636ab8c1_isrForCrosstalkSources_997030_20.stderr parsl.dataflow.dflow INFO: Task 664 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fd21e11aa08c5214bad6d62ae602da7 with result from task 664 parsl.dataflow.dflow DEBUG: Task 664 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2d2971f0-5357-427c-bfff-62a71e267c6f_isrForCrosstalkSources_996981_4.stdout parsl.dataflow.dflow INFO: Standard error for task 664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2d2971f0-5357-427c-bfff-62a71e267c6f_isrForCrosstalkSources_996981_4.stderr parsl.dataflow.dflow INFO: Task 659 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7afeb32312bd1bc4add96ae98dfbce8d with result from task 659 parsl.dataflow.dflow DEBUG: Task 659 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/97d1eef5-d34e-42fb-8a03-6647f112f847_isrForCrosstalkSources_996981_13.stdout parsl.dataflow.dflow INFO: Standard error for task 659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/97d1eef5-d34e-42fb-8a03-6647f112f847_isrForCrosstalkSources_996981_13.stderr parsl.dataflow.dflow INFO: Task 700 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0bbf2f6450305eff7d9460338d6e9b58 with result from task 700 parsl.dataflow.dflow DEBUG: Task 700 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/56e10972-8a3f-4942-8e31-915507104720_isrForCrosstalkSources_996961_34.stdout parsl.dataflow.dflow INFO: Standard error for task 700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/56e10972-8a3f-4942-8e31-915507104720_isrForCrosstalkSources_996961_34.stderr 2024-03-15 13:52:53 proc_lsst.multi:170 [INFO] [multi] job_id 17075338 proc_lsst.multi INFO: [multi] job_id 17075338 2024-03-15 13:52:53 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 208 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 208 2024-03-15 13:52:53 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:53 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075338 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075338 parsl.executors.status_handling DEBUG: Launched block 263 on executor multi with job ID 17075338 parsl.executors.status_handling INFO: Allocated block ID 264 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-264 2024-03-15 13:52:53 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-264 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-264 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 208 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 208 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:53 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=264 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:53 proc_lsst.multi:170 [INFO] [multi] job_id 17075344 proc_lsst.multi INFO: [multi] job_id 17075344 2024-03-15 13:52:53 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 209 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 209 2024-03-15 13:52:53 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:53 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075344 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075344 parsl.executors.status_handling DEBUG: Launched block 264 on executor multi with job ID 17075344 parsl.executors.status_handling INFO: Allocated block ID 265 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-265 2024-03-15 13:52:53 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-265 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-265 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 209 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 209 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:53 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=265 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 625 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6da9a0ff8fa2d2a9725050d476f0cb80 with result from task 625 parsl.dataflow.dflow DEBUG: Task 625 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/313fde2d-b413-4f2c-97e7-2daad3c2961f_isrForCrosstalkSources_997006_22.stdout parsl.dataflow.dflow INFO: Standard error for task 625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/313fde2d-b413-4f2c-97e7-2daad3c2961f_isrForCrosstalkSources_997006_22.stderr parsl.dataflow.dflow INFO: Task 705 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d9c73c05e0985809a2d54c3dca3c154 with result from task 705 parsl.dataflow.dflow DEBUG: Task 705 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b0ca83a8-b352-4c00-a177-c1d69eb2315b_isrForCrosstalkSources_997031_34.stdout parsl.dataflow.dflow INFO: Standard error for task 705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b0ca83a8-b352-4c00-a177-c1d69eb2315b_isrForCrosstalkSources_997031_34.stderr 2024-03-15 13:52:53 proc_lsst.multi:170 [INFO] [multi] job_id 17075345 proc_lsst.multi INFO: [multi] job_id 17075345 2024-03-15 13:52:53 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 210 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 210 2024-03-15 13:52:53 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:53 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075345 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075345 parsl.executors.status_handling DEBUG: Launched block 265 on executor multi with job ID 17075345 parsl.executors.status_handling INFO: Allocated block ID 266 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-266 2024-03-15 13:52:53 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-266 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-266 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:53 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:53 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 210 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 210 2024-03-15 13:52:53 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:53 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=266 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 710 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dfb4b01563a3208a222150c44b53a715 with result from task 710 parsl.dataflow.dflow DEBUG: Task 710 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/485ff063-438d-4298-b666-a31276a97ce6_isrForCrosstalkSources_997010_20.stdout parsl.dataflow.dflow INFO: Standard error for task 710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/485ff063-438d-4298-b666-a31276a97ce6_isrForCrosstalkSources_997010_20.stderr parsl.dataflow.dflow INFO: Task 674 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65369756c46d644eafbacbf0d498770d with result from task 674 parsl.dataflow.dflow DEBUG: Task 674 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/e82e3242-8816-4939-a078-88a670954699_isrForCrosstalkSources_997004_13.stdout parsl.dataflow.dflow INFO: Standard error for task 674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/e82e3242-8816-4939-a078-88a670954699_isrForCrosstalkSources_997004_13.stderr 2024-03-15 13:52:54 proc_lsst.multi:170 [INFO] [multi] job_id 17075346 proc_lsst.multi INFO: [multi] job_id 17075346 2024-03-15 13:52:54 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 211 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 211 2024-03-15 13:52:54 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:54 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075346 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075346 parsl.executors.status_handling DEBUG: Launched block 266 on executor multi with job ID 17075346 parsl.executors.status_handling INFO: Allocated block ID 267 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-267 2024-03-15 13:52:54 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-267 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-267 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 211 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 211 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:54 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=267 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 738 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63dceabf2f84787d865a51ac7c71b90b with result from task 738 parsl.dataflow.dflow DEBUG: Task 738 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5cbcca52-ceed-481b-9518-59d3f2a7d1cb_isrForCrosstalkSources_997003_52.stdout parsl.dataflow.dflow INFO: Standard error for task 738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5cbcca52-ceed-481b-9518-59d3f2a7d1cb_isrForCrosstalkSources_997003_52.stderr 2024-03-15 13:52:54 proc_lsst.multi:170 [INFO] [multi] job_id 17075347 proc_lsst.multi INFO: [multi] job_id 17075347 2024-03-15 13:52:54 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 212 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 212 2024-03-15 13:52:54 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:54 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075347 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075347 parsl.executors.status_handling DEBUG: Launched block 267 on executor multi with job ID 17075347 parsl.executors.status_handling INFO: Allocated block ID 268 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-268 2024-03-15 13:52:54 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-268 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-268 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:54 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:54 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 212 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 212 2024-03-15 13:52:54 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:54 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=268 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 740 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 293fb803235e2a298b3e6b2854e95595 with result from task 740 parsl.dataflow.dflow DEBUG: Task 740 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4060b6cb-eb16-4c7e-851f-048a98dc2ec1_isrForCrosstalkSources_996957_4.stdout parsl.dataflow.dflow INFO: Standard error for task 740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4060b6cb-eb16-4c7e-851f-048a98dc2ec1_isrForCrosstalkSources_996957_4.stderr parsl.dataflow.dflow INFO: Task 693 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2e501b045646e1e32b667877796036b with result from task 693 parsl.dataflow.dflow DEBUG: Task 693 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/a6a87cc7-7487-4226-8326-9740b0777fc4_isrForCrosstalkSources_996998_21.stdout parsl.dataflow.dflow INFO: Standard error for task 693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/a6a87cc7-7487-4226-8326-9740b0777fc4_isrForCrosstalkSources_996998_21.stderr parsl.dataflow.dflow INFO: Task 730 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dee687fe481e088c58e0e7e9637decb7 with result from task 730 parsl.dataflow.dflow DEBUG: Task 730 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/a58016b5-65fd-4938-be47-3872357cc888_isrForCrosstalkSources_996966_17.stdout parsl.dataflow.dflow INFO: Standard error for task 730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/a58016b5-65fd-4938-be47-3872357cc888_isrForCrosstalkSources_996966_17.stderr 2024-03-15 13:52:55 proc_lsst.multi:170 [INFO] [multi] job_id 17075348 proc_lsst.multi INFO: [multi] job_id 17075348 2024-03-15 13:52:55 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 213 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 213 2024-03-15 13:52:55 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:55 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075348 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075348 parsl.executors.status_handling DEBUG: Launched block 268 on executor multi with job ID 17075348 parsl.executors.status_handling INFO: Allocated block ID 269 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-269 2024-03-15 13:52:55 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-269 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-269 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 213 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 213 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:55 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=269 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 698 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51fb4ef1b1bae807384c04cc6f5f192d with result from task 698 parsl.dataflow.dflow DEBUG: Task 698 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f43cd18-1e97-4e33-86bc-847f52f59f85_isrForCrosstalkSources_996994_13.stdout parsl.dataflow.dflow INFO: Standard error for task 698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f43cd18-1e97-4e33-86bc-847f52f59f85_isrForCrosstalkSources_996994_13.stderr parsl.dataflow.dflow INFO: Task 636 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f105b2cc0f3cd537dddce9e0b60b028 with result from task 636 parsl.dataflow.dflow DEBUG: Task 636 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7eb9faac-674a-4616-b7b0-eaa1a67edef5_isrForCrosstalkSources_997011_9.stdout parsl.dataflow.dflow INFO: Standard error for task 636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7eb9faac-674a-4616-b7b0-eaa1a67edef5_isrForCrosstalkSources_997011_9.stderr parsl.dataflow.dflow INFO: Task 769 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 057fc24d34bb6696654712c3dbeefc48 with result from task 769 parsl.dataflow.dflow DEBUG: Task 769 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c53edd47-42cb-44eb-b02a-9e7381b06bed_isrForCrosstalkSources_996968_5.stdout parsl.dataflow.dflow INFO: Standard error for task 769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c53edd47-42cb-44eb-b02a-9e7381b06bed_isrForCrosstalkSources_996968_5.stderr 2024-03-15 13:52:55 proc_lsst.multi:170 [INFO] [multi] job_id 17075351 proc_lsst.multi INFO: [multi] job_id 17075351 2024-03-15 13:52:55 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 214 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 214 2024-03-15 13:52:55 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:55 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075351 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075351 parsl.executors.status_handling DEBUG: Launched block 269 on executor multi with job ID 17075351 parsl.executors.status_handling INFO: Allocated block ID 270 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-270 2024-03-15 13:52:55 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-270 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-270 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:55 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:55 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 214 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 214 2024-03-15 13:52:55 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:55 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=270 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:52:56 proc_lsst.multi:170 [INFO] [multi] job_id 17075353 proc_lsst.multi INFO: [multi] job_id 17075353 2024-03-15 13:52:56 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 215 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 215 2024-03-15 13:52:56 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:56 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075353 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075353 parsl.executors.status_handling DEBUG: Launched block 270 on executor multi with job ID 17075353 parsl.executors.status_handling INFO: Allocated block ID 271 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-271 2024-03-15 13:52:56 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-271 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-271 2024-03-15 13:52:56 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:56 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:56 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:56 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:56 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:56 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 215 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 215 2024-03-15 13:52:56 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:56 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=271 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 682 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 193373f5e4d3975e4334c889296585dc with result from task 682 parsl.dataflow.dflow DEBUG: Task 682 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c7d2c6cd-c324-44dc-bebc-4844f7a669ca_isrForCrosstalkSources_997000_36.stdout parsl.dataflow.dflow INFO: Standard error for task 682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c7d2c6cd-c324-44dc-bebc-4844f7a669ca_isrForCrosstalkSources_997000_36.stderr parsl.dataflow.dflow INFO: Task 703 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 729cbb7f4007ad4f97868003e790b442 with result from task 703 parsl.dataflow.dflow DEBUG: Task 703 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c80a20e5-e462-4054-85b7-bf5a6cf48e93_isrForCrosstalkSources_997021_14.stdout parsl.dataflow.dflow INFO: Standard error for task 703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c80a20e5-e462-4054-85b7-bf5a6cf48e93_isrForCrosstalkSources_997021_14.stderr 2024-03-15 13:52:57 proc_lsst.multi:170 [INFO] [multi] job_id 17075354 proc_lsst.multi INFO: [multi] job_id 17075354 2024-03-15 13:52:57 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 216 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 216 2024-03-15 13:52:57 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:57 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075354 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075354 parsl.executors.status_handling DEBUG: Launched block 271 on executor multi with job ID 17075354 parsl.executors.status_handling INFO: Allocated block ID 272 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-272 2024-03-15 13:52:57 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-272 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-272 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 216 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 216 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:57 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=272 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 526 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd189571e5aca29b8eb9e35e4d314af5 with result from task 526 parsl.dataflow.dflow DEBUG: Task 526 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ea180c96-c1e4-4722-838c-95be09a8d437_isrForCrosstalkSources_997029_19.stdout parsl.dataflow.dflow INFO: Standard error for task 526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ea180c96-c1e4-4722-838c-95be09a8d437_isrForCrosstalkSources_997029_19.stderr parsl.dataflow.dflow INFO: Task 524 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89eb6e57d7d51a2ad8a58216860384df with result from task 524 parsl.dataflow.dflow DEBUG: Task 524 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/432f9f79-1618-4f30-8516-3e6e863a93e3_isrForCrosstalkSources_996979_45.stdout parsl.dataflow.dflow INFO: Standard error for task 524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/432f9f79-1618-4f30-8516-3e6e863a93e3_isrForCrosstalkSources_996979_45.stderr parsl.dataflow.dflow INFO: Task 522 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 158cbe5dc2d26449b91302a2df3e634c with result from task 522 parsl.dataflow.dflow DEBUG: Task 522 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/eb232e8c-9874-4faf-ab40-e36e240c8fa9_isrForCrosstalkSources_996971_19.stdout parsl.dataflow.dflow INFO: Standard error for task 522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/eb232e8c-9874-4faf-ab40-e36e240c8fa9_isrForCrosstalkSources_996971_19.stderr parsl.dataflow.dflow INFO: Task 527 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5e4dd19bfa538169c98a1235597753d with result from task 527 parsl.dataflow.dflow DEBUG: Task 527 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/4d692560-61eb-4462-9c12-d049963fea68_isrForCrosstalkSources_996985_3.stdout parsl.dataflow.dflow INFO: Standard error for task 527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/4d692560-61eb-4462-9c12-d049963fea68_isrForCrosstalkSources_996985_3.stderr parsl.dataflow.dflow INFO: Task 523 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48b0e307dfee560a87e11984892d2a18 with result from task 523 parsl.dataflow.dflow DEBUG: Task 523 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3be1543c-bd96-4a30-ab07-d388dc77bbdd_isrForCrosstalkSources_996963_28.stdout parsl.dataflow.dflow INFO: Standard error for task 523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3be1543c-bd96-4a30-ab07-d388dc77bbdd_isrForCrosstalkSources_996963_28.stderr parsl.dataflow.dflow INFO: Task 520 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a23766e06c337f4438d2771518df665e with result from task 520 parsl.dataflow.dflow DEBUG: Task 520 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e26b6584-787d-4f6e-9817-5bb88fb891af_isrForCrosstalkSources_996984_48.stdout parsl.dataflow.dflow INFO: Standard error for task 520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e26b6584-787d-4f6e-9817-5bb88fb891af_isrForCrosstalkSources_996984_48.stderr parsl.dataflow.dflow INFO: Task 675 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 67bf639537160db410a4a762fec6ee9b with result from task 675 parsl.dataflow.dflow DEBUG: Task 675 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d8a601f9-8c34-4515-87fa-1769fc1ee7c6_isrForCrosstalkSources_997000_5.stdout parsl.dataflow.dflow INFO: Standard error for task 675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d8a601f9-8c34-4515-87fa-1769fc1ee7c6_isrForCrosstalkSources_997000_5.stderr parsl.dataflow.dflow INFO: Task 777 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef5f19f7f87a03aff29670bc5876acd6 with result from task 777 parsl.dataflow.dflow DEBUG: Task 777 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/e8bf53e6-3a2a-49f6-a57b-eb04ae6443b6_isrForCrosstalkSources_997014_2.stdout parsl.dataflow.dflow INFO: Standard error for task 777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/e8bf53e6-3a2a-49f6-a57b-eb04ae6443b6_isrForCrosstalkSources_997014_2.stderr parsl.dataflow.dflow INFO: Task 673 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9afbe21cdac247b803c2b7cd163f8ea with result from task 673 parsl.dataflow.dflow DEBUG: Task 673 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/bcc5f387-e13c-4c42-a62b-78ce755cae5c_isrForCrosstalkSources_996975_21.stdout parsl.dataflow.dflow INFO: Standard error for task 673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/bcc5f387-e13c-4c42-a62b-78ce755cae5c_isrForCrosstalkSources_996975_21.stderr parsl.dataflow.dflow INFO: Task 784 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 05cb7c50dbb823996e5a4af3a6000568 with result from task 784 parsl.dataflow.dflow DEBUG: Task 784 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/c6da1623-7bce-4fdf-83c7-ad16f107948e_isrForCrosstalkSources_996952_33.stdout parsl.dataflow.dflow INFO: Standard error for task 784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/c6da1623-7bce-4fdf-83c7-ad16f107948e_isrForCrosstalkSources_996952_33.stderr parsl.dataflow.dflow INFO: Task 794 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86c3e9a7501f8dc0579b115664632991 with result from task 794 parsl.dataflow.dflow DEBUG: Task 794 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8ea1eed6-b1b2-4d9e-85ab-33af451c26f3_isrForCrosstalkSources_997013_34.stdout parsl.dataflow.dflow INFO: Standard error for task 794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8ea1eed6-b1b2-4d9e-85ab-33af451c26f3_isrForCrosstalkSources_997013_34.stderr parsl.dataflow.dflow INFO: Task 525 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99b24b48755eb3b8726882ed9e1e3d10 with result from task 525 parsl.dataflow.dflow DEBUG: Task 525 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/35c68243-a1ca-4bdd-b7aa-f9f04ecc2e84_isrForCrosstalkSources_996988_45.stdout parsl.dataflow.dflow INFO: Standard error for task 525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/35c68243-a1ca-4bdd-b7aa-f9f04ecc2e84_isrForCrosstalkSources_996988_45.stderr 2024-03-15 13:52:57 proc_lsst.multi:170 [INFO] [multi] job_id 17075355 proc_lsst.multi INFO: [multi] job_id 17075355 2024-03-15 13:52:57 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 217 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 217 2024-03-15 13:52:57 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:57 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075355 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075355 parsl.executors.status_handling DEBUG: Launched block 272 on executor multi with job ID 17075355 parsl.executors.status_handling INFO: Allocated block ID 273 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-273 2024-03-15 13:52:57 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-273 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-273 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:57 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:57 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 217 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 217 2024-03-15 13:52:57 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:57 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=273 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 694 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a116205c97be8bfab1626660ca5a5d7 with result from task 694 parsl.dataflow.dflow DEBUG: Task 694 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/bf79eb25-1944-4d8e-867e-d6b1e6862488_isrForCrosstalkSources_996970_13.stdout parsl.dataflow.dflow INFO: Standard error for task 694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/bf79eb25-1944-4d8e-867e-d6b1e6862488_isrForCrosstalkSources_996970_13.stderr parsl.dataflow.dflow INFO: Task 521 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3cff5dd9059e740116f6f77f4afc3a70 with result from task 521 parsl.dataflow.dflow DEBUG: Task 521 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e84e5307-afdf-4e41-a46b-fbe17bba9b1a_isrForCrosstalkSources_996954_25.stdout parsl.dataflow.dflow INFO: Standard error for task 521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e84e5307-afdf-4e41-a46b-fbe17bba9b1a_isrForCrosstalkSources_996954_25.stderr 2024-03-15 13:52:58 proc_lsst.multi:170 [INFO] [multi] job_id 17075357 proc_lsst.multi INFO: [multi] job_id 17075357 2024-03-15 13:52:58 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 218 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 218 2024-03-15 13:52:58 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:58 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075357 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075357 parsl.executors.status_handling DEBUG: Launched block 273 on executor multi with job ID 17075357 parsl.executors.status_handling INFO: Allocated block ID 274 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-274 2024-03-15 13:52:58 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-274 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-274 2024-03-15 13:52:58 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:58 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:58 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:58 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:58 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:58 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 218 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 218 2024-03-15 13:52:58 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:58 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=274 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 654 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa24fd5b891dbe63db8fbae2f05aeb6b with result from task 654 parsl.dataflow.dflow DEBUG: Task 654 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/32f03f31-8e8a-4c24-a07c-1c90ed882ee6_isrForCrosstalkSources_996964_56.stdout parsl.dataflow.dflow INFO: Standard error for task 654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/32f03f31-8e8a-4c24-a07c-1c90ed882ee6_isrForCrosstalkSources_996964_56.stderr parsl.dataflow.dflow INFO: Task 695 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ddc89f6d8a3f6b03933231fa378053b7 with result from task 695 parsl.dataflow.dflow DEBUG: Task 695 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/8778cd7c-c41b-42ec-a626-e4da331bf9ab_isrForCrosstalkSources_997028_19.stdout parsl.dataflow.dflow INFO: Standard error for task 695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/8778cd7c-c41b-42ec-a626-e4da331bf9ab_isrForCrosstalkSources_997028_19.stderr parsl.dataflow.dflow INFO: Task 707 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9cd963c6f2aea8bf4b04623d1ad6e0d with result from task 707 parsl.dataflow.dflow DEBUG: Task 707 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/366d183b-c079-47e9-9d03-a0646e91523e_isrForCrosstalkSources_996953_56.stdout parsl.dataflow.dflow INFO: Standard error for task 707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/366d183b-c079-47e9-9d03-a0646e91523e_isrForCrosstalkSources_996953_56.stderr parsl.dataflow.dflow INFO: Task 701 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 413fb8d6ac5c30711be30278d26b5688 with result from task 701 parsl.dataflow.dflow DEBUG: Task 701 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c988af9d-09d5-40dc-b511-3d942e320a37_isrForCrosstalkSources_996960_13.stdout parsl.dataflow.dflow INFO: Standard error for task 701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c988af9d-09d5-40dc-b511-3d942e320a37_isrForCrosstalkSources_996960_13.stderr 2024-03-15 13:52:59 proc_lsst.multi:170 [INFO] [multi] job_id 17075358 proc_lsst.multi INFO: [multi] job_id 17075358 2024-03-15 13:52:59 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 219 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 219 2024-03-15 13:52:59 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:59 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075358 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075358 parsl.executors.status_handling DEBUG: Launched block 274 on executor multi with job ID 17075358 parsl.executors.status_handling INFO: Allocated block ID 275 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-275 2024-03-15 13:52:59 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-275 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-275 2024-03-15 13:52:59 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:52:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:52:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:52:59 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:52:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:52:59 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:52:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:52:59 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:52:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:52:59 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:52:59 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 219 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 219 2024-03-15 13:52:59 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:52:59 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=275 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 713 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2cc78bf8ee0bd148a4614aa5772a2516 with result from task 713 parsl.dataflow.dflow DEBUG: Task 713 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7fa03746-907b-40fa-b67e-7c6618b809ee_isrForCrosstalkSources_997021_10.stdout parsl.dataflow.dflow INFO: Standard error for task 713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7fa03746-907b-40fa-b67e-7c6618b809ee_isrForCrosstalkSources_997021_10.stderr parsl.dataflow.dflow INFO: Task 717 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bebeb8625a066ed3e3e76cc09bc1a1fa with result from task 717 parsl.dataflow.dflow DEBUG: Task 717 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2d15abdd-0034-4c2f-a38b-a03a5c65b083_isrForCrosstalkSources_997006_51.stdout parsl.dataflow.dflow INFO: Standard error for task 717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2d15abdd-0034-4c2f-a38b-a03a5c65b083_isrForCrosstalkSources_997006_51.stderr parsl.dataflow.dflow INFO: Task 778 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 40d92487d4e79b0127b8d261d81602ee with result from task 778 parsl.dataflow.dflow DEBUG: Task 778 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/16f44b68-10f3-4358-a356-dae81d8cf8dc_isrForCrosstalkSources_997000_58.stdout parsl.dataflow.dflow INFO: Standard error for task 778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/16f44b68-10f3-4358-a356-dae81d8cf8dc_isrForCrosstalkSources_997000_58.stderr 2024-03-15 13:53:00 proc_lsst.multi:170 [INFO] [multi] job_id 17075359 proc_lsst.multi INFO: [multi] job_id 17075359 2024-03-15 13:53:00 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 220 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 220 2024-03-15 13:53:00 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:00 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075359 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075359 parsl.executors.status_handling DEBUG: Launched block 275 on executor multi with job ID 17075359 parsl.executors.status_handling INFO: Allocated block ID 276 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-276 2024-03-15 13:53:00 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-276 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-276 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 220 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 220 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:00 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=276 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:53:00 proc_lsst.multi:170 [INFO] [multi] job_id 17075360 proc_lsst.multi INFO: [multi] job_id 17075360 2024-03-15 13:53:00 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 221 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 221 2024-03-15 13:53:00 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:00 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075360 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075360 parsl.executors.status_handling DEBUG: Launched block 276 on executor multi with job ID 17075360 parsl.executors.status_handling INFO: Allocated block ID 277 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-277 2024-03-15 13:53:00 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-277 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-277 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:00 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:00 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 221 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 221 2024-03-15 13:53:00 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:00 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=277 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 790 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry faffe2ea6f23bd834643841fb67525d8 with result from task 790 parsl.dataflow.dflow DEBUG: Task 790 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/700cd715-a166-4943-b595-8e6c847049ab_isrForCrosstalkSources_996992_27.stdout parsl.dataflow.dflow INFO: Standard error for task 790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/700cd715-a166-4943-b595-8e6c847049ab_isrForCrosstalkSources_996992_27.stderr parsl.dataflow.dflow INFO: Task 782 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 306cf27890a22d4248b9863e8163ef9a with result from task 782 parsl.dataflow.dflow DEBUG: Task 782 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/b780aa8b-3d10-4c66-918f-662798d9304b_isrForCrosstalkSources_996967_37.stdout parsl.dataflow.dflow INFO: Standard error for task 782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/b780aa8b-3d10-4c66-918f-662798d9304b_isrForCrosstalkSources_996967_37.stderr parsl.dataflow.dflow INFO: Task 766 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a572cad1174dfd529f5a1025555bd16 with result from task 766 parsl.dataflow.dflow DEBUG: Task 766 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/45166a13-267f-4448-8f5a-7e33794df11c_isrForCrosstalkSources_996972_36.stdout parsl.dataflow.dflow INFO: Standard error for task 766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/45166a13-267f-4448-8f5a-7e33794df11c_isrForCrosstalkSources_996972_36.stderr parsl.dataflow.dflow INFO: Task 667 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4ea86205caaa4e46e5e75248e49c7c8 with result from task 667 parsl.dataflow.dflow DEBUG: Task 667 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/98b8409d-5955-42f4-a13e-69f7192d78a6_isrForCrosstalkSources_997018_17.stdout parsl.dataflow.dflow INFO: Standard error for task 667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/98b8409d-5955-42f4-a13e-69f7192d78a6_isrForCrosstalkSources_997018_17.stderr parsl.dataflow.dflow INFO: Task 781 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5cc3fd357893129b4091bb9cb9ee5090 with result from task 781 2024-03-15 13:53:01 proc_lsst.multi:170 [INFO] [multi] job_id 17075364 proc_lsst.multi INFO: [multi] job_id 17075364 2024-03-15 13:53:01 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 222 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 222 2024-03-15 13:53:01 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:01 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075364 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075364 parsl.executors.status_handling DEBUG: Launched block 277 on executor multi with job ID 17075364 parsl.executors.status_handling INFO: Allocated block ID 278 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-278 2024-03-15 13:53:01 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-278 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-278 2024-03-15 13:53:01 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:01 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:01 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:01 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:01 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:01 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 222 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 222 2024-03-15 13:53:01 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:01 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=278 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 781 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e8e831cd-ff0c-43b1-9273-7e3b666cf5e4_isrForCrosstalkSources_996993_16.stdout parsl.dataflow.dflow INFO: Standard error for task 781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e8e831cd-ff0c-43b1-9273-7e3b666cf5e4_isrForCrosstalkSources_996993_16.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 744 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9bee68cc0feb3b17383800e58374cdc2 with result from task 744 parsl.dataflow.dflow DEBUG: Task 744 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b3e85931-9c94-4708-8fbc-20b136d4da70_isrForCrosstalkSources_996972_50.stdout parsl.dataflow.dflow INFO: Standard error for task 744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b3e85931-9c94-4708-8fbc-20b136d4da70_isrForCrosstalkSources_996972_50.stderr parsl.dataflow.dflow INFO: Task 708 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fba6e8e7562bc7022a28d2024830575d with result from task 708 parsl.dataflow.dflow DEBUG: Task 708 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/179bce96-e714-4bde-aebd-05871fde32a2_isrForCrosstalkSources_996959_41.stdout parsl.dataflow.dflow INFO: Standard error for task 708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/179bce96-e714-4bde-aebd-05871fde32a2_isrForCrosstalkSources_996959_41.stderr parsl.dataflow.dflow INFO: Task 686 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39207d59dcd0a2a209a3e73fc2ab4069 with result from task 686 parsl.dataflow.dflow DEBUG: Task 686 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/44f2089c-212b-4c0c-adc7-e08f2a2db08c_isrForCrosstalkSources_997029_18.stdout parsl.dataflow.dflow INFO: Standard error for task 686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/44f2089c-212b-4c0c-adc7-e08f2a2db08c_isrForCrosstalkSources_997029_18.stderr 2024-03-15 13:53:02 proc_lsst.multi:170 [INFO] [multi] job_id 17075368 proc_lsst.multi INFO: [multi] job_id 17075368 2024-03-15 13:53:02 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 223 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 223 2024-03-15 13:53:02 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:02 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075368 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075368 parsl.executors.status_handling DEBUG: Launched block 278 on executor multi with job ID 17075368 parsl.executors.status_handling INFO: Allocated block ID 279 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-279 2024-03-15 13:53:02 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-279 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-279 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 223 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 223 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:02 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=279 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 775 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 28ed076030bd56d6a8505e7b1bebdde1 with result from task 775 parsl.dataflow.dflow DEBUG: Task 775 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/debf0699-64a4-43cf-accd-e49fe07a74e0_isrForCrosstalkSources_996977_15.stdout parsl.dataflow.dflow INFO: Standard error for task 775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/debf0699-64a4-43cf-accd-e49fe07a74e0_isrForCrosstalkSources_996977_15.stderr parsl.dataflow.dflow INFO: Task 739 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82711a38d8624030a82f707f0dd61eed with result from task 739 parsl.dataflow.dflow DEBUG: Task 739 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/71503f6d-5c98-4f17-a16e-0124f2b499e9_isrForCrosstalkSources_996986_6.stdout parsl.dataflow.dflow INFO: Standard error for task 739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/71503f6d-5c98-4f17-a16e-0124f2b499e9_isrForCrosstalkSources_996986_6.stderr parsl.dataflow.dflow INFO: Task 724 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ccd37ea0643cd81ded0574039c045b5 with result from task 724 parsl.dataflow.dflow DEBUG: Task 724 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cddaddde-7eb0-400e-a7b5-5d0a8972918c_isrForCrosstalkSources_996957_45.stdout parsl.dataflow.dflow INFO: Standard error for task 724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cddaddde-7eb0-400e-a7b5-5d0a8972918c_isrForCrosstalkSources_996957_45.stderr parsl.dataflow.dflow INFO: Task 729 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e0805b0fadba4cd19315d64f382c6bec with result from task 729 parsl.dataflow.dflow DEBUG: Task 729 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0fda6158-4a83-46ac-a7e6-149dad795d06_isrForCrosstalkSources_996958_39.stdout parsl.dataflow.dflow INFO: Standard error for task 729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0fda6158-4a83-46ac-a7e6-149dad795d06_isrForCrosstalkSources_996958_39.stderr parsl.dataflow.dflow INFO: Task 761 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6acc6e764ce8128074d3a49c2be6b52d with result from task 761 parsl.dataflow.dflow DEBUG: Task 761 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2fd96589-64e5-40ee-9b61-742a62dd8bbe_isrForCrosstalkSources_996989_33.stdout parsl.dataflow.dflow INFO: Standard error for task 761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2fd96589-64e5-40ee-9b61-742a62dd8bbe_isrForCrosstalkSources_996989_33.stderr parsl.dataflow.dflow INFO: Task 723 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 80fbe28f4343667c5369b5c1f617393a with result from task 723 parsl.dataflow.dflow DEBUG: Task 723 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/cd911220-a76b-48ce-b002-d20e91cb4b8c_isrForCrosstalkSources_996972_24.stdout parsl.dataflow.dflow INFO: Standard error for task 723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/cd911220-a76b-48ce-b002-d20e91cb4b8c_isrForCrosstalkSources_996972_24.stderr parsl.dataflow.dflow INFO: Task 779 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 508f10ce245bb447d656b76e4c085b7f with result from task 779 parsl.dataflow.dflow DEBUG: Task 779 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c1bb0696-daae-4774-b325-e0be466157d0_isrForCrosstalkSources_996977_39.stdout parsl.dataflow.dflow INFO: Standard error for task 779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c1bb0696-daae-4774-b325-e0be466157d0_isrForCrosstalkSources_996977_39.stderr 2024-03-15 13:53:02 proc_lsst.multi:170 [INFO] [multi] job_id 17075372 proc_lsst.multi INFO: [multi] job_id 17075372 2024-03-15 13:53:02 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 224 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 224 2024-03-15 13:53:02 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:02 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075372 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075372 parsl.executors.status_handling DEBUG: Launched block 279 on executor multi with job ID 17075372 parsl.executors.status_handling INFO: Allocated block ID 280 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-280 2024-03-15 13:53:02 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-280 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-280 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:02 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:02 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 224 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 224 2024-03-15 13:53:02 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:02 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=280 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 704 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c32e99e0305773795959fab196cd94d with result from task 704 parsl.dataflow.dflow DEBUG: Task 704 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1c9296de-c7cb-46d9-addf-d5c6eb35dab0_isrForCrosstalkSources_996998_14.stdout parsl.dataflow.dflow INFO: Standard error for task 704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1c9296de-c7cb-46d9-addf-d5c6eb35dab0_isrForCrosstalkSources_996998_14.stderr parsl.dataflow.dflow INFO: Task 763 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a0d7909e1fd81c656c816f859e07575 with result from task 763 parsl.dataflow.dflow DEBUG: Task 763 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a4b86c7b-54af-41b6-a1a0-acea979d056e_isrForCrosstalkSources_996964_8.stdout parsl.dataflow.dflow INFO: Standard error for task 763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a4b86c7b-54af-41b6-a1a0-acea979d056e_isrForCrosstalkSources_996964_8.stderr 2024-03-15 13:53:03 proc_lsst.multi:170 [INFO] [multi] job_id 17075376 proc_lsst.multi INFO: [multi] job_id 17075376 2024-03-15 13:53:03 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 225 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 225 2024-03-15 13:53:03 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:03 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075376 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075376 parsl.executors.status_handling DEBUG: Launched block 280 on executor multi with job ID 17075376 parsl.executors.status_handling INFO: Allocated block ID 281 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-281 2024-03-15 13:53:03 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-281 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-281 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 225 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 225 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:03 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=281 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 681 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8b7afb1da7b938ae39055dda088c509 with result from task 681 parsl.dataflow.dflow DEBUG: Task 681 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/91b49bd0-1c1a-4688-9cb4-e3c1ac025c74_isrForCrosstalkSources_996969_35.stdout parsl.dataflow.dflow INFO: Standard error for task 681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/91b49bd0-1c1a-4688-9cb4-e3c1ac025c74_isrForCrosstalkSources_996969_35.stderr 2024-03-15 13:53:03 proc_lsst.multi:170 [INFO] [multi] job_id 17075379 proc_lsst.multi INFO: [multi] job_id 17075379 2024-03-15 13:53:03 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 226 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 226 2024-03-15 13:53:03 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:03 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075379 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075379 parsl.executors.status_handling DEBUG: Launched block 281 on executor multi with job ID 17075379 parsl.executors.status_handling INFO: Allocated block ID 282 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-282 2024-03-15 13:53:03 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-282 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-282 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:03 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:03 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 226 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 226 2024-03-15 13:53:03 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:03 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=282 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 649 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6eceac8f4a141e2a9b642bb3e48f80f6 with result from task 649 parsl.dataflow.dflow DEBUG: Task 649 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f02c1d17-412b-4f93-94de-a34e55ba132a_isrForCrosstalkSources_997031_12.stdout parsl.dataflow.dflow INFO: Standard error for task 649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f02c1d17-412b-4f93-94de-a34e55ba132a_isrForCrosstalkSources_997031_12.stderr parsl.dataflow.dflow INFO: Task 799 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 76217ff52a35847dc94a4ef3073ed6cc with result from task 799 parsl.dataflow.dflow DEBUG: Task 799 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/8ecfeb72-a41a-4dfc-8584-a94b5692a41b_isrForCrosstalkSources_996962_4.stdout parsl.dataflow.dflow INFO: Standard error for task 799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/8ecfeb72-a41a-4dfc-8584-a94b5692a41b_isrForCrosstalkSources_996962_4.stderr parsl.dataflow.dflow INFO: Task 697 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ba4216315fd71e93c773bc9752affde with result from task 697 parsl.dataflow.dflow DEBUG: Task 697 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/8749896f-1486-456f-b290-f20bfdd265a1_isrForCrosstalkSources_996958_2.stdout parsl.dataflow.dflow INFO: Standard error for task 697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/8749896f-1486-456f-b290-f20bfdd265a1_isrForCrosstalkSources_996958_2.stderr parsl.dataflow.dflow INFO: Task 679 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 311bec8e2a42e6be1a0d25ff3856c0a1 with result from task 679 parsl.dataflow.dflow DEBUG: Task 679 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7a8cac0a-8d43-4a9f-b4d2-0c16e8d03bad_isrForCrosstalkSources_996994_18.stdout parsl.dataflow.dflow INFO: Standard error for task 679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7a8cac0a-8d43-4a9f-b4d2-0c16e8d03bad_isrForCrosstalkSources_996994_18.stderr parsl.dataflow.dflow INFO: Task 722 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c59df0c2479d4d6c7fbf0627b89e2a4 with result from task 722 parsl.dataflow.dflow DEBUG: Task 722 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/775a802d-a535-4ace-89ef-777968dabb9e_isrForCrosstalkSources_997007_22.stdout parsl.dataflow.dflow INFO: Standard error for task 722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/775a802d-a535-4ace-89ef-777968dabb9e_isrForCrosstalkSources_997007_22.stderr parsl.dataflow.dflow INFO: Task 768 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 436b8287878fb8c9ac1f4429c1698457 with result from task 768 parsl.dataflow.dflow DEBUG: Task 768 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2e1718f5-4cfc-4944-aa03-c8dce4e0494b_isrForCrosstalkSources_996956_9.stdout parsl.dataflow.dflow INFO: Standard error for task 768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2e1718f5-4cfc-4944-aa03-c8dce4e0494b_isrForCrosstalkSources_996956_9.stderr parsl.dataflow.dflow INFO: Task 719 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 599b249eac47d6cb0879004ad577d711 with result from task 719 parsl.dataflow.dflow DEBUG: Task 719 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b227bab8-5490-44de-a88f-54601b12eb04_isrForCrosstalkSources_997016_16.stdout parsl.dataflow.dflow INFO: Standard error for task 719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b227bab8-5490-44de-a88f-54601b12eb04_isrForCrosstalkSources_997016_16.stderr 2024-03-15 13:53:04 proc_lsst.multi:170 [INFO] [multi] job_id 17075380 proc_lsst.multi INFO: [multi] job_id 17075380 2024-03-15 13:53:04 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 227 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 227 2024-03-15 13:53:04 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:04 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075380 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075380 parsl.executors.status_handling DEBUG: Launched block 282 on executor multi with job ID 17075380 parsl.executors.status_handling INFO: Allocated block ID 283 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-283 2024-03-15 13:53:04 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-283 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-283 2024-03-15 13:53:04 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:04 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:04 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:04 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:04 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:04 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 227 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 227 2024-03-15 13:53:04 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:04 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=283 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 795 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc300fc44b6ed377655c32bbc22fce0b with result from task 795 parsl.dataflow.dflow DEBUG: Task 795 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/58d0c6e7-d862-4666-bc02-d4310f44e0c7_isrForCrosstalkSources_996977_24.stdout parsl.dataflow.dflow INFO: Standard error for task 795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/58d0c6e7-d862-4666-bc02-d4310f44e0c7_isrForCrosstalkSources_996977_24.stderr parsl.dataflow.dflow INFO: Task 716 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d894a61eb609554949179b0c5981ad6 with result from task 716 parsl.dataflow.dflow DEBUG: Task 716 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/15bae4ec-4f64-4f05-850e-37b0370e99cf_isrForCrosstalkSources_996954_21.stdout parsl.dataflow.dflow INFO: Standard error for task 716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/15bae4ec-4f64-4f05-850e-37b0370e99cf_isrForCrosstalkSources_996954_21.stderr parsl.dataflow.dflow INFO: Task 736 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a40757c23f84f3d1fa425c4ce099293 with result from task 736 parsl.dataflow.dflow DEBUG: Task 736 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/fdf5b9fa-fd81-48ff-b021-38dbeeafca75_isrForCrosstalkSources_996985_38.stdout parsl.dataflow.dflow INFO: Standard error for task 736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/fdf5b9fa-fd81-48ff-b021-38dbeeafca75_isrForCrosstalkSources_996985_38.stderr parsl.dataflow.dflow INFO: Task 773 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2ddcddacade1351b9c3b3f29647a05d with result from task 773 parsl.dataflow.dflow DEBUG: Task 773 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/87b4c794-4d93-45a7-8678-753d50bae62e_isrForCrosstalkSources_996962_6.stdout parsl.dataflow.dflow INFO: Standard error for task 773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/87b4c794-4d93-45a7-8678-753d50bae62e_isrForCrosstalkSources_996962_6.stderr 2024-03-15 13:53:05 proc_lsst.multi:170 [INFO] [multi] job_id 17075381 proc_lsst.multi INFO: [multi] job_id 17075381 2024-03-15 13:53:05 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 228 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 228 2024-03-15 13:53:05 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:05 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075381 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075381 parsl.executors.status_handling DEBUG: Launched block 283 on executor multi with job ID 17075381 parsl.executors.status_handling INFO: Allocated block ID 284 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-284 2024-03-15 13:53:05 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-284 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-284 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 228 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 228 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:05 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=284 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 699 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6fa4c1a250c443b3fd2bc754cbf2540 with result from task 699 parsl.dataflow.dflow DEBUG: Task 699 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7beca4d5-7630-4f98-8ba7-51f910f11463_isrForCrosstalkSources_997003_53.stdout parsl.dataflow.dflow INFO: Standard error for task 699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/7beca4d5-7630-4f98-8ba7-51f910f11463_isrForCrosstalkSources_997003_53.stderr parsl.dataflow.dflow INFO: Task 684 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abb51fad989b4c89e64aa8e7decefc89 with result from task 684 parsl.dataflow.dflow DEBUG: Task 684 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/301f955c-1ad0-4452-8452-8429c2bf6e48_isrForCrosstalkSources_996996_53.stdout parsl.dataflow.dflow INFO: Standard error for task 684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/301f955c-1ad0-4452-8452-8429c2bf6e48_isrForCrosstalkSources_996996_53.stderr parsl.dataflow.dflow INFO: Task 702 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba82a2c1ce0c3f1191f9fe3691d19318 with result from task 702 parsl.dataflow.dflow DEBUG: Task 702 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/20b77816-2e70-479c-ab74-170471c55479_isrForCrosstalkSources_996985_49.stdout parsl.dataflow.dflow INFO: Standard error for task 702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/20b77816-2e70-479c-ab74-170471c55479_isrForCrosstalkSources_996985_49.stderr parsl.dataflow.dflow INFO: Task 661 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91512e256b0b9f3a98ab2036985e67db with result from task 661 parsl.dataflow.dflow DEBUG: Task 661 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/4b92a6a6-cb85-47d7-b09b-1f807156d17f_isrForCrosstalkSources_997016_5.stdout parsl.dataflow.dflow INFO: Standard error for task 661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/4b92a6a6-cb85-47d7-b09b-1f807156d17f_isrForCrosstalkSources_997016_5.stderr parsl.dataflow.dflow INFO: Task 813 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8a555856c5c8527db9c2f8cbaa05627 with result from task 813 parsl.dataflow.dflow DEBUG: Task 813 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/246e1e9b-918c-4077-a5b2-a62816bc909e_isrForCrosstalkSources_997024_23.stdout parsl.dataflow.dflow INFO: Standard error for task 813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/246e1e9b-918c-4077-a5b2-a62816bc909e_isrForCrosstalkSources_997024_23.stderr parsl.dataflow.dflow INFO: Task 747 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c16d8caebbe903703396d56ef567369 with result from task 747 parsl.dataflow.dflow DEBUG: Task 747 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/84dbcd9a-094d-4594-a4c7-2301a8c76629_isrForCrosstalkSources_996969_29.stdout parsl.dataflow.dflow INFO: Standard error for task 747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/84dbcd9a-094d-4594-a4c7-2301a8c76629_isrForCrosstalkSources_996969_29.stderr parsl.dataflow.dflow INFO: Task 780 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d451d93c5901754e84b24018d34a4742 with result from task 780 parsl.dataflow.dflow DEBUG: Task 780 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/545db4f5-47dc-4582-94ee-625ac165677f_isrForCrosstalkSources_996990_10.stdout parsl.dataflow.dflow INFO: Standard error for task 780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/545db4f5-47dc-4582-94ee-625ac165677f_isrForCrosstalkSources_996990_10.stderr parsl.dataflow.dflow INFO: Task 746 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e5d66479190f63d9ff0da46c284d73f with result from task 746 parsl.dataflow.dflow DEBUG: Task 746 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/10af904c-7c2b-47b4-80e5-c3237d265540_isrForCrosstalkSources_996964_30.stdout parsl.dataflow.dflow INFO: Standard error for task 746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/10af904c-7c2b-47b4-80e5-c3237d265540_isrForCrosstalkSources_996964_30.stderr parsl.dataflow.dflow INFO: Task 783 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 697e0bb20faf4ea5ad604283c5df10be with result from task 783 parsl.dataflow.dflow DEBUG: Task 783 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d753e2e6-c4dd-4ef3-94c3-3c60bca29be7_isrForCrosstalkSources_996985_15.stdout parsl.dataflow.dflow INFO: Standard error for task 783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d753e2e6-c4dd-4ef3-94c3-3c60bca29be7_isrForCrosstalkSources_996985_15.stderr parsl.dataflow.dflow INFO: Task 653 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e98204978d27bbd67d09cb5554e4b83 with result from task 653 parsl.dataflow.dflow DEBUG: Task 653 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ef058422-086a-4ce2-bf5d-9eabe27128bb_isrForCrosstalkSources_996980_56.stdout parsl.dataflow.dflow INFO: Standard error for task 653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ef058422-086a-4ce2-bf5d-9eabe27128bb_isrForCrosstalkSources_996980_56.stderr parsl.dataflow.dflow INFO: Task 657 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e65566a4837094ac1441a797b5c0b7d7 with result from task 657 parsl.dataflow.dflow DEBUG: Task 657 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/bce97935-7ad9-4808-b21c-5bc3d74e4fc1_isrForCrosstalkSources_997003_29.stdout parsl.dataflow.dflow INFO: Standard error for task 657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/bce97935-7ad9-4808-b21c-5bc3d74e4fc1_isrForCrosstalkSources_997003_29.stderr 2024-03-15 13:53:05 proc_lsst.multi:170 [INFO] [multi] job_id 17075382 proc_lsst.multi INFO: [multi] job_id 17075382 2024-03-15 13:53:05 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 229 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 229 2024-03-15 13:53:05 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:05 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075382 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075382 parsl.executors.status_handling DEBUG: Launched block 284 on executor multi with job ID 17075382 parsl.executors.status_handling INFO: Allocated block ID 285 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-285 2024-03-15 13:53:05 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-285 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-285 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:05 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:05 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 229 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 229 2024-03-15 13:53:05 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:05 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=285 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 760 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 995978100c0cdc4e5587f1764931beb2 with result from task 760 parsl.dataflow.dflow DEBUG: Task 760 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/fce7f8a5-da10-4148-b7b3-da1d861588b4_isrForCrosstalkSources_996982_4.stdout parsl.dataflow.dflow INFO: Standard error for task 760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/fce7f8a5-da10-4148-b7b3-da1d861588b4_isrForCrosstalkSources_996982_4.stderr parsl.dataflow.dflow INFO: Task 748 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1352c14c28ab6ee2d2dd868745a3a661 with result from task 748 parsl.dataflow.dflow DEBUG: Task 748 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1139286-4e58-4369-978d-d4d2137d777c_isrForCrosstalkSources_996975_5.stdout parsl.dataflow.dflow INFO: Standard error for task 748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1139286-4e58-4369-978d-d4d2137d777c_isrForCrosstalkSources_996975_5.stderr parsl.dataflow.dflow INFO: Task 715 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 133b89e2ea80e2aa4107720cbe59d8a7 with result from task 715 parsl.dataflow.dflow DEBUG: Task 715 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7dae67b8-eac3-4394-bd80-13227a1f7526_isrForCrosstalkSources_996993_36.stdout parsl.dataflow.dflow INFO: Standard error for task 715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7dae67b8-eac3-4394-bd80-13227a1f7526_isrForCrosstalkSources_996993_36.stderr parsl.dataflow.dflow INFO: Task 757 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 370dc73d017e3c0cfe52536428b85c0d with result from task 757 parsl.dataflow.dflow DEBUG: Task 757 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9434eb78-b555-4660-b271-6977c4f16001_isrForCrosstalkSources_996968_56.stdout parsl.dataflow.dflow INFO: Standard error for task 757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9434eb78-b555-4660-b271-6977c4f16001_isrForCrosstalkSources_996968_56.stderr parsl.dataflow.dflow INFO: Task 791 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 951d17adba18ca9bdc09cb31aaf51546 with result from task 791 parsl.dataflow.dflow DEBUG: Task 791 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b6562ccc-e5ab-492b-8f72-061b0f564a5f_isrForCrosstalkSources_997025_59.stdout parsl.dataflow.dflow INFO: Standard error for task 791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b6562ccc-e5ab-492b-8f72-061b0f564a5f_isrForCrosstalkSources_997025_59.stderr 2024-03-15 13:53:06 proc_lsst.multi:170 [INFO] [multi] job_id 17075384 proc_lsst.multi INFO: [multi] job_id 17075384 2024-03-15 13:53:06 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 230 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 230 2024-03-15 13:53:06 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:06 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075384 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075384 parsl.executors.status_handling DEBUG: Launched block 285 on executor multi with job ID 17075384 parsl.executors.status_handling INFO: Allocated block ID 286 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-286 2024-03-15 13:53:06 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-286 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-286 2024-03-15 13:53:06 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:06 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:06 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:06 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:06 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:06 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 230 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 230 2024-03-15 13:53:06 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:06 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=286 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 691 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f09c13fe2d87f0dd5a5129eee578079 with result from task 691 parsl.dataflow.dflow DEBUG: Task 691 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/34f7622f-9021-4910-8bfd-60bdfc40c88d_isrForCrosstalkSources_997000_15.stdout parsl.dataflow.dflow INFO: Standard error for task 691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/34f7622f-9021-4910-8bfd-60bdfc40c88d_isrForCrosstalkSources_997000_15.stderr parsl.dataflow.dflow INFO: Task 642 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d616f6997ab5e08d502fd0a548ceff7 with result from task 642 parsl.dataflow.dflow DEBUG: Task 642 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/846bd7b3-69d1-4dbf-8887-2b95a0b221c3_isrForCrosstalkSources_996999_5.stdout parsl.dataflow.dflow INFO: Standard error for task 642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/846bd7b3-69d1-4dbf-8887-2b95a0b221c3_isrForCrosstalkSources_996999_5.stderr parsl.dataflow.dflow INFO: Task 643 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 336df94ab49025f5e7c1e19dd5c3a861 with result from task 643 parsl.dataflow.dflow DEBUG: Task 643 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/6dcba814-aa6b-4d5e-a507-5a302056a4d2_isrForCrosstalkSources_996984_16.stdout parsl.dataflow.dflow INFO: Standard error for task 643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/6dcba814-aa6b-4d5e-a507-5a302056a4d2_isrForCrosstalkSources_996984_16.stderr parsl.dataflow.dflow INFO: Task 745 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 771b0089ce83b0678771e5dd2daf2b9c with result from task 745 parsl.dataflow.dflow DEBUG: Task 745 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/d2e3b29f-0ba3-43e9-8169-72ff63dcf368_isrForCrosstalkSources_997018_36.stdout parsl.dataflow.dflow INFO: Standard error for task 745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/d2e3b29f-0ba3-43e9-8169-72ff63dcf368_isrForCrosstalkSources_997018_36.stderr parsl.dataflow.dflow INFO: Task 645 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac71fbf6e67e9df755ca5336aed49a0e with result from task 645 parsl.dataflow.dflow DEBUG: Task 645 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/61495e3b-748d-4f77-ad13-177073a2941d_isrForCrosstalkSources_996967_12.stdout parsl.dataflow.dflow INFO: Standard error for task 645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/61495e3b-748d-4f77-ad13-177073a2941d_isrForCrosstalkSources_996967_12.stderr parsl.dataflow.dflow INFO: Task 628 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d23d5c53b6c7ca007bb15991c92ca265 with result from task 628 parsl.dataflow.dflow DEBUG: Task 628 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ba028b4a-40d6-4e2f-9a95-553106e31df5_isrForCrosstalkSources_996988_9.stdout parsl.dataflow.dflow INFO: Standard error for task 628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ba028b4a-40d6-4e2f-9a95-553106e31df5_isrForCrosstalkSources_996988_9.stderr parsl.dataflow.dflow INFO: Task 742 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a10241ba52dffe645b701433836ce7a with result from task 742 parsl.dataflow.dflow DEBUG: Task 742 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2bf62534-e40a-4a13-be76-a6935a3e4234_isrForCrosstalkSources_997003_61.stdout parsl.dataflow.dflow INFO: Standard error for task 742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2bf62534-e40a-4a13-be76-a6935a3e4234_isrForCrosstalkSources_997003_61.stderr parsl.dataflow.dflow INFO: Task 810 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e94e0b48e77e4dba39dddcd4d24e264a with result from task 810 2024-03-15 13:53:07 proc_lsst.multi:170 [INFO] [multi] job_id 17075385 proc_lsst.multi INFO: [multi] job_id 17075385 2024-03-15 13:53:07 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 231 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 231 2024-03-15 13:53:07 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:07 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075385 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075385 parsl.executors.status_handling DEBUG: Launched block 286 on executor multi with job ID 17075385 parsl.executors.status_handling INFO: Allocated block ID 287 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-287 2024-03-15 13:53:07 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-287 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-287 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 231 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 231 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:07 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=287 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow DEBUG: Task 810 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7b3501ae-d361-4032-a20f-355b031c92a4_isrForCrosstalkSources_996973_36.stdout parsl.dataflow.dflow INFO: Standard error for task 810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7b3501ae-d361-4032-a20f-355b031c92a4_isrForCrosstalkSources_996973_36.stderr parsl.dataflow.dflow INFO: Task 793 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70e551464c81756abde88f3645f9dac2 with result from task 793 parsl.dataflow.dflow DEBUG: Task 793 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/44d479f7-29a5-4c88-b070-f6c9cfbcafac_isrForCrosstalkSources_996982_52.stdout parsl.dataflow.dflow INFO: Standard error for task 793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/44d479f7-29a5-4c88-b070-f6c9cfbcafac_isrForCrosstalkSources_996982_52.stderr parsl.dataflow.dflow INFO: Task 714 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac661ad56c269fa6ac759a05723a6e24 with result from task 714 parsl.dataflow.dflow DEBUG: Task 714 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/19e1b954-1026-4ba2-89a5-c0931ca434cf_isrForCrosstalkSources_997012_54.stdout parsl.dataflow.dflow INFO: Standard error for task 714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/19e1b954-1026-4ba2-89a5-c0931ca434cf_isrForCrosstalkSources_997012_54.stderr parsl.dataflow.dflow INFO: Task 734 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4231f9b013438081de8f13a3e68e12bb with result from task 734 parsl.dataflow.dflow DEBUG: Task 734 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/ac360468-9c08-4fd6-9782-9fbbd6c213ba_isrForCrosstalkSources_997004_12.stdout parsl.dataflow.dflow INFO: Standard error for task 734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/ac360468-9c08-4fd6-9782-9fbbd6c213ba_isrForCrosstalkSources_997004_12.stderr 2024-03-15 13:53:07 proc_lsst.multi:170 [INFO] [multi] job_id 17075386 proc_lsst.multi INFO: [multi] job_id 17075386 2024-03-15 13:53:07 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 232 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 232 2024-03-15 13:53:07 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:07 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075386 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075386 parsl.executors.status_handling DEBUG: Launched block 287 on executor multi with job ID 17075386 parsl.executors.status_handling INFO: Allocated block ID 288 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-288 2024-03-15 13:53:07 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-288 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-288 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:07 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:07 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 232 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 232 2024-03-15 13:53:07 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:07 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=288 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 765 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4c053c121ad483479e4e3c63ee0cc90 with result from task 765 parsl.dataflow.dflow DEBUG: Task 765 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/32a42997-e2bf-41a7-ab7c-ebc48f43061c_isrForCrosstalkSources_996958_5.stdout parsl.dataflow.dflow INFO: Standard error for task 765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/32a42997-e2bf-41a7-ab7c-ebc48f43061c_isrForCrosstalkSources_996958_5.stderr parsl.dataflow.dflow INFO: Task 733 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81e3d94b82a19b8b7d04cd0eb43a898b with result from task 733 parsl.dataflow.dflow DEBUG: Task 733 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/aebd9bac-c45d-4f37-9c0c-0b23144198a0_isrForCrosstalkSources_996966_43.stdout parsl.dataflow.dflow INFO: Standard error for task 733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/aebd9bac-c45d-4f37-9c0c-0b23144198a0_isrForCrosstalkSources_996966_43.stderr parsl.dataflow.dflow INFO: Task 678 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f7e51198d398e39548649ec1c42b90b with result from task 678 parsl.dataflow.dflow DEBUG: Task 678 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ee724f90-b04f-48d9-90bb-607a055649be_isrForCrosstalkSources_997002_35.stdout parsl.dataflow.dflow INFO: Standard error for task 678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ee724f90-b04f-48d9-90bb-607a055649be_isrForCrosstalkSources_997002_35.stderr 2024-03-15 13:53:08 proc_lsst.multi:170 [INFO] [multi] job_id 17075387 proc_lsst.multi INFO: [multi] job_id 17075387 2024-03-15 13:53:08 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 233 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 233 2024-03-15 13:53:08 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:08 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075387 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075387 parsl.executors.status_handling DEBUG: Launched block 288 on executor multi with job ID 17075387 parsl.executors.status_handling INFO: Allocated block ID 289 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-289 2024-03-15 13:53:08 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-289 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-289 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 233 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 233 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:08 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=289 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 801 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b650adf849e98f10488a520f5aa01b06 with result from task 801 parsl.dataflow.dflow DEBUG: Task 801 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/006c502f-e828-4527-8d71-31812f33eaef_isrForCrosstalkSources_996958_15.stdout parsl.dataflow.dflow INFO: Standard error for task 801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/006c502f-e828-4527-8d71-31812f33eaef_isrForCrosstalkSources_996958_15.stderr parsl.dataflow.dflow INFO: Task 709 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f8861b5f6dc07ee4f862b49176526611 with result from task 709 parsl.dataflow.dflow DEBUG: Task 709 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7665e6fe-9f69-491f-a215-99f4aa00c887_isrForCrosstalkSources_996989_48.stdout parsl.dataflow.dflow INFO: Standard error for task 709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7665e6fe-9f69-491f-a215-99f4aa00c887_isrForCrosstalkSources_996989_48.stderr 2024-03-15 13:53:08 proc_lsst.multi:170 [INFO] [multi] job_id 17075388 proc_lsst.multi INFO: [multi] job_id 17075388 2024-03-15 13:53:08 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 234 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 234 2024-03-15 13:53:08 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:08 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075388 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075388 parsl.executors.status_handling DEBUG: Launched block 289 on executor multi with job ID 17075388 parsl.executors.status_handling INFO: Allocated block ID 290 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-290 2024-03-15 13:53:08 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-290 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-290 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:08 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:08 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 234 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 234 2024-03-15 13:53:08 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:08 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=290 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 696 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f858753a92ec661f0e0c2edf0ac1b208 with result from task 696 parsl.dataflow.dflow DEBUG: Task 696 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/af962c92-5307-4ad8-a3e8-be6425543a47_isrForCrosstalkSources_997005_48.stdout parsl.dataflow.dflow INFO: Standard error for task 696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/af962c92-5307-4ad8-a3e8-be6425543a47_isrForCrosstalkSources_997005_48.stderr parsl.dataflow.dflow INFO: Task 822 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 083afae08706653b9de6adf13ca2d8ea with result from task 822 parsl.dataflow.dflow DEBUG: Task 822 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/ebd52f2f-849b-4e7c-877f-a1c4cc38208e_isrForCrosstalkSources_997026_15.stdout parsl.dataflow.dflow INFO: Standard error for task 822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/ebd52f2f-849b-4e7c-877f-a1c4cc38208e_isrForCrosstalkSources_997026_15.stderr parsl.dataflow.dflow INFO: Task 767 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8bdc6f828981ad07504acbc3e0c78d15 with result from task 767 parsl.dataflow.dflow DEBUG: Task 767 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/202f49a2-3b3f-49ad-8654-5cb95e6ac5ff_isrForCrosstalkSources_997009_28.stdout parsl.dataflow.dflow INFO: Standard error for task 767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/202f49a2-3b3f-49ad-8654-5cb95e6ac5ff_isrForCrosstalkSources_997009_28.stderr parsl.dataflow.dflow INFO: Task 672 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e5dadbd677556af4350d654ee85a9766 with result from task 672 parsl.dataflow.dflow DEBUG: Task 672 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/aebd6608-5238-4177-85b1-985460e884b6_isrForCrosstalkSources_996963_48.stdout parsl.dataflow.dflow INFO: Standard error for task 672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/aebd6608-5238-4177-85b1-985460e884b6_isrForCrosstalkSources_996963_48.stderr parsl.dataflow.dflow INFO: Task 770 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 716d5d912c3f84e0700aa3e79106ce1c with result from task 770 parsl.dataflow.dflow DEBUG: Task 770 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/71766269-67a9-4c01-8495-3e818c8ab35d_isrForCrosstalkSources_997025_43.stdout parsl.dataflow.dflow INFO: Standard error for task 770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/71766269-67a9-4c01-8495-3e818c8ab35d_isrForCrosstalkSources_997025_43.stderr parsl.dataflow.dflow INFO: Task 774 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bbefd56808831a3954dcac778d929391 with result from task 774 parsl.dataflow.dflow DEBUG: Task 774 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/badde207-b509-4d67-88e3-5250e1242828_isrForCrosstalkSources_997014_57.stdout parsl.dataflow.dflow INFO: Standard error for task 774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/badde207-b509-4d67-88e3-5250e1242828_isrForCrosstalkSources_997014_57.stderr 2024-03-15 13:53:09 proc_lsst.multi:170 [INFO] [multi] job_id 17075392 proc_lsst.multi INFO: [multi] job_id 17075392 2024-03-15 13:53:09 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 235 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 235 2024-03-15 13:53:09 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:09 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075392 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075392 parsl.executors.status_handling DEBUG: Launched block 290 on executor multi with job ID 17075392 parsl.executors.status_handling INFO: Allocated block ID 291 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-291 2024-03-15 13:53:09 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-291 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-291 2024-03-15 13:53:09 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:09 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:09 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:09 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:09 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:09 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 235 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 235 2024-03-15 13:53:09 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:09 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=291 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 832 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa734ab9b94b4a23c4feb2268c37ba88 with result from task 832 parsl.dataflow.dflow DEBUG: Task 832 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9d5edf9a-f23b-4ab3-a4f0-955f9a8e3f0e_isrForCrosstalkSources_996986_40.stdout parsl.dataflow.dflow INFO: Standard error for task 832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9d5edf9a-f23b-4ab3-a4f0-955f9a8e3f0e_isrForCrosstalkSources_996986_40.stderr parsl.dataflow.dflow INFO: Task 752 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a17f7919c5bcbd40a8a53ea0eb2cd087 with result from task 752 parsl.dataflow.dflow DEBUG: Task 752 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e22e2eba-f3be-4ec1-8158-705cafa11ccd_isrForCrosstalkSources_997016_18.stdout parsl.dataflow.dflow INFO: Standard error for task 752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e22e2eba-f3be-4ec1-8158-705cafa11ccd_isrForCrosstalkSources_997016_18.stderr parsl.dataflow.dflow INFO: Task 718 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b06bc667de5ce66853ae1ea6707c5e0f with result from task 718 parsl.dataflow.dflow DEBUG: Task 718 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/8bf7653a-b6e5-4ebc-91bc-3b4a71629c1e_isrForCrosstalkSources_997010_44.stdout parsl.dataflow.dflow INFO: Standard error for task 718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/8bf7653a-b6e5-4ebc-91bc-3b4a71629c1e_isrForCrosstalkSources_997010_44.stderr parsl.dataflow.dflow INFO: Task 640 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85896d41a086835458c8581e0107e276 with result from task 640 parsl.dataflow.dflow DEBUG: Task 640 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8612bf38-2d88-4d92-9612-96d779132327_isrForCrosstalkSources_996967_54.stdout parsl.dataflow.dflow INFO: Standard error for task 640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8612bf38-2d88-4d92-9612-96d779132327_isrForCrosstalkSources_996967_54.stderr parsl.dataflow.dflow INFO: Task 720 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53d19db1e1c87a3adeb4c0d2146f903a with result from task 720 parsl.dataflow.dflow DEBUG: Task 720 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/29679dab-e3a6-499c-aace-b1226520c540_isrForCrosstalkSources_997030_35.stdout parsl.dataflow.dflow INFO: Standard error for task 720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/29679dab-e3a6-499c-aace-b1226520c540_isrForCrosstalkSources_997030_35.stderr parsl.dataflow.dflow INFO: Task 756 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 09df370561300dc719dd5ea3a0d1a264 with result from task 756 parsl.dataflow.dflow DEBUG: Task 756 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/6cbf75e4-74c1-4d02-abcd-67336408aaef_isrForCrosstalkSources_996978_23.stdout parsl.dataflow.dflow INFO: Standard error for task 756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/6cbf75e4-74c1-4d02-abcd-67336408aaef_isrForCrosstalkSources_996978_23.stderr parsl.dataflow.dflow INFO: Task 633 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df3cf6565b9655060768d701e9b1844a with result from task 633 parsl.dataflow.dflow DEBUG: Task 633 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/91e45c7e-58d9-4037-83c1-218e17130dd1_isrForCrosstalkSources_996996_28.stdout parsl.dataflow.dflow INFO: Standard error for task 633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/91e45c7e-58d9-4037-83c1-218e17130dd1_isrForCrosstalkSources_996996_28.stderr parsl.dataflow.dflow INFO: Task 762 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db2278a08db7cd1fd848eb8f63c3864a with result from task 762 parsl.dataflow.dflow DEBUG: Task 762 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/828843b4-dab3-4146-bfe7-3a63447088ec_isrForCrosstalkSources_997003_6.stdout parsl.dataflow.dflow INFO: Standard error for task 762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/828843b4-dab3-4146-bfe7-3a63447088ec_isrForCrosstalkSources_997003_6.stderr parsl.dataflow.dflow INFO: Task 764 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d60f984655bac47b845b2f56dd1e99a0 with result from task 764 parsl.dataflow.dflow DEBUG: Task 764 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/4276a8f6-5e75-4b12-b29a-1d60776c3ab2_isrForCrosstalkSources_997003_28.stdout parsl.dataflow.dflow INFO: Standard error for task 764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/4276a8f6-5e75-4b12-b29a-1d60776c3ab2_isrForCrosstalkSources_997003_28.stderr parsl.dataflow.dflow INFO: Task 731 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 214a9bddb05b121f98202555a695cc28 with result from task 731 parsl.dataflow.dflow DEBUG: Task 731 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/2c305899-5c07-42ea-a995-7361ef1bfb20_isrForCrosstalkSources_996986_36.stdout parsl.dataflow.dflow INFO: Standard error for task 731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/2c305899-5c07-42ea-a995-7361ef1bfb20_isrForCrosstalkSources_996986_36.stderr parsl.dataflow.dflow INFO: Task 677 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a1070ca590f816aacde4adec7748fb3 with result from task 677 parsl.dataflow.dflow DEBUG: Task 677 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ee3fd40f-a58c-4c0c-8c21-1a5fbe9b301c_isrForCrosstalkSources_996991_48.stdout parsl.dataflow.dflow INFO: Standard error for task 677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ee3fd40f-a58c-4c0c-8c21-1a5fbe9b301c_isrForCrosstalkSources_996991_48.stderr parsl.dataflow.dflow INFO: Task 817 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7390a322ba03540a7e0a97cd94420339 with result from task 817 parsl.dataflow.dflow DEBUG: Task 817 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/9be6e759-af82-4494-8656-bb07fe2defc2_isrForCrosstalkSources_996980_5.stdout parsl.dataflow.dflow INFO: Standard error for task 817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/9be6e759-af82-4494-8656-bb07fe2defc2_isrForCrosstalkSources_996980_5.stderr 2024-03-15 13:53:10 proc_lsst.multi:170 [INFO] [multi] job_id 17075394 proc_lsst.multi INFO: [multi] job_id 17075394 2024-03-15 13:53:10 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 236 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 236 2024-03-15 13:53:10 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:10 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075394 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075394 parsl.executors.status_handling DEBUG: Launched block 291 on executor multi with job ID 17075394 parsl.executors.status_handling INFO: Allocated block ID 292 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-292 2024-03-15 13:53:10 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-292 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-292 2024-03-15 13:53:10 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:10 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:10 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:10 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:10 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:10 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 236 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 236 2024-03-15 13:53:10 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:10 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=292 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 759 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e82c38920000adef0746513b25bb2b4e with result from task 759 parsl.dataflow.dflow DEBUG: Task 759 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0b908e9f-fe75-4820-87e9-fd356e1cd22f_isrForCrosstalkSources_996997_9.stdout parsl.dataflow.dflow INFO: Standard error for task 759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0b908e9f-fe75-4820-87e9-fd356e1cd22f_isrForCrosstalkSources_996997_9.stderr parsl.dataflow.dflow INFO: Task 711 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7580fdc2b9fd7450001d408a81fe7e79 with result from task 711 parsl.dataflow.dflow DEBUG: Task 711 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f1119573-4f98-40c0-89c1-8b337fdc0b8d_isrForCrosstalkSources_997032_37.stdout parsl.dataflow.dflow INFO: Standard error for task 711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f1119573-4f98-40c0-89c1-8b337fdc0b8d_isrForCrosstalkSources_997032_37.stderr parsl.dataflow.dflow INFO: Task 669 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce3b4657cd32688440c55f240d699fc6 with result from task 669 parsl.dataflow.dflow DEBUG: Task 669 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2d4da697-f669-47c6-b566-c4992377d23f_isrForCrosstalkSources_997007_38.stdout parsl.dataflow.dflow INFO: Standard error for task 669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2d4da697-f669-47c6-b566-c4992377d23f_isrForCrosstalkSources_997007_38.stderr parsl.dataflow.dflow INFO: Task 826 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a85efb35eba5608bd00f86afc2787021 with result from task 826 parsl.dataflow.dflow DEBUG: Task 826 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/34eeafc9-861c-4550-9423-cbb389edea2d_isrForCrosstalkSources_996957_36.stdout parsl.dataflow.dflow INFO: Standard error for task 826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/34eeafc9-861c-4550-9423-cbb389edea2d_isrForCrosstalkSources_996957_36.stderr parsl.dataflow.dflow INFO: Task 842 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3606504e08ba87c4547b0b8cd01359ff with result from task 842 parsl.dataflow.dflow DEBUG: Task 842 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c9e6976a-92c0-4856-a2ed-21520273d226_isrForCrosstalkSources_997016_58.stdout parsl.dataflow.dflow INFO: Standard error for task 842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c9e6976a-92c0-4856-a2ed-21520273d226_isrForCrosstalkSources_997016_58.stderr parsl.dataflow.dflow INFO: Task 712 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe0ecce70094d9b7ec24559b2fbaf977 with result from task 712 parsl.dataflow.dflow DEBUG: Task 712 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/3f436e99-cc32-45f3-9419-113603cc6227_isrForCrosstalkSources_996964_52.stdout parsl.dataflow.dflow INFO: Standard error for task 712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/3f436e99-cc32-45f3-9419-113603cc6227_isrForCrosstalkSources_996964_52.stderr parsl.dataflow.dflow INFO: Task 754 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry baf8378d6cc91ea66d80b6a14062343a with result from task 754 parsl.dataflow.dflow DEBUG: Task 754 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/252c5470-d1e5-4e8f-a9db-80ea1b7f9e93_isrForCrosstalkSources_996964_15.stdout parsl.dataflow.dflow INFO: Standard error for task 754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/252c5470-d1e5-4e8f-a9db-80ea1b7f9e93_isrForCrosstalkSources_996964_15.stderr 2024-03-15 13:53:11 proc_lsst.multi:170 [INFO] [multi] job_id 17075399 proc_lsst.multi INFO: [multi] job_id 17075399 2024-03-15 13:53:11 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 237 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 237 2024-03-15 13:53:11 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:11 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075399 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075399 parsl.executors.status_handling DEBUG: Launched block 292 on executor multi with job ID 17075399 parsl.executors.status_handling INFO: Allocated block ID 293 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-293 2024-03-15 13:53:11 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-293 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-293 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 237 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 237 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:11 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=293 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 776 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 335ce041791dd596f3cb78e4464d954e with result from task 776 parsl.dataflow.dflow DEBUG: Task 776 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/dde3d9f7-347b-4551-9538-33cb702568de_isrForCrosstalkSources_996988_53.stdout parsl.dataflow.dflow INFO: Standard error for task 776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/dde3d9f7-347b-4551-9538-33cb702568de_isrForCrosstalkSources_996988_53.stderr parsl.dataflow.dflow INFO: Task 743 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a3a7feef812b69afe9f5277a84dba53a with result from task 743 parsl.dataflow.dflow DEBUG: Task 743 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f3cf5636-4535-48a0-95c8-fa24184d4d9d_isrForCrosstalkSources_997032_29.stdout parsl.dataflow.dflow INFO: Standard error for task 743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f3cf5636-4535-48a0-95c8-fa24184d4d9d_isrForCrosstalkSources_997032_29.stderr parsl.dataflow.dflow INFO: Task 755 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d31778d9ffc4e30ae7dd9f7845351afb with result from task 755 parsl.dataflow.dflow DEBUG: Task 755 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/8f54438f-9418-48a1-95ac-b693782fcde9_isrForCrosstalkSources_997020_16.stdout parsl.dataflow.dflow INFO: Standard error for task 755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/8f54438f-9418-48a1-95ac-b693782fcde9_isrForCrosstalkSources_997020_16.stderr parsl.dataflow.dflow INFO: Task 812 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d4f5b30c304f5956784e81a10e369ab with result from task 812 parsl.dataflow.dflow DEBUG: Task 812 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/818774ee-fa26-41d7-ac56-a5355bf18587_isrForCrosstalkSources_997002_41.stdout parsl.dataflow.dflow INFO: Standard error for task 812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/818774ee-fa26-41d7-ac56-a5355bf18587_isrForCrosstalkSources_997002_41.stderr parsl.dataflow.dflow INFO: Task 856 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9db9b880dd992520737555988be6fa27 with result from task 856 parsl.dataflow.dflow DEBUG: Task 856 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4bd46426-edf0-4290-b8ca-e4c850bc00d8_isrForCrosstalkSources_996980_19.stdout parsl.dataflow.dflow INFO: Standard error for task 856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4bd46426-edf0-4290-b8ca-e4c850bc00d8_isrForCrosstalkSources_996980_19.stderr parsl.dataflow.dflow INFO: Task 789 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7bc523e89ad68f8edfb7e4f5f074d9b7 with result from task 789 parsl.dataflow.dflow DEBUG: Task 789 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e8c0ad20-4cf3-4860-b019-7868679050a3_isrForCrosstalkSources_996978_12.stdout parsl.dataflow.dflow INFO: Standard error for task 789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e8c0ad20-4cf3-4860-b019-7868679050a3_isrForCrosstalkSources_996978_12.stderr parsl.dataflow.dflow INFO: Task 671 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 952ecc04faeaa7b8d26d5a2c57321ab9 with result from task 671 parsl.dataflow.dflow DEBUG: Task 671 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d9ab12b3-225f-48ab-82dc-795bab87cb07_isrForCrosstalkSources_997003_62.stdout parsl.dataflow.dflow INFO: Standard error for task 671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d9ab12b3-225f-48ab-82dc-795bab87cb07_isrForCrosstalkSources_997003_62.stderr parsl.dataflow.dflow INFO: Task 689 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 636ae4ca63bfa6b93286a6b4e43e5a13 with result from task 689 parsl.dataflow.dflow DEBUG: Task 689 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/06cb429b-3ebd-4ec0-b38d-e3deca191b0f_isrForCrosstalkSources_996958_29.stdout parsl.dataflow.dflow INFO: Standard error for task 689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/06cb429b-3ebd-4ec0-b38d-e3deca191b0f_isrForCrosstalkSources_996958_29.stderr parsl.dataflow.dflow INFO: Task 802 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af730f98d7fe3b7a5c28b00f3531b190 with result from task 802 parsl.dataflow.dflow DEBUG: Task 802 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/94479306-858d-445d-8219-78a0805f7120_isrForCrosstalkSources_997007_16.stdout parsl.dataflow.dflow INFO: Standard error for task 802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/94479306-858d-445d-8219-78a0805f7120_isrForCrosstalkSources_997007_16.stderr parsl.dataflow.dflow INFO: Task 816 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a9faafaa2ae93ac9bacd172b49603c6a with result from task 816 parsl.dataflow.dflow DEBUG: Task 816 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b7e635bb-0962-43f3-afde-f90d9fb9f4af_isrForCrosstalkSources_997003_46.stdout parsl.dataflow.dflow INFO: Standard error for task 816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b7e635bb-0962-43f3-afde-f90d9fb9f4af_isrForCrosstalkSources_997003_46.stderr 2024-03-15 13:53:11 proc_lsst.multi:170 [INFO] [multi] job_id 17075400 proc_lsst.multi INFO: [multi] job_id 17075400 2024-03-15 13:53:11 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 238 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 238 2024-03-15 13:53:11 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:11 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075400 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075400 parsl.executors.status_handling DEBUG: Launched block 293 on executor multi with job ID 17075400 parsl.executors.status_handling INFO: Allocated block ID 294 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-294 2024-03-15 13:53:11 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-294 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-294 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:11 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:11 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 238 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 238 2024-03-15 13:53:11 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:11 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=294 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 726 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9490b7656e6406cba3e11e637c09d460 with result from task 726 parsl.dataflow.dflow DEBUG: Task 726 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/38a91b94-9359-4b47-9379-edfc1b53b7f7_isrForCrosstalkSources_996976_30.stdout parsl.dataflow.dflow INFO: Standard error for task 726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/38a91b94-9359-4b47-9379-edfc1b53b7f7_isrForCrosstalkSources_996976_30.stderr parsl.dataflow.dflow INFO: Task 646 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f46bfd5ac87afcdc960575dc5a2d61ff with result from task 646 parsl.dataflow.dflow DEBUG: Task 646 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/b002214a-327e-4187-ae0d-758c3697038b_isrForCrosstalkSources_996970_42.stdout parsl.dataflow.dflow INFO: Standard error for task 646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/b002214a-327e-4187-ae0d-758c3697038b_isrForCrosstalkSources_996970_42.stderr parsl.dataflow.dflow INFO: Task 753 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 36a7e80620d11b410e3423fceb604db9 with result from task 753 parsl.dataflow.dflow DEBUG: Task 753 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b4d32665-a611-45c9-9ee7-dc9a42936497_isrForCrosstalkSources_996966_16.stdout parsl.dataflow.dflow INFO: Standard error for task 753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b4d32665-a611-45c9-9ee7-dc9a42936497_isrForCrosstalkSources_996966_16.stderr parsl.dataflow.dflow INFO: Task 751 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7b109b71db9c897464c39fb18161e9e with result from task 751 parsl.dataflow.dflow DEBUG: Task 751 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/8559b384-612b-4f70-a527-95d8c6d8976b_isrForCrosstalkSources_997012_37.stdout parsl.dataflow.dflow INFO: Standard error for task 751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/8559b384-612b-4f70-a527-95d8c6d8976b_isrForCrosstalkSources_997012_37.stderr 2024-03-15 13:53:12 proc_lsst.multi:170 [INFO] [multi] job_id 17075401 proc_lsst.multi INFO: [multi] job_id 17075401 2024-03-15 13:53:12 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 239 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 239 2024-03-15 13:53:12 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:12 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075401 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075401 parsl.executors.status_handling DEBUG: Launched block 294 on executor multi with job ID 17075401 parsl.executors.status_handling INFO: Allocated block ID 295 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-295 2024-03-15 13:53:12 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-295 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-295 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 239 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 239 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:12 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=295 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 811 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed6f0c895b36d3773725dd7fd3a4e471 with result from task 811 parsl.dataflow.dflow DEBUG: Task 811 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/efa72a08-f035-48b5-9263-a874ea514e72_isrForCrosstalkSources_997007_6.stdout parsl.dataflow.dflow INFO: Standard error for task 811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/efa72a08-f035-48b5-9263-a874ea514e72_isrForCrosstalkSources_997007_6.stderr parsl.dataflow.dflow INFO: Task 786 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4e56b5966c13a730ca0260f87960209 with result from task 786 parsl.dataflow.dflow DEBUG: Task 786 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4f5e74bb-c5b6-4f3c-bfc6-8484ff1e99dd_isrForCrosstalkSources_997014_43.stdout parsl.dataflow.dflow INFO: Standard error for task 786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4f5e74bb-c5b6-4f3c-bfc6-8484ff1e99dd_isrForCrosstalkSources_997014_43.stderr parsl.dataflow.dflow INFO: Task 647 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0bc79f41083eb6d980f61ab4a1fdf7ff with result from task 647 parsl.dataflow.dflow DEBUG: Task 647 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ae752355-8191-4c55-b084-2c0f86802d20_isrForCrosstalkSources_996972_28.stdout parsl.dataflow.dflow INFO: Standard error for task 647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ae752355-8191-4c55-b084-2c0f86802d20_isrForCrosstalkSources_996972_28.stderr parsl.dataflow.dflow INFO: Task 725 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7747b46c8932aff0cf63f027ff5fbda with result from task 725 parsl.dataflow.dflow DEBUG: Task 725 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/558e0422-ac27-459d-8453-11807c82e534_isrForCrosstalkSources_996987_48.stdout parsl.dataflow.dflow INFO: Standard error for task 725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/558e0422-ac27-459d-8453-11807c82e534_isrForCrosstalkSources_996987_48.stderr parsl.dataflow.dflow INFO: Task 855 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6997a60a76ff25a8dd03b53cce27cce1 with result from task 855 parsl.dataflow.dflow DEBUG: Task 855 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/15761c2f-5f8b-4842-b424-d9dcd48f5355_isrForCrosstalkSources_996971_40.stdout parsl.dataflow.dflow INFO: Standard error for task 855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/15761c2f-5f8b-4842-b424-d9dcd48f5355_isrForCrosstalkSources_996971_40.stderr parsl.dataflow.dflow INFO: Task 656 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83884a378bdeead44e0a7dab89f9c011 with result from task 656 parsl.dataflow.dflow DEBUG: Task 656 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/9880ec23-1dd6-48de-84c1-b94bd324c917_isrForCrosstalkSources_996969_7.stdout parsl.dataflow.dflow INFO: Standard error for task 656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/9880ec23-1dd6-48de-84c1-b94bd324c917_isrForCrosstalkSources_996969_7.stderr parsl.dataflow.dflow INFO: Task 706 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65017617e42b4e115ebdf3dd9662b662 with result from task 706 parsl.dataflow.dflow DEBUG: Task 706 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/446a01de-1032-4867-93f9-915b4a804e41_isrForCrosstalkSources_997001_62.stdout parsl.dataflow.dflow INFO: Standard error for task 706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/446a01de-1032-4867-93f9-915b4a804e41_isrForCrosstalkSources_997001_62.stderr 2024-03-15 13:53:12 proc_lsst.multi:170 [INFO] [multi] job_id 17075405 proc_lsst.multi INFO: [multi] job_id 17075405 2024-03-15 13:53:12 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 240 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 240 2024-03-15 13:53:12 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:12 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075405 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075405 parsl.executors.status_handling DEBUG: Launched block 295 on executor multi with job ID 17075405 parsl.executors.status_handling INFO: Allocated block ID 296 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-296 2024-03-15 13:53:12 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-296 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-296 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:12 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:12 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 240 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 240 2024-03-15 13:53:12 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:12 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=296 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 846 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0de30b96b7118663caaa98ecae48aeb4 with result from task 846 parsl.dataflow.dflow DEBUG: Task 846 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d46630ab-2b66-48bb-a667-12dee0a04f11_isrForCrosstalkSources_997023_27.stdout parsl.dataflow.dflow INFO: Standard error for task 846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d46630ab-2b66-48bb-a667-12dee0a04f11_isrForCrosstalkSources_997023_27.stderr parsl.dataflow.dflow INFO: Task 772 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7b30d702e808b1e09d0bbdc79fb9176 with result from task 772 parsl.dataflow.dflow DEBUG: Task 772 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/5f3645c8-f2f8-4555-b3ff-fd58aad9e8c4_isrForCrosstalkSources_996970_61.stdout parsl.dataflow.dflow INFO: Standard error for task 772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/5f3645c8-f2f8-4555-b3ff-fd58aad9e8c4_isrForCrosstalkSources_996970_61.stderr parsl.dataflow.dflow INFO: Task 727 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fbb249bfb1f7eb085a01215864e31b0d with result from task 727 parsl.dataflow.dflow DEBUG: Task 727 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d6c5af40-86e5-48d4-a43b-44de9337cb8c_isrForCrosstalkSources_996990_55.stdout parsl.dataflow.dflow INFO: Standard error for task 727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d6c5af40-86e5-48d4-a43b-44de9337cb8c_isrForCrosstalkSources_996990_55.stderr parsl.dataflow.dflow INFO: Task 652 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d39b5641193c7830c9c5df82f9d1ce4 with result from task 652 parsl.dataflow.dflow DEBUG: Task 652 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6435b05f-39e2-424c-9651-7e4bd1687d8b_isrForCrosstalkSources_997005_50.stdout parsl.dataflow.dflow INFO: Standard error for task 652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6435b05f-39e2-424c-9651-7e4bd1687d8b_isrForCrosstalkSources_997005_50.stderr 2024-03-15 13:53:13 proc_lsst.multi:170 [INFO] [multi] job_id 17075408 proc_lsst.multi INFO: [multi] job_id 17075408 2024-03-15 13:53:13 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 241 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 241 2024-03-15 13:53:13 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:13 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075408 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075408 parsl.executors.status_handling DEBUG: Launched block 296 on executor multi with job ID 17075408 parsl.executors.status_handling INFO: Allocated block ID 297 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-297 2024-03-15 13:53:13 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-297 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-297 2024-03-15 13:53:13 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:13 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:13 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:13 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:13 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:13 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 241 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 241 2024-03-15 13:53:13 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:13 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=297 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 839 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21cae9655f2acd3512a7fdda0318f9c8 with result from task 839 parsl.dataflow.dflow DEBUG: Task 839 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4ed2f7ae-eae1-4ec7-9a29-4ed1bad01eaf_isrForCrosstalkSources_996957_39.stdout parsl.dataflow.dflow INFO: Standard error for task 839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4ed2f7ae-eae1-4ec7-9a29-4ed1bad01eaf_isrForCrosstalkSources_996957_39.stderr parsl.dataflow.dflow INFO: Task 883 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56cb29e6dfe11d2ebdf9c01f5017a83e with result from task 883 parsl.dataflow.dflow DEBUG: Task 883 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c4e8592d-cd9a-4848-b286-d692e6d05385_isrForCrosstalkSources_996961_26.stdout parsl.dataflow.dflow INFO: Standard error for task 883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c4e8592d-cd9a-4848-b286-d692e6d05385_isrForCrosstalkSources_996961_26.stderr parsl.dataflow.dflow INFO: Task 683 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c916bf80bc92f9740a1e8aff6a55e8c3 with result from task 683 parsl.dataflow.dflow DEBUG: Task 683 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/e076199e-7953-4027-aa42-284101a28f71_isrForCrosstalkSources_996980_47.stdout parsl.dataflow.dflow INFO: Standard error for task 683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/e076199e-7953-4027-aa42-284101a28f71_isrForCrosstalkSources_996980_47.stderr parsl.dataflow.dflow INFO: Task 651 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ac6b32fc5e64b01bf635ce70d84366a with result from task 651 parsl.dataflow.dflow DEBUG: Task 651 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1ea7cc0f-904d-42a1-ad71-bc1f04907aed_isrForCrosstalkSources_997027_44.stdout parsl.dataflow.dflow INFO: Standard error for task 651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1ea7cc0f-904d-42a1-ad71-bc1f04907aed_isrForCrosstalkSources_997027_44.stderr parsl.dataflow.dflow INFO: Task 658 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84b8abe0cd214f9b702479f00358e717 with result from task 658 parsl.dataflow.dflow DEBUG: Task 658 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/5f046451-1242-4eb6-ab2f-36c95f601a78_isrForCrosstalkSources_997026_58.stdout parsl.dataflow.dflow INFO: Standard error for task 658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/5f046451-1242-4eb6-ab2f-36c95f601a78_isrForCrosstalkSources_997026_58.stderr parsl.dataflow.dflow INFO: Task 750 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bdfdfa9063da13b3873890af182417d5 with result from task 750 parsl.dataflow.dflow DEBUG: Task 750 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/594f4542-c809-4a14-a17a-b1a6b13a424a_isrForCrosstalkSources_997033_48.stdout parsl.dataflow.dflow INFO: Standard error for task 750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/594f4542-c809-4a14-a17a-b1a6b13a424a_isrForCrosstalkSources_997033_48.stderr parsl.dataflow.dflow INFO: Task 864 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 896ca41621eaf5bd01bffd82c4e2f30a with result from task 864 parsl.dataflow.dflow DEBUG: Task 864 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6e2335ae-aa6e-4a63-842e-fc256d554263_isrForCrosstalkSources_997017_34.stdout parsl.dataflow.dflow INFO: Standard error for task 864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6e2335ae-aa6e-4a63-842e-fc256d554263_isrForCrosstalkSources_997017_34.stderr parsl.dataflow.dflow INFO: Task 900 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a18dd296a151d9331e17ae24c0eaded with result from task 900 parsl.dataflow.dflow DEBUG: Task 900 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7eb842b6-5218-4889-9022-0090fa97523e_isrForCrosstalkSources_996988_20.stdout parsl.dataflow.dflow INFO: Standard error for task 900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7eb842b6-5218-4889-9022-0090fa97523e_isrForCrosstalkSources_996988_20.stderr parsl.dataflow.dflow INFO: Task 685 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4d80d18797a1bf92951d78fc59af4a01 with result from task 685 parsl.dataflow.dflow DEBUG: Task 685 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1910f109-f449-430c-8708-256f8583f883_isrForCrosstalkSources_997014_53.stdout parsl.dataflow.dflow INFO: Standard error for task 685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1910f109-f449-430c-8708-256f8583f883_isrForCrosstalkSources_997014_53.stderr 2024-03-15 13:53:15 proc_lsst.multi:170 [INFO] [multi] job_id 17075417 proc_lsst.multi INFO: [multi] job_id 17075417 2024-03-15 13:53:15 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 242 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 242 2024-03-15 13:53:15 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:15 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075417 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075417 parsl.executors.status_handling DEBUG: Launched block 297 on executor multi with job ID 17075417 parsl.executors.status_handling INFO: Allocated block ID 298 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-298 2024-03-15 13:53:15 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-298 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-298 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 242 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 242 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:15 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=298 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 660 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2cbc067d234701a6e3bd9b2f9bbc28e6 with result from task 660 parsl.dataflow.dflow DEBUG: Task 660 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/00ad1a39-6428-44ac-a3da-9bf371beeae6_isrForCrosstalkSources_997021_62.stdout parsl.dataflow.dflow INFO: Standard error for task 660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/00ad1a39-6428-44ac-a3da-9bf371beeae6_isrForCrosstalkSources_997021_62.stderr parsl.dataflow.dflow INFO: Task 644 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 025135034275e904ffba3219dddc05b4 with result from task 644 parsl.dataflow.dflow DEBUG: Task 644 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/5781801d-8d9e-4af4-9762-4a81937f3835_isrForCrosstalkSources_996995_62.stdout parsl.dataflow.dflow INFO: Standard error for task 644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/5781801d-8d9e-4af4-9762-4a81937f3835_isrForCrosstalkSources_996995_62.stderr parsl.dataflow.dflow INFO: Task 623 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19e5268f4bcc08491507b56c18430beb with result from task 623 parsl.dataflow.dflow DEBUG: Task 623 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/dc215cf0-4845-4d56-b557-c49445e32497_isrForCrosstalkSources_996973_57.stdout parsl.dataflow.dflow INFO: Standard error for task 623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/dc215cf0-4845-4d56-b557-c49445e32497_isrForCrosstalkSources_996973_57.stderr parsl.dataflow.dflow INFO: Task 836 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03beb55d75c02d131fc6bb23a5e63f51 with result from task 836 parsl.dataflow.dflow DEBUG: Task 836 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/c13335b2-693e-4a28-9e8e-9f8728f6c372_isrForCrosstalkSources_997005_6.stdout parsl.dataflow.dflow INFO: Standard error for task 836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/c13335b2-693e-4a28-9e8e-9f8728f6c372_isrForCrosstalkSources_997005_6.stderr parsl.dataflow.dflow INFO: Task 676 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e3148023ba867422c347c42eb442c04 with result from task 676 parsl.dataflow.dflow DEBUG: Task 676 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/8337531e-fcc7-4703-9f89-71838b0f5877_isrForCrosstalkSources_996959_48.stdout parsl.dataflow.dflow INFO: Standard error for task 676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/8337531e-fcc7-4703-9f89-71838b0f5877_isrForCrosstalkSources_996959_48.stderr parsl.dataflow.dflow INFO: Task 787 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ae083fb263f8523139448364ce471bc with result from task 787 parsl.dataflow.dflow DEBUG: Task 787 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/4ddf1c30-7b66-4be6-970f-a63f64223494_isrForCrosstalkSources_997008_42.stdout parsl.dataflow.dflow INFO: Standard error for task 787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/4ddf1c30-7b66-4be6-970f-a63f64223494_isrForCrosstalkSources_997008_42.stderr parsl.dataflow.dflow INFO: Task 835 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7cf6682dffb228d54f435d337c4699f with result from task 835 parsl.dataflow.dflow DEBUG: Task 835 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/185e9fd6-e6a3-4784-a9bf-8449da8cd8ba_isrForCrosstalkSources_997014_40.stdout parsl.dataflow.dflow INFO: Standard error for task 835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/185e9fd6-e6a3-4784-a9bf-8449da8cd8ba_isrForCrosstalkSources_997014_40.stderr parsl.dataflow.dflow INFO: Task 648 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12fdb1f2a207c51be279a5d402216ab8 with result from task 648 parsl.dataflow.dflow DEBUG: Task 648 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a372db81-8893-4c7e-991d-8cd90f2f2be2_isrForCrosstalkSources_996993_62.stdout parsl.dataflow.dflow INFO: Standard error for task 648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a372db81-8893-4c7e-991d-8cd90f2f2be2_isrForCrosstalkSources_996993_62.stderr 2024-03-15 13:53:15 proc_lsst.multi:170 [INFO] [multi] job_id 17075428 proc_lsst.multi INFO: [multi] job_id 17075428 2024-03-15 13:53:15 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 243 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 243 2024-03-15 13:53:15 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:15 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075428 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075428 parsl.executors.status_handling DEBUG: Launched block 298 on executor multi with job ID 17075428 parsl.executors.status_handling INFO: Allocated block ID 299 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-299 2024-03-15 13:53:15 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-299 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-299 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:15 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:15 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 243 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 243 2024-03-15 13:53:15 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:15 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=299 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 909 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a9d6323247980087320e36c901aafda with result from task 909 parsl.dataflow.dflow DEBUG: Task 909 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fc58858c-81b5-4a78-b30d-f591547542d1_isrForCrosstalkSources_996961_53.stdout parsl.dataflow.dflow INFO: Standard error for task 909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fc58858c-81b5-4a78-b30d-f591547542d1_isrForCrosstalkSources_996961_53.stderr parsl.dataflow.dflow INFO: Task 732 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1bcd6274a0f94fd105ab7aa44745b008 with result from task 732 parsl.dataflow.dflow DEBUG: Task 732 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/96d318c1-a8b3-41e0-825a-44e9e7a7835d_isrForCrosstalkSources_997025_54.stdout parsl.dataflow.dflow INFO: Standard error for task 732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/96d318c1-a8b3-41e0-825a-44e9e7a7835d_isrForCrosstalkSources_997025_54.stderr parsl.dataflow.dflow INFO: Task 840 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fa04e4a2f8bed4d4e87fd50b3954599 with result from task 840 parsl.dataflow.dflow DEBUG: Task 840 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/bf4daddf-5aad-46ca-8893-96027080de84_isrForCrosstalkSources_996980_1.stdout parsl.dataflow.dflow INFO: Standard error for task 840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/bf4daddf-5aad-46ca-8893-96027080de84_isrForCrosstalkSources_996980_1.stderr parsl.dataflow.dflow INFO: Task 728 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96a06b8967f68bee6d4b8155b7d74225 with result from task 728 parsl.dataflow.dflow DEBUG: Task 728 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e14f11fa-5694-42e0-807a-7ad4a7047b2f_isrForCrosstalkSources_996968_53.stdout parsl.dataflow.dflow INFO: Standard error for task 728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e14f11fa-5694-42e0-807a-7ad4a7047b2f_isrForCrosstalkSources_996968_53.stderr parsl.dataflow.dflow INFO: Task 785 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3aad201192270bdf434d7c25bb42180e with result from task 785 parsl.dataflow.dflow DEBUG: Task 785 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/af74facd-f12e-4dea-be9c-419dae812455_isrForCrosstalkSources_996997_42.stdout parsl.dataflow.dflow INFO: Standard error for task 785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/af74facd-f12e-4dea-be9c-419dae812455_isrForCrosstalkSources_996997_42.stderr parsl.dataflow.dflow INFO: Task 692 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c1e257b4fb98b34eb021d7cf0d9fa65 with result from task 692 parsl.dataflow.dflow DEBUG: Task 692 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4e47a46d-9d31-455c-bc5c-ea330a06e626_isrForCrosstalkSources_997023_57.stdout parsl.dataflow.dflow INFO: Standard error for task 692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4e47a46d-9d31-455c-bc5c-ea330a06e626_isrForCrosstalkSources_997023_57.stderr parsl.dataflow.dflow INFO: Task 828 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5512db4d9395998ad90736ef97465a1 with result from task 828 parsl.dataflow.dflow DEBUG: Task 828 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2aaa8ecd-1d9a-4c12-a67f-5d8d04bc3429_isrForCrosstalkSources_996962_11.stdout parsl.dataflow.dflow INFO: Standard error for task 828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2aaa8ecd-1d9a-4c12-a67f-5d8d04bc3429_isrForCrosstalkSources_996962_11.stderr 2024-03-15 13:53:16 proc_lsst.multi:170 [INFO] [multi] job_id 17075430 proc_lsst.multi INFO: [multi] job_id 17075430 2024-03-15 13:53:16 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 244 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 244 2024-03-15 13:53:16 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:16 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075430 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075430 parsl.executors.status_handling DEBUG: Launched block 299 on executor multi with job ID 17075430 parsl.executors.status_handling INFO: Allocated block ID 300 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-300 2024-03-15 13:53:16 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-300 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-300 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 244 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 244 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:16 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=300 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 800 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 34376c9583d4c0016f378b81c78882fc with result from task 800 parsl.dataflow.dflow DEBUG: Task 800 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/b1852634-80ec-474b-8338-7e3e3c37d9a0_isrForCrosstalkSources_996961_5.stdout parsl.dataflow.dflow INFO: Standard error for task 800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/b1852634-80ec-474b-8338-7e3e3c37d9a0_isrForCrosstalkSources_996961_5.stderr parsl.dataflow.dflow INFO: Task 837 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e12968ac1f0bf5caaa658c5e76e53091 with result from task 837 parsl.dataflow.dflow DEBUG: Task 837 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/2da20d0b-d507-42ae-937a-83373d5e7f6e_isrForCrosstalkSources_996967_16.stdout parsl.dataflow.dflow INFO: Standard error for task 837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/2da20d0b-d507-42ae-937a-83373d5e7f6e_isrForCrosstalkSources_996967_16.stderr parsl.dataflow.dflow INFO: Task 885 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c83e84a2ea4583102125bd7d112afae0 with result from task 885 parsl.dataflow.dflow DEBUG: Task 885 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5f86934a-6c23-4622-b8c4-d083df4e64da_isrForCrosstalkSources_996981_10.stdout parsl.dataflow.dflow INFO: Standard error for task 885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5f86934a-6c23-4622-b8c4-d083df4e64da_isrForCrosstalkSources_996981_10.stderr parsl.dataflow.dflow INFO: Task 804 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f6a03c8b02e450aaa90625cc06061c1 with result from task 804 parsl.dataflow.dflow DEBUG: Task 804 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/d083fdf7-aab9-4caf-8821-fd25b6ce2e8b_isrForCrosstalkSources_997011_11.stdout parsl.dataflow.dflow INFO: Standard error for task 804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/d083fdf7-aab9-4caf-8821-fd25b6ce2e8b_isrForCrosstalkSources_997011_11.stderr parsl.dataflow.dflow INFO: Task 875 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bdfa010ebd8e7b1722636b7d371c4aa2 with result from task 875 parsl.dataflow.dflow DEBUG: Task 875 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ec4a5c9f-aaa3-44ab-981d-e91b2c42f03e_isrForCrosstalkSources_997013_18.stdout parsl.dataflow.dflow INFO: Standard error for task 875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ec4a5c9f-aaa3-44ab-981d-e91b2c42f03e_isrForCrosstalkSources_997013_18.stderr parsl.dataflow.dflow INFO: Task 838 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 983abc9ae1aa0d0ee3ccc9678d6cd34c with result from task 838 parsl.dataflow.dflow DEBUG: Task 838 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2c2973c9-2af5-4305-b08d-f6a6b778fcb8_isrForCrosstalkSources_997006_28.stdout parsl.dataflow.dflow INFO: Standard error for task 838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2c2973c9-2af5-4305-b08d-f6a6b778fcb8_isrForCrosstalkSources_997006_28.stderr parsl.dataflow.dflow INFO: Task 890 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e0894935fe867e46f70ed68b78142e4 with result from task 890 parsl.dataflow.dflow DEBUG: Task 890 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/c23b3f79-7fcd-447f-8435-1f25f3137205_isrForCrosstalkSources_996971_2.stdout parsl.dataflow.dflow INFO: Standard error for task 890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/c23b3f79-7fcd-447f-8435-1f25f3137205_isrForCrosstalkSources_996971_2.stderr parsl.dataflow.dflow INFO: Task 848 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 286cbb1460d79f9126ad3a3afebb00ae with result from task 848 parsl.dataflow.dflow DEBUG: Task 848 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/0fcb4ad5-a808-4600-bd33-6bf8441f6544_isrForCrosstalkSources_997006_61.stdout parsl.dataflow.dflow INFO: Standard error for task 848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/0fcb4ad5-a808-4600-bd33-6bf8441f6544_isrForCrosstalkSources_997006_61.stderr parsl.dataflow.dflow INFO: Task 758 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a65731536648e6163b97b60ee37ecc4 with result from task 758 parsl.dataflow.dflow DEBUG: Task 758 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/bf898f6c-75b5-47dc-b836-73cc3adf541a_isrForCrosstalkSources_996985_53.stdout parsl.dataflow.dflow INFO: Standard error for task 758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/bf898f6c-75b5-47dc-b836-73cc3adf541a_isrForCrosstalkSources_996985_53.stderr 2024-03-15 13:53:16 proc_lsst.multi:170 [INFO] [multi] job_id 17075431 proc_lsst.multi INFO: [multi] job_id 17075431 2024-03-15 13:53:16 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 245 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 245 2024-03-15 13:53:16 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:16 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075431 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075431 parsl.executors.status_handling DEBUG: Launched block 300 on executor multi with job ID 17075431 parsl.executors.status_handling INFO: Allocated block ID 301 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-301 2024-03-15 13:53:16 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-301 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-301 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:16 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:16 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 245 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 245 2024-03-15 13:53:16 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:16 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=301 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 803 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ea8d29d8cb361bc9a1b5c028f0f8817 with result from task 803 parsl.dataflow.dflow DEBUG: Task 803 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/b6482575-edb6-4d6f-b639-ab7a755da88e_isrForCrosstalkSources_997011_48.stdout parsl.dataflow.dflow INFO: Standard error for task 803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/b6482575-edb6-4d6f-b639-ab7a755da88e_isrForCrosstalkSources_997011_48.stderr parsl.dataflow.dflow INFO: Task 792 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d4694ec1bab359bec9bcc3e587f4bb2 with result from task 792 parsl.dataflow.dflow DEBUG: Task 792 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/672cc3a2-77d3-4b25-925f-6eb907483ab1_isrForCrosstalkSources_996984_53.stdout parsl.dataflow.dflow INFO: Standard error for task 792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/672cc3a2-77d3-4b25-925f-6eb907483ab1_isrForCrosstalkSources_996984_53.stderr parsl.dataflow.dflow INFO: Task 735 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86b079fd2e413411a400b45c534fd9ae with result from task 735 parsl.dataflow.dflow DEBUG: Task 735 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/f571cc35-6735-49a6-9cb1-8a1bf9b54533_isrForCrosstalkSources_997010_62.stdout parsl.dataflow.dflow INFO: Standard error for task 735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/f571cc35-6735-49a6-9cb1-8a1bf9b54533_isrForCrosstalkSources_997010_62.stderr 2024-03-15 13:53:17 proc_lsst.multi:170 [INFO] [multi] job_id 17075432 proc_lsst.multi INFO: [multi] job_id 17075432 2024-03-15 13:53:17 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 246 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 246 2024-03-15 13:53:17 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:17 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075432 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075432 parsl.executors.status_handling DEBUG: Launched block 301 on executor multi with job ID 17075432 parsl.executors.status_handling INFO: Allocated block ID 302 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-302 2024-03-15 13:53:17 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-302 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-302 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 246 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 246 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:17 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=302 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 690 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 255b5f1254b5c872e678b31d91999285 with result from task 690 parsl.dataflow.dflow DEBUG: Task 690 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/3baab248-11b4-4c04-8cfe-97d639c9b0e7_isrForCrosstalkSources_997026_61.stdout parsl.dataflow.dflow INFO: Standard error for task 690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/3baab248-11b4-4c04-8cfe-97d639c9b0e7_isrForCrosstalkSources_997026_61.stderr parsl.dataflow.dflow INFO: Task 670 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fe828176a1d941603dbcde347072599 with result from task 670 parsl.dataflow.dflow DEBUG: Task 670 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7bae6981-4dd1-46b2-ae8f-f140a500dad8_isrForCrosstalkSources_996989_59.stdout parsl.dataflow.dflow INFO: Standard error for task 670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7bae6981-4dd1-46b2-ae8f-f140a500dad8_isrForCrosstalkSources_996989_59.stderr parsl.dataflow.dflow INFO: Task 680 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 473c23e47551f26356a30b5e800e7365 with result from task 680 parsl.dataflow.dflow DEBUG: Task 680 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/46d16f4a-e765-49c7-b273-0f4de994c281_isrForCrosstalkSources_996975_61.stdout parsl.dataflow.dflow INFO: Standard error for task 680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/46d16f4a-e765-49c7-b273-0f4de994c281_isrForCrosstalkSources_996975_61.stderr 2024-03-15 13:53:17 proc_lsst.multi:170 [INFO] [multi] job_id 17075433 proc_lsst.multi INFO: [multi] job_id 17075433 2024-03-15 13:53:17 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 247 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 247 2024-03-15 13:53:17 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:17 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075433 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075433 parsl.executors.status_handling DEBUG: Launched block 302 on executor multi with job ID 17075433 parsl.executors.status_handling INFO: Allocated block ID 303 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-303 2024-03-15 13:53:17 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-303 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-303 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:17 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:17 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 247 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 247 2024-03-15 13:53:17 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:17 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=303 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files 2024-03-15 13:53:18 proc_lsst.multi:170 [INFO] [multi] job_id 17075434 proc_lsst.multi INFO: [multi] job_id 17075434 2024-03-15 13:53:18 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 248 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 248 2024-03-15 13:53:18 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:18 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075434 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075434 parsl.executors.status_handling DEBUG: Launched block 303 on executor multi with job ID 17075434 parsl.executors.status_handling INFO: Allocated block ID 304 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-304 2024-03-15 13:53:18 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-304 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-304 2024-03-15 13:53:18 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:18 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:18 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:18 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:18 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:18 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 248 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 248 2024-03-15 13:53:18 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:18 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=304 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 922 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 112a95e25a1a22a8d1c41b96392d31ab with result from task 922 parsl.dataflow.dflow DEBUG: Task 922 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2fde8ca9-48a9-4773-ac52-2bc96d935b7f_isrForCrosstalkSources_996969_60.stdout parsl.dataflow.dflow INFO: Standard error for task 922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2fde8ca9-48a9-4773-ac52-2bc96d935b7f_isrForCrosstalkSources_996969_60.stderr parsl.dataflow.dflow INFO: Task 908 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fca2e944a4925da6b996e045af6e8f9d with result from task 908 parsl.dataflow.dflow DEBUG: Task 908 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/fde1c162-ee5f-467d-b6c0-ae62b9bb62a0_isrForCrosstalkSources_997020_7.stdout parsl.dataflow.dflow INFO: Standard error for task 908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/fde1c162-ee5f-467d-b6c0-ae62b9bb62a0_isrForCrosstalkSources_997020_7.stderr parsl.dataflow.dflow INFO: Task 788 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a52589be9c753cfc47910f5b7c5885e4 with result from task 788 2024-03-15 13:53:19 proc_lsst.multi:170 [INFO] [multi] job_id 17075435 proc_lsst.multi INFO: [multi] job_id 17075435 2024-03-15 13:53:19 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 249 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 249 2024-03-15 13:53:19 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:19 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075435 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075435 parsl.executors.status_handling DEBUG: Launched block 304 on executor multi with job ID 17075435 parsl.executors.status_handling INFO: Allocated block ID 305 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-305 2024-03-15 13:53:19 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-305 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-305 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 249 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 249 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:19 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=305 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.dataflow.dflow DEBUG: Task 788 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/cae5e6d9-3f54-4ad0-8c5d-fd8a2d713d26_isrForCrosstalkSources_996991_20.stdout parsl.dataflow.dflow INFO: Standard error for task 788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/cae5e6d9-3f54-4ad0-8c5d-fd8a2d713d26_isrForCrosstalkSources_996991_20.stderr parsl.dataflow.dflow INFO: Task 889 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 729af41cd3da3d252154e1117afae1fb with result from task 889 parsl.dataflow.dflow DEBUG: Task 889 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d459c8fb-384e-40de-be28-77f93a7c8efc_isrForCrosstalkSources_997004_32.stdout parsl.dataflow.dflow INFO: Standard error for task 889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d459c8fb-384e-40de-be28-77f93a7c8efc_isrForCrosstalkSources_997004_32.stderr parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 915 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 52ea5a712ed06a63e18739021ad05a44 with result from task 915 parsl.dataflow.dflow DEBUG: Task 915 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3b8aa232-48da-4b71-9bcd-4307a2e93b53_isrForCrosstalkSources_996955_14.stdout parsl.dataflow.dflow INFO: Standard error for task 915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3b8aa232-48da-4b71-9bcd-4307a2e93b53_isrForCrosstalkSources_996955_14.stderr 2024-03-15 13:53:19 proc_lsst.multi:170 [INFO] [multi] job_id 17075436 proc_lsst.multi INFO: [multi] job_id 17075436 2024-03-15 13:53:19 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 250 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 250 2024-03-15 13:53:19 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:19 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075436 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075436 parsl.executors.status_handling DEBUG: Launched block 305 on executor multi with job ID 17075436 parsl.executors.status_handling INFO: Allocated block ID 306 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-306 2024-03-15 13:53:19 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-306 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-306 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:19 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:19 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 250 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 250 2024-03-15 13:53:19 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:19 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=306 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 929 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd28dd8f12a0d0cd7cc7041edc40a37d with result from task 929 parsl.dataflow.dflow DEBUG: Task 929 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/8a133999-f74f-4c7b-a11b-40897282332c_isrForCrosstalkSources_997033_26.stdout parsl.dataflow.dflow INFO: Standard error for task 929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/8a133999-f74f-4c7b-a11b-40897282332c_isrForCrosstalkSources_997033_26.stderr parsl.dataflow.dflow INFO: Task 807 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 06c0f3cfad7bf7936258ea812347595f with result from task 807 parsl.dataflow.dflow DEBUG: Task 807 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1c4bd8cb-4563-4530-8cd3-0264d44aeac4_isrForCrosstalkSources_996968_8.stdout parsl.dataflow.dflow INFO: Standard error for task 807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1c4bd8cb-4563-4530-8cd3-0264d44aeac4_isrForCrosstalkSources_996968_8.stderr parsl.dataflow.dflow INFO: Task 829 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30118957d46fb81a68e807260f2bcff5 with result from task 829 parsl.dataflow.dflow DEBUG: Task 829 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/a22eaf35-29f7-4a92-ab0c-e7501fc5400e_isrForCrosstalkSources_996962_25.stdout parsl.dataflow.dflow INFO: Standard error for task 829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/a22eaf35-29f7-4a92-ab0c-e7501fc5400e_isrForCrosstalkSources_996962_25.stderr parsl.dataflow.dflow INFO: Task 831 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a08a9dfd46c4c802d7cde4cc1c54e974 with result from task 831 parsl.dataflow.dflow DEBUG: Task 831 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e9a80a65-dfe8-4bc7-a1a9-f1201ed33657_isrForCrosstalkSources_996964_51.stdout parsl.dataflow.dflow INFO: Standard error for task 831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e9a80a65-dfe8-4bc7-a1a9-f1201ed33657_isrForCrosstalkSources_996964_51.stderr parsl.dataflow.dflow INFO: Task 946 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03da7ef3449ac6db1adf5c86736ccccd with result from task 946 parsl.dataflow.dflow DEBUG: Task 946 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/4424fceb-97a4-4029-aad6-7a770827164a_isrForCrosstalkSources_996953_23.stdout parsl.dataflow.dflow INFO: Standard error for task 946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/4424fceb-97a4-4029-aad6-7a770827164a_isrForCrosstalkSources_996953_23.stderr 2024-03-15 13:53:20 proc_lsst.multi:170 [INFO] [multi] job_id 17075438 proc_lsst.multi INFO: [multi] job_id 17075438 2024-03-15 13:53:20 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 251 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 251 2024-03-15 13:53:20 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:20 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075438 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075438 parsl.executors.status_handling DEBUG: Launched block 306 on executor multi with job ID 17075438 parsl.executors.status_handling INFO: Allocated block ID 307 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-307 2024-03-15 13:53:20 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-307 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-307 2024-03-15 13:53:20 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:20 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:20 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:20 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:20 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:20 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 251 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 251 2024-03-15 13:53:20 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:20 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=307 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 873 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e45b93f6ed6bd5e5959be1f66a03aac with result from task 873 parsl.dataflow.dflow DEBUG: Task 873 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/67c58fca-9983-4ac5-8b81-c3e427823f83_isrForCrosstalkSources_997003_39.stdout parsl.dataflow.dflow INFO: Standard error for task 873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/67c58fca-9983-4ac5-8b81-c3e427823f83_isrForCrosstalkSources_997003_39.stderr parsl.dataflow.dflow INFO: Task 926 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be4dd31a6c80491de491edc54f8a8c90 with result from task 926 parsl.dataflow.dflow DEBUG: Task 926 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7c70072d-e1b6-4599-945d-e8f1b2086a42_isrForCrosstalkSources_996991_25.stdout parsl.dataflow.dflow INFO: Standard error for task 926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7c70072d-e1b6-4599-945d-e8f1b2086a42_isrForCrosstalkSources_996991_25.stderr parsl.dataflow.dflow INFO: Task 895 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c43a99b912a8d2c3d761a0f944550065 with result from task 895 parsl.dataflow.dflow DEBUG: Task 895 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/db96f8d0-dafb-42a1-b45f-dcbc3798812e_isrForCrosstalkSources_997009_32.stdout parsl.dataflow.dflow INFO: Standard error for task 895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/db96f8d0-dafb-42a1-b45f-dcbc3798812e_isrForCrosstalkSources_997009_32.stderr parsl.dataflow.dflow INFO: Task 841 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63740c5294d92374fefa9e5be90f8508 with result from task 841 parsl.dataflow.dflow DEBUG: Task 841 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6658601f-552d-49ff-8931-1158da6844f3_isrForCrosstalkSources_996953_8.stdout parsl.dataflow.dflow INFO: Standard error for task 841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6658601f-552d-49ff-8931-1158da6844f3_isrForCrosstalkSources_996953_8.stderr parsl.dataflow.dflow INFO: Task 957 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da29ee98e03cacdaa0cf78bcadbdc322 with result from task 957 parsl.dataflow.dflow DEBUG: Task 957 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/0f5c2fba-7ea8-4fde-87da-ad7063e898ef_isrForCrosstalkSources_997011_33.stdout parsl.dataflow.dflow INFO: Standard error for task 957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/0f5c2fba-7ea8-4fde-87da-ad7063e898ef_isrForCrosstalkSources_997011_33.stderr 2024-03-15 13:53:21 proc_lsst.multi:170 [INFO] [multi] job_id 17075440 proc_lsst.multi INFO: [multi] job_id 17075440 2024-03-15 13:53:21 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 252 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 252 2024-03-15 13:53:21 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:21 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075440 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075440 parsl.executors.status_handling DEBUG: Launched block 307 on executor multi with job ID 17075440 parsl.executors.status_handling INFO: Allocated block ID 308 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-308 2024-03-15 13:53:21 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-308 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-308 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 252 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 252 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:21 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=308 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 947 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 981900bd5ab851d7b44fd9bf4879bae7 with result from task 947 parsl.dataflow.dflow DEBUG: Task 947 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/25584aba-5fed-4a74-ab62-e9e70a4843ff_isrForCrosstalkSources_996996_27.stdout parsl.dataflow.dflow INFO: Standard error for task 947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/25584aba-5fed-4a74-ab62-e9e70a4843ff_isrForCrosstalkSources_996996_27.stderr parsl.dataflow.dflow INFO: Task 849 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 943205567d13dbe12e95ca91bbca6a43 with result from task 849 parsl.dataflow.dflow DEBUG: Task 849 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5f843ac6-a6f8-40ac-94ac-a36cea010fd2_isrForCrosstalkSources_997021_23.stdout parsl.dataflow.dflow INFO: Standard error for task 849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5f843ac6-a6f8-40ac-94ac-a36cea010fd2_isrForCrosstalkSources_997021_23.stderr parsl.dataflow.dflow INFO: Task 844 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 14a54057b41a2f95180e180c1c1e2eed with result from task 844 parsl.dataflow.dflow DEBUG: Task 844 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6e3c1ab9-cf44-4e9d-8e03-125bdd2ea00c_isrForCrosstalkSources_997005_23.stdout parsl.dataflow.dflow INFO: Standard error for task 844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6e3c1ab9-cf44-4e9d-8e03-125bdd2ea00c_isrForCrosstalkSources_997005_23.stderr parsl.dataflow.dflow INFO: Task 884 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1aecc53a54a648885b8d63ea3f16c30c with result from task 884 parsl.dataflow.dflow DEBUG: Task 884 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/76b604b9-ce9d-49b0-8c3e-8222e0d94859_isrForCrosstalkSources_997015_56.stdout parsl.dataflow.dflow INFO: Standard error for task 884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/76b604b9-ce9d-49b0-8c3e-8222e0d94859_isrForCrosstalkSources_997015_56.stderr parsl.dataflow.dflow INFO: Task 888 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5780fd166239f198dbcbadf0bc88d73a with result from task 888 parsl.dataflow.dflow DEBUG: Task 888 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/77334703-578e-4c8b-9a24-36fd39683198_isrForCrosstalkSources_996978_4.stdout parsl.dataflow.dflow INFO: Standard error for task 888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/77334703-578e-4c8b-9a24-36fd39683198_isrForCrosstalkSources_996978_4.stderr 2024-03-15 13:53:21 proc_lsst.multi:170 [INFO] [multi] job_id 17075441 proc_lsst.multi INFO: [multi] job_id 17075441 2024-03-15 13:53:21 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 253 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 253 2024-03-15 13:53:21 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:21 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075441 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075441 parsl.executors.status_handling DEBUG: Launched block 308 on executor multi with job ID 17075441 parsl.executors.status_handling INFO: Allocated block ID 309 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-309 2024-03-15 13:53:21 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-309 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-309 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:21 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:21 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 253 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 253 2024-03-15 13:53:21 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:21 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=309 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 820 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 127cfb039c9f39e7b8c931c9164f72ed with result from task 820 parsl.dataflow.dflow DEBUG: Task 820 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/34e4c126-96b9-43bb-8203-4d277e11f6bb_isrForCrosstalkSources_997007_19.stdout parsl.dataflow.dflow INFO: Standard error for task 820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/34e4c126-96b9-43bb-8203-4d277e11f6bb_isrForCrosstalkSources_997007_19.stderr parsl.dataflow.dflow INFO: Task 911 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af514919c93f6f68ff3405f31937332a with result from task 911 parsl.dataflow.dflow DEBUG: Task 911 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/dbb5486f-ae04-4c75-b035-4bbb3e32b302_isrForCrosstalkSources_997027_46.stdout parsl.dataflow.dflow INFO: Standard error for task 911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/dbb5486f-ae04-4c75-b035-4bbb3e32b302_isrForCrosstalkSources_997027_46.stderr parsl.dataflow.dflow INFO: Task 797 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry afd01e6b4f1c787999be7aee04e92768 with result from task 797 parsl.dataflow.dflow DEBUG: Task 797 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c2b0b837-351f-4e7f-bfd6-aaa309679fea_isrForCrosstalkSources_996961_6.stdout parsl.dataflow.dflow INFO: Standard error for task 797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c2b0b837-351f-4e7f-bfd6-aaa309679fea_isrForCrosstalkSources_996961_6.stderr parsl.dataflow.dflow INFO: Task 923 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1622607b3fa7c95b1b575b795f85627 with result from task 923 parsl.dataflow.dflow DEBUG: Task 923 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/e385377b-55b4-4048-9da6-9277e9af94ad_isrForCrosstalkSources_996977_8.stdout parsl.dataflow.dflow INFO: Standard error for task 923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/e385377b-55b4-4048-9da6-9277e9af94ad_isrForCrosstalkSources_996977_8.stderr parsl.dataflow.dflow INFO: Task 827 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d51822258f39e743c81378c028eb1db with result from task 827 parsl.dataflow.dflow DEBUG: Task 827 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/04a5e965-7e86-4333-9950-38f947af7b29_isrForCrosstalkSources_997004_17.stdout parsl.dataflow.dflow INFO: Standard error for task 827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/04a5e965-7e86-4333-9950-38f947af7b29_isrForCrosstalkSources_997004_17.stderr parsl.dataflow.dflow INFO: Task 942 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81bfb60ad69c4d13b90b033da4a5a1b5 with result from task 942 parsl.dataflow.dflow DEBUG: Task 942 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c0404c2d-d74a-4436-a668-3320043bf59f_isrForCrosstalkSources_996955_32.stdout parsl.dataflow.dflow INFO: Standard error for task 942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c0404c2d-d74a-4436-a668-3320043bf59f_isrForCrosstalkSources_996955_32.stderr 2024-03-15 13:53:22 proc_lsst.multi:170 [INFO] [multi] job_id 17075443 proc_lsst.multi INFO: [multi] job_id 17075443 2024-03-15 13:53:22 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 254 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 254 2024-03-15 13:53:22 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:22 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075443 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075443 parsl.executors.status_handling DEBUG: Launched block 309 on executor multi with job ID 17075443 parsl.executors.status_handling INFO: Allocated block ID 310 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-310 2024-03-15 13:53:22 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-310 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-310 2024-03-15 13:53:22 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:22 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:22 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:22 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:22 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:22 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 254 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 254 2024-03-15 13:53:22 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:22 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=310 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 834 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6eb679667e6d666ef2eb61d9e0160825 with result from task 834 parsl.dataflow.dflow DEBUG: Task 834 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/f6d2cea5-fc74-44da-984b-29c0176bedd2_isrForCrosstalkSources_996997_27.stdout parsl.dataflow.dflow INFO: Standard error for task 834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/f6d2cea5-fc74-44da-984b-29c0176bedd2_isrForCrosstalkSources_996997_27.stderr 2024-03-15 13:53:23 proc_lsst.multi:170 [INFO] [multi] job_id 17075444 proc_lsst.multi INFO: [multi] job_id 17075444 2024-03-15 13:53:23 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 255 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 255 2024-03-15 13:53:23 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:23 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075444 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075444 parsl.executors.status_handling DEBUG: Launched block 310 on executor multi with job ID 17075444 parsl.executors.status_handling INFO: Allocated block ID 311 parsl.executors.status_handling DEBUG: Submitting to provider with job_name parsl.multi.block-311 2024-03-15 13:53:23 proc_lsst.multi:153 [INFO] [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-311 proc_lsst.multi INFO: [multi] got submit process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 1 parsl.multi.block-311 2024-03-15 13:53:23 proc_lsst.multi:162 [INFO] [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 proc_lsst.multi INFO: [multi] local process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn --max_workers 2 2024-03-15 13:53:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 1 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 1 2024-03-15 13:53:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 1 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 1 2024-03-15 13:53:23 proc_lsst.multi:162 [INFO] [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] astro process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 30 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 30 2024-03-15 13:53:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 30 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 30 2024-03-15 13:53:23 proc_lsst.multi:162 [INFO] [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_rtx6k process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 5 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 5 2024-03-15 13:53:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 5 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 5 2024-03-15 13:53:23 proc_lsst.multi:162 [INFO] [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] escience_a40 process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 20 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 20 2024-03-15 13:53:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 20 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 20 2024-03-15 13:53:23 proc_lsst.multi:162 [INFO] [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn proc_lsst.multi INFO: [multi] checkpoint process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn 2024-03-15 13:53:23 proc_lsst.multi:163 [INFO] [multi] len(self.providers[provider].resources) 255 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 255 2024-03-15 13:53:23 proc_lsst.multi:164 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:23 proc_lsst.multi:166 [INFO] [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint proc_lsst.multi INFO: [multi] submitting process_worker_pool.py -a 10.64.65.9,10.64.129.9,169.254.95.120,127.0.0.1,198.48.92.26,n3009 -p 0 -c 1.0 -m None --poll 10 --task_port=54667 --result_port=54177 --logdir=/mmfs1/gscratch/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/runinfo/000/multi --block_id=311 --hb_period=30 --hb_threshold=120 --cpu-affinity none --available-accelerators --start-method spawn to checkpoint parsl.providers.slurm.slurm DEBUG: Requesting one block with 1 nodes parsl.providers.slurm.slurm DEBUG: Writing submit script parsl.providers.slurm.slurm DEBUG: moving files parsl.dataflow.dflow INFO: Task 870 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb54e92eb721e7840567890d898a5340 with result from task 870 parsl.dataflow.dflow DEBUG: Task 870 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a37801f2-ba16-4ceb-8024-f10a781bae34_isrForCrosstalkSources_996952_6.stdout parsl.dataflow.dflow INFO: Standard error for task 870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a37801f2-ba16-4ceb-8024-f10a781bae34_isrForCrosstalkSources_996952_6.stderr parsl.dataflow.dflow INFO: Task 854 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b7e5fd1940b87b9f719896ace3c22fb with result from task 854 parsl.dataflow.dflow DEBUG: Task 854 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/cd284f19-3ba6-4d71-a6eb-47d17fad29a0_isrForCrosstalkSources_997033_25.stdout parsl.dataflow.dflow INFO: Standard error for task 854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/cd284f19-3ba6-4d71-a6eb-47d17fad29a0_isrForCrosstalkSources_997033_25.stderr parsl.dataflow.dflow INFO: Task 858 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c53330bde1632434f304776caf45b595 with result from task 858 parsl.dataflow.dflow DEBUG: Task 858 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/36416d78-1716-4301-812f-4838f5fafd5e_isrForCrosstalkSources_996985_51.stdout parsl.dataflow.dflow INFO: Standard error for task 858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/36416d78-1716-4301-812f-4838f5fafd5e_isrForCrosstalkSources_996985_51.stderr parsl.dataflow.dflow INFO: Task 956 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2df30a26ea0c5ed45fc540ecb4d79ae with result from task 956 parsl.dataflow.dflow DEBUG: Task 956 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/aff0a313-e807-4580-a899-79bc95e86bb5_isrForCrosstalkSources_996986_13.stdout parsl.dataflow.dflow INFO: Standard error for task 956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/aff0a313-e807-4580-a899-79bc95e86bb5_isrForCrosstalkSources_996986_13.stderr 2024-03-15 13:53:23 proc_lsst.multi:170 [INFO] [multi] job_id 17075446 proc_lsst.multi INFO: [multi] job_id 17075446 2024-03-15 13:53:23 proc_lsst.multi:171 [INFO] [multi] len(self.providers[provider].resources) 256 proc_lsst.multi INFO: [multi] len(self.providers[provider].resources) 256 2024-03-15 13:53:23 proc_lsst.multi:172 [INFO] [multi] self.providers[provider].max_blocks 256 proc_lsst.multi INFO: [multi] self.providers[provider].max_blocks 256 2024-03-15 13:53:23 proc_lsst.multi:178 [INFO] [multi] provider checkpoint accepted submit and returned 17075446 proc_lsst.multi INFO: [multi] provider checkpoint accepted submit and returned 17075446 parsl.executors.status_handling DEBUG: Launched block 311 on executor multi with job ID 17075446 parsl.process_loggers DEBUG: Normal ending for _general_strategy on thread JobStatusPoller-Timer-Thread-22709913200592 2024-03-15 13:53:23 proc_lsst.multi:146 [INFO] found job 91028 in provider local proc_lsst.multi INFO: found job 91028 in provider local parsl.dataflow.dflow INFO: Task 886 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d82c2c17d2ca45d4bedd7a5dfd0708c8 with result from task 886 parsl.dataflow.dflow DEBUG: Task 886 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/38a84872-3357-4a00-88ad-1da6b5d1c5f8_isrForCrosstalkSources_996980_12.stdout parsl.dataflow.dflow INFO: Standard error for task 886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/38a84872-3357-4a00-88ad-1da6b5d1c5f8_isrForCrosstalkSources_996980_12.stderr parsl.dataflow.dflow INFO: Task 809 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57974aedd4d91f86ba323c01d3d7890e with result from task 809 parsl.dataflow.dflow DEBUG: Task 809 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/35c8333b-e080-4fc1-8a41-9b659503163e_isrForCrosstalkSources_996952_1.stdout parsl.dataflow.dflow INFO: Standard error for task 809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/35c8333b-e080-4fc1-8a41-9b659503163e_isrForCrosstalkSources_996952_1.stderr 2024-03-15 13:53:23 proc_lsst.multi:146 [INFO] found job 17074833 in provider astro proc_lsst.multi INFO: found job 17074833 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 936 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4004ec68ada679c8adae2fa1824988b0 with result from task 936 parsl.dataflow.dflow DEBUG: Task 936 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/111f3b68-51ab-4e85-8095-794eb6d8a311_isrForCrosstalkSources_997021_8.stdout parsl.dataflow.dflow INFO: Standard error for task 936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/111f3b68-51ab-4e85-8095-794eb6d8a311_isrForCrosstalkSources_997021_8.stderr parsl.dataflow.dflow INFO: Task 939 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44b9c7524a65b091f98d7b3b571878d9 with result from task 939 parsl.dataflow.dflow DEBUG: Task 939 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a446093a-fba3-43c4-9e09-9bb6ed38f921_isrForCrosstalkSources_996999_31.stdout parsl.dataflow.dflow INFO: Standard error for task 939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a446093a-fba3-43c4-9e09-9bb6ed38f921_isrForCrosstalkSources_996999_31.stderr parsl.dataflow.dflow INFO: Task 917 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 196be02bd45815fe7a6e3ba5d0bcf27d with result from task 917 parsl.dataflow.dflow DEBUG: Task 917 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/1d1341b6-5b62-4ce2-8383-35c8348fafe3_isrForCrosstalkSources_997030_27.stdout parsl.dataflow.dflow INFO: Standard error for task 917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/1d1341b6-5b62-4ce2-8383-35c8348fafe3_isrForCrosstalkSources_997030_27.stderr parsl.dataflow.dflow INFO: Task 862 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91c0f957d2bb4d67471348a5a42779ec with result from task 862 parsl.dataflow.dflow DEBUG: Task 862 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/689785a9-9627-4338-982b-bfe7d9be7bb6_isrForCrosstalkSources_996981_23.stdout parsl.dataflow.dflow INFO: Standard error for task 862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/689785a9-9627-4338-982b-bfe7d9be7bb6_isrForCrosstalkSources_996981_23.stderr parsl.dataflow.dflow INFO: Task 805 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c97ed97a1aa2ae23ff1bd910813183c with result from task 805 parsl.dataflow.dflow DEBUG: Task 805 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/a38b8586-5a0d-454b-bf40-a1668e1869c0_isrForCrosstalkSources_996990_15.stdout parsl.dataflow.dflow INFO: Standard error for task 805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/a38b8586-5a0d-454b-bf40-a1668e1869c0_isrForCrosstalkSources_996990_15.stderr parsl.dataflow.dflow INFO: Task 852 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3bd8192bd4cc94b62cfd7c566b9838af with result from task 852 parsl.dataflow.dflow DEBUG: Task 852 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/42a28c6a-255b-4c36-afc4-30528cf0b391_isrForCrosstalkSources_997001_20.stdout parsl.dataflow.dflow INFO: Standard error for task 852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/42a28c6a-255b-4c36-afc4-30528cf0b391_isrForCrosstalkSources_997001_20.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:24 proc_lsst.multi:146 [INFO] found job 17074834 in provider astro proc_lsst.multi INFO: found job 17074834 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 934 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1b61b80e09a95eb6c6f02d36531b506 with result from task 934 parsl.dataflow.dflow DEBUG: Task 934 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1a7a1d78-d8d4-4f91-af49-49d2dbfe9aa7_isrForCrosstalkSources_997032_51.stdout parsl.dataflow.dflow INFO: Standard error for task 934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1a7a1d78-d8d4-4f91-af49-49d2dbfe9aa7_isrForCrosstalkSources_997032_51.stderr parsl.dataflow.dflow INFO: Task 974 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d2d46b0160b941e626aa902ee884143 with result from task 974 parsl.dataflow.dflow DEBUG: Task 974 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/de958e11-141a-4c2b-ba32-24fc726d36b5_isrForCrosstalkSources_996982_3.stdout parsl.dataflow.dflow INFO: Standard error for task 974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/de958e11-141a-4c2b-ba32-24fc726d36b5_isrForCrosstalkSources_996982_3.stderr parsl.dataflow.dflow INFO: Task 966 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0345043f73492bf3129fb6c06c112c35 with result from task 966 parsl.dataflow.dflow DEBUG: Task 966 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b8a52336-afaf-4efa-8c7a-338f61b3200d_isrForCrosstalkSources_996975_33.stdout parsl.dataflow.dflow INFO: Standard error for task 966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b8a52336-afaf-4efa-8c7a-338f61b3200d_isrForCrosstalkSources_996975_33.stderr parsl.dataflow.dflow INFO: Task 918 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c05999fbfb9b5d0a1dea23a11c22dc7 with result from task 918 parsl.dataflow.dflow DEBUG: Task 918 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c3133489-ce35-4e1b-9119-35853ac32603_isrForCrosstalkSources_997028_12.stdout parsl.dataflow.dflow INFO: Standard error for task 918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c3133489-ce35-4e1b-9119-35853ac32603_isrForCrosstalkSources_997028_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:25 proc_lsst.multi:146 [INFO] found job 17074835 in provider astro proc_lsst.multi INFO: found job 17074835 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 913 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b34bbd5ddcabe7ddda673ad7299d3e8 with result from task 913 parsl.dataflow.dflow DEBUG: Task 913 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/997e7b38-4c40-42ef-b591-12d25ae07f44_isrForCrosstalkSources_997022_22.stdout parsl.dataflow.dflow INFO: Standard error for task 913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/997e7b38-4c40-42ef-b591-12d25ae07f44_isrForCrosstalkSources_997022_22.stderr parsl.dataflow.dflow INFO: Task 865 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d28e8cfc82ca2ca8d5043192902c33bf with result from task 865 parsl.dataflow.dflow DEBUG: Task 865 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4952ac8e-eece-479e-9f84-32a5cf52589c_isrForCrosstalkSources_997023_44.stdout parsl.dataflow.dflow INFO: Standard error for task 865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4952ac8e-eece-479e-9f84-32a5cf52589c_isrForCrosstalkSources_997023_44.stderr parsl.dataflow.dflow INFO: Task 898 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d77c3307d4c8e60a4723994ae126f0f7 with result from task 898 parsl.dataflow.dflow DEBUG: Task 898 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e9f6317e-3d05-4641-8f85-b5833ff56456_isrForCrosstalkSources_997030_33.stdout parsl.dataflow.dflow INFO: Standard error for task 898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e9f6317e-3d05-4641-8f85-b5833ff56456_isrForCrosstalkSources_997030_33.stderr parsl.dataflow.dflow INFO: Task 912 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 339ce5edc321bcd5beab1644c5e500e9 with result from task 912 parsl.dataflow.dflow DEBUG: Task 912 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e19a96f9-c988-4d6c-9e43-1bc79d3e2b48_isrForCrosstalkSources_997023_36.stdout parsl.dataflow.dflow INFO: Standard error for task 912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e19a96f9-c988-4d6c-9e43-1bc79d3e2b48_isrForCrosstalkSources_997023_36.stderr parsl.dataflow.dflow INFO: Task 821 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 66216a777dfb26b60f47509bcecbebd8 with result from task 821 parsl.dataflow.dflow DEBUG: Task 821 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/97a69c8e-eb02-4e8b-a3cc-688f6bd7a023_isrForCrosstalkSources_996961_46.stdout parsl.dataflow.dflow INFO: Standard error for task 821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/97a69c8e-eb02-4e8b-a3cc-688f6bd7a023_isrForCrosstalkSources_996961_46.stderr parsl.dataflow.dflow INFO: Task 945 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b122dc71b12d421082fc17cc1fe87fb with result from task 945 parsl.dataflow.dflow DEBUG: Task 945 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9453da9d-f41a-4dd9-90a4-fea80089a8b5_isrForCrosstalkSources_996986_35.stdout parsl.dataflow.dflow INFO: Standard error for task 945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9453da9d-f41a-4dd9-90a4-fea80089a8b5_isrForCrosstalkSources_996986_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:26 proc_lsst.multi:146 [INFO] found job 17074836 in provider astro proc_lsst.multi INFO: found job 17074836 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 951 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef75ca4546eda875893cc623c8ac441d with result from task 951 parsl.dataflow.dflow DEBUG: Task 951 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/00e97e3c-a66c-47df-b116-9588638758c2_isrForCrosstalkSources_997021_12.stdout parsl.dataflow.dflow INFO: Standard error for task 951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/00e97e3c-a66c-47df-b116-9588638758c2_isrForCrosstalkSources_997021_12.stderr parsl.dataflow.dflow INFO: Task 927 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59e3f8e92218a472cca5f63d0c1c847a with result from task 927 parsl.dataflow.dflow DEBUG: Task 927 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ef95b0da-aa84-4db1-9648-c6b5d84ba963_isrForCrosstalkSources_997029_12.stdout parsl.dataflow.dflow INFO: Standard error for task 927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ef95b0da-aa84-4db1-9648-c6b5d84ba963_isrForCrosstalkSources_997029_12.stderr parsl.dataflow.dflow INFO: Task 796 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59c0804df24b6ed08164ab02f555f8a9 with result from task 796 parsl.dataflow.dflow DEBUG: Task 796 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/598cbe5a-1c50-4684-8d5c-2b48860ddfdb_isrForCrosstalkSources_997025_58.stdout parsl.dataflow.dflow INFO: Standard error for task 796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/598cbe5a-1c50-4684-8d5c-2b48860ddfdb_isrForCrosstalkSources_997025_58.stderr parsl.dataflow.dflow INFO: Task 992 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea06cb919fdf4dc1c2f5acfba5a19dde with result from task 992 parsl.dataflow.dflow DEBUG: Task 992 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2f3b592f-e4c6-4201-9ace-f8a419f8887b_isrForCrosstalkSources_997002_24.stdout parsl.dataflow.dflow INFO: Standard error for task 992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2f3b592f-e4c6-4201-9ace-f8a419f8887b_isrForCrosstalkSources_997002_24.stderr parsl.dataflow.dflow INFO: Task 948 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 204b7173cd885069b3d43891208d0764 with result from task 948 parsl.dataflow.dflow DEBUG: Task 948 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/0bda1630-816b-4915-a9b9-d4fabb3a4efd_isrForCrosstalkSources_996955_1.stdout parsl.dataflow.dflow INFO: Standard error for task 948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/0bda1630-816b-4915-a9b9-d4fabb3a4efd_isrForCrosstalkSources_996955_1.stderr parsl.dataflow.dflow INFO: Task 860 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b198d6dd548fc07a9cd3d98cdd6a5b43 with result from task 860 parsl.dataflow.dflow DEBUG: Task 860 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/33af21de-41ef-4aff-9c57-69232f5a3d80_isrForCrosstalkSources_997001_12.stdout parsl.dataflow.dflow INFO: Standard error for task 860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/33af21de-41ef-4aff-9c57-69232f5a3d80_isrForCrosstalkSources_997001_12.stderr parsl.dataflow.dflow INFO: Task 906 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a664e4326dc50c4d6012793209d25d9 with result from task 906 parsl.dataflow.dflow DEBUG: Task 906 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/f995f856-fe08-4fe6-ad96-203b4b5c1746_isrForCrosstalkSources_997021_28.stdout parsl.dataflow.dflow INFO: Standard error for task 906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/f995f856-fe08-4fe6-ad96-203b4b5c1746_isrForCrosstalkSources_997021_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:26 proc_lsst.multi:146 [INFO] found job 17074837 in provider astro proc_lsst.multi INFO: found job 17074837 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 863 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b376361d0c9b261951e166ccd0023db with result from task 863 parsl.dataflow.dflow DEBUG: Task 863 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/5d41d401-6a3f-4ee1-b6c6-21388a081e72_isrForCrosstalkSources_997029_41.stdout parsl.dataflow.dflow INFO: Standard error for task 863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/5d41d401-6a3f-4ee1-b6c6-21388a081e72_isrForCrosstalkSources_997029_41.stderr parsl.dataflow.dflow INFO: Task 960 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 071cc1db5379cd51efea2e60dd6df0eb with result from task 960 parsl.dataflow.dflow DEBUG: Task 960 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/f2a5a851-3b91-4ae5-a229-7719bc13c087_isrForCrosstalkSources_996994_57.stdout parsl.dataflow.dflow INFO: Standard error for task 960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/f2a5a851-3b91-4ae5-a229-7719bc13c087_isrForCrosstalkSources_996994_57.stderr parsl.dataflow.dflow INFO: Task 1007 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8cbb0fe8d3a6973f582f8a027e4230a with result from task 1007 parsl.dataflow.dflow DEBUG: Task 1007 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f27091cd-4b22-4ae4-b72f-4d932870c187_isrForCrosstalkSources_997005_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f27091cd-4b22-4ae4-b72f-4d932870c187_isrForCrosstalkSources_997005_51.stderr parsl.dataflow.dflow INFO: Task 971 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a5274dbb273aa8fdb8e95a6ca9cae2d with result from task 971 parsl.dataflow.dflow DEBUG: Task 971 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d2598782-d453-4711-bb5a-e6206682df75_isrForCrosstalkSources_996979_23.stdout parsl.dataflow.dflow INFO: Standard error for task 971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d2598782-d453-4711-bb5a-e6206682df75_isrForCrosstalkSources_996979_23.stderr parsl.dataflow.dflow INFO: Task 861 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 528049d80139259db2ff20a99323ff89 with result from task 861 parsl.dataflow.dflow DEBUG: Task 861 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/48fe4706-01ff-4eae-9b85-81da83342006_isrForCrosstalkSources_996998_42.stdout parsl.dataflow.dflow INFO: Standard error for task 861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/48fe4706-01ff-4eae-9b85-81da83342006_isrForCrosstalkSources_996998_42.stderr parsl.dataflow.dflow INFO: Task 891 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 410883af4a2fb07a1ae03f7aff26d3e0 with result from task 891 parsl.dataflow.dflow DEBUG: Task 891 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/078bb60e-9f36-45f0-bdd0-ab665851e6ab_isrForCrosstalkSources_996958_11.stdout parsl.dataflow.dflow INFO: Standard error for task 891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/078bb60e-9f36-45f0-bdd0-ab665851e6ab_isrForCrosstalkSources_996958_11.stderr parsl.dataflow.dflow INFO: Task 808 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 28e8b028efcc7bb6a0fdfae2892e335c with result from task 808 parsl.dataflow.dflow DEBUG: Task 808 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/3ea77ef0-961d-471b-86a7-acd2d31e5b66_isrForCrosstalkSources_996983_5.stdout parsl.dataflow.dflow INFO: Standard error for task 808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/3ea77ef0-961d-471b-86a7-acd2d31e5b66_isrForCrosstalkSources_996983_5.stderr parsl.dataflow.dflow INFO: Task 919 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8cac75de1b3ccfc361e693d11f857e4e with result from task 919 parsl.dataflow.dflow DEBUG: Task 919 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/dad2fd71-6043-457f-a8de-9d213df1eae4_isrForCrosstalkSources_996983_2.stdout parsl.dataflow.dflow INFO: Standard error for task 919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/dad2fd71-6043-457f-a8de-9d213df1eae4_isrForCrosstalkSources_996983_2.stderr parsl.dataflow.dflow INFO: Task 871 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 048678a0f781e5c9d842328124125306 with result from task 871 parsl.dataflow.dflow DEBUG: Task 871 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/11403d84-e6ae-4f71-b35a-9d068e881a29_isrForCrosstalkSources_996958_23.stdout parsl.dataflow.dflow INFO: Standard error for task 871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/11403d84-e6ae-4f71-b35a-9d068e881a29_isrForCrosstalkSources_996958_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:27 proc_lsst.multi:146 [INFO] found job 17074838 in provider astro proc_lsst.multi INFO: found job 17074838 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 943 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5466914f9a830ea76da0688f4af11b2f with result from task 943 parsl.dataflow.dflow DEBUG: Task 943 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/29ba8115-abf7-4e9c-8471-7e4ce157a2a8_isrForCrosstalkSources_997004_36.stdout parsl.dataflow.dflow INFO: Standard error for task 943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/29ba8115-abf7-4e9c-8471-7e4ce157a2a8_isrForCrosstalkSources_997004_36.stderr parsl.dataflow.dflow INFO: Task 938 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 426bfe04e5176bb97a0df22f7fe7e907 with result from task 938 parsl.dataflow.dflow DEBUG: Task 938 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/728fdc0c-67b6-4d91-9ee9-45a9eab62ac6_isrForCrosstalkSources_996993_43.stdout parsl.dataflow.dflow INFO: Standard error for task 938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/728fdc0c-67b6-4d91-9ee9-45a9eab62ac6_isrForCrosstalkSources_996993_43.stderr parsl.dataflow.dflow INFO: Task 798 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d55e5c2cc130f539d8085380088f88b4 with result from task 798 parsl.dataflow.dflow DEBUG: Task 798 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/0a4d0421-3c81-4309-91ea-8f9123077bd0_isrForCrosstalkSources_996987_47.stdout parsl.dataflow.dflow INFO: Standard error for task 798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/0a4d0421-3c81-4309-91ea-8f9123077bd0_isrForCrosstalkSources_996987_47.stderr parsl.dataflow.dflow INFO: Task 897 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4d0278ea22ffe7a67f5432c7685c7f96 with result from task 897 parsl.dataflow.dflow DEBUG: Task 897 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/51af2634-d344-403a-b1e5-2a55d5dd94a1_isrForCrosstalkSources_997025_49.stdout parsl.dataflow.dflow INFO: Standard error for task 897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/51af2634-d344-403a-b1e5-2a55d5dd94a1_isrForCrosstalkSources_997025_49.stderr parsl.dataflow.dflow INFO: Task 892 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 34cd882dc65a5dc67c7a62b7875005f4 with result from task 892 parsl.dataflow.dflow DEBUG: Task 892 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/af87b50b-6fab-4f2a-a259-203dd4af9cba_isrForCrosstalkSources_996996_2.stdout parsl.dataflow.dflow INFO: Standard error for task 892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/af87b50b-6fab-4f2a-a259-203dd4af9cba_isrForCrosstalkSources_996996_2.stderr parsl.dataflow.dflow INFO: Task 859 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a19ac2cd88fdfb294bf25da70ccc40a with result from task 859 parsl.dataflow.dflow DEBUG: Task 859 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/23923201-b7b8-4937-993c-5d959842cbb9_isrForCrosstalkSources_996977_31.stdout parsl.dataflow.dflow INFO: Standard error for task 859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/23923201-b7b8-4937-993c-5d959842cbb9_isrForCrosstalkSources_996977_31.stderr parsl.dataflow.dflow INFO: Task 881 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5781bc3e80b769bde60b57fd8c143e3b with result from task 881 parsl.dataflow.dflow DEBUG: Task 881 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/48b9f70b-51df-44cd-8f7d-b9a91415ef73_isrForCrosstalkSources_996974_29.stdout parsl.dataflow.dflow INFO: Standard error for task 881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/48b9f70b-51df-44cd-8f7d-b9a91415ef73_isrForCrosstalkSources_996974_29.stderr parsl.dataflow.dflow INFO: Task 907 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d6d13c552f2b5fe83eb0fd74cc01b4e5 with result from task 907 parsl.dataflow.dflow DEBUG: Task 907 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a8452c15-0488-4d97-a457-f46751b429e4_isrForCrosstalkSources_997015_60.stdout parsl.dataflow.dflow INFO: Standard error for task 907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a8452c15-0488-4d97-a457-f46751b429e4_isrForCrosstalkSources_997015_60.stderr parsl.dataflow.dflow INFO: Task 954 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 41beb22dba8367d953106f6e1642bfea with result from task 954 parsl.dataflow.dflow DEBUG: Task 954 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/61f6b43e-b955-4f96-a1e3-68adc9ec9b02_isrForCrosstalkSources_997004_14.stdout parsl.dataflow.dflow INFO: Standard error for task 954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/61f6b43e-b955-4f96-a1e3-68adc9ec9b02_isrForCrosstalkSources_997004_14.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:28 proc_lsst.multi:146 [INFO] found job 17074839 in provider astro proc_lsst.multi INFO: found job 17074839 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 806 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea745a675a18f014157d1f292ea449b4 with result from task 806 parsl.dataflow.dflow DEBUG: Task 806 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/29a80fad-a022-4229-8047-c000121b02f7_isrForCrosstalkSources_997025_57.stdout parsl.dataflow.dflow INFO: Standard error for task 806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/29a80fad-a022-4229-8047-c000121b02f7_isrForCrosstalkSources_997025_57.stderr parsl.dataflow.dflow INFO: Task 847 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11e164e39c42f3857e5dd0fd16f5e207 with result from task 847 parsl.dataflow.dflow DEBUG: Task 847 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b096b323-343f-4f50-aa25-7b1c92389a86_isrForCrosstalkSources_997025_36.stdout parsl.dataflow.dflow INFO: Standard error for task 847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/b096b323-343f-4f50-aa25-7b1c92389a86_isrForCrosstalkSources_997025_36.stderr parsl.dataflow.dflow INFO: Task 905 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b25f278bff2577ea004f846a4d25003 with result from task 905 parsl.dataflow.dflow DEBUG: Task 905 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7b234ece-b643-4e1a-a225-f281c59125c5_isrForCrosstalkSources_997013_48.stdout parsl.dataflow.dflow INFO: Standard error for task 905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7b234ece-b643-4e1a-a225-f281c59125c5_isrForCrosstalkSources_997013_48.stderr parsl.dataflow.dflow INFO: Task 903 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8af66b6097780988cdf937c69f56994f with result from task 903 parsl.dataflow.dflow DEBUG: Task 903 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/3391b787-e7d6-413f-9360-b73f6a5a0ca6_isrForCrosstalkSources_997013_36.stdout parsl.dataflow.dflow INFO: Standard error for task 903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/3391b787-e7d6-413f-9360-b73f6a5a0ca6_isrForCrosstalkSources_997013_36.stderr parsl.dataflow.dflow INFO: Task 977 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3bc5e5134ad80f510ab7df8b185b197c with result from task 977 parsl.dataflow.dflow DEBUG: Task 977 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/3ee415a1-fc3b-410c-8224-1ced65a98fdd_isrForCrosstalkSources_997013_46.stdout parsl.dataflow.dflow INFO: Standard error for task 977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/3ee415a1-fc3b-410c-8224-1ced65a98fdd_isrForCrosstalkSources_997013_46.stderr parsl.dataflow.dflow INFO: Task 980 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78d546967b2993e3715e99bc61129a68 with result from task 980 parsl.dataflow.dflow DEBUG: Task 980 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/09ba95d0-37cd-4dc9-ae9b-90c2d96faa14_isrForCrosstalkSources_996965_4.stdout parsl.dataflow.dflow INFO: Standard error for task 980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/09ba95d0-37cd-4dc9-ae9b-90c2d96faa14_isrForCrosstalkSources_996965_4.stderr parsl.dataflow.dflow INFO: Task 933 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75e4b9f853baa4be88053979757b322f with result from task 933 parsl.dataflow.dflow DEBUG: Task 933 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/8717e401-5bba-4bf0-8bd9-3618ba143791_isrForCrosstalkSources_996975_19.stdout parsl.dataflow.dflow INFO: Standard error for task 933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/8717e401-5bba-4bf0-8bd9-3618ba143791_isrForCrosstalkSources_996975_19.stderr parsl.dataflow.dflow INFO: Task 910 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26b50c3732c8682eed2b3cd933d39cba with result from task 910 parsl.dataflow.dflow DEBUG: Task 910 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e5fbbbf1-d1ba-495c-bc37-186355434bc6_isrForCrosstalkSources_996968_29.stdout parsl.dataflow.dflow INFO: Standard error for task 910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e5fbbbf1-d1ba-495c-bc37-186355434bc6_isrForCrosstalkSources_996968_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:29 proc_lsst.multi:146 [INFO] found job 17074841 in provider astro proc_lsst.multi INFO: found job 17074841 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 818 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d92bbc5a779b5e34d31d90866ef866c7 with result from task 818 parsl.dataflow.dflow DEBUG: Task 818 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/afb6144d-047f-4e43-ab0f-826fae86a24e_isrForCrosstalkSources_996994_29.stdout parsl.dataflow.dflow INFO: Standard error for task 818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/afb6144d-047f-4e43-ab0f-826fae86a24e_isrForCrosstalkSources_996994_29.stderr parsl.dataflow.dflow INFO: Task 887 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 694f8ab1a0427b2e747ea4192cee735f with result from task 887 parsl.dataflow.dflow DEBUG: Task 887 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cca988b9-6b14-4ca7-99cc-82e2e308309e_isrForCrosstalkSources_996957_48.stdout parsl.dataflow.dflow INFO: Standard error for task 887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cca988b9-6b14-4ca7-99cc-82e2e308309e_isrForCrosstalkSources_996957_48.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:29 proc_lsst.multi:146 [INFO] found job 17074842 in provider astro proc_lsst.multi INFO: found job 17074842 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 931 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3ad9737b2c4b6e44402a483b793199b4 with result from task 931 parsl.dataflow.dflow DEBUG: Task 931 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/1962d897-9a31-4aa2-9be4-4f9e6068c218_isrForCrosstalkSources_996985_16.stdout parsl.dataflow.dflow INFO: Standard error for task 931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/1962d897-9a31-4aa2-9be4-4f9e6068c218_isrForCrosstalkSources_996985_16.stderr parsl.dataflow.dflow INFO: Task 1013 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b3d5f097addf33a4f4a8e55a9d22e7e with result from task 1013 parsl.dataflow.dflow DEBUG: Task 1013 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b5174a84-2e66-4630-9127-6ce0460d6750_isrForCrosstalkSources_997033_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b5174a84-2e66-4630-9127-6ce0460d6750_isrForCrosstalkSources_997033_34.stderr parsl.dataflow.dflow INFO: Task 937 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e73616af61dc6e3122c60ab820dd6e3 with result from task 937 parsl.dataflow.dflow DEBUG: Task 937 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/2a1fd48f-c9cd-48d4-a285-e84959d050a6_isrForCrosstalkSources_996992_24.stdout parsl.dataflow.dflow INFO: Standard error for task 937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/2a1fd48f-c9cd-48d4-a285-e84959d050a6_isrForCrosstalkSources_996992_24.stderr parsl.dataflow.dflow INFO: Task 950 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fc524459d75f4fc851f9f067cdf6c1e3 with result from task 950 parsl.dataflow.dflow DEBUG: Task 950 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/19b8f33f-7551-406c-a2eb-cf15e1ac1448_isrForCrosstalkSources_997031_44.stdout parsl.dataflow.dflow INFO: Standard error for task 950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/19b8f33f-7551-406c-a2eb-cf15e1ac1448_isrForCrosstalkSources_997031_44.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:30 proc_lsst.multi:146 [INFO] found job 17074843 in provider astro proc_lsst.multi INFO: found job 17074843 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 894 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3778b3774d9d61af5f54ca6bb119607c with result from task 894 parsl.dataflow.dflow DEBUG: Task 894 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8543486d-22f9-415b-96c3-51e5b7360b6c_isrForCrosstalkSources_996973_3.stdout parsl.dataflow.dflow INFO: Standard error for task 894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8543486d-22f9-415b-96c3-51e5b7360b6c_isrForCrosstalkSources_996973_3.stderr parsl.dataflow.dflow INFO: Task 882 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11d93210664b8cfc3e0afeeb71cfdca8 with result from task 882 parsl.dataflow.dflow DEBUG: Task 882 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ca5003ea-f12f-43db-b9cf-9b09969b76e9_isrForCrosstalkSources_997001_57.stdout parsl.dataflow.dflow INFO: Standard error for task 882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ca5003ea-f12f-43db-b9cf-9b09969b76e9_isrForCrosstalkSources_997001_57.stderr parsl.dataflow.dflow INFO: Task 850 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 517723fc197b5e3dbe3a16ace3723059 with result from task 850 parsl.dataflow.dflow DEBUG: Task 850 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/1a2a22e5-6895-4062-91ba-34c3df47123c_isrForCrosstalkSources_996972_46.stdout parsl.dataflow.dflow INFO: Standard error for task 850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/1a2a22e5-6895-4062-91ba-34c3df47123c_isrForCrosstalkSources_996972_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:30 proc_lsst.multi:146 [INFO] found job 17074850 in provider astro proc_lsst.multi INFO: found job 17074850 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 944 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ceff5de246c62815b14b7e346ac76f2d with result from task 944 parsl.dataflow.dflow DEBUG: Task 944 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/785d3d84-4a30-410b-8fbb-7b42c7ed8f24_isrForCrosstalkSources_996965_1.stdout parsl.dataflow.dflow INFO: Standard error for task 944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/785d3d84-4a30-410b-8fbb-7b42c7ed8f24_isrForCrosstalkSources_996965_1.stderr parsl.dataflow.dflow INFO: Task 952 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d461e79f2fca97707de0d88e9afd5ada with result from task 952 parsl.dataflow.dflow DEBUG: Task 952 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7124b386-05b0-40dd-b684-10c86bb690bc_isrForCrosstalkSources_996993_6.stdout parsl.dataflow.dflow INFO: Standard error for task 952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7124b386-05b0-40dd-b684-10c86bb690bc_isrForCrosstalkSources_996993_6.stderr parsl.dataflow.dflow INFO: Task 830 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c4e90831fb2a81b693812786fe7998e with result from task 830 parsl.dataflow.dflow DEBUG: Task 830 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7fb78965-0fb9-442e-829e-a4b1d442f337_isrForCrosstalkSources_997028_22.stdout parsl.dataflow.dflow INFO: Standard error for task 830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7fb78965-0fb9-442e-829e-a4b1d442f337_isrForCrosstalkSources_997028_22.stderr parsl.dataflow.dflow INFO: Task 1003 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff5e63bd767ce839f709ea63974c21d2 with result from task 1003 parsl.dataflow.dflow DEBUG: Task 1003 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a20996f2-b061-4d02-8a84-43f8406d308a_isrForCrosstalkSources_996965_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a20996f2-b061-4d02-8a84-43f8406d308a_isrForCrosstalkSources_996965_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:31 proc_lsst.multi:146 [INFO] found job 17074851 in provider astro proc_lsst.multi INFO: found job 17074851 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 868 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b10cf47d33b2a8e489b4dee39d255ff with result from task 868 parsl.dataflow.dflow DEBUG: Task 868 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/73f958b4-a0fe-47c5-b3d9-f47cd9b05766_isrForCrosstalkSources_996984_56.stdout parsl.dataflow.dflow INFO: Standard error for task 868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/73f958b4-a0fe-47c5-b3d9-f47cd9b05766_isrForCrosstalkSources_996984_56.stderr parsl.dataflow.dflow INFO: Task 866 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9c9569119eaf919a143ec8a656989b2 with result from task 866 parsl.dataflow.dflow DEBUG: Task 866 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2edc0ca3-5bba-4eb6-9d13-9c4a50929fe7_isrForCrosstalkSources_997015_41.stdout parsl.dataflow.dflow INFO: Standard error for task 866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2edc0ca3-5bba-4eb6-9d13-9c4a50929fe7_isrForCrosstalkSources_997015_41.stderr parsl.dataflow.dflow INFO: Task 904 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce591dc577eaae47bbd13a462c78fff8 with result from task 904 parsl.dataflow.dflow DEBUG: Task 904 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/b8a7c629-9f33-4cfa-be03-3995fc682ead_isrForCrosstalkSources_996978_36.stdout parsl.dataflow.dflow INFO: Standard error for task 904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/b8a7c629-9f33-4cfa-be03-3995fc682ead_isrForCrosstalkSources_996978_36.stderr parsl.dataflow.dflow INFO: Task 857 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf072a122c645a573e91abd591041c62 with result from task 857 parsl.dataflow.dflow DEBUG: Task 857 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/50478ef0-320b-4825-9d41-caf945eed432_isrForCrosstalkSources_996998_36.stdout parsl.dataflow.dflow INFO: Standard error for task 857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/50478ef0-320b-4825-9d41-caf945eed432_isrForCrosstalkSources_996998_36.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:32 proc_lsst.multi:146 [INFO] found job 17074852 in provider astro proc_lsst.multi INFO: found job 17074852 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 941 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33e852df8ccea38610343c6cafc5f5b4 with result from task 941 parsl.dataflow.dflow DEBUG: Task 941 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/32cf3899-2294-4ec6-9913-cdc29a062131_isrForCrosstalkSources_997014_39.stdout parsl.dataflow.dflow INFO: Standard error for task 941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/32cf3899-2294-4ec6-9913-cdc29a062131_isrForCrosstalkSources_997014_39.stderr parsl.dataflow.dflow INFO: Task 872 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8906d80646c22c66b4b148c8b96d8c16 with result from task 872 parsl.dataflow.dflow DEBUG: Task 872 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f9e149c2-36c4-4ec8-950f-98df23e21f3a_isrForCrosstalkSources_996985_41.stdout parsl.dataflow.dflow INFO: Standard error for task 872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f9e149c2-36c4-4ec8-950f-98df23e21f3a_isrForCrosstalkSources_996985_41.stderr parsl.dataflow.dflow INFO: Task 930 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 697b1d92b2f7903f2fab05e575cf379b with result from task 930 parsl.dataflow.dflow DEBUG: Task 930 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/025e9d6c-a480-4ad9-91df-2931701b4066_isrForCrosstalkSources_996985_50.stdout parsl.dataflow.dflow INFO: Standard error for task 930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/025e9d6c-a480-4ad9-91df-2931701b4066_isrForCrosstalkSources_996985_50.stderr parsl.dataflow.dflow INFO: Task 1023 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d188a74c607376bc4169d385328d9031 with result from task 1023 parsl.dataflow.dflow DEBUG: Task 1023 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/4e70c73f-e627-46b1-a80d-f1a86afe63a9_isrForCrosstalkSources_997004_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/4e70c73f-e627-46b1-a80d-f1a86afe63a9_isrForCrosstalkSources_997004_42.stderr parsl.dataflow.dflow INFO: Task 845 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3502d2cfeea2e86f2b2612577da421f8 with result from task 845 parsl.dataflow.dflow DEBUG: Task 845 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/9c55f14b-f320-4604-8769-c46ac3d164d4_isrForCrosstalkSources_996976_60.stdout parsl.dataflow.dflow INFO: Standard error for task 845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/9c55f14b-f320-4604-8769-c46ac3d164d4_isrForCrosstalkSources_996976_60.stderr parsl.dataflow.dflow INFO: Task 878 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7538fa45e1a983a15370074cb27cfe4 with result from task 878 parsl.dataflow.dflow DEBUG: Task 878 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9badf252-476e-481c-b06e-2da33f4bd4dd_isrForCrosstalkSources_997000_11.stdout parsl.dataflow.dflow INFO: Standard error for task 878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9badf252-476e-481c-b06e-2da33f4bd4dd_isrForCrosstalkSources_997000_11.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:32 proc_lsst.multi:146 [INFO] found job 17074853 in provider astro proc_lsst.multi INFO: found job 17074853 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 880 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fb4fc98c0f9568e373041933366c10e with result from task 880 parsl.dataflow.dflow DEBUG: Task 880 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f0651d85-f576-4ba7-9643-a18a2a503875_isrForCrosstalkSources_996958_43.stdout parsl.dataflow.dflow INFO: Standard error for task 880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f0651d85-f576-4ba7-9643-a18a2a503875_isrForCrosstalkSources_996958_43.stderr parsl.dataflow.dflow INFO: Task 932 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86f3b0f19e80d0f4d0a2f366463af216 with result from task 932 parsl.dataflow.dflow DEBUG: Task 932 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d13dc73f-ceb9-428b-82c2-69eac9d602a7_isrForCrosstalkSources_996981_41.stdout parsl.dataflow.dflow INFO: Standard error for task 932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d13dc73f-ceb9-428b-82c2-69eac9d602a7_isrForCrosstalkSources_996981_41.stderr parsl.dataflow.dflow INFO: Task 925 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 991f503a489c835bf546ff3287054671 with result from task 925 parsl.dataflow.dflow DEBUG: Task 925 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c3ce31bc-355d-4d69-8b7f-98b70e992bae_isrForCrosstalkSources_996991_4.stdout parsl.dataflow.dflow INFO: Standard error for task 925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c3ce31bc-355d-4d69-8b7f-98b70e992bae_isrForCrosstalkSources_996991_4.stderr parsl.dataflow.dflow INFO: Task 981 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4232769ef59cce868c72581ac0dda9b8 with result from task 981 parsl.dataflow.dflow DEBUG: Task 981 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/91536af8-bcea-4317-8a53-7b525e037f84_isrForCrosstalkSources_997017_4.stdout parsl.dataflow.dflow INFO: Standard error for task 981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/91536af8-bcea-4317-8a53-7b525e037f84_isrForCrosstalkSources_997017_4.stderr parsl.dataflow.dflow INFO: Task 1000 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be78e7f0a228e3d3c2bbfdb315798dc1 with result from task 1000 parsl.dataflow.dflow DEBUG: Task 1000 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4a2bbdc3-6400-4c09-8a8c-68609e66b859_isrForCrosstalkSources_996972_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/4a2bbdc3-6400-4c09-8a8c-68609e66b859_isrForCrosstalkSources_996972_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:33 proc_lsst.multi:146 [INFO] found job 17074854 in provider astro proc_lsst.multi INFO: found job 17074854 in provider astro parsl.dataflow.dflow INFO: Task 823 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.memoization DEBUG: Storing app cache entry cd9c7a396d361c44b6737b9468b971c4 with result from task 823 parsl.dataflow.dflow DEBUG: Task 823 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c6e46aa4-8e53-4034-bc66-85a1a3c612dc_isrForCrosstalkSources_996994_43.stdout parsl.dataflow.dflow INFO: Standard error for task 823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c6e46aa4-8e53-4034-bc66-85a1a3c612dc_isrForCrosstalkSources_996994_43.stderr parsl.dataflow.dflow INFO: Task 935 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a530a14ceea4e321be4a71b85c804bf with result from task 935 parsl.dataflow.dflow DEBUG: Task 935 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/aaf2b3bf-7086-49f7-b85e-099dbb31c755_isrForCrosstalkSources_997009_46.stdout parsl.dataflow.dflow INFO: Standard error for task 935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/aaf2b3bf-7086-49f7-b85e-099dbb31c755_isrForCrosstalkSources_997009_46.stderr parsl.dataflow.dflow INFO: Task 853 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 108d6bbd2d978d631576498d1965c79d with result from task 853 parsl.dataflow.dflow DEBUG: Task 853 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e50c54d5-1d64-49b5-a7a4-0422c3c54774_isrForCrosstalkSources_996998_55.stdout parsl.dataflow.dflow INFO: Standard error for task 853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e50c54d5-1d64-49b5-a7a4-0422c3c54774_isrForCrosstalkSources_996998_55.stderr parsl.dataflow.dflow INFO: Task 1015 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 791f9d8880da23e60304db0089e8b8c2 with result from task 1015 parsl.dataflow.dflow DEBUG: Task 1015 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1420c740-b583-4719-8133-c8f812eee047_isrForCrosstalkSources_996961_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1420c740-b583-4719-8133-c8f812eee047_isrForCrosstalkSources_996961_24.stderr parsl.dataflow.dflow INFO: Task 975 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 340f525da12e201db3b895478853dac4 with result from task 975 parsl.dataflow.dflow DEBUG: Task 975 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/896b5f44-57f5-457a-a256-764a3de2ec54_isrForCrosstalkSources_996994_1.stdout parsl.dataflow.dflow INFO: Standard error for task 975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/896b5f44-57f5-457a-a256-764a3de2ec54_isrForCrosstalkSources_996994_1.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:33 proc_lsst.multi:146 [INFO] found job 17074855 in provider astro proc_lsst.multi INFO: found job 17074855 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:34 proc_lsst.multi:146 [INFO] found job 17074856 in provider astro proc_lsst.multi INFO: found job 17074856 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 833 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4dfc2b8d7abbf8e156f5a0154e0fa7c with result from task 833 parsl.dataflow.dflow DEBUG: Task 833 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e228ea8a-7954-4b0b-ba98-88e02702bc48_isrForCrosstalkSources_996996_31.stdout parsl.dataflow.dflow INFO: Standard error for task 833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e228ea8a-7954-4b0b-ba98-88e02702bc48_isrForCrosstalkSources_996996_31.stderr parsl.dataflow.dflow INFO: Task 962 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27c73a527f08759bd23eea611a0cc4a5 with result from task 962 parsl.dataflow.dflow DEBUG: Task 962 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0c05e74c-f6ea-4c5b-b600-f994ab6eb528_isrForCrosstalkSources_997023_18.stdout parsl.dataflow.dflow INFO: Standard error for task 962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0c05e74c-f6ea-4c5b-b600-f994ab6eb528_isrForCrosstalkSources_997023_18.stderr parsl.dataflow.dflow INFO: Task 921 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 95009d483da348ad0a246e03abf661e3 with result from task 921 parsl.dataflow.dflow DEBUG: Task 921 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c71431db-4def-49ca-9305-e39d306248a0_isrForCrosstalkSources_996965_57.stdout parsl.dataflow.dflow INFO: Standard error for task 921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c71431db-4def-49ca-9305-e39d306248a0_isrForCrosstalkSources_996965_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 843 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 24bd06dca7c9dd9657a89fc6b7b12fe5 with result from task 843 parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 843 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:34 proc_lsst.multi:146 [INFO] found job 17074857 in provider astro proc_lsst.multi INFO: found job 17074857 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/9d01e152-0937-479c-b08e-cd72618e5803_isrForCrosstalkSources_997007_18.stdout parsl.dataflow.dflow INFO: Standard error for task 843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/9d01e152-0937-479c-b08e-cd72618e5803_isrForCrosstalkSources_997007_18.stderr parsl.dataflow.dflow INFO: Task 869 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e487ef719eae24caca31ef3e86a57d88 with result from task 869 parsl.dataflow.dflow DEBUG: Task 869 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e1c00d3e-fb58-42bd-98fa-d8f3bf11ff8f_isrForCrosstalkSources_996984_38.stdout parsl.dataflow.dflow INFO: Standard error for task 869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e1c00d3e-fb58-42bd-98fa-d8f3bf11ff8f_isrForCrosstalkSources_996984_38.stderr parsl.dataflow.dflow INFO: Task 984 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d0c1534b8a33cbe602cf4e9440e3e85 with result from task 984 parsl.dataflow.dflow DEBUG: Task 984 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f4762ba3-ffad-4b61-84d8-bbce4d473d20_isrForCrosstalkSources_996966_9.stdout parsl.dataflow.dflow INFO: Standard error for task 984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f4762ba3-ffad-4b61-84d8-bbce4d473d20_isrForCrosstalkSources_996966_9.stderr parsl.dataflow.dflow INFO: Task 819 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62c21ec60dc4fb72298ab860af7c4834 with result from task 819 parsl.dataflow.dflow DEBUG: Task 819 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/8f725df4-c9b0-4008-970c-2d4f2d609438_isrForCrosstalkSources_996990_50.stdout parsl.dataflow.dflow INFO: Standard error for task 819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/8f725df4-c9b0-4008-970c-2d4f2d609438_isrForCrosstalkSources_996990_50.stderr parsl.dataflow.dflow INFO: Task 928 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5dbda31aaa162755b3a6a13ba496569b with result from task 928 parsl.dataflow.dflow DEBUG: Task 928 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/9001ec41-bc01-48ca-8bfb-12f590ae50b7_isrForCrosstalkSources_997016_10.stdout parsl.dataflow.dflow INFO: Standard error for task 928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/9001ec41-bc01-48ca-8bfb-12f590ae50b7_isrForCrosstalkSources_997016_10.stderr parsl.dataflow.dflow INFO: Task 970 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5335296dc5a9287b29ccd49d7c0efcdf with result from task 970 parsl.dataflow.dflow DEBUG: Task 970 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/5c54f6b0-03bd-431a-85c1-d6bcb1d1ff3b_isrForCrosstalkSources_996958_38.stdout parsl.dataflow.dflow INFO: Standard error for task 970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/5c54f6b0-03bd-431a-85c1-d6bcb1d1ff3b_isrForCrosstalkSources_996958_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:35 proc_lsst.multi:146 [INFO] found job 17074859 in provider astro proc_lsst.multi INFO: found job 17074859 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 1054 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bddf30f64eb816e26321127b7c229293 with result from task 1054 parsl.dataflow.dflow DEBUG: Task 1054 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c1169a95-95bf-4361-b017-a2639bf9fef7_isrForCrosstalkSources_996995_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c1169a95-95bf-4361-b017-a2639bf9fef7_isrForCrosstalkSources_996995_16.stderr parsl.dataflow.dflow INFO: Task 953 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f38f7008a940f5ed44dce9e6c7416e20 with result from task 953 parsl.dataflow.dflow DEBUG: Task 953 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/db6ef0cf-6d43-415a-9adc-ef9828138cbc_isrForCrosstalkSources_996961_57.stdout parsl.dataflow.dflow INFO: Standard error for task 953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/db6ef0cf-6d43-415a-9adc-ef9828138cbc_isrForCrosstalkSources_996961_57.stderr parsl.dataflow.dflow INFO: Task 877 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae3a98d47db5da13d6790f406915dd58 with result from task 877 parsl.dataflow.dflow DEBUG: Task 877 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/64adc7cd-8992-415f-80fc-bbfd93498ac9_isrForCrosstalkSources_997024_53.stdout parsl.dataflow.dflow INFO: Standard error for task 877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/64adc7cd-8992-415f-80fc-bbfd93498ac9_isrForCrosstalkSources_997024_53.stderr parsl.dataflow.dflow INFO: Task 961 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8770315730ceaf41b910f2b1f3c3b6a6 with result from task 961 parsl.dataflow.dflow DEBUG: Task 961 checkpointed parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/38181a03-a93a-4c44-a4c1-d6aab1bcb5cd_isrForCrosstalkSources_996953_37.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/38181a03-a93a-4c44-a4c1-d6aab1bcb5cd_isrForCrosstalkSources_996953_37.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:35 proc_lsst.multi:146 [INFO] found job 17074861 in provider astro proc_lsst.multi INFO: found job 17074861 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 940 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c7125a7fbefe0b126dad48f6730a262 with result from task 940 parsl.dataflow.dflow DEBUG: Task 940 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fbeb28bb-2796-4912-8da8-c8eea1b5ce9f_isrForCrosstalkSources_996955_7.stdout parsl.dataflow.dflow INFO: Standard error for task 940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fbeb28bb-2796-4912-8da8-c8eea1b5ce9f_isrForCrosstalkSources_996955_7.stderr parsl.dataflow.dflow INFO: Task 874 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ea29cbfb5f923fad0520e284540028f with result from task 874 parsl.dataflow.dflow DEBUG: Task 874 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/f2934c9f-84cc-446b-a378-698eb026f235_isrForCrosstalkSources_997011_38.stdout parsl.dataflow.dflow INFO: Standard error for task 874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/f2934c9f-84cc-446b-a378-698eb026f235_isrForCrosstalkSources_997011_38.stderr parsl.dataflow.dflow INFO: Task 991 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 408cde82e7c50b6252351c317a120278 with result from task 991 parsl.dataflow.dflow DEBUG: Task 991 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2f225d23-beb4-4747-937e-f815b20bde4a_isrForCrosstalkSources_997002_42.stdout parsl.dataflow.dflow INFO: Standard error for task 991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2f225d23-beb4-4747-937e-f815b20bde4a_isrForCrosstalkSources_997002_42.stderr parsl.dataflow.dflow INFO: Task 982 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d73c2930db83ab87089607aa6ef9c2c with result from task 982 parsl.dataflow.dflow DEBUG: Task 982 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/d514d75b-dc56-46f8-94bd-085054f8e06c_isrForCrosstalkSources_997029_2.stdout parsl.dataflow.dflow INFO: Standard error for task 982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/d514d75b-dc56-46f8-94bd-085054f8e06c_isrForCrosstalkSources_997029_2.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:36 proc_lsst.multi:146 [INFO] found job 17074863 in provider astro proc_lsst.multi INFO: found job 17074863 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 902 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23c66001caf9e0afa4dc16a1ba4504d1 with result from task 902 parsl.dataflow.dflow DEBUG: Task 902 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5a22932e-b79d-4f9a-a9c7-b45b46f519c1_isrForCrosstalkSources_996964_49.stdout parsl.dataflow.dflow INFO: Standard error for task 902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5a22932e-b79d-4f9a-a9c7-b45b46f519c1_isrForCrosstalkSources_996964_49.stderr parsl.dataflow.dflow INFO: Task 963 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a203555b33e93f8faaf6a7756b903ab with result from task 963 parsl.dataflow.dflow DEBUG: Task 963 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/c3516cc3-03a2-4e2d-9243-444ddb00266b_isrForCrosstalkSources_996953_45.stdout parsl.dataflow.dflow INFO: Standard error for task 963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/c3516cc3-03a2-4e2d-9243-444ddb00266b_isrForCrosstalkSources_996953_45.stderr parsl.dataflow.dflow INFO: Task 924 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 98edde3ce5384d4581f4af1b240d0ead with result from task 924 parsl.dataflow.dflow DEBUG: Task 924 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/5085bafe-fe2f-4389-94d5-8fb1ca3b7acb_isrForCrosstalkSources_997029_3.stdout parsl.dataflow.dflow INFO: Standard error for task 924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/5085bafe-fe2f-4389-94d5-8fb1ca3b7acb_isrForCrosstalkSources_997029_3.stderr parsl.dataflow.dflow INFO: Task 1031 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 74fcb469bc0059a2fff2a5803d7edb4a with result from task 1031 parsl.dataflow.dflow DEBUG: Task 1031 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/314a81ea-0985-4b99-8f71-bc48df093cf0_isrForCrosstalkSources_997031_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/314a81ea-0985-4b99-8f71-bc48df093cf0_isrForCrosstalkSources_997031_49.stderr parsl.dataflow.dflow INFO: Task 1009 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 434ecc5ed78f265bcf9af96a42905478 with result from task 1009 parsl.dataflow.dflow DEBUG: Task 1009 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/40575bad-dd1c-4a56-ba7a-8b4bf0c7c62c_isrForCrosstalkSources_996954_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/40575bad-dd1c-4a56-ba7a-8b4bf0c7c62c_isrForCrosstalkSources_996954_32.stderr parsl.dataflow.dflow INFO: Task 1048 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0151cfd8b017936c3474cb4c39b9dd63 with result from task 1048 parsl.dataflow.dflow DEBUG: Task 1048 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/707049ab-7b2e-4945-bb92-91652d324fd9_isrForCrosstalkSources_997007_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/707049ab-7b2e-4945-bb92-91652d324fd9_isrForCrosstalkSources_997007_14.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:36 proc_lsst.multi:146 [INFO] found job 17074865 in provider astro proc_lsst.multi INFO: found job 17074865 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 1044 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 518458aa6b25229761314b525c775736 with result from task 1044 parsl.dataflow.dflow DEBUG: Task 1044 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/28018e0a-fb60-43cd-ab9f-51925779fb95_isrForCrosstalkSources_996996_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/28018e0a-fb60-43cd-ab9f-51925779fb95_isrForCrosstalkSources_996996_39.stderr parsl.dataflow.dflow INFO: Task 814 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9aac46b8d656920c3375d3b4c7ac4d7f with result from task 814 parsl.dataflow.dflow DEBUG: Task 814 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/48b63da0-497c-4602-ab9f-d80de55d6443_isrForCrosstalkSources_996966_28.stdout parsl.dataflow.dflow INFO: Standard error for task 814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/48b63da0-497c-4602-ab9f-d80de55d6443_isrForCrosstalkSources_996966_28.stderr parsl.dataflow.dflow INFO: Task 1037 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f6684a0d881ed8334eb5e0b2820cc1db with result from task 1037 parsl.dataflow.dflow DEBUG: Task 1037 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/ff99ca74-62c3-4bbc-8c00-2a739d41bbbe_isrForCrosstalkSources_997012_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/ff99ca74-62c3-4bbc-8c00-2a739d41bbbe_isrForCrosstalkSources_997012_50.stderr parsl.dataflow.dflow INFO: Task 1066 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18b252e4cf20a375d7a7edbfbc8dc7f2 with result from task 1066 parsl.dataflow.dflow DEBUG: Task 1066 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b8c7af25-ab61-4c7e-87e7-c704ee26363e_isrForCrosstalkSources_996953_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b8c7af25-ab61-4c7e-87e7-c704ee26363e_isrForCrosstalkSources_996953_43.stderr parsl.dataflow.dflow INFO: Task 1022 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0712032f201e6749dc5fb8373999b59b with result from task 1022 parsl.dataflow.dflow DEBUG: Task 1022 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/43d87b41-531e-4119-b95f-99dc9b5d9053_isrForCrosstalkSources_996971_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/43d87b41-531e-4119-b95f-99dc9b5d9053_isrForCrosstalkSources_996971_34.stderr parsl.dataflow.dflow INFO: Task 1053 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e45c0346b5a48edacf1d38bbf1f60251 with result from task 1053 parsl.dataflow.dflow DEBUG: Task 1053 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/26700e75-540a-4599-98d0-21bdeedded5c_isrForCrosstalkSources_996987_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/26700e75-540a-4599-98d0-21bdeedded5c_isrForCrosstalkSources_996987_40.stderr parsl.dataflow.dflow INFO: Task 996 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 932f6f2bc02de2d9c7d23e059f2201b6 with result from task 996 parsl.dataflow.dflow DEBUG: Task 996 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/5894c832-b65e-4854-8ba2-b85516b355f0_isrForCrosstalkSources_996984_40.stdout parsl.dataflow.dflow INFO: Standard error for task 996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/5894c832-b65e-4854-8ba2-b85516b355f0_isrForCrosstalkSources_996984_40.stderr parsl.dataflow.dflow INFO: Task 958 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c9e498ae087aefbdb7ab4ac8795b4f5 with result from task 958 parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:37 proc_lsst.multi:146 [INFO] found job 17074869 in provider astro proc_lsst.multi INFO: found job 17074869 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow DEBUG: Task 958 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/1f30fa2e-d1a0-48ef-8ae3-af0f0a90d06a_isrForCrosstalkSources_996976_53.stdout parsl.dataflow.dflow INFO: Standard error for task 958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/1f30fa2e-d1a0-48ef-8ae3-af0f0a90d06a_isrForCrosstalkSources_996976_53.stderr parsl.dataflow.dflow INFO: Task 876 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a0cd4e5f4052c26c892c643a78a5a3f with result from task 876 parsl.dataflow.dflow DEBUG: Task 876 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/140fcced-5a0b-4934-ab88-eaa82c764c31_isrForCrosstalkSources_996962_35.stdout parsl.dataflow.dflow INFO: Standard error for task 876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/140fcced-5a0b-4934-ab88-eaa82c764c31_isrForCrosstalkSources_996962_35.stderr parsl.dataflow.dflow INFO: Task 1063 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90862a848ba6410c373946cfe1f960fd with result from task 1063 parsl.dataflow.dflow DEBUG: Task 1063 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f4681f4-5833-4ac3-b7ff-d48b1b16c415_isrForCrosstalkSources_996994_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f4681f4-5833-4ac3-b7ff-d48b1b16c415_isrForCrosstalkSources_996994_25.stderr parsl.dataflow.dflow INFO: Task 896 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e24302ab3ed7dba1fd12aa6bb00792b6 with result from task 896 parsl.dataflow.dflow DEBUG: Task 896 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/577b0b5b-f25a-4459-93bf-44ce45f67398_isrForCrosstalkSources_996980_11.stdout parsl.dataflow.dflow INFO: Standard error for task 896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/577b0b5b-f25a-4459-93bf-44ce45f67398_isrForCrosstalkSources_996980_11.stderr parsl.dataflow.dflow INFO: Task 976 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 34eaf940942d182cdb6ec5098a9713eb with result from task 976 parsl.dataflow.dflow DEBUG: Task 976 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/9262851d-aa2b-453c-aebc-6d2499f3ee44_isrForCrosstalkSources_996992_53.stdout parsl.dataflow.dflow INFO: Standard error for task 976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/9262851d-aa2b-453c-aebc-6d2499f3ee44_isrForCrosstalkSources_996992_53.stderr parsl.dataflow.dflow INFO: Task 988 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a76f6bd94638768342b98200667b1329 with result from task 988 parsl.dataflow.dflow DEBUG: Task 988 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/60f36c43-cac2-4c61-8d4a-2455f24eb560_isrForCrosstalkSources_996973_18.stdout parsl.dataflow.dflow INFO: Standard error for task 988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/60f36c43-cac2-4c61-8d4a-2455f24eb560_isrForCrosstalkSources_996973_18.stderr parsl.dataflow.dflow INFO: Task 851 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13a97d8d239e36caddbc8a9a4fe88d00 with result from task 851 parsl.dataflow.dflow DEBUG: Task 851 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d6e81ec2-6454-44bb-8b5d-eae93d50ef46_isrForCrosstalkSources_997024_30.stdout parsl.dataflow.dflow INFO: Standard error for task 851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d6e81ec2-6454-44bb-8b5d-eae93d50ef46_isrForCrosstalkSources_997024_30.stderr parsl.dataflow.dflow INFO: Task 1008 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58772f85d1263984dc8266d45e6de0df with result from task 1008 parsl.dataflow.dflow DEBUG: Task 1008 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/790381ee-f65f-4ec0-b1d1-1e3b0c57a2ca_isrForCrosstalkSources_996982_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/790381ee-f65f-4ec0-b1d1-1e3b0c57a2ca_isrForCrosstalkSources_996982_21.stderr parsl.dataflow.dflow INFO: Task 985 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c6e817430ac188ae1ee06a9d15985aee with result from task 985 parsl.dataflow.dflow DEBUG: Task 985 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/9a4f0394-75a2-4967-a976-c0c1008ecd10_isrForCrosstalkSources_997025_3.stdout parsl.dataflow.dflow INFO: Standard error for task 985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/9a4f0394-75a2-4967-a976-c0c1008ecd10_isrForCrosstalkSources_997025_3.stderr parsl.dataflow.dflow INFO: Task 916 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3527e62dce6545272b4db0ac597fe59f with result from task 916 parsl.dataflow.dflow DEBUG: Task 916 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d0360362-1b2d-40ae-9eb6-7b3308659148_isrForCrosstalkSources_996981_48.stdout parsl.dataflow.dflow INFO: Standard error for task 916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d0360362-1b2d-40ae-9eb6-7b3308659148_isrForCrosstalkSources_996981_48.stderr parsl.dataflow.dflow INFO: Task 965 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9cdeb76d793d116bad3554d68f5e0d31 with result from task 965 parsl.dataflow.dflow DEBUG: Task 965 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/857f8d43-09d6-479c-a7ee-25463438db04_isrForCrosstalkSources_996959_53.stdout parsl.dataflow.dflow INFO: Standard error for task 965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/857f8d43-09d6-479c-a7ee-25463438db04_isrForCrosstalkSources_996959_53.stderr parsl.dataflow.dflow INFO: Task 914 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 282f91d765e9267938ae7cf76ff256b7 with result from task 914 parsl.dataflow.dflow DEBUG: Task 914 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d93610fe-e27a-40f6-b543-1be9487f0f00_isrForCrosstalkSources_997030_49.stdout parsl.dataflow.dflow INFO: Standard error for task 914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d93610fe-e27a-40f6-b543-1be9487f0f00_isrForCrosstalkSources_997030_49.stderr parsl.dataflow.dflow INFO: Task 959 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f002b410ccebd385422b147d5271b175 with result from task 959 parsl.dataflow.dflow DEBUG: Task 959 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3fcaab23-b11e-4367-9a7d-a38ba8adfca3_isrForCrosstalkSources_996959_43.stdout parsl.dataflow.dflow INFO: Standard error for task 959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3fcaab23-b11e-4367-9a7d-a38ba8adfca3_isrForCrosstalkSources_996959_43.stderr parsl.dataflow.dflow INFO: Task 893 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39ec81c76a19bfe9a96dddb8f6947ebe with result from task 893 parsl.dataflow.dflow DEBUG: Task 893 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4ce504f4-b2f5-48f0-9398-725b4f11cecb_isrForCrosstalkSources_996966_47.stdout parsl.dataflow.dflow INFO: Standard error for task 893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4ce504f4-b2f5-48f0-9398-725b4f11cecb_isrForCrosstalkSources_996966_47.stderr parsl.dataflow.dflow INFO: Task 1029 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d10e6402de9d797af41901fa97586d39 with result from task 1029 parsl.dataflow.dflow DEBUG: Task 1029 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ac2bbdfe-1776-43f8-81e6-cfb29dd6f9a7_isrForCrosstalkSources_996981_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ac2bbdfe-1776-43f8-81e6-cfb29dd6f9a7_isrForCrosstalkSources_996981_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:38 proc_lsst.multi:146 [INFO] found job 17074870 in provider astro proc_lsst.multi INFO: found job 17074870 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 1001 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d41aa72dc9e99805510505a288b8a46c with result from task 1001 parsl.dataflow.dflow DEBUG: Task 1001 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f8b87dd6-81eb-45d1-a93a-116aa4a969f1_isrForCrosstalkSources_996958_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f8b87dd6-81eb-45d1-a93a-116aa4a969f1_isrForCrosstalkSources_996958_16.stderr parsl.dataflow.dflow INFO: Task 1005 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fcfe59c45d2e59c7ec997c3096abc49 with result from task 1005 parsl.dataflow.dflow DEBUG: Task 1005 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/64133487-7aae-4b3b-89af-a22b5bbf3ee7_isrForCrosstalkSources_996977_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/64133487-7aae-4b3b-89af-a22b5bbf3ee7_isrForCrosstalkSources_996977_6.stderr parsl.dataflow.dflow INFO: Task 1014 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f6096727f07cc0dbc6242a0183f285ba with result from task 1014 parsl.dataflow.dflow DEBUG: Task 1014 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/f410303f-ce0d-427f-a951-070cd02104d2_isrForCrosstalkSources_997000_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/f410303f-ce0d-427f-a951-070cd02104d2_isrForCrosstalkSources_997000_39.stderr parsl.dataflow.dflow INFO: Task 967 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01e62e60fef330dd0ea3a3241dff6ec3 with result from task 967 parsl.dataflow.dflow DEBUG: Task 967 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/0a991ddc-2e7d-46f0-ac8c-92ac0c1de089_isrForCrosstalkSources_996996_5.stdout parsl.dataflow.dflow INFO: Standard error for task 967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/0a991ddc-2e7d-46f0-ac8c-92ac0c1de089_isrForCrosstalkSources_996996_5.stderr parsl.dataflow.dflow INFO: Task 867 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d445df06c2a123b6728c1ad01c1f770 with result from task 867 parsl.dataflow.dflow DEBUG: Task 867 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/e093d761-6ee3-4fe8-8053-12c6a8b01634_isrForCrosstalkSources_997014_61.stdout parsl.dataflow.dflow INFO: Standard error for task 867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/e093d761-6ee3-4fe8-8053-12c6a8b01634_isrForCrosstalkSources_997014_61.stderr parsl.dataflow.dflow INFO: Task 1062 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 76ff027d2d2c42d8643e2b08e12318d2 with result from task 1062 parsl.dataflow.dflow DEBUG: Task 1062 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/c919d350-9c16-4228-a5a5-576a766808d5_isrForCrosstalkSources_997022_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/c919d350-9c16-4228-a5a5-576a766808d5_isrForCrosstalkSources_997022_16.stderr parsl.dataflow.dflow INFO: Task 815 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b545cbdb61142d89eedef653fe0b2691 with result from task 815 parsl.dataflow.dflow DEBUG: Task 815 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/15d3b7ab-c217-4330-be82-ec92cce97aeb_isrForCrosstalkSources_997008_55.stdout parsl.dataflow.dflow INFO: Standard error for task 815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/15d3b7ab-c217-4330-be82-ec92cce97aeb_isrForCrosstalkSources_997008_55.stderr parsl.dataflow.dflow INFO: Task 1006 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e464d55b9f367e5f07eb34ea6dad658e with result from task 1006 parsl.dataflow.dflow DEBUG: Task 1006 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/0ff3723d-e379-4e7e-9a12-56bc5412a4e9_isrForCrosstalkSources_996974_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/0ff3723d-e379-4e7e-9a12-56bc5412a4e9_isrForCrosstalkSources_996974_43.stderr parsl.dataflow.dflow INFO: Task 1071 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e42975613d0d7db46850b4c51712bc6d with result from task 1071 parsl.dataflow.dflow DEBUG: Task 1071 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/de56bb61-8e41-4044-9098-5ffd5634982e_isrForCrosstalkSources_996996_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/de56bb61-8e41-4044-9098-5ffd5634982e_isrForCrosstalkSources_996996_34.stderr parsl.dataflow.dflow INFO: Task 983 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab093286dc68716a84b45981671a7857 with result from task 983 parsl.dataflow.dflow DEBUG: Task 983 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/19984382-a14a-4a2f-a9f0-4f20261346e1_isrForCrosstalkSources_997028_1.stdout parsl.dataflow.dflow INFO: Standard error for task 983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/19984382-a14a-4a2f-a9f0-4f20261346e1_isrForCrosstalkSources_997028_1.stderr parsl.dataflow.dflow INFO: Task 994 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d474cddd7034f857119d62e0607c2234 with result from task 994 parsl.dataflow.dflow DEBUG: Task 994 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/baf870db-1a53-4c27-8217-bdcea29ab72f_isrForCrosstalkSources_997024_37.stdout parsl.dataflow.dflow INFO: Standard error for task 994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/baf870db-1a53-4c27-8217-bdcea29ab72f_isrForCrosstalkSources_997024_37.stderr parsl.dataflow.dflow INFO: Task 920 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b81083c116ccff4a6926f7903a992934 with result from task 920 parsl.dataflow.dflow DEBUG: Task 920 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/18dd25a3-5c0e-4a14-8a8a-ce0559cb4679_isrForCrosstalkSources_997014_59.stdout parsl.dataflow.dflow INFO: Standard error for task 920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/18dd25a3-5c0e-4a14-8a8a-ce0559cb4679_isrForCrosstalkSources_997014_59.stderr parsl.dataflow.dflow INFO: Task 879 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca52247284882d7c29acd7b39efdb6cf with result from task 879 parsl.dataflow.dflow DEBUG: Task 879 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/69ec2d5a-0744-4ff5-bb18-5c43f9ae9624_isrForCrosstalkSources_997022_44.stdout parsl.dataflow.dflow INFO: Standard error for task 879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/69ec2d5a-0744-4ff5-bb18-5c43f9ae9624_isrForCrosstalkSources_997022_44.stderr parsl.dataflow.dflow INFO: Task 968 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 167dd9fc91c8d20d5d32c3a1f3f1bac6 with result from task 968 parsl.dataflow.dflow DEBUG: Task 968 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a5acff24-9cd8-470e-8432-9a6d95da5f77_isrForCrosstalkSources_996965_52.stdout parsl.dataflow.dflow INFO: Standard error for task 968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a5acff24-9cd8-470e-8432-9a6d95da5f77_isrForCrosstalkSources_996965_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:39 proc_lsst.multi:146 [INFO] found job 17074871 in provider astro proc_lsst.multi INFO: found job 17074871 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 998 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03e31216959be796b5aa78a8672e9892 with result from task 998 parsl.dataflow.dflow DEBUG: Task 998 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/64d0b8fd-aee3-413c-ba72-b9b8efe60eb2_isrForCrosstalkSources_996964_54.stdout parsl.dataflow.dflow INFO: Standard error for task 998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/64d0b8fd-aee3-413c-ba72-b9b8efe60eb2_isrForCrosstalkSources_996964_54.stderr parsl.dataflow.dflow INFO: Task 824 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d3187a681d635e87c0d035385903bbe2 with result from task 824 parsl.dataflow.dflow DEBUG: Task 824 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d944a5e8-cb0c-4b53-8fc3-5af685ed39e4_isrForCrosstalkSources_996997_61.stdout parsl.dataflow.dflow INFO: Standard error for task 824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d944a5e8-cb0c-4b53-8fc3-5af685ed39e4_isrForCrosstalkSources_996997_61.stderr parsl.dataflow.dflow INFO: Task 969 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ae9f139c462e00f34ced9acb8c8755b with result from task 969 parsl.dataflow.dflow DEBUG: Task 969 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/90fb61e7-5746-42a0-bf4a-2499756302f6_isrForCrosstalkSources_997032_58.stdout parsl.dataflow.dflow INFO: Standard error for task 969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/90fb61e7-5746-42a0-bf4a-2499756302f6_isrForCrosstalkSources_997032_58.stderr parsl.dataflow.dflow INFO: Task 1018 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd14daf8ae3560686010868d41308b77 with result from task 1018 parsl.dataflow.dflow DEBUG: Task 1018 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/ab43e665-6862-4ff0-96ab-9037d10a9975_isrForCrosstalkSources_997008_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/ab43e665-6862-4ff0-96ab-9037d10a9975_isrForCrosstalkSources_997008_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:40 proc_lsst.multi:146 [INFO] found job 17074872 in provider astro proc_lsst.multi INFO: found job 17074872 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 1004 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba6f603a99634578e7e3f4122973e907 with result from task 1004 parsl.dataflow.dflow DEBUG: Task 1004 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/8cf58eac-e2c1-4078-9dfa-5fe597f014e7_isrForCrosstalkSources_997014_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/8cf58eac-e2c1-4078-9dfa-5fe597f014e7_isrForCrosstalkSources_997014_48.stderr parsl.dataflow.dflow INFO: Task 990 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f09596d8cf3b21aa188122519e7dbee with result from task 990 parsl.dataflow.dflow DEBUG: Task 990 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7cbe3ea6-f0d7-4e1b-a1ca-5311e26b2794_isrForCrosstalkSources_997029_28.stdout parsl.dataflow.dflow INFO: Standard error for task 990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7cbe3ea6-f0d7-4e1b-a1ca-5311e26b2794_isrForCrosstalkSources_997029_28.stderr parsl.dataflow.dflow INFO: Task 989 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e51af4503ad4ca39d849cca173eae88 with result from task 989 parsl.dataflow.dflow DEBUG: Task 989 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/10569c16-8a35-4310-ab81-123817ed8006_isrForCrosstalkSources_997011_28.stdout parsl.dataflow.dflow INFO: Standard error for task 989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/10569c16-8a35-4310-ab81-123817ed8006_isrForCrosstalkSources_997011_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 972 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 048b5d9fc3b2b0a753e7d7ab9b5cb80c with result from task 972 parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 972 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/56d95c02-2ca7-43dd-a260-359961adf940_isrForCrosstalkSources_997005_2.stdout parsl.dataflow.dflow INFO: Standard error for task 972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/56d95c02-2ca7-43dd-a260-359961adf940_isrForCrosstalkSources_997005_2.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:41 proc_lsst.multi:146 [INFO] found job 17074873 in provider astro proc_lsst.multi INFO: found job 17074873 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 1069 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a299593769788dbae782b1d5c203f33f with result from task 1069 parsl.dataflow.dflow DEBUG: Task 1069 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0cd7a8fd-01a1-4da2-b085-aedcc3dd2d5e_isrForCrosstalkSources_997012_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0cd7a8fd-01a1-4da2-b085-aedcc3dd2d5e_isrForCrosstalkSources_997012_1.stderr parsl.dataflow.dflow INFO: Task 979 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4938d14cc043fc9b44a8d6b75a0727eb with result from task 979 parsl.dataflow.dflow DEBUG: Task 979 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/d734e273-f818-4767-bd07-21473beb5cec_isrForCrosstalkSources_997028_57.stdout parsl.dataflow.dflow INFO: Standard error for task 979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/d734e273-f818-4767-bd07-21473beb5cec_isrForCrosstalkSources_997028_57.stderr parsl.dataflow.dflow INFO: Task 899 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dcfc383b90bc31544e46fb067eaf5b27 with result from task 899 parsl.dataflow.dflow DEBUG: Task 899 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/69751147-9f54-490b-997e-a6ed8d44ce2d_isrForCrosstalkSources_996994_60.stdout parsl.dataflow.dflow INFO: Standard error for task 899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/69751147-9f54-490b-997e-a6ed8d44ce2d_isrForCrosstalkSources_996994_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:41 proc_lsst.multi:146 [INFO] found job 17074876 in provider astro proc_lsst.multi INFO: found job 17074876 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 901 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf469f55167ad53062c75b6593d3e24c with result from task 901 parsl.dataflow.dflow DEBUG: Task 901 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7e21db97-62ca-45c8-badc-a527bc7fe846_isrForCrosstalkSources_996963_43.stdout parsl.dataflow.dflow INFO: Standard error for task 901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7e21db97-62ca-45c8-badc-a527bc7fe846_isrForCrosstalkSources_996963_43.stderr parsl.dataflow.dflow INFO: Task 1116 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ffe6633bfe59741a6a29b34e7c13c169 with result from task 1116 parsl.dataflow.dflow DEBUG: Task 1116 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8ab1b09c-d3b9-4138-a0d4-4b9b8ba811e1_isrForCrosstalkSources_996977_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8ab1b09c-d3b9-4138-a0d4-4b9b8ba811e1_isrForCrosstalkSources_996977_56.stderr parsl.dataflow.dflow INFO: Task 986 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c0b95f674e1e89457d05b87cf01c2647 with result from task 986 parsl.dataflow.dflow DEBUG: Task 986 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b68369c8-c0f1-477a-adbb-2db7efa65c72_isrForCrosstalkSources_996997_43.stdout parsl.dataflow.dflow INFO: Standard error for task 986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b68369c8-c0f1-477a-adbb-2db7efa65c72_isrForCrosstalkSources_996997_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:42 proc_lsst.multi:146 [INFO] found job 17074877 in provider astro proc_lsst.multi INFO: found job 17074877 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 973 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a9b66b37cd87cdd536b922cd3f00e03a with result from task 973 parsl.dataflow.dflow DEBUG: Task 973 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e34f723e-63c4-46e9-8bc5-58a7fd8e60bd_isrForCrosstalkSources_997032_45.stdout parsl.dataflow.dflow INFO: Standard error for task 973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e34f723e-63c4-46e9-8bc5-58a7fd8e60bd_isrForCrosstalkSources_997032_45.stderr parsl.dataflow.dflow INFO: Task 1002 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c8354b73fb744645bcac61c094f7cde with result from task 1002 parsl.dataflow.dflow DEBUG: Task 1002 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a0aceac6-ce33-45d5-9d89-c35859ac69eb_isrForCrosstalkSources_996994_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a0aceac6-ce33-45d5-9d89-c35859ac69eb_isrForCrosstalkSources_996994_17.stderr parsl.dataflow.dflow INFO: Task 1085 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4bd205300ac0fd82704019d33ec9e1f with result from task 1085 parsl.dataflow.dflow DEBUG: Task 1085 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5168432e-c6d1-438e-9a56-763bdf74fe4f_isrForCrosstalkSources_996987_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5168432e-c6d1-438e-9a56-763bdf74fe4f_isrForCrosstalkSources_996987_6.stderr parsl.dataflow.dflow INFO: Task 964 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c77259d64e4b02c4f4b1ef8248eee65 with result from task 964 parsl.dataflow.dflow DEBUG: Task 964 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b30d7062-9b36-4f3e-a053-075412539de9_isrForCrosstalkSources_996997_58.stdout parsl.dataflow.dflow INFO: Standard error for task 964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b30d7062-9b36-4f3e-a053-075412539de9_isrForCrosstalkSources_996997_58.stderr parsl.dataflow.dflow INFO: Task 825 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c97f096253d598b98a5f79935bbdc40 with result from task 825 parsl.dataflow.dflow DEBUG: Task 825 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ab44d992-939c-490d-b845-8fa7e925cc22_isrForCrosstalkSources_996998_57.stdout parsl.dataflow.dflow INFO: Standard error for task 825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ab44d992-939c-490d-b845-8fa7e925cc22_isrForCrosstalkSources_996998_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:43 proc_lsst.multi:146 [INFO] found job 17074879 in provider astro proc_lsst.multi INFO: found job 17074879 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 999 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96c137b9a5d35d2f7c341c0aefb9cb71 with result from task 999 parsl.dataflow.dflow DEBUG: Task 999 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cefa7ecc-e766-4409-81a7-2ff5146ef836_isrForCrosstalkSources_997024_58.stdout parsl.dataflow.dflow INFO: Standard error for task 999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cefa7ecc-e766-4409-81a7-2ff5146ef836_isrForCrosstalkSources_997024_58.stderr parsl.dataflow.dflow INFO: Task 955 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54faf8c9d9e17260ce9a3bb035cf7e6a with result from task 955 parsl.dataflow.dflow DEBUG: Task 955 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/1d6f811d-28b6-4dc9-bcbf-79076b3b2982_isrForCrosstalkSources_997023_58.stdout parsl.dataflow.dflow INFO: Standard error for task 955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/1d6f811d-28b6-4dc9-bcbf-79076b3b2982_isrForCrosstalkSources_997023_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:43 proc_lsst.multi:146 [INFO] found job 17074880 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074880 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 1108 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6bfa488de71cd32a607864ba85a6a67 with result from task 1108 parsl.dataflow.dflow DEBUG: Task 1108 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/078cbd60-f777-4a7a-99bb-dc7a8052c7a7_isrForCrosstalkSources_996972_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/078cbd60-f777-4a7a-99bb-dc7a8052c7a7_isrForCrosstalkSources_996972_4.stderr parsl.dataflow.dflow INFO: Task 987 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 160ddecd94855bff5c5673e96e2346ae with result from task 987 parsl.dataflow.dflow DEBUG: Task 987 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2acf96d9-5db0-4d6b-aad0-696a43a12d13_isrForCrosstalkSources_996974_57.stdout parsl.dataflow.dflow INFO: Standard error for task 987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2acf96d9-5db0-4d6b-aad0-696a43a12d13_isrForCrosstalkSources_996974_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:44 proc_lsst.multi:146 [INFO] found job 17074881 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074881 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 1026 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b62975285c537a7b8493f48dfe59e01 with result from task 1026 parsl.dataflow.dflow DEBUG: Task 1026 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/11f627bd-f78a-46d7-bb5a-1c10bf42e2a7_isrForCrosstalkSources_996975_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/11f627bd-f78a-46d7-bb5a-1c10bf42e2a7_isrForCrosstalkSources_996975_26.stderr parsl.dataflow.dflow INFO: Task 1138 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d8579b86e9e83e8498c66d60e4570f0e with result from task 1138 parsl.dataflow.dflow DEBUG: Task 1138 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/64c89996-0dc1-4f0b-914c-cf9dbf9aa3d5_isrForCrosstalkSources_996954_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/64c89996-0dc1-4f0b-914c-cf9dbf9aa3d5_isrForCrosstalkSources_996954_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:45 proc_lsst.multi:146 [INFO] found job 17074887 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074887 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 1058 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd9f7817d58ae29d3d801306a5eaf604 with result from task 1058 parsl.dataflow.dflow DEBUG: Task 1058 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d7f2a9e0-8e8a-433c-a901-1e721bb88022_isrForCrosstalkSources_996981_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d7f2a9e0-8e8a-433c-a901-1e721bb88022_isrForCrosstalkSources_996981_27.stderr parsl.dataflow.dflow INFO: Task 1025 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd8a5d6157df224c3fc1f744d0fae7ea with result from task 1025 parsl.dataflow.dflow DEBUG: Task 1025 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/984f7ebf-9eba-4eac-a804-dd8b240023de_isrForCrosstalkSources_996982_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/984f7ebf-9eba-4eac-a804-dd8b240023de_isrForCrosstalkSources_996982_25.stderr parsl.dataflow.dflow INFO: Task 1100 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18f6560d317f1a3d92e6983ba3619208 with result from task 1100 parsl.dataflow.dflow DEBUG: Task 1100 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0ab9e3cc-448b-48a7-9d9f-5bba644ed622_isrForCrosstalkSources_997010_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0ab9e3cc-448b-48a7-9d9f-5bba644ed622_isrForCrosstalkSources_997010_25.stderr parsl.dataflow.dflow INFO: Task 1092 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ab72f45a0a24f6888b788559e9f71cc with result from task 1092 parsl.dataflow.dflow DEBUG: Task 1092 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/89e82789-e4c6-4953-b531-bba6ecf3d058_isrForCrosstalkSources_997020_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/89e82789-e4c6-4953-b531-bba6ecf3d058_isrForCrosstalkSources_997020_34.stderr parsl.dataflow.dflow INFO: Task 1120 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d598ca8065d5277061f1ce91bcff47e7 with result from task 1120 parsl.dataflow.dflow DEBUG: Task 1120 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/d8905dc6-fbe6-4137-a9a8-79a0894311ee_isrForCrosstalkSources_996956_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/d8905dc6-fbe6-4137-a9a8-79a0894311ee_isrForCrosstalkSources_996956_34.stderr parsl.dataflow.dflow INFO: Task 1064 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f2ba32a6a1bd73ab2a90d66fcbb9e2a7 with result from task 1064 parsl.dataflow.dflow DEBUG: Task 1064 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d27d987a-ee4d-4bb7-88b3-4996962dbc65_isrForCrosstalkSources_996985_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d27d987a-ee4d-4bb7-88b3-4996962dbc65_isrForCrosstalkSources_996985_27.stderr parsl.dataflow.dflow INFO: Task 1073 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ff97752d9a542b5f1b4cb1969460768 with result from task 1073 parsl.dataflow.dflow DEBUG: Task 1073 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d1ae974e-38fe-42c8-8f13-18b49c72841a_isrForCrosstalkSources_997017_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d1ae974e-38fe-42c8-8f13-18b49c72841a_isrForCrosstalkSources_997017_33.stderr parsl.dataflow.dflow INFO: Task 949 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f2be0d0f77498dc04d41bb92a358d55 with result from task 949 parsl.dataflow.dflow DEBUG: Task 949 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f479662c-592c-4942-91f9-928e35774d61_isrForCrosstalkSources_996995_49.stdout parsl.dataflow.dflow INFO: Standard error for task 949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f479662c-592c-4942-91f9-928e35774d61_isrForCrosstalkSources_996995_49.stderr parsl.dataflow.dflow INFO: Task 1082 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6418cc50550a4eb311fc681ca2c0951a with result from task 1082 parsl.dataflow.dflow DEBUG: Task 1082 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/7b74aa89-d895-4bb1-8b3e-46b4e03b3d2a_isrForCrosstalkSources_997032_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/7b74aa89-d895-4bb1-8b3e-46b4e03b3d2a_isrForCrosstalkSources_997032_34.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:45 proc_lsst.multi:146 [INFO] found job 17074888 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074888 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 1027 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38a4864723a1cb681f7165a53bcdd623 with result from task 1027 parsl.dataflow.dflow DEBUG: Task 1027 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bf70c3b8-187d-4003-873f-cd169b799936_isrForCrosstalkSources_996959_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bf70c3b8-187d-4003-873f-cd169b799936_isrForCrosstalkSources_996959_13.stderr parsl.dataflow.dflow INFO: Task 993 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c16b55a6a6fc960aa2cb51a29f2f8bc with result from task 993 parsl.dataflow.dflow DEBUG: Task 993 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/45328959-e1e2-4dcb-9f8a-bfe0cc80aa2b_isrForCrosstalkSources_996965_32.stdout parsl.dataflow.dflow INFO: Standard error for task 993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/45328959-e1e2-4dcb-9f8a-bfe0cc80aa2b_isrForCrosstalkSources_996965_32.stderr parsl.dataflow.dflow INFO: Task 1118 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90755bfc3759cfb8ec89cbc2d0e87c2c with result from task 1118 parsl.dataflow.dflow DEBUG: Task 1118 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5b77f48f-5214-465e-8b88-e87aea8be8d5_isrForCrosstalkSources_997017_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5b77f48f-5214-465e-8b88-e87aea8be8d5_isrForCrosstalkSources_997017_45.stderr parsl.dataflow.dflow INFO: Task 1043 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58b21958eae28dcda2e0f117681ef7da with result from task 1043 parsl.dataflow.dflow DEBUG: Task 1043 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e4b3fbbd-75ca-4359-87d5-c355efee7f05_isrForCrosstalkSources_996954_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e4b3fbbd-75ca-4359-87d5-c355efee7f05_isrForCrosstalkSources_996954_9.stderr parsl.dataflow.dflow INFO: Task 1155 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fe6f653cd37763a4bb3eae35b989a76 with result from task 1155 parsl.dataflow.dflow DEBUG: Task 1155 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/db0a931f-3cfa-4c79-8ff0-c1b32d7cb8e8_isrForCrosstalkSources_996955_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/db0a931f-3cfa-4c79-8ff0-c1b32d7cb8e8_isrForCrosstalkSources_996955_46.stderr parsl.dataflow.dflow INFO: Task 1070 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad38e7a40c75c575a5699d7f15df223b with result from task 1070 parsl.dataflow.dflow DEBUG: Task 1070 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/a93283c9-82c2-4b3f-aaf2-d36601834058_isrForCrosstalkSources_997004_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/a93283c9-82c2-4b3f-aaf2-d36601834058_isrForCrosstalkSources_997004_26.stderr parsl.dataflow.dflow INFO: Task 1090 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 95731eb4364c2678801e1d240e988dc9 with result from task 1090 parsl.dataflow.dflow DEBUG: Task 1090 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/1815ab39-727e-4c85-94c7-180f146b36ca_isrForCrosstalkSources_996954_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/1815ab39-727e-4c85-94c7-180f146b36ca_isrForCrosstalkSources_996954_11.stderr parsl.dataflow.dflow INFO: Task 1089 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f1b9b349d58a46072af6a1524a82ebe2 with result from task 1089 parsl.dataflow.dflow DEBUG: Task 1089 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c3918c1d-3c72-496e-8d68-13b17fd0cf57_isrForCrosstalkSources_996977_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c3918c1d-3c72-496e-8d68-13b17fd0cf57_isrForCrosstalkSources_996977_16.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:46 proc_lsst.multi:146 [INFO] found job 17074889 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074889 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 1080 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2833162be16b0cfb42c39f8a4aaf88c1 with result from task 1080 parsl.dataflow.dflow DEBUG: Task 1080 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/bc893f69-0e9c-4267-8ac3-8e3b9c4b0765_isrForCrosstalkSources_996982_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/bc893f69-0e9c-4267-8ac3-8e3b9c4b0765_isrForCrosstalkSources_996982_45.stderr parsl.dataflow.dflow INFO: Task 1143 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08eb23218fbe3989e8d405f7a1865562 with result from task 1143 parsl.dataflow.dflow DEBUG: Task 1143 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/28b7b053-8b8e-4e93-b09c-86e0f8f2a7d0_isrForCrosstalkSources_996993_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/28b7b053-8b8e-4e93-b09c-86e0f8f2a7d0_isrForCrosstalkSources_996993_20.stderr parsl.dataflow.dflow INFO: Task 1021 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8cfde0a841964e9868afb127f681238 with result from task 1021 parsl.dataflow.dflow DEBUG: Task 1021 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a332ed7d-aec8-4953-a5e8-a025b3cbce6a_isrForCrosstalkSources_996978_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a332ed7d-aec8-4953-a5e8-a025b3cbce6a_isrForCrosstalkSources_996978_39.stderr parsl.dataflow.dflow INFO: Task 1035 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f876ce87487cacbc31b5ebb2a746fa1f with result from task 1035 parsl.dataflow.dflow DEBUG: Task 1035 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/75fa1e19-71fd-4d95-9e2b-04a4a7ca2d14_isrForCrosstalkSources_996985_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/75fa1e19-71fd-4d95-9e2b-04a4a7ca2d14_isrForCrosstalkSources_996985_8.stderr parsl.dataflow.dflow INFO: Task 1129 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c1d8d5e53ca4dd8e8c14c20f478bf86 with result from task 1129 parsl.dataflow.dflow DEBUG: Task 1129 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/31a1548a-42de-404b-8383-604e5b8519bf_isrForCrosstalkSources_997001_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/31a1548a-42de-404b-8383-604e5b8519bf_isrForCrosstalkSources_997001_46.stderr parsl.dataflow.dflow INFO: Task 1127 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a2e3612d914ef9b184ba0a7089a3e25 with result from task 1127 parsl.dataflow.dflow DEBUG: Task 1127 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/a0c198e7-9ad0-4f1b-9266-11548a3a230d_isrForCrosstalkSources_996986_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/a0c198e7-9ad0-4f1b-9266-11548a3a230d_isrForCrosstalkSources_996986_43.stderr parsl.dataflow.dflow INFO: Task 1133 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 591d48b5270fb855b6ec71717ef4fad7 with result from task 1133 parsl.dataflow.dflow DEBUG: Task 1133 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a56d2b43-ab2a-4695-96cd-51627cdd6761_isrForCrosstalkSources_996970_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a56d2b43-ab2a-4695-96cd-51627cdd6761_isrForCrosstalkSources_996970_51.stderr parsl.dataflow.dflow INFO: Task 1159 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16d6695a3879e4d89dc85f43ba47ed13 with result from task 1159 parsl.dataflow.dflow DEBUG: Task 1159 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6946f26f-1efc-4c16-afb6-fd8efc143c46_isrForCrosstalkSources_997030_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6946f26f-1efc-4c16-afb6-fd8efc143c46_isrForCrosstalkSources_997030_26.stderr parsl.dataflow.dflow INFO: Task 1149 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d542932144fbedab94f8749f32ebfc05 with result from task 1149 parsl.dataflow.dflow DEBUG: Task 1149 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/594d6f13-16bb-434f-9d4d-a321158a0201_isrForCrosstalkSources_996955_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/594d6f13-16bb-434f-9d4d-a321158a0201_isrForCrosstalkSources_996955_27.stderr parsl.dataflow.dflow INFO: Task 1038 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a906711ab11fac132f5bf4ab65fc9f48 with result from task 1038 parsl.dataflow.dflow DEBUG: Task 1038 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/1f8aed8d-c2d6-42ac-9e9c-30d8fdf58479_isrForCrosstalkSources_997026_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/1f8aed8d-c2d6-42ac-9e9c-30d8fdf58479_isrForCrosstalkSources_997026_51.stderr parsl.dataflow.dflow INFO: Task 1173 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f5a027c59cb05252cdeb86c72feee17 with result from task 1173 parsl.dataflow.dflow DEBUG: Task 1173 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2fadd61b-19dc-4ada-95b7-89a90bb9c66e_isrForCrosstalkSources_996956_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2fadd61b-19dc-4ada-95b7-89a90bb9c66e_isrForCrosstalkSources_996956_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:48 proc_lsst.multi:146 [INFO] found job 17074894 in provider escience_a40 proc_lsst.multi INFO: found job 17074894 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1137 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c99767be51ebaa1dbdb6c1926116a0c with result from task 1137 parsl.dataflow.dflow DEBUG: Task 1137 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1b4dd511-db2e-44ab-ac56-ebadd32576e2_isrForCrosstalkSources_997005_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1b4dd511-db2e-44ab-ac56-ebadd32576e2_isrForCrosstalkSources_997005_22.stderr parsl.dataflow.dflow INFO: Task 1034 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ee29b6071ee6d7c0666e3fc5b3487937 with result from task 1034 parsl.dataflow.dflow DEBUG: Task 1034 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/bb88595a-5800-4383-98b4-83b513157d8d_isrForCrosstalkSources_996972_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/bb88595a-5800-4383-98b4-83b513157d8d_isrForCrosstalkSources_996972_31.stderr parsl.dataflow.dflow INFO: Task 1117 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dfe17e4fdc6af4cd064bf4ce4ad0f356 with result from task 1117 parsl.dataflow.dflow DEBUG: Task 1117 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/9caa13e0-2187-4ecf-9869-43ab061fa53b_isrForCrosstalkSources_996975_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/9caa13e0-2187-4ecf-9869-43ab061fa53b_isrForCrosstalkSources_996975_39.stderr parsl.dataflow.dflow INFO: Task 1106 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8622d997382721d96fca59aedcb35b9f with result from task 1106 parsl.dataflow.dflow DEBUG: Task 1106 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/af76bdac-b9c4-4a5a-a6df-bcbf9e42487a_isrForCrosstalkSources_997011_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/af76bdac-b9c4-4a5a-a6df-bcbf9e42487a_isrForCrosstalkSources_997011_17.stderr parsl.dataflow.dflow INFO: Task 1051 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 47cef778df8972038c5b91ff578d63f4 with result from task 1051 parsl.dataflow.dflow DEBUG: Task 1051 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b3842d3f-203c-4221-978f-c5b75be588f5_isrForCrosstalkSources_996980_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b3842d3f-203c-4221-978f-c5b75be588f5_isrForCrosstalkSources_996980_22.stderr parsl.dataflow.dflow INFO: Task 1059 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd85e4889554438dbb9d65e22ece57be with result from task 1059 parsl.dataflow.dflow DEBUG: Task 1059 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3d456af9-b89b-417d-9404-265a8e320e46_isrForCrosstalkSources_996988_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3d456af9-b89b-417d-9404-265a8e320e46_isrForCrosstalkSources_996988_32.stderr parsl.dataflow.dflow INFO: Task 1060 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb27831e09bd473d3464645a43130203 with result from task 1060 parsl.dataflow.dflow DEBUG: Task 1060 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ea2b423e-bf43-47ab-b386-b39905bcdcab_isrForCrosstalkSources_996992_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ea2b423e-bf43-47ab-b386-b39905bcdcab_isrForCrosstalkSources_996992_16.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:49 proc_lsst.multi:146 [INFO] found job 17074895 in provider escience_a40 proc_lsst.multi INFO: found job 17074895 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1074 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26613f7620d15b4ff6ea1d4c3552da77 with result from task 1074 parsl.dataflow.dflow DEBUG: Task 1074 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/6e7ade40-ec51-428f-814a-311b36300613_isrForCrosstalkSources_997000_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/6e7ade40-ec51-428f-814a-311b36300613_isrForCrosstalkSources_997000_45.stderr parsl.dataflow.dflow INFO: Task 1194 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5804a610837afa3f96e4df2d1f7c9ad7 with result from task 1194 parsl.dataflow.dflow DEBUG: Task 1194 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ca0df4df-3c0b-4c5e-85e9-40164d27ba93_isrForCrosstalkSources_996982_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ca0df4df-3c0b-4c5e-85e9-40164d27ba93_isrForCrosstalkSources_996982_27.stderr parsl.dataflow.dflow INFO: Task 995 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba4875a73dcf88250ac77d6ede2181c7 with result from task 995 parsl.dataflow.dflow DEBUG: Task 995 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/67cf5956-c253-4bc1-8551-2494e2b4ba73_isrForCrosstalkSources_996973_10.stdout parsl.dataflow.dflow INFO: Standard error for task 995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/67cf5956-c253-4bc1-8551-2494e2b4ba73_isrForCrosstalkSources_996973_10.stderr parsl.dataflow.dflow INFO: Task 1157 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 128a29c7d0c4fb905f0b895c71e5bd9c with result from task 1157 parsl.dataflow.dflow DEBUG: Task 1157 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/17ee4e50-ec0c-46c6-909f-d8d0730766bd_isrForCrosstalkSources_997020_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/17ee4e50-ec0c-46c6-909f-d8d0730766bd_isrForCrosstalkSources_997020_18.stderr parsl.dataflow.dflow INFO: Task 1196 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab889be10fab4fbe686ea7d2568c475e with result from task 1196 parsl.dataflow.dflow DEBUG: Task 1196 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/184027d2-008b-4f5b-a807-28cc8eaa22e3_isrForCrosstalkSources_996958_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/184027d2-008b-4f5b-a807-28cc8eaa22e3_isrForCrosstalkSources_996958_25.stderr parsl.dataflow.dflow INFO: Task 1172 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f6fe8917f12b800150e284790ba8cd21 with result from task 1172 parsl.dataflow.dflow DEBUG: Task 1172 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/823e0feb-8628-4bbd-9b09-08299f51bf41_isrForCrosstalkSources_996962_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/823e0feb-8628-4bbd-9b09-08299f51bf41_isrForCrosstalkSources_996962_45.stderr parsl.dataflow.dflow INFO: Task 1041 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 249c4d6430c6b5436b3e3b63d9908309 with result from task 1041 parsl.dataflow.dflow DEBUG: Task 1041 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/59bfe8bd-3760-43c1-bbfe-806ccd0c7bdc_isrForCrosstalkSources_997027_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/59bfe8bd-3760-43c1-bbfe-806ccd0c7bdc_isrForCrosstalkSources_997027_28.stderr parsl.dataflow.dflow INFO: Task 1151 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cbcf2866811cc8e59fa3180ae55a11ec with result from task 1151 parsl.dataflow.dflow DEBUG: Task 1151 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/a63efe0d-7b0d-4b80-899d-f4438838409d_isrForCrosstalkSources_997010_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/a63efe0d-7b0d-4b80-899d-f4438838409d_isrForCrosstalkSources_997010_5.stderr parsl.dataflow.dflow INFO: Task 1084 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78800121da82ac7da8c6aeade1da7d15 with result from task 1084 parsl.dataflow.dflow DEBUG: Task 1084 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/dbaf200c-a880-4485-962a-d5f337d5f674_isrForCrosstalkSources_996981_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/dbaf200c-a880-4485-962a-d5f337d5f674_isrForCrosstalkSources_996981_9.stderr parsl.dataflow.dflow INFO: Task 1016 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea633a7aa7607555a5e3a6d17a59468f with result from task 1016 parsl.dataflow.dflow DEBUG: Task 1016 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d3d6f35f-352b-4904-b7d3-03519cded0c0_isrForCrosstalkSources_996991_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d3d6f35f-352b-4904-b7d3-03519cded0c0_isrForCrosstalkSources_996991_39.stderr parsl.dataflow.dflow INFO: Task 1086 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc96c19c6e72492780a3495a16fa1bf9 with result from task 1086 parsl.dataflow.dflow DEBUG: Task 1086 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/98cef2ec-2bd5-443f-8c74-a11a09f3f8c7_isrForCrosstalkSources_996967_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/98cef2ec-2bd5-443f-8c74-a11a09f3f8c7_isrForCrosstalkSources_996967_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:50 proc_lsst.multi:146 [INFO] found job 17074897 in provider escience_a40 proc_lsst.multi INFO: found job 17074897 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1141 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0fe1f1ff10850dee871793d587e497bf with result from task 1141 parsl.dataflow.dflow DEBUG: Task 1141 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e2ae19ce-e8a1-4f58-b24a-5479bbeb2160_isrForCrosstalkSources_997026_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e2ae19ce-e8a1-4f58-b24a-5479bbeb2160_isrForCrosstalkSources_997026_5.stderr parsl.dataflow.dflow INFO: Task 1055 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0b5dae1426f082eed6190241a1dcc859 with result from task 1055 parsl.dataflow.dflow DEBUG: Task 1055 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d405770a-d8cd-4db9-af73-c18104d92c32_isrForCrosstalkSources_997026_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d405770a-d8cd-4db9-af73-c18104d92c32_isrForCrosstalkSources_997026_44.stderr parsl.dataflow.dflow INFO: Task 1040 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65e09efeac3fe64f171f5e1780979386 with result from task 1040 parsl.dataflow.dflow DEBUG: Task 1040 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/cb0cc08b-6ec5-4e8d-b56c-986d8eec255e_isrForCrosstalkSources_996981_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/cb0cc08b-6ec5-4e8d-b56c-986d8eec255e_isrForCrosstalkSources_996981_32.stderr parsl.dataflow.dflow INFO: Task 1163 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32fc4049bd9aa606c61574d0897425f6 with result from task 1163 parsl.dataflow.dflow DEBUG: Task 1163 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/11f30f70-f027-4ee8-bfc6-f6e695faa5d9_isrForCrosstalkSources_997011_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/11f30f70-f027-4ee8-bfc6-f6e695faa5d9_isrForCrosstalkSources_997011_53.stderr parsl.dataflow.dflow INFO: Task 1186 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4bf766cc0bc348e72be368bcbaa9b45 with result from task 1186 parsl.dataflow.dflow DEBUG: Task 1186 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/42e6f484-c6da-4b24-9809-5ab452f2c455_isrForCrosstalkSources_997029_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/42e6f484-c6da-4b24-9809-5ab452f2c455_isrForCrosstalkSources_997029_33.stderr parsl.dataflow.dflow INFO: Task 1095 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62ac55e4558dc70a358a899d18727c7c with result from task 1095 parsl.dataflow.dflow DEBUG: Task 1095 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/b9b8c5cc-6252-4141-867e-bd0424ae9109_isrForCrosstalkSources_997018_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/b9b8c5cc-6252-4141-867e-bd0424ae9109_isrForCrosstalkSources_997018_4.stderr parsl.dataflow.dflow INFO: Task 1102 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ab03a9958263030d5e5fc6d65833145 with result from task 1102 parsl.dataflow.dflow DEBUG: Task 1102 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/47764378-8ef0-4390-a62e-81b0d3317ecd_isrForCrosstalkSources_997017_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/47764378-8ef0-4390-a62e-81b0d3317ecd_isrForCrosstalkSources_997017_18.stderr parsl.dataflow.dflow INFO: Task 1154 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9f145782a85fe4ed29fb31d29cc141bb with result from task 1154 parsl.dataflow.dflow DEBUG: Task 1154 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7b56c0a5-2df3-4de7-9bd7-104eff6a2d85_isrForCrosstalkSources_996993_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7b56c0a5-2df3-4de7-9bd7-104eff6a2d85_isrForCrosstalkSources_996993_2.stderr parsl.dataflow.dflow INFO: Task 1067 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fd5178e044e465432c4bc76335ebc5a with result from task 1067 parsl.dataflow.dflow DEBUG: Task 1067 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d29dc867-1e90-4e68-bf73-1a0e6af726fa_isrForCrosstalkSources_996971_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d29dc867-1e90-4e68-bf73-1a0e6af726fa_isrForCrosstalkSources_996971_7.stderr parsl.dataflow.dflow INFO: Task 1152 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7f8f073407c0a990b9247bea838c6f8 with result from task 1152 parsl.dataflow.dflow DEBUG: Task 1152 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f35908ed-3989-4654-b277-1f27876a69f0_isrForCrosstalkSources_997020_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f35908ed-3989-4654-b277-1f27876a69f0_isrForCrosstalkSources_997020_35.stderr parsl.dataflow.dflow INFO: Task 1050 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8019d3dba20a264a4a75899862bc7aa7 with result from task 1050 parsl.dataflow.dflow DEBUG: Task 1050 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/9502f89d-f46c-4b94-ad22-536b67e03d61_isrForCrosstalkSources_997013_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/9502f89d-f46c-4b94-ad22-536b67e03d61_isrForCrosstalkSources_997013_45.stderr parsl.dataflow.dflow INFO: Task 1091 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fca19318a864578867ad4640d80f2af with result from task 1091 parsl.dataflow.dflow DEBUG: Task 1091 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/543c9b31-9a37-457b-8ab4-c29f31626f14_isrForCrosstalkSources_996994_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/543c9b31-9a37-457b-8ab4-c29f31626f14_isrForCrosstalkSources_996994_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:50 proc_lsst.multi:146 [INFO] found job 17074898 in provider escience_a40 proc_lsst.multi INFO: found job 17074898 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1178 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ea2af3298cda08be1a5e4ab3a653275 with result from task 1178 parsl.dataflow.dflow DEBUG: Task 1178 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/f828a135-221c-4c54-921e-9af628af3a8b_isrForCrosstalkSources_996954_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/f828a135-221c-4c54-921e-9af628af3a8b_isrForCrosstalkSources_996954_46.stderr parsl.dataflow.dflow INFO: Task 1083 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b5d7910b219262c463f6bfdef82d859 with result from task 1083 parsl.dataflow.dflow DEBUG: Task 1083 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/ae99867e-84f5-4de2-898b-6b29453dab97_isrForCrosstalkSources_997025_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/ae99867e-84f5-4de2-898b-6b29453dab97_isrForCrosstalkSources_997025_62.stderr parsl.dataflow.dflow INFO: Task 978 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9dbb096357ec7201a6e46e9ccc95ff99 with result from task 978 parsl.dataflow.dflow DEBUG: Task 978 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/5b18c679-e874-4398-b98f-fb537dfafb7e_isrForCrosstalkSources_997032_42.stdout parsl.dataflow.dflow INFO: Standard error for task 978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/5b18c679-e874-4398-b98f-fb537dfafb7e_isrForCrosstalkSources_997032_42.stderr parsl.dataflow.dflow INFO: Task 1039 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ea5669006e0a732472c255b59263899 with result from task 1039 parsl.dataflow.dflow DEBUG: Task 1039 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e0dbcf7b-8a38-4609-a427-1f42c3746965_isrForCrosstalkSources_996961_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e0dbcf7b-8a38-4609-a427-1f42c3746965_isrForCrosstalkSources_996961_59.stderr parsl.dataflow.dflow INFO: Task 1110 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1421fe6b6cc6497a4bdcb5131195fd8c with result from task 1110 parsl.dataflow.dflow DEBUG: Task 1110 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a55b7bbb-b782-4482-9ece-e9aa31c0f74b_isrForCrosstalkSources_997000_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a55b7bbb-b782-4482-9ece-e9aa31c0f74b_isrForCrosstalkSources_997000_48.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:51 proc_lsst.multi:146 [INFO] found job 17074932 in provider escience_a40 proc_lsst.multi INFO: found job 17074932 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1052 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25eb84e46bd0de29c2727584e2819f4d with result from task 1052 parsl.dataflow.dflow DEBUG: Task 1052 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/213fc11d-81fa-44a2-9cf8-3ff61cf27512_isrForCrosstalkSources_997025_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/213fc11d-81fa-44a2-9cf8-3ff61cf27512_isrForCrosstalkSources_997025_56.stderr parsl.dataflow.dflow INFO: Task 1144 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e8c6fe5949d6ad4da28654d4b70f979 with result from task 1144 parsl.dataflow.dflow DEBUG: Task 1144 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/e70921fc-c229-4b8a-8c1c-d0a6c8dc3fee_isrForCrosstalkSources_997013_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/e70921fc-c229-4b8a-8c1c-d0a6c8dc3fee_isrForCrosstalkSources_997013_11.stderr parsl.dataflow.dflow INFO: Task 1161 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02c8bec20ad2ebabd449e7eaba470628 with result from task 1161 parsl.dataflow.dflow DEBUG: Task 1161 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/24d583d4-a8af-48f4-983a-76e164c75572_isrForCrosstalkSources_997019_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/24d583d4-a8af-48f4-983a-76e164c75572_isrForCrosstalkSources_997019_23.stderr parsl.dataflow.dflow INFO: Task 1200 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 805e484fe4f8596ccefa31ac95e904b8 with result from task 1200 parsl.dataflow.dflow DEBUG: Task 1200 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7314f359-5bd9-4d62-9cb3-97cebcefb2ab_isrForCrosstalkSources_997025_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7314f359-5bd9-4d62-9cb3-97cebcefb2ab_isrForCrosstalkSources_997025_16.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:52 proc_lsst.multi:146 [INFO] found job 17074933 in provider escience_a40 proc_lsst.multi INFO: found job 17074933 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1179 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1301e4b6c7546284b5a30f8357e11885 with result from task 1179 parsl.dataflow.dflow DEBUG: Task 1179 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7b755f60-0448-4c26-aefe-0a8fa6883c1a_isrForCrosstalkSources_996974_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7b755f60-0448-4c26-aefe-0a8fa6883c1a_isrForCrosstalkSources_996974_56.stderr parsl.dataflow.dflow INFO: Task 1158 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d958c4bea070e570fb24b6d5898f1075 with result from task 1158 parsl.dataflow.dflow DEBUG: Task 1158 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e938bb47-0523-4446-a7e7-af83238ffa89_isrForCrosstalkSources_997022_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e938bb47-0523-4446-a7e7-af83238ffa89_isrForCrosstalkSources_997022_18.stderr parsl.dataflow.dflow INFO: Task 1097 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13ff227fd0573203ec5f53a610cf6df1 with result from task 1097 parsl.dataflow.dflow DEBUG: Task 1097 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/26d0c6b3-cacb-4c17-83af-41005d8dba8a_isrForCrosstalkSources_996983_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/26d0c6b3-cacb-4c17-83af-41005d8dba8a_isrForCrosstalkSources_996983_6.stderr parsl.dataflow.dflow INFO: Task 1182 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59ff57f35e04900a76c56b198d104791 with result from task 1182 parsl.dataflow.dflow DEBUG: Task 1182 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/29093b24-aac8-4290-9850-15a290c888e7_isrForCrosstalkSources_996979_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/29093b24-aac8-4290-9850-15a290c888e7_isrForCrosstalkSources_996979_47.stderr parsl.dataflow.dflow INFO: Task 1103 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c0adf99d92505a33c748b23f1c59eae with result from task 1103 parsl.dataflow.dflow DEBUG: Task 1103 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/819ebf35-579a-4098-9a6c-e13c54999685_isrForCrosstalkSources_997025_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/819ebf35-579a-4098-9a6c-e13c54999685_isrForCrosstalkSources_997025_42.stderr parsl.dataflow.dflow INFO: Task 1076 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 386d9682889cbb194c85260eaf26fe90 with result from task 1076 parsl.dataflow.dflow DEBUG: Task 1076 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3b4c81f7-cb3d-43c0-ae7a-7bb97a3bfca7_isrForCrosstalkSources_996959_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3b4c81f7-cb3d-43c0-ae7a-7bb97a3bfca7_isrForCrosstalkSources_996959_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:52 proc_lsst.multi:146 [INFO] found job 17074934 in provider escience_a40 proc_lsst.multi INFO: found job 17074934 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1068 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d6acc7c55907196f635e2e2b74c0c95 with result from task 1068 parsl.dataflow.dflow DEBUG: Task 1068 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/805f8b5a-937d-42f1-a0d5-fe4757e002c8_isrForCrosstalkSources_996965_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/805f8b5a-937d-42f1-a0d5-fe4757e002c8_isrForCrosstalkSources_996965_21.stderr parsl.dataflow.dflow INFO: Task 1107 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2731f8cf9b45fbc3a8ed0e253baebe23 with result from task 1107 parsl.dataflow.dflow DEBUG: Task 1107 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/dc7844b2-f034-4302-8c20-86fa14ea0331_isrForCrosstalkSources_997010_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/dc7844b2-f034-4302-8c20-86fa14ea0331_isrForCrosstalkSources_997010_33.stderr parsl.dataflow.dflow INFO: Task 1049 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 037eb83588913ecd53a3eebecb0aef10 with result from task 1049 parsl.dataflow.dflow DEBUG: Task 1049 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6f3c43ae-20c6-4e36-abcd-503e9e802852_isrForCrosstalkSources_996983_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6f3c43ae-20c6-4e36-abcd-503e9e802852_isrForCrosstalkSources_996983_34.stderr parsl.dataflow.dflow INFO: Task 1112 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d345a7c4486675c71097bc19334301a0 with result from task 1112 parsl.dataflow.dflow DEBUG: Task 1112 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/7d1daedb-bc0c-4ef0-a801-d0bf7eb823e5_isrForCrosstalkSources_996992_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/7d1daedb-bc0c-4ef0-a801-d0bf7eb823e5_isrForCrosstalkSources_996992_34.stderr parsl.dataflow.dflow INFO: Task 1126 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 417d35ebc5d361fce61e7123a66bd806 with result from task 1126 parsl.dataflow.dflow DEBUG: Task 1126 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e3e36a87-ab49-404b-8419-33998435197a_isrForCrosstalkSources_997005_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e3e36a87-ab49-404b-8419-33998435197a_isrForCrosstalkSources_997005_37.stderr parsl.dataflow.dflow INFO: Task 1020 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aca989ca66353c0958cdfb9de06aed29 with result from task 1020 parsl.dataflow.dflow DEBUG: Task 1020 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7da26623-734d-4265-966b-446581b285e1_isrForCrosstalkSources_996968_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7da26623-734d-4265-966b-446581b285e1_isrForCrosstalkSources_996968_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:53 proc_lsst.multi:146 [INFO] found job 17074935 in provider escience_a40 proc_lsst.multi INFO: found job 17074935 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1030 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18cebfca1aa9a6b4f6637b1b819644b1 with result from task 1030 parsl.dataflow.dflow DEBUG: Task 1030 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/6cd0833c-bb02-449d-937b-36bfb541f4a2_isrForCrosstalkSources_997014_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/6cd0833c-bb02-449d-937b-36bfb541f4a2_isrForCrosstalkSources_997014_30.stderr parsl.dataflow.dflow INFO: Task 1017 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d5170a7d167873189f299228cd6f9d5 with result from task 1017 parsl.dataflow.dflow DEBUG: Task 1017 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d078fc2d-b6af-4c89-9616-6606a629b4a4_isrForCrosstalkSources_996966_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d078fc2d-b6af-4c89-9616-6606a629b4a4_isrForCrosstalkSources_996966_44.stderr parsl.dataflow.dflow INFO: Task 1101 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fdeaa90cc7610e17812510a37cfe075 with result from task 1101 parsl.dataflow.dflow DEBUG: Task 1101 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b49430e5-607f-411f-8100-6a46baf9123a_isrForCrosstalkSources_996964_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b49430e5-607f-411f-8100-6a46baf9123a_isrForCrosstalkSources_996964_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:53 proc_lsst.multi:146 [INFO] found job 17074936 in provider escience_a40 proc_lsst.multi INFO: found job 17074936 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1010 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 787cb914c58fe99bc4266bc9978bf55a with result from task 1010 parsl.dataflow.dflow DEBUG: Task 1010 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/42189ab0-7b0d-4a49-9ea8-c360cea2c695_isrForCrosstalkSources_997011_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/42189ab0-7b0d-4a49-9ea8-c360cea2c695_isrForCrosstalkSources_997011_41.stderr parsl.dataflow.dflow INFO: Task 1111 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51b966f34f315b3cace0c797da997203 with result from task 1111 parsl.dataflow.dflow DEBUG: Task 1111 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/66d0fb60-6bc9-4692-be4f-809e1c0c66db_isrForCrosstalkSources_996997_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/66d0fb60-6bc9-4692-be4f-809e1c0c66db_isrForCrosstalkSources_996997_4.stderr parsl.dataflow.dflow INFO: Task 1201 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cfde1d9f70fbca26f6353fda72452cf2 with result from task 1201 parsl.dataflow.dflow DEBUG: Task 1201 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/23d25687-64f6-4101-a9e0-6b8238c715cd_isrForCrosstalkSources_996983_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/23d25687-64f6-4101-a9e0-6b8238c715cd_isrForCrosstalkSources_996983_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:54 proc_lsst.multi:146 [INFO] found job 17074937 in provider escience_a40 proc_lsst.multi INFO: found job 17074937 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1124 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a61ff1144798c6d7a3b09265eef8721c with result from task 1124 parsl.dataflow.dflow DEBUG: Task 1124 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/ab58644f-4b15-485f-ae74-d58fcfe458dc_isrForCrosstalkSources_996985_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/ab58644f-4b15-485f-ae74-d58fcfe458dc_isrForCrosstalkSources_996985_34.stderr parsl.dataflow.dflow INFO: Task 1011 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 828c5743d0239fc537b4b6b1d44135bd with result from task 1011 parsl.dataflow.dflow DEBUG: Task 1011 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b67fc318-170b-4f53-9502-2faded0d9e27_isrForCrosstalkSources_996972_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b67fc318-170b-4f53-9502-2faded0d9e27_isrForCrosstalkSources_996972_9.stderr parsl.dataflow.dflow INFO: Task 1087 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73226312a59f1daafd014fc04446a216 with result from task 1087 parsl.dataflow.dflow DEBUG: Task 1087 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/732c7440-1e97-43c2-9383-48565eeffcba_isrForCrosstalkSources_997016_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/732c7440-1e97-43c2-9383-48565eeffcba_isrForCrosstalkSources_997016_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:54 proc_lsst.multi:146 [INFO] found job 17074938 in provider escience_a40 proc_lsst.multi INFO: found job 17074938 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1099 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 617b487a05ef4bfc7b1326c32225aea2 with result from task 1099 parsl.dataflow.dflow DEBUG: Task 1099 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/97b6692c-ad34-48c1-ac2d-ed373151552e_isrForCrosstalkSources_996974_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/97b6692c-ad34-48c1-ac2d-ed373151552e_isrForCrosstalkSources_996974_8.stderr parsl.dataflow.dflow INFO: Task 1212 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac87167752830cffc03db1bd8e35a93d with result from task 1212 parsl.dataflow.dflow DEBUG: Task 1212 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/39894ad4-2e88-4e09-a808-6f0eba5f2dc2_isrForCrosstalkSources_996966_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/39894ad4-2e88-4e09-a808-6f0eba5f2dc2_isrForCrosstalkSources_996966_21.stderr parsl.dataflow.dflow INFO: Task 1170 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4eed2730c31c749688ad20dc189ecd8 with result from task 1170 parsl.dataflow.dflow DEBUG: Task 1170 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8fb406ca-193b-4fde-92fb-7d7faff51e28_isrForCrosstalkSources_996963_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8fb406ca-193b-4fde-92fb-7d7faff51e28_isrForCrosstalkSources_996963_19.stderr parsl.dataflow.dflow INFO: Task 1065 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93dcde49c2b2611b6bbec6b736a3b20c with result from task 1065 parsl.dataflow.dflow DEBUG: Task 1065 checkpointed parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 1065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/41773c26-932f-4c59-a10a-7628dbde377a_isrForCrosstalkSources_996990_39.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 1065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/41773c26-932f-4c59-a10a-7628dbde377a_isrForCrosstalkSources_996990_39.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 1208 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 802451328e01a3d65ca1d1a27a858882 with result from task 1208 parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 1208 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 1208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/ead171b2-cef5-4568-9f6e-3a84fc52f274_isrForCrosstalkSources_997018_34.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:55 proc_lsst.multi:146 [INFO] found job 17074939 in provider escience_a40 proc_lsst.multi INFO: found job 17074939 in provider escience_a40 parsl.dataflow.dflow INFO: Standard error for task 1208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/ead171b2-cef5-4568-9f6e-3a84fc52f274_isrForCrosstalkSources_997018_34.stderr parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1046 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 174fc165cbdeb1a5c296be70b286325b with result from task 1046 parsl.dataflow.dflow DEBUG: Task 1046 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f84438b6-6749-4118-befa-6fa3b3481111_isrForCrosstalkSources_996971_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f84438b6-6749-4118-befa-6fa3b3481111_isrForCrosstalkSources_996971_62.stderr parsl.dataflow.dflow INFO: Task 1189 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e42ca05a76ade63b31c3a2eea8fa90e with result from task 1189 parsl.dataflow.dflow DEBUG: Task 1189 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fafaf523-fc1b-466e-b836-f4a0d2a169e4_isrForCrosstalkSources_996955_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fafaf523-fc1b-466e-b836-f4a0d2a169e4_isrForCrosstalkSources_996955_21.stderr parsl.dataflow.dflow INFO: Task 1056 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ad76d4dac1d4a9d6d3cc7f4e5af0bc7 with result from task 1056 parsl.dataflow.dflow DEBUG: Task 1056 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/d849dd15-cb64-4647-952f-a2457ed167c4_isrForCrosstalkSources_996961_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/d849dd15-cb64-4647-952f-a2457ed167c4_isrForCrosstalkSources_996961_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:55 proc_lsst.multi:146 [INFO] found job 17074940 in provider escience_a40 proc_lsst.multi INFO: found job 17074940 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1190 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91350a0b6ff653ea47fbe736b97bc243 with result from task 1190 parsl.dataflow.dflow DEBUG: Task 1190 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/887801d3-f83f-4caf-9b02-cf4b276a3807_isrForCrosstalkSources_997003_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/887801d3-f83f-4caf-9b02-cf4b276a3807_isrForCrosstalkSources_997003_13.stderr parsl.dataflow.dflow INFO: Task 1032 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5cff2846a399b20edc195b5e219a738 with result from task 1032 parsl.dataflow.dflow DEBUG: Task 1032 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/cfd0c747-71f3-40b6-a6cc-d35103ea13e5_isrForCrosstalkSources_997025_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/cfd0c747-71f3-40b6-a6cc-d35103ea13e5_isrForCrosstalkSources_997025_11.stderr parsl.dataflow.dflow INFO: Task 1146 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf6248c05126bd50c4f2f3809e9755cf with result from task 1146 parsl.dataflow.dflow DEBUG: Task 1146 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/7fbeeba6-5452-4e76-a9d4-06a311cea8b4_isrForCrosstalkSources_996959_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/7fbeeba6-5452-4e76-a9d4-06a311cea8b4_isrForCrosstalkSources_996959_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:56 proc_lsst.multi:146 [INFO] found job 17074942 in provider escience_a40 proc_lsst.multi INFO: found job 17074942 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1012 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 892994c111ed05564811b8fe00f45eb4 with result from task 1012 parsl.dataflow.dflow DEBUG: Task 1012 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/63ca5720-1bb2-4dd1-ad21-c408785f2a55_isrForCrosstalkSources_996988_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/63ca5720-1bb2-4dd1-ad21-c408785f2a55_isrForCrosstalkSources_996988_30.stderr parsl.dataflow.dflow INFO: Task 1024 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b2b81ee2aa2f41e0f9cac70b2ae61c3 with result from task 1024 parsl.dataflow.dflow DEBUG: Task 1024 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d8218b56-96f6-40e4-a347-93de56259202_isrForCrosstalkSources_996987_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d8218b56-96f6-40e4-a347-93de56259202_isrForCrosstalkSources_996987_55.stderr parsl.dataflow.dflow INFO: Task 1128 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 335fe23d638b1cb21d39fecc3570e0ef with result from task 1128 parsl.dataflow.dflow DEBUG: Task 1128 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c6f0f579-e782-480d-84ca-a1d3d6d15b87_isrForCrosstalkSources_997020_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c6f0f579-e782-480d-84ca-a1d3d6d15b87_isrForCrosstalkSources_997020_45.stderr parsl.dataflow.dflow INFO: Task 1077 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75ec9a7757ae60f592020f261ec888d2 with result from task 1077 parsl.dataflow.dflow DEBUG: Task 1077 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/17e28cfa-e183-4420-bf75-5e68da6bf9bf_isrForCrosstalkSources_996982_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/17e28cfa-e183-4420-bf75-5e68da6bf9bf_isrForCrosstalkSources_996982_39.stderr parsl.dataflow.dflow INFO: Task 1081 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d545d4141ce96b796a85a355a4c2d6a6 with result from task 1081 parsl.dataflow.dflow DEBUG: Task 1081 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/08d7c441-5637-4f9e-b855-61f0651b9dd9_isrForCrosstalkSources_997024_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/08d7c441-5637-4f9e-b855-61f0651b9dd9_isrForCrosstalkSources_997024_7.stderr parsl.dataflow.dflow INFO: Task 1028 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97e23bb656ad250e0832fabd58e4d993 with result from task 1028 parsl.dataflow.dflow DEBUG: Task 1028 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cea42da8-b66c-43e2-a890-a22c29d17a3b_isrForCrosstalkSources_996983_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cea42da8-b66c-43e2-a890-a22c29d17a3b_isrForCrosstalkSources_996983_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:56 proc_lsst.multi:146 [INFO] found job 17074974 in provider escience_a40 proc_lsst.multi INFO: found job 17074974 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1209 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9ccc716059e7f1558a2f94cdc3a9167 with result from task 1209 parsl.dataflow.dflow DEBUG: Task 1209 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3a45069f-0b9e-4a60-9414-87d2ee54caf1_isrForCrosstalkSources_996999_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3a45069f-0b9e-4a60-9414-87d2ee54caf1_isrForCrosstalkSources_996999_55.stderr parsl.dataflow.dflow INFO: Task 1191 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3be1e36faf897267a8de437707e00eb4 with result from task 1191 parsl.dataflow.dflow DEBUG: Task 1191 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/97122055-3346-4709-95c4-0150403e6538_isrForCrosstalkSources_997010_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/97122055-3346-4709-95c4-0150403e6538_isrForCrosstalkSources_997010_11.stderr parsl.dataflow.dflow INFO: Task 1139 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd7a63d130d120ab778e8e6afd3594c3 with result from task 1139 parsl.dataflow.dflow DEBUG: Task 1139 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/aa709e19-52fb-44e0-a9e5-a2388db2f529_isrForCrosstalkSources_996956_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/aa709e19-52fb-44e0-a9e5-a2388db2f529_isrForCrosstalkSources_996956_53.stderr parsl.dataflow.dflow INFO: Task 1036 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 209afe62c75fa33f9217b4f99965726a with result from task 1036 parsl.dataflow.dflow DEBUG: Task 1036 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/22559425-67fa-4727-942a-0576f6a9609a_isrForCrosstalkSources_997030_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/22559425-67fa-4727-942a-0576f6a9609a_isrForCrosstalkSources_997030_43.stderr parsl.dataflow.dflow INFO: Task 1226 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 513b6682d86cc6e86fa80a6dad3b7d7c with result from task 1226 parsl.dataflow.dflow DEBUG: Task 1226 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/d7ee40c8-5419-4a15-80ab-0f866e8e4da5_isrForCrosstalkSources_997019_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/d7ee40c8-5419-4a15-80ab-0f866e8e4da5_isrForCrosstalkSources_997019_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:57 proc_lsst.multi:146 [INFO] found job 17075003 in provider escience_a40 proc_lsst.multi INFO: found job 17075003 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1019 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 368e51c228b79d09f284dba3c8d058b8 with result from task 1019 parsl.dataflow.dflow DEBUG: Task 1019 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/d7cd5b54-0403-4f9b-8f5d-c0893dde9c17_isrForCrosstalkSources_997013_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/d7cd5b54-0403-4f9b-8f5d-c0893dde9c17_isrForCrosstalkSources_997013_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:58 proc_lsst.multi:146 [INFO] found job 17075018 in provider escience_a40 proc_lsst.multi INFO: found job 17075018 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1202 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d14f7756d148539a04dfb8f993a20db2 with result from task 1202 parsl.dataflow.dflow DEBUG: Task 1202 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/0eb02d11-02ff-40ef-8f49-3f17d9f7966a_isrForCrosstalkSources_996967_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/0eb02d11-02ff-40ef-8f49-3f17d9f7966a_isrForCrosstalkSources_996967_33.stderr parsl.dataflow.dflow INFO: Task 1168 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 813ce8f2468437a5818f73ffcc03e752 with result from task 1168 parsl.dataflow.dflow DEBUG: Task 1168 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/aad7d0cc-2490-4b27-9d7e-55bc9fe143cb_isrForCrosstalkSources_996996_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/aad7d0cc-2490-4b27-9d7e-55bc9fe143cb_isrForCrosstalkSources_996996_14.stderr parsl.dataflow.dflow INFO: Task 1045 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cacc47fa145ed6f0798ba1a455a10c49 with result from task 1045 parsl.dataflow.dflow DEBUG: Task 1045 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/8163d928-6f89-4f52-8d01-a7d2f4f09f5f_isrForCrosstalkSources_996978_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/8163d928-6f89-4f52-8d01-a7d2f4f09f5f_isrForCrosstalkSources_996978_58.stderr parsl.dataflow.dflow INFO: Task 1057 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e7b003fab546611bfacaeb85ab4c16b with result from task 1057 parsl.dataflow.dflow DEBUG: Task 1057 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f6fbbdbc-f9c9-4d13-b914-ea9c5d033162_isrForCrosstalkSources_996973_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f6fbbdbc-f9c9-4d13-b914-ea9c5d033162_isrForCrosstalkSources_996973_60.stderr parsl.dataflow.dflow INFO: Task 1135 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 629ae47a2cd0ebe8d2ca3f0115b6ffa6 with result from task 1135 parsl.dataflow.dflow DEBUG: Task 1135 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4f85b213-46a5-47c9-8856-ca1c4f505de2_isrForCrosstalkSources_996976_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4f85b213-46a5-47c9-8856-ca1c4f505de2_isrForCrosstalkSources_996976_9.stderr parsl.dataflow.dflow INFO: Task 1131 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b2c7067ff1630e378d83fcfae7f5ec3 with result from task 1131 parsl.dataflow.dflow DEBUG: Task 1131 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c0c9a97f-2b12-4c32-b053-fe4e0a8f3a68_isrForCrosstalkSources_997004_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c0c9a97f-2b12-4c32-b053-fe4e0a8f3a68_isrForCrosstalkSources_997004_44.stderr parsl.dataflow.dflow INFO: Task 1222 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21e30f5661fa628dc023b1cec68eac29 with result from task 1222 parsl.dataflow.dflow DEBUG: Task 1222 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/d15d1658-95e7-4b35-ae58-3aca3d3dc485_isrForCrosstalkSources_996957_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/d15d1658-95e7-4b35-ae58-3aca3d3dc485_isrForCrosstalkSources_996957_14.stderr parsl.dataflow.dflow INFO: Task 1104 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13dab32d19f808c55ef210f1025857f9 with result from task 1104 parsl.dataflow.dflow DEBUG: Task 1104 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/d08e9b66-c334-457b-984f-a496427d6f5d_isrForCrosstalkSources_996955_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/d08e9b66-c334-457b-984f-a496427d6f5d_isrForCrosstalkSources_996955_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:59 proc_lsst.multi:146 [INFO] found job 17075019 in provider escience_a40 proc_lsst.multi INFO: found job 17075019 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1230 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7096ce7c3f10a969bc7e7ba73e3eb69c with result from task 1230 parsl.dataflow.dflow DEBUG: Task 1230 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/c90fd4c8-50b0-44ab-9ed0-0e7ef528f916_isrForCrosstalkSources_996963_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/c90fd4c8-50b0-44ab-9ed0-0e7ef528f916_isrForCrosstalkSources_996963_11.stderr parsl.dataflow.dflow INFO: Task 1142 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c250704ec5dffdc590635065535e1391 with result from task 1142 parsl.dataflow.dflow DEBUG: Task 1142 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b2dccd33-a1e1-4399-8e19-13f7272fff75_isrForCrosstalkSources_996962_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b2dccd33-a1e1-4399-8e19-13f7272fff75_isrForCrosstalkSources_996962_10.stderr parsl.dataflow.dflow INFO: Task 1169 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef8c448a0c4a76e03dd68aacc1bd8e57 with result from task 1169 parsl.dataflow.dflow DEBUG: Task 1169 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/2cc2f660-29d4-4d63-a1f5-d0b59eea3216_isrForCrosstalkSources_996992_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/2cc2f660-29d4-4d63-a1f5-d0b59eea3216_isrForCrosstalkSources_996992_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:53:59 proc_lsst.multi:146 [INFO] found job 17075020 in provider escience_a40 proc_lsst.multi INFO: found job 17075020 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1105 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d445cfb2290127acca34327afd003823 with result from task 1105 parsl.dataflow.dflow DEBUG: Task 1105 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/1c368f6c-f1c7-4e6b-a687-5c75b31f2812_isrForCrosstalkSources_996976_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/1c368f6c-f1c7-4e6b-a687-5c75b31f2812_isrForCrosstalkSources_996976_46.stderr parsl.dataflow.dflow INFO: Task 1205 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 820cb2ecb5208636577a882ee546bbd2 with result from task 1205 parsl.dataflow.dflow DEBUG: Task 1205 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/0eb4ea75-99ce-433d-ab38-879a4c7b83fd_isrForCrosstalkSources_997031_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/0eb4ea75-99ce-433d-ab38-879a4c7b83fd_isrForCrosstalkSources_997031_56.stderr parsl.dataflow.dflow INFO: Task 1171 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6218abd8760f15e396ce77548f1d5568 with result from task 1171 parsl.dataflow.dflow DEBUG: Task 1171 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/bd4859e9-b8aa-43f2-8587-5ca72fc9fd7e_isrForCrosstalkSources_996989_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/bd4859e9-b8aa-43f2-8587-5ca72fc9fd7e_isrForCrosstalkSources_996989_55.stderr parsl.dataflow.dflow INFO: Task 1164 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 308555dd8c61a0ccfb590070a09a72ea with result from task 1164 parsl.dataflow.dflow DEBUG: Task 1164 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d98fb72f-67db-47ff-9cc7-a47386895cf1_isrForCrosstalkSources_997001_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d98fb72f-67db-47ff-9cc7-a47386895cf1_isrForCrosstalkSources_997001_39.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:00 proc_lsst.multi:146 [INFO] found job 17075021 in provider escience_a40 proc_lsst.multi INFO: found job 17075021 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 1079 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e81d7dd0879c98da52f934e903ad034 with result from task 1079 parsl.dataflow.dflow DEBUG: Task 1079 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4d3dada1-ce8c-4401-af86-57d70f127aac_isrForCrosstalkSources_997018_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4d3dada1-ce8c-4401-af86-57d70f127aac_isrForCrosstalkSources_997018_37.stderr parsl.dataflow.dflow INFO: Task 1125 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2eda869d6e624d88e3a53a07f8376021 with result from task 1125 parsl.dataflow.dflow DEBUG: Task 1125 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d38c02b6-5d32-48d4-9eef-89e854b55534_isrForCrosstalkSources_996997_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d38c02b6-5d32-48d4-9eef-89e854b55534_isrForCrosstalkSources_996997_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:00 proc_lsst.multi:146 [INFO] found job 17075022 in provider checkpoint proc_lsst.multi INFO: found job 17075022 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1075 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01cd70e4b69cd0efdfdae7dda25eaab7 with result from task 1075 parsl.dataflow.dflow DEBUG: Task 1075 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9eaedfff-ff44-4955-b80f-a2614e93cfdf_isrForCrosstalkSources_996995_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9eaedfff-ff44-4955-b80f-a2614e93cfdf_isrForCrosstalkSources_996995_53.stderr parsl.dataflow.dflow INFO: Task 1156 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d940cd76dcf54f0f43e5a4258be66fb with result from task 1156 parsl.dataflow.dflow DEBUG: Task 1156 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b19d1ef0-5441-41f3-b8fb-e04e361b8d2b_isrForCrosstalkSources_997016_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b19d1ef0-5441-41f3-b8fb-e04e361b8d2b_isrForCrosstalkSources_997016_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:01 proc_lsst.multi:146 [INFO] found job 17075030 in provider checkpoint proc_lsst.multi INFO: found job 17075030 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1098 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2e78fa7f28842b0375b9babe6040e41 with result from task 1098 parsl.dataflow.dflow DEBUG: Task 1098 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d4a70fd8-325e-494b-b693-58dd87f950bf_isrForCrosstalkSources_996982_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d4a70fd8-325e-494b-b693-58dd87f950bf_isrForCrosstalkSources_996982_12.stderr parsl.dataflow.dflow INFO: Task 1033 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de23b1e2fe88828edca083beaabfb890 with result from task 1033 parsl.dataflow.dflow DEBUG: Task 1033 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/245475c8-48dc-4a31-88c4-c497de45483a_isrForCrosstalkSources_997008_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/245475c8-48dc-4a31-88c4-c497de45483a_isrForCrosstalkSources_997008_2.stderr parsl.dataflow.dflow INFO: Task 1136 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb087f41af8774723121368863bfd756 with result from task 1136 parsl.dataflow.dflow DEBUG: Task 1136 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b7bb596b-38fd-48da-a8fb-a45a4f31d541_isrForCrosstalkSources_997003_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b7bb596b-38fd-48da-a8fb-a45a4f31d541_isrForCrosstalkSources_997003_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1255 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 572fe4260e79bae26d202270003ff004 with result from task 1255 parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1255 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e152647a-a845-4eda-9b09-164037dd2846_isrForCrosstalkSources_996985_2.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e152647a-a845-4eda-9b09-164037dd2846_isrForCrosstalkSources_996985_2.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:01 proc_lsst.multi:146 [INFO] found job 17075031 in provider checkpoint proc_lsst.multi INFO: found job 17075031 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1235 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6a05a3c99ea7ef12caf19883d2bc452 with result from task 1235 parsl.dataflow.dflow DEBUG: Task 1235 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c9f24141-ed61-4dcb-b9c5-52b5f357b025_isrForCrosstalkSources_996985_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c9f24141-ed61-4dcb-b9c5-52b5f357b025_isrForCrosstalkSources_996985_55.stderr parsl.dataflow.dflow INFO: Task 1251 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 922002c5aa0ec1357f31a068bf726ca9 with result from task 1251 parsl.dataflow.dflow DEBUG: Task 1251 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6185a494-953e-4992-9eb7-3488ebe0d28c_isrForCrosstalkSources_997010_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6185a494-953e-4992-9eb7-3488ebe0d28c_isrForCrosstalkSources_997010_51.stderr parsl.dataflow.dflow INFO: Task 1279 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 499e342d97b5eacfd85083addafc6389 with result from task 1279 parsl.dataflow.dflow DEBUG: Task 1279 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b1ee44f1-4c6b-4394-86e4-dfeadba08318_isrForCrosstalkSources_996972_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/b1ee44f1-4c6b-4394-86e4-dfeadba08318_isrForCrosstalkSources_996972_8.stderr parsl.dataflow.dflow INFO: Task 1140 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0cd7d8f9f057afa0c25b62adf70ee18a with result from task 1140 parsl.dataflow.dflow DEBUG: Task 1140 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/78108d1a-ad6a-47da-9141-12e17d6a5d69_isrForCrosstalkSources_996962_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/78108d1a-ad6a-47da-9141-12e17d6a5d69_isrForCrosstalkSources_996962_29.stderr parsl.dataflow.dflow INFO: Task 1167 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6412466317878c36cc2e15d969d3aac8 with result from task 1167 parsl.dataflow.dflow DEBUG: Task 1167 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b2111f0c-297d-4d02-97c4-1226b1cab4ed_isrForCrosstalkSources_997030_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b2111f0c-297d-4d02-97c4-1226b1cab4ed_isrForCrosstalkSources_997030_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:02 proc_lsst.multi:146 [INFO] found job 17075032 in provider checkpoint proc_lsst.multi INFO: found job 17075032 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1094 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15a68c04c7d432c975f6b89cf1c1788b with result from task 1094 parsl.dataflow.dflow DEBUG: Task 1094 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ffbbf283-35c3-474e-8169-da64432f608d_isrForCrosstalkSources_996986_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ffbbf283-35c3-474e-8169-da64432f608d_isrForCrosstalkSources_996986_41.stderr parsl.dataflow.dflow INFO: Task 1147 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fc67e839094c5f61f5246f09430a399a with result from task 1147 parsl.dataflow.dflow DEBUG: Task 1147 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/974eddc5-4261-467b-8be3-3350eb01e88b_isrForCrosstalkSources_996972_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/974eddc5-4261-467b-8be3-3350eb01e88b_isrForCrosstalkSources_996972_19.stderr parsl.dataflow.dflow INFO: Task 1206 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac478fd24779e551e09f0cc6690a182e with result from task 1206 parsl.dataflow.dflow DEBUG: Task 1206 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a3ca6510-a34a-4a02-8f39-ec82d6654db5_isrForCrosstalkSources_996961_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a3ca6510-a34a-4a02-8f39-ec82d6654db5_isrForCrosstalkSources_996961_17.stderr parsl.dataflow.dflow INFO: Task 1266 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1534eb0384ffd0e754a657760cbbf62c with result from task 1266 parsl.dataflow.dflow DEBUG: Task 1266 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/830dd92a-dedb-4fea-9d5c-7408583c201d_isrForCrosstalkSources_996965_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/830dd92a-dedb-4fea-9d5c-7408583c201d_isrForCrosstalkSources_996965_50.stderr parsl.dataflow.dflow INFO: Task 1114 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 37d2a27ace51100dc00824afbe4c9885 with result from task 1114 parsl.dataflow.dflow DEBUG: Task 1114 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4ec175dd-a44b-4800-86ec-4feab3945b88_isrForCrosstalkSources_996995_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4ec175dd-a44b-4800-86ec-4feab3945b88_isrForCrosstalkSources_996995_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1150 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry a7296708174289708739315fa0a91d76 with result from task 1150 parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1150 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/51859399-caba-4961-ad68-9ab6453b5181_isrForCrosstalkSources_996953_49.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/51859399-caba-4961-ad68-9ab6453b5181_isrForCrosstalkSources_996953_49.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1180 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry a7cf1aaebff50501a5533f2dc23c389d with result from task 1180 parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 1180 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:02 proc_lsst.multi:146 [INFO] found job 17075033 in provider checkpoint proc_lsst.multi INFO: found job 17075033 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/061d4a9c-fbe4-432d-a179-102381ef8405_isrForCrosstalkSources_996955_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/061d4a9c-fbe4-432d-a179-102381ef8405_isrForCrosstalkSources_996955_28.stderr parsl.dataflow.dflow INFO: Task 1121 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f5391f66c976f5a59bdb108333d677b with result from task 1121 parsl.dataflow.dflow DEBUG: Task 1121 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/5b979aae-f932-405b-a9aa-24c1c60198fd_isrForCrosstalkSources_996986_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/5b979aae-f932-405b-a9aa-24c1c60198fd_isrForCrosstalkSources_996986_5.stderr parsl.dataflow.dflow INFO: Task 1185 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c92bdfafaf907e25f610378919b9bed with result from task 1185 parsl.dataflow.dflow DEBUG: Task 1185 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/77a72adf-b9a7-44be-a2c5-ee12523ff770_isrForCrosstalkSources_996954_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/77a72adf-b9a7-44be-a2c5-ee12523ff770_isrForCrosstalkSources_996954_3.stderr parsl.dataflow.dflow INFO: Task 1176 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e8dcb502b07c7f45f50fc9f3e85d1da with result from task 1176 parsl.dataflow.dflow DEBUG: Task 1176 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/501ea7ee-9171-44ed-9c96-10189d3a75e5_isrForCrosstalkSources_997002_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/501ea7ee-9171-44ed-9c96-10189d3a75e5_isrForCrosstalkSources_997002_18.stderr parsl.dataflow.dflow INFO: Task 1042 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99ad1b7a5e1ef2029cdbd22df20b6388 with result from task 1042 parsl.dataflow.dflow DEBUG: Task 1042 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d419ddf5-fe04-4bd4-90a1-6d7a84b5bb29_isrForCrosstalkSources_996981_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d419ddf5-fe04-4bd4-90a1-6d7a84b5bb29_isrForCrosstalkSources_996981_7.stderr parsl.dataflow.dflow INFO: Task 1166 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7c40f793fa70927938ef5efde4f723e with result from task 1166 parsl.dataflow.dflow DEBUG: Task 1166 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/7ab97a65-ac21-4c0b-8269-892678ff3cb3_isrForCrosstalkSources_996977_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/7ab97a65-ac21-4c0b-8269-892678ff3cb3_isrForCrosstalkSources_996977_58.stderr parsl.dataflow.dflow INFO: Task 1227 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d304641628670634683b847a4d3dbb67 with result from task 1227 parsl.dataflow.dflow DEBUG: Task 1227 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9056e112-aee5-451d-8342-90fdf5fcf584_isrForCrosstalkSources_996997_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9056e112-aee5-451d-8342-90fdf5fcf584_isrForCrosstalkSources_996997_17.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:03 proc_lsst.multi:146 [INFO] found job 17075034 in provider checkpoint proc_lsst.multi INFO: found job 17075034 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:04 proc_lsst.multi:146 [INFO] found job 17075035 in provider checkpoint proc_lsst.multi INFO: found job 17075035 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1193 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4735563ea45b17957e63b225126aadf7 with result from task 1193 parsl.dataflow.dflow DEBUG: Task 1193 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/fd47f819-cf57-478b-a574-6835a5de6117_isrForCrosstalkSources_996971_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/fd47f819-cf57-478b-a574-6835a5de6117_isrForCrosstalkSources_996971_46.stderr parsl.dataflow.dflow INFO: Task 1268 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0816196881daa4c77d9b284c57f77943 with result from task 1268 parsl.dataflow.dflow DEBUG: Task 1268 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/97930b67-466f-4029-8c4f-b74191b01e2d_isrForCrosstalkSources_996956_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/97930b67-466f-4029-8c4f-b74191b01e2d_isrForCrosstalkSources_996956_23.stderr parsl.dataflow.dflow INFO: Task 1096 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50f092b4438426e64e5319970f82f412 with result from task 1096 parsl.dataflow.dflow DEBUG: Task 1096 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/58c5c2dd-a3ad-4d87-89f4-c3e973d52ef8_isrForCrosstalkSources_996977_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/58c5c2dd-a3ad-4d87-89f4-c3e973d52ef8_isrForCrosstalkSources_996977_48.stderr parsl.dataflow.dflow INFO: Task 1184 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e0262539b8fe2382a001a23c1d8b2af with result from task 1184 parsl.dataflow.dflow DEBUG: Task 1184 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f43a009c-2b64-4d06-960f-7fe396fab50d_isrForCrosstalkSources_996971_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f43a009c-2b64-4d06-960f-7fe396fab50d_isrForCrosstalkSources_996971_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:04 proc_lsst.multi:146 [INFO] found job 17075038 in provider checkpoint proc_lsst.multi INFO: found job 17075038 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1264 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 046bb81ae58217e602f6b1b2f62b871a with result from task 1264 parsl.dataflow.dflow DEBUG: Task 1264 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/217b0639-cadf-4d20-bed7-73adc057b671_isrForCrosstalkSources_996977_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/217b0639-cadf-4d20-bed7-73adc057b671_isrForCrosstalkSources_996977_4.stderr parsl.dataflow.dflow INFO: Task 1210 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 578ed6751e07724172b067cbc1b47996 with result from task 1210 parsl.dataflow.dflow DEBUG: Task 1210 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d8e71428-b76d-4604-863a-5e96068eedb5_isrForCrosstalkSources_996995_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d8e71428-b76d-4604-863a-5e96068eedb5_isrForCrosstalkSources_996995_1.stderr parsl.dataflow.dflow INFO: Task 1061 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19902662f8ef35afdd13c08ff2a1f3d3 with result from task 1061 parsl.dataflow.dflow DEBUG: Task 1061 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c55301a5-84ef-43cb-9131-651fbddd7bbe_isrForCrosstalkSources_997000_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c55301a5-84ef-43cb-9131-651fbddd7bbe_isrForCrosstalkSources_997000_9.stderr parsl.dataflow.dflow INFO: Task 1302 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b457641dc1f9b2b85c6e8c866440c52 with result from task 1302 parsl.dataflow.dflow DEBUG: Task 1302 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/26cd148f-9f36-4c4b-8e22-162596923e08_isrForCrosstalkSources_997020_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/26cd148f-9f36-4c4b-8e22-162596923e08_isrForCrosstalkSources_997020_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:05 proc_lsst.multi:146 [INFO] found job 17075041 in provider checkpoint proc_lsst.multi INFO: found job 17075041 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1188 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f6456678f96933bf649089ef84303f44 with result from task 1188 parsl.dataflow.dflow DEBUG: Task 1188 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/065afaa0-15e3-4c2d-8f7a-9bee862ebc23_isrForCrosstalkSources_996967_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/065afaa0-15e3-4c2d-8f7a-9bee862ebc23_isrForCrosstalkSources_996967_1.stderr parsl.dataflow.dflow INFO: Task 1203 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a658f6bd77b52e788b0dd775978b5151 with result from task 1203 parsl.dataflow.dflow DEBUG: Task 1203 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c0ada6bf-91f5-4071-afdb-52f307ca9a2e_isrForCrosstalkSources_996973_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c0ada6bf-91f5-4071-afdb-52f307ca9a2e_isrForCrosstalkSources_996973_47.stderr parsl.dataflow.dflow INFO: Task 1280 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5854b1a41e1f99555caf1a42d2562491 with result from task 1280 parsl.dataflow.dflow DEBUG: Task 1280 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c273b913-5dfd-4ee5-b33a-023d0061288a_isrForCrosstalkSources_996992_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c273b913-5dfd-4ee5-b33a-023d0061288a_isrForCrosstalkSources_996992_7.stderr parsl.dataflow.dflow INFO: Task 1204 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5563f1a68fe2ba71b49a5135837a3f7d with result from task 1204 parsl.dataflow.dflow DEBUG: Task 1204 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/34636636-b1e3-4de7-b813-ef2550382ea3_isrForCrosstalkSources_997003_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/34636636-b1e3-4de7-b813-ef2550382ea3_isrForCrosstalkSources_997003_1.stderr parsl.dataflow.dflow INFO: Task 1278 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b3738f6b20c9a05ca02a5028086dbeab with result from task 1278 parsl.dataflow.dflow DEBUG: Task 1278 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1d81105a-a7db-4993-97c2-ed281d48d342_isrForCrosstalkSources_997022_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1d81105a-a7db-4993-97c2-ed281d48d342_isrForCrosstalkSources_997022_48.stderr parsl.dataflow.dflow INFO: Task 1160 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4c0fba2427cc3939c8fb8db39ea4631 with result from task 1160 parsl.dataflow.dflow DEBUG: Task 1160 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d810fba2-d99b-4e9e-9476-e500e4b4a7ee_isrForCrosstalkSources_997010_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d810fba2-d99b-4e9e-9476-e500e4b4a7ee_isrForCrosstalkSources_997010_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:05 proc_lsst.multi:146 [INFO] found job 17075042 in provider checkpoint proc_lsst.multi INFO: found job 17075042 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1130 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9d9267446fea07abdbb5a458fcedf67 with result from task 1130 parsl.dataflow.dflow DEBUG: Task 1130 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a942a5b2-95e7-4f86-b5ca-c7402e3f1fcb_isrForCrosstalkSources_997011_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a942a5b2-95e7-4f86-b5ca-c7402e3f1fcb_isrForCrosstalkSources_997011_3.stderr parsl.dataflow.dflow INFO: Task 1175 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23507354b9292c6f3d3d92f5bc6e9df8 with result from task 1175 parsl.dataflow.dflow DEBUG: Task 1175 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/44b348a3-ab23-463c-967a-485e737a6b6b_isrForCrosstalkSources_996964_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/44b348a3-ab23-463c-967a-485e737a6b6b_isrForCrosstalkSources_996964_42.stderr parsl.dataflow.dflow INFO: Task 1330 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a14f1181f22da9f0b64e4087b8f1a85 with result from task 1330 parsl.dataflow.dflow DEBUG: Task 1330 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/416df719-3918-43c2-9187-6e5d7d31030b_isrForCrosstalkSources_996957_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/416df719-3918-43c2-9187-6e5d7d31030b_isrForCrosstalkSources_996957_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:06 proc_lsst.multi:146 [INFO] found job 17075046 in provider checkpoint proc_lsst.multi INFO: found job 17075046 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1283 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 34996a8bb437af436e390519e609aa5c with result from task 1283 parsl.dataflow.dflow DEBUG: Task 1283 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c436a085-14d6-437e-be3f-57b61c4283da_isrForCrosstalkSources_996960_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c436a085-14d6-437e-be3f-57b61c4283da_isrForCrosstalkSources_996960_7.stderr parsl.dataflow.dflow INFO: Task 1078 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 393dd813baab0bd01b62b8bdef91d79e with result from task 1078 parsl.dataflow.dflow DEBUG: Task 1078 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/47c09d99-21b7-4c6a-9692-a2008d0d3765_isrForCrosstalkSources_996966_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/47c09d99-21b7-4c6a-9692-a2008d0d3765_isrForCrosstalkSources_996966_29.stderr parsl.dataflow.dflow INFO: Task 1047 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9cf49c5bf02d59d178482c0f24d4f0b with result from task 1047 parsl.dataflow.dflow DEBUG: Task 1047 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/48429f34-1150-4ce4-a5aa-91cadea7b2e3_isrForCrosstalkSources_996965_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/48429f34-1150-4ce4-a5aa-91cadea7b2e3_isrForCrosstalkSources_996965_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:07 proc_lsst.multi:146 [INFO] found job 17075047 in provider checkpoint proc_lsst.multi INFO: found job 17075047 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1187 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fd43f2f617b8b180e2f8bd1db561796 with result from task 1187 parsl.dataflow.dflow DEBUG: Task 1187 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6332bfa1-0f00-4f5e-8bed-ec6aacc4227f_isrForCrosstalkSources_996983_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6332bfa1-0f00-4f5e-8bed-ec6aacc4227f_isrForCrosstalkSources_996983_38.stderr parsl.dataflow.dflow INFO: Task 1270 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82ddbdb2e84c57009d597f296667b22c with result from task 1270 parsl.dataflow.dflow DEBUG: Task 1270 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7eb32139-4a18-41d3-b141-3dad6ed510f7_isrForCrosstalkSources_996975_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7eb32139-4a18-41d3-b141-3dad6ed510f7_isrForCrosstalkSources_996975_13.stderr parsl.dataflow.dflow INFO: Task 1199 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00f55431d55822dea215d8c731eedaf3 with result from task 1199 parsl.dataflow.dflow DEBUG: Task 1199 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/bcb74919-14b8-4d8f-9f15-f3ed29823127_isrForCrosstalkSources_997020_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/bcb74919-14b8-4d8f-9f15-f3ed29823127_isrForCrosstalkSources_997020_51.stderr parsl.dataflow.dflow INFO: Task 1277 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e838239541bb50151f981c14e96e373 with result from task 1277 parsl.dataflow.dflow DEBUG: Task 1277 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/9be88a99-6663-4aff-a935-113b9503c7c1_isrForCrosstalkSources_997009_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/9be88a99-6663-4aff-a935-113b9503c7c1_isrForCrosstalkSources_997009_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1119 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 33c7a8f237a95d6ee2a7a4dd2e793d49 with result from task 1119 parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1119 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d13afab9-1758-4e01-af80-18ca3bff2ba0_isrForCrosstalkSources_997003_57.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d13afab9-1758-4e01-af80-18ca3bff2ba0_isrForCrosstalkSources_997003_57.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:07 proc_lsst.multi:146 [INFO] found job 17075048 in provider checkpoint proc_lsst.multi INFO: found job 17075048 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1181 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42451e0d3c40a30b21504706e9cc7867 with result from task 1181 parsl.dataflow.dflow DEBUG: Task 1181 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/dc6e75b6-c543-470e-87f8-70a9b37964c9_isrForCrosstalkSources_996963_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/dc6e75b6-c543-470e-87f8-70a9b37964c9_isrForCrosstalkSources_996963_62.stderr parsl.dataflow.dflow INFO: Task 1267 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ccbf8317418675fd66b12ddea512eaf with result from task 1267 parsl.dataflow.dflow DEBUG: Task 1267 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/59fa65e4-eb51-4ff8-b8b8-d741c6461fc1_isrForCrosstalkSources_996969_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/59fa65e4-eb51-4ff8-b8b8-d741c6461fc1_isrForCrosstalkSources_996969_42.stderr parsl.dataflow.dflow INFO: Task 1162 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 71130af273e81f915f892295eda3e7c8 with result from task 1162 parsl.dataflow.dflow DEBUG: Task 1162 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/94bffaba-2460-4e36-a5a4-845fa7a42b46_isrForCrosstalkSources_996989_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/94bffaba-2460-4e36-a5a4-845fa7a42b46_isrForCrosstalkSources_996989_42.stderr parsl.dataflow.dflow INFO: Task 1311 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7bbf3854d3914720267ddb099209929 with result from task 1311 parsl.dataflow.dflow DEBUG: Task 1311 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/da14c176-7358-4465-bdef-b0d060dd7d3b_isrForCrosstalkSources_996978_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/da14c176-7358-4465-bdef-b0d060dd7d3b_isrForCrosstalkSources_996978_25.stderr parsl.dataflow.dflow INFO: Task 1132 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 47ef86121f3869907ba189eeecc124ec with result from task 1132 parsl.dataflow.dflow DEBUG: Task 1132 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/abc005b3-085c-4e1b-9ec8-b7798158ab26_isrForCrosstalkSources_996995_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/abc005b3-085c-4e1b-9ec8-b7798158ab26_isrForCrosstalkSources_996995_50.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:08 proc_lsst.multi:146 [INFO] found job 17075049 in provider checkpoint proc_lsst.multi INFO: found job 17075049 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1165 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7db8f61cbe2a346d37c32942f7239143 with result from task 1165 parsl.dataflow.dflow DEBUG: Task 1165 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6439a416-fe1d-4e98-a644-fc141dc5a846_isrForCrosstalkSources_996952_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6439a416-fe1d-4e98-a644-fc141dc5a846_isrForCrosstalkSources_996952_31.stderr parsl.dataflow.dflow INFO: Task 1197 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa3928a4192612002faab8dfdfca272d with result from task 1197 parsl.dataflow.dflow DEBUG: Task 1197 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/03bab785-8838-493b-bed4-b503221cd3ec_isrForCrosstalkSources_997011_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/03bab785-8838-493b-bed4-b503221cd3ec_isrForCrosstalkSources_997011_2.stderr parsl.dataflow.dflow INFO: Task 1326 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 450d9d400bc14597c661e0404db34076 with result from task 1326 parsl.dataflow.dflow DEBUG: Task 1326 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ba15d08f-9d63-4b7b-846a-f5d7942ac4f1_isrForCrosstalkSources_996960_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ba15d08f-9d63-4b7b-846a-f5d7942ac4f1_isrForCrosstalkSources_996960_35.stderr parsl.dataflow.dflow INFO: Task 1123 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0b87c867498e7bf60718db03175f8239 with result from task 1123 parsl.dataflow.dflow DEBUG: Task 1123 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/ccbe72e2-fe76-4ec7-8c19-1e623d632590_isrForCrosstalkSources_997032_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/ccbe72e2-fe76-4ec7-8c19-1e623d632590_isrForCrosstalkSources_997032_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:08 proc_lsst.multi:146 [INFO] found job 17075050 in provider checkpoint proc_lsst.multi INFO: found job 17075050 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1228 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db1dfb8831c8199c782e971da3431e42 with result from task 1228 parsl.dataflow.dflow DEBUG: Task 1228 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/9c2b0afc-03f9-43b7-bfee-a8d98e3c80b1_isrForCrosstalkSources_996994_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/9c2b0afc-03f9-43b7-bfee-a8d98e3c80b1_isrForCrosstalkSources_996994_4.stderr parsl.dataflow.dflow INFO: Task 1113 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08adae0ff424f155274ca0199569cea1 with result from task 1113 parsl.dataflow.dflow DEBUG: Task 1113 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/d094433f-cb56-40db-ad16-02e40e67947b_isrForCrosstalkSources_996963_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/d094433f-cb56-40db-ad16-02e40e67947b_isrForCrosstalkSources_996963_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:09 proc_lsst.multi:146 [INFO] found job 17075052 in provider checkpoint proc_lsst.multi INFO: found job 17075052 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1093 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry babc269d09fcaa6f86f239134dd6cab5 with result from task 1093 parsl.dataflow.dflow DEBUG: Task 1093 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/0de5cd82-cfb5-4b94-b28c-761cc7193e77_isrForCrosstalkSources_996969_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/0de5cd82-cfb5-4b94-b28c-761cc7193e77_isrForCrosstalkSources_996969_59.stderr parsl.dataflow.dflow INFO: Task 1122 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4db81a5ceaa69fe09c3173caa200c4c6 with result from task 1122 parsl.dataflow.dflow DEBUG: Task 1122 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c3bae945-6dec-4259-8a6b-107282cb88f9_isrForCrosstalkSources_997020_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c3bae945-6dec-4259-8a6b-107282cb88f9_isrForCrosstalkSources_997020_49.stderr parsl.dataflow.dflow INFO: Task 1072 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c99af1620e0142e9c62b0c9deb414b3 with result from task 1072 parsl.dataflow.dflow DEBUG: Task 1072 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/0f0e289a-12f7-4c54-a6c4-6233c62cb7ea_isrForCrosstalkSources_997033_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/0f0e289a-12f7-4c54-a6c4-6233c62cb7ea_isrForCrosstalkSources_997033_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 1221 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 8ae0bc4655941ffd6200c026976b4aa6 with result from task 1221 parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1221 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/421f31cb-9c21-4531-8873-d4692508f4c6_isrForCrosstalkSources_996961_13.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/421f31cb-9c21-4531-8873-d4692508f4c6_isrForCrosstalkSources_996961_13.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1261 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 48eb2b629e1a157bdc5e3e00dfa56b17 with result from task 1261 parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1261 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/bf4198ce-99fe-411f-bb20-26d848799635_isrForCrosstalkSources_996961_16.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/bf4198ce-99fe-411f-bb20-26d848799635_isrForCrosstalkSources_996961_16.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:09 proc_lsst.multi:146 [INFO] found job 17075053 in provider checkpoint proc_lsst.multi INFO: found job 17075053 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1207 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 686968aa0f6742d93eeb66402af6e222 with result from task 1207 parsl.dataflow.dflow DEBUG: Task 1207 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/526e8cc8-50ed-48e9-bfb9-e467d7cfd690_isrForCrosstalkSources_997033_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/526e8cc8-50ed-48e9-bfb9-e467d7cfd690_isrForCrosstalkSources_997033_57.stderr parsl.dataflow.dflow INFO: Task 1245 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f336b059ad1025a7d8770a2f5c01b415 with result from task 1245 parsl.dataflow.dflow DEBUG: Task 1245 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/575b4c42-a842-4953-b74d-213ba3155460_isrForCrosstalkSources_997026_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/575b4c42-a842-4953-b74d-213ba3155460_isrForCrosstalkSources_997026_24.stderr parsl.dataflow.dflow INFO: Task 1198 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 347a91337ee09da38ef0c643f5dc3689 with result from task 1198 parsl.dataflow.dflow DEBUG: Task 1198 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/30431b6a-f8fc-444f-bab9-e41c5df87b42_isrForCrosstalkSources_997033_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/30431b6a-f8fc-444f-bab9-e41c5df87b42_isrForCrosstalkSources_997033_62.stderr parsl.dataflow.dflow INFO: Task 997 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78822c7ea41c84297afb80c73b5825c4 with result from task 997 parsl.dataflow.dflow DEBUG: Task 997 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/0cae0463-6a35-443d-a154-7e1bf55ff6be_isrForCrosstalkSources_997027_50.stdout parsl.dataflow.dflow INFO: Standard error for task 997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/0cae0463-6a35-443d-a154-7e1bf55ff6be_isrForCrosstalkSources_997027_50.stderr parsl.dataflow.dflow INFO: Task 1177 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ebd36a244d7c9a656475383ce719932 with result from task 1177 parsl.dataflow.dflow DEBUG: Task 1177 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0d2aebba-fb20-492d-a8fd-531fc9af53fa_isrForCrosstalkSources_996970_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0d2aebba-fb20-492d-a8fd-531fc9af53fa_isrForCrosstalkSources_996970_60.stderr parsl.dataflow.dflow INFO: Task 1088 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f290a2ada76f136400a680e10e8f36e7 with result from task 1088 parsl.dataflow.dflow DEBUG: Task 1088 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/bf975c85-7b94-40ef-9516-2ee260959340_isrForCrosstalkSources_996990_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/bf975c85-7b94-40ef-9516-2ee260959340_isrForCrosstalkSources_996990_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:10 proc_lsst.multi:146 [INFO] found job 17075054 in provider checkpoint proc_lsst.multi INFO: found job 17075054 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1259 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb41c42283789126a88a5ffd6cf52197 with result from task 1259 parsl.dataflow.dflow DEBUG: Task 1259 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e7785ecd-3395-4285-877f-1070d9818af5_isrForCrosstalkSources_997002_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e7785ecd-3395-4285-877f-1070d9818af5_isrForCrosstalkSources_997002_17.stderr parsl.dataflow.dflow INFO: Task 1234 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 071c76547716a3632010097b1243882a with result from task 1234 parsl.dataflow.dflow DEBUG: Task 1234 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b79768bf-1e7b-4f9c-b4be-a582692aa1a2_isrForCrosstalkSources_997007_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b79768bf-1e7b-4f9c-b4be-a582692aa1a2_isrForCrosstalkSources_997007_53.stderr parsl.dataflow.dflow INFO: Task 1328 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a3403f84f56bb93aab7918aa7d9b6ee2 with result from task 1328 parsl.dataflow.dflow DEBUG: Task 1328 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/83fe8707-adf6-4c88-9c4b-7e698502bca8_isrForCrosstalkSources_997006_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/83fe8707-adf6-4c88-9c4b-7e698502bca8_isrForCrosstalkSources_997006_14.stderr parsl.dataflow.dflow INFO: Task 1229 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f8f534f961bc04a78cb4daedb45a2ee with result from task 1229 parsl.dataflow.dflow DEBUG: Task 1229 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f6342740-da7d-42f9-8dd5-dd0436aa8692_isrForCrosstalkSources_996953_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f6342740-da7d-42f9-8dd5-dd0436aa8692_isrForCrosstalkSources_996953_13.stderr parsl.dataflow.dflow INFO: Task 1306 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b6c6f82970b747e980336595f1a445c with result from task 1306 parsl.dataflow.dflow DEBUG: Task 1306 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/242fde74-344c-45e7-bfb9-3ea2a8c07141_isrForCrosstalkSources_996960_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/242fde74-344c-45e7-bfb9-3ea2a8c07141_isrForCrosstalkSources_996960_45.stderr parsl.dataflow.dflow INFO: Task 1219 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 981d3563ae56585f9cb5b02b0be613bd with result from task 1219 parsl.dataflow.dflow DEBUG: Task 1219 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/cda0079f-9047-4a2b-bdd2-a9aa621d4cc6_isrForCrosstalkSources_997007_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/cda0079f-9047-4a2b-bdd2-a9aa621d4cc6_isrForCrosstalkSources_997007_12.stderr parsl.dataflow.dflow INFO: Task 1109 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8ddec831732ea334a628f1aa32be2c3 with result from task 1109 parsl.dataflow.dflow DEBUG: Task 1109 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f4d14f27-3471-4c0b-ac04-7dbb7f74bae9_isrForCrosstalkSources_996977_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f4d14f27-3471-4c0b-ac04-7dbb7f74bae9_isrForCrosstalkSources_996977_43.stderr parsl.dataflow.dflow INFO: Task 1192 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9820298e7a5a858cdc038d95fae81940 with result from task 1192 parsl.dataflow.dflow DEBUG: Task 1192 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/11d83f7b-b722-4734-b4e4-a2712138e405_isrForCrosstalkSources_997013_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/11d83f7b-b722-4734-b4e4-a2712138e405_isrForCrosstalkSources_997013_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:11 proc_lsst.multi:146 [INFO] found job 17075055 in provider checkpoint proc_lsst.multi INFO: found job 17075055 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1216 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb288db349c3ebe8ba7008cac42853e3 with result from task 1216 parsl.dataflow.dflow DEBUG: Task 1216 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/d99f32b3-4edf-4558-948b-a1517925f342_isrForCrosstalkSources_996953_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/d99f32b3-4edf-4558-948b-a1517925f342_isrForCrosstalkSources_996953_10.stderr parsl.dataflow.dflow INFO: Task 1148 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b1284ffc892843ec941b99ad24875f7 with result from task 1148 parsl.dataflow.dflow DEBUG: Task 1148 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/822ab246-c73c-4a1c-af65-13c03a284b89_isrForCrosstalkSources_997032_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/822ab246-c73c-4a1c-af65-13c03a284b89_isrForCrosstalkSources_997032_44.stderr parsl.dataflow.dflow INFO: Task 1262 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f91c1584c65778a2d1d6c61907e2865 with result from task 1262 parsl.dataflow.dflow DEBUG: Task 1262 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/22fa8b9f-a47c-47bb-aaf2-281abd50617e_isrForCrosstalkSources_996991_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/22fa8b9f-a47c-47bb-aaf2-281abd50617e_isrForCrosstalkSources_996991_3.stderr parsl.dataflow.dflow INFO: Task 1174 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a9d72c517ffcab5ca3af86b1594768fd with result from task 1174 parsl.dataflow.dflow DEBUG: Task 1174 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/21fcb382-9b78-4d6f-b456-a0e17531b1cb_isrForCrosstalkSources_997028_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/21fcb382-9b78-4d6f-b456-a0e17531b1cb_isrForCrosstalkSources_997028_58.stderr parsl.dataflow.dflow INFO: Task 1340 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c5d85a3203a8710d216885a568564ea5 with result from task 1340 parsl.dataflow.dflow DEBUG: Task 1340 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e551ecd5-d027-4155-8307-d3180ac31696_isrForCrosstalkSources_997020_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e551ecd5-d027-4155-8307-d3180ac31696_isrForCrosstalkSources_997020_6.stderr parsl.dataflow.dflow INFO: Task 1195 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae33bda43c60ddda022ed4e812f0ea56 with result from task 1195 parsl.dataflow.dflow DEBUG: Task 1195 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/8e043630-0666-4bc6-89c3-1b9f92b1bf65_isrForCrosstalkSources_997004_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/8e043630-0666-4bc6-89c3-1b9f92b1bf65_isrForCrosstalkSources_997004_61.stderr parsl.dataflow.dflow INFO: Task 1309 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9b3d5f1b3ba738171a398b0acf8297b with result from task 1309 parsl.dataflow.dflow DEBUG: Task 1309 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/de3fd8a7-6533-4454-a5d4-78926fa77dc0_isrForCrosstalkSources_996990_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/de3fd8a7-6533-4454-a5d4-78926fa77dc0_isrForCrosstalkSources_996990_44.stderr parsl.dataflow.dflow INFO: Task 1260 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8e7f34b82741e6f28b26a2719a1ff8b with result from task 1260 parsl.dataflow.dflow DEBUG: Task 1260 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c84f3837-7d7b-4f5b-8751-774a697b2454_isrForCrosstalkSources_996966_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c84f3837-7d7b-4f5b-8751-774a697b2454_isrForCrosstalkSources_996966_24.stderr parsl.dataflow.dflow INFO: Task 1246 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 037a20aba751f1ff7bebce0f7663ce38 with result from task 1246 parsl.dataflow.dflow DEBUG: Task 1246 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8574ae50-2678-41e9-b97e-9d90c0f28fac_isrForCrosstalkSources_996971_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8574ae50-2678-41e9-b97e-9d90c0f28fac_isrForCrosstalkSources_996971_28.stderr parsl.dataflow.dflow INFO: Task 1290 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 751f065466220a75f3e5815e9b1c0f8c with result from task 1290 parsl.dataflow.dflow DEBUG: Task 1290 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/49742881-bcff-41e0-adb5-6c25dd439105_isrForCrosstalkSources_997015_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/49742881-bcff-41e0-adb5-6c25dd439105_isrForCrosstalkSources_997015_30.stderr parsl.dataflow.dflow INFO: Task 1365 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ef1e1cd9a86ba180f40e6560e2bbc37 with result from task 1365 parsl.dataflow.dflow DEBUG: Task 1365 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/d185549d-7f51-4bbe-8a44-7f8028eac3e3_isrForCrosstalkSources_996952_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/d185549d-7f51-4bbe-8a44-7f8028eac3e3_isrForCrosstalkSources_996952_29.stderr parsl.dataflow.dflow INFO: Task 1284 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b11c5e2de031bb08cc0f7781a9e82e58 with result from task 1284 parsl.dataflow.dflow DEBUG: Task 1284 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/687cea21-e716-4f86-b411-275f9391a3b9_isrForCrosstalkSources_997018_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/687cea21-e716-4f86-b411-275f9391a3b9_isrForCrosstalkSources_997018_28.stderr parsl.dataflow.dflow INFO: Task 1257 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90a7f2277d3a849c8d043eb75dc4feec with result from task 1257 parsl.dataflow.dflow DEBUG: Task 1257 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/89917c3f-f4e7-41b7-976b-fd87fb551174_isrForCrosstalkSources_996996_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/89917c3f-f4e7-41b7-976b-fd87fb551174_isrForCrosstalkSources_996996_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1134 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry d83a3a75971287efe2341499ae4a1342 with result from task 1134 parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1134 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/805db196-a535-4b99-8887-2267bd0f3f55_isrForCrosstalkSources_996991_62.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/805db196-a535-4b99-8887-2267bd0f3f55_isrForCrosstalkSources_996991_62.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:12 proc_lsst.multi:146 [INFO] found job 17075056 in provider checkpoint proc_lsst.multi INFO: found job 17075056 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1211 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 348b76d4717039d910006c578c02a911 with result from task 1211 parsl.dataflow.dflow DEBUG: Task 1211 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/aca5642c-6c2a-4791-8dd4-0f737f8c6162_isrForCrosstalkSources_997000_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/aca5642c-6c2a-4791-8dd4-0f737f8c6162_isrForCrosstalkSources_997000_16.stderr parsl.dataflow.dflow INFO: Task 1269 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d0c336531256d58c9f071ee0d7211bed with result from task 1269 parsl.dataflow.dflow DEBUG: Task 1269 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4a7b1532-05f8-42f4-a8b0-4805bd3407f1_isrForCrosstalkSources_996966_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4a7b1532-05f8-42f4-a8b0-4805bd3407f1_isrForCrosstalkSources_996966_31.stderr parsl.dataflow.dflow INFO: Task 1276 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c98d59ac788f858ad94537fd88a7a80f with result from task 1276 parsl.dataflow.dflow DEBUG: Task 1276 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0425416a-d00d-4d78-83f6-2fc4d1d8b413_isrForCrosstalkSources_997020_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0425416a-d00d-4d78-83f6-2fc4d1d8b413_isrForCrosstalkSources_997020_58.stderr parsl.dataflow.dflow INFO: Task 1362 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c75b2c47f72068798aed7359d589f9b4 with result from task 1362 parsl.dataflow.dflow DEBUG: Task 1362 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d0d40a2-cb5d-4423-8809-5d90e42880d5_isrForCrosstalkSources_996995_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d0d40a2-cb5d-4423-8809-5d90e42880d5_isrForCrosstalkSources_996995_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:13 proc_lsst.multi:146 [INFO] found job 17075057 in provider checkpoint proc_lsst.multi INFO: found job 17075057 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1254 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5cd12e8411f7586efb805aa055a0ca65 with result from task 1254 parsl.dataflow.dflow DEBUG: Task 1254 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/2d04903f-10eb-4163-93e4-238dc05c9428_isrForCrosstalkSources_996997_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/2d04903f-10eb-4163-93e4-238dc05c9428_isrForCrosstalkSources_996997_23.stderr parsl.dataflow.dflow INFO: Task 1388 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c0ef4d3e6866adf9fb5d014583c8ea34 with result from task 1388 parsl.dataflow.dflow DEBUG: Task 1388 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b9a446b6-4f02-41cd-ae12-2d7873c01406_isrForCrosstalkSources_996957_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b9a446b6-4f02-41cd-ae12-2d7873c01406_isrForCrosstalkSources_996957_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1282 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry e03874dba5e7a66d1ee500ec2d5a51c3 with result from task 1282 parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1282 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/afffa461-e355-42d8-aa84-5e29e97b090a_isrForCrosstalkSources_997014_9.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/afffa461-e355-42d8-aa84-5e29e97b090a_isrForCrosstalkSources_997014_9.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1265 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 109c49211d76cb970c9b807a18d33b18 with result from task 1265 parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1265 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4fce2dce-b4ec-4d58-967a-11306098b4ba_isrForCrosstalkSources_997015_1.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4fce2dce-b4ec-4d58-967a-11306098b4ba_isrForCrosstalkSources_997015_1.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:14 proc_lsst.multi:146 [INFO] found job 17075058 in provider checkpoint proc_lsst.multi INFO: found job 17075058 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1115 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2f257b5be253faf287ba73f337eca3c with result from task 1115 parsl.dataflow.dflow DEBUG: Task 1115 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/cc342ad8-55ee-4d3e-a078-b7c6d33c319f_isrForCrosstalkSources_996980_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/cc342ad8-55ee-4d3e-a078-b7c6d33c319f_isrForCrosstalkSources_996980_58.stderr parsl.dataflow.dflow INFO: Task 1363 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a26ccb198654ec7981c9f19d2353ff3 with result from task 1363 parsl.dataflow.dflow DEBUG: Task 1363 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/77d4c0e6-c86b-4112-b64c-26258e0a1952_isrForCrosstalkSources_996961_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/77d4c0e6-c86b-4112-b64c-26258e0a1952_isrForCrosstalkSources_996961_40.stderr parsl.dataflow.dflow INFO: Task 1145 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 194ef014250c74b3d8d53fad882a6bc5 with result from task 1145 parsl.dataflow.dflow DEBUG: Task 1145 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/2e92b2bd-5c4b-43d1-8e17-c4521691fde8_isrForCrosstalkSources_996985_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/2e92b2bd-5c4b-43d1-8e17-c4521691fde8_isrForCrosstalkSources_996985_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:14 proc_lsst.multi:146 [INFO] found job 17075059 in provider checkpoint proc_lsst.multi INFO: found job 17075059 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1253 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62ebbbfd596d1751756baa02fa37f4e2 with result from task 1253 parsl.dataflow.dflow DEBUG: Task 1253 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/931f9440-cfdc-41d2-9ebb-188be10d9093_isrForCrosstalkSources_996965_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/931f9440-cfdc-41d2-9ebb-188be10d9093_isrForCrosstalkSources_996965_30.stderr parsl.dataflow.dflow INFO: Task 1237 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4852461472deee8d7872cfe875feff4 with result from task 1237 parsl.dataflow.dflow DEBUG: Task 1237 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4714a952-867b-49ec-b322-924de3a3cf4f_isrForCrosstalkSources_996962_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4714a952-867b-49ec-b322-924de3a3cf4f_isrForCrosstalkSources_996962_31.stderr parsl.dataflow.dflow INFO: Task 1397 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba206af9550d5c1b6e6a5ee0a4740885 with result from task 1397 parsl.dataflow.dflow DEBUG: Task 1397 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/9698b4f4-5cf7-4f47-a439-f993aa9b2885_isrForCrosstalkSources_996999_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/9698b4f4-5cf7-4f47-a439-f993aa9b2885_isrForCrosstalkSources_996999_35.stderr parsl.dataflow.dflow INFO: Task 1241 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ceccb1974555595857e144b69962f2e8 with result from task 1241 parsl.dataflow.dflow DEBUG: Task 1241 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d502e38d-5503-45bd-a35a-0bb35d6af0cd_isrForCrosstalkSources_996995_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d502e38d-5503-45bd-a35a-0bb35d6af0cd_isrForCrosstalkSources_996995_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:15 proc_lsst.multi:146 [INFO] found job 17075060 in provider checkpoint proc_lsst.multi INFO: found job 17075060 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1379 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e29b580380db42961e9408ff21da99e6 with result from task 1379 parsl.dataflow.dflow DEBUG: Task 1379 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0cb75091-38b3-40f5-89d6-d995dc091ab7_isrForCrosstalkSources_996973_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0cb75091-38b3-40f5-89d6-d995dc091ab7_isrForCrosstalkSources_996973_19.stderr parsl.dataflow.dflow INFO: Task 1240 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d20f200078e227fbd94adfd9aba67e4 with result from task 1240 parsl.dataflow.dflow DEBUG: Task 1240 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/3887aac0-b62f-400b-a0bf-0c25a2a81e78_isrForCrosstalkSources_997000_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/3887aac0-b62f-400b-a0bf-0c25a2a81e78_isrForCrosstalkSources_997000_19.stderr parsl.dataflow.dflow INFO: Task 1292 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9eccffda882360b9af0ea0217b1f3c0 with result from task 1292 parsl.dataflow.dflow DEBUG: Task 1292 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/ca852a3d-401f-4c61-a53b-f3364061bcf4_isrForCrosstalkSources_997018_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/ca852a3d-401f-4c61-a53b-f3364061bcf4_isrForCrosstalkSources_997018_29.stderr parsl.dataflow.dflow INFO: Task 1294 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c9f9b4e0c2b8f84d3768e9a0496e8c1 with result from task 1294 parsl.dataflow.dflow DEBUG: Task 1294 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0c807557-ed83-4b82-a960-78278892c434_isrForCrosstalkSources_997007_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0c807557-ed83-4b82-a960-78278892c434_isrForCrosstalkSources_997007_40.stderr parsl.dataflow.dflow INFO: Task 1364 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d9e93c662e3c8374f591190480c4527 with result from task 1364 parsl.dataflow.dflow DEBUG: Task 1364 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3f3fd5fb-e653-4c86-9c11-7ba1236f3dc6_isrForCrosstalkSources_997033_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3f3fd5fb-e653-4c86-9c11-7ba1236f3dc6_isrForCrosstalkSources_997033_24.stderr parsl.dataflow.dflow INFO: Task 1371 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08cbd5e75c9fbdc004606b697852eeb4 with result from task 1371 parsl.dataflow.dflow DEBUG: Task 1371 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/9264a22f-5fc3-41b7-9955-5b490d8f945b_isrForCrosstalkSources_996966_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/9264a22f-5fc3-41b7-9955-5b490d8f945b_isrForCrosstalkSources_996966_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:16 proc_lsst.multi:146 [INFO] found job 17075061 in provider checkpoint proc_lsst.multi INFO: found job 17075061 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1325 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a53850b96bc77a6f42686c33c34cefe with result from task 1325 parsl.dataflow.dflow DEBUG: Task 1325 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3d4e266d-2fd4-4df3-8798-8d8aa46eb147_isrForCrosstalkSources_996969_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3d4e266d-2fd4-4df3-8798-8d8aa46eb147_isrForCrosstalkSources_996969_13.stderr parsl.dataflow.dflow INFO: Task 1231 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1dca868625bc8a490086149556b974f with result from task 1231 parsl.dataflow.dflow DEBUG: Task 1231 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/6a03c5af-d51e-448d-bfc2-a21331087a1f_isrForCrosstalkSources_996967_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/6a03c5af-d51e-448d-bfc2-a21331087a1f_isrForCrosstalkSources_996967_43.stderr parsl.dataflow.dflow INFO: Task 1183 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54d07f18c5a9d36884b867d3f3841c50 with result from task 1183 parsl.dataflow.dflow DEBUG: Task 1183 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/48db3dfa-8f63-475c-94ac-72777aeae281_isrForCrosstalkSources_996998_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/48db3dfa-8f63-475c-94ac-72777aeae281_isrForCrosstalkSources_996998_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:16 proc_lsst.multi:146 [INFO] found job 17075062 in provider checkpoint proc_lsst.multi INFO: found job 17075062 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1215 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f46da0aad1059b87e0526330666954e with result from task 1215 parsl.dataflow.dflow DEBUG: Task 1215 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fe080c47-3a4d-469b-b9b8-12c6f5d76912_isrForCrosstalkSources_996961_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fe080c47-3a4d-469b-b9b8-12c6f5d76912_isrForCrosstalkSources_996961_48.stderr parsl.dataflow.dflow INFO: Task 1382 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 05714e02edf8a5372294b08fcdbb428d with result from task 1382 parsl.dataflow.dflow DEBUG: Task 1382 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/a90c5157-90d6-42bc-9763-3bac58828c75_isrForCrosstalkSources_997031_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/a90c5157-90d6-42bc-9763-3bac58828c75_isrForCrosstalkSources_997031_4.stderr parsl.dataflow.dflow INFO: Task 1249 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7d6222174a7435a2515a8f1b73a2817 with result from task 1249 parsl.dataflow.dflow DEBUG: Task 1249 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/506f3c55-a429-42f8-bc81-3115e242075a_isrForCrosstalkSources_997017_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/506f3c55-a429-42f8-bc81-3115e242075a_isrForCrosstalkSources_997017_40.stderr parsl.dataflow.dflow INFO: Task 1308 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b431235813293c94eed1bbb92f4a10f2 with result from task 1308 parsl.dataflow.dflow DEBUG: Task 1308 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/064ac10b-5c3e-49f4-802f-d914c06c50ef_isrForCrosstalkSources_996994_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/064ac10b-5c3e-49f4-802f-d914c06c50ef_isrForCrosstalkSources_996994_24.stderr parsl.dataflow.dflow INFO: Task 1392 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15d3e90e844c1f84eb55ee74df727cc2 with result from task 1392 parsl.dataflow.dflow DEBUG: Task 1392 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/8f56e55d-d91b-4dfc-a9fb-74f5dc64b2be_isrForCrosstalkSources_996986_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/8f56e55d-d91b-4dfc-a9fb-74f5dc64b2be_isrForCrosstalkSources_996986_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:17 proc_lsst.multi:146 [INFO] found job 17075063 in provider checkpoint proc_lsst.multi INFO: found job 17075063 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1243 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0fee61f5fcfce96841c6d2c3b0bdbf7 with result from task 1243 parsl.dataflow.dflow DEBUG: Task 1243 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/82c26b7c-be60-4471-b912-cd52c1917b5f_isrForCrosstalkSources_996979_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/82c26b7c-be60-4471-b912-cd52c1917b5f_isrForCrosstalkSources_996979_11.stderr parsl.dataflow.dflow INFO: Task 1263 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10d5c075d7ef178dca61dc98c7eb5851 with result from task 1263 parsl.dataflow.dflow DEBUG: Task 1263 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/359315b4-cf2b-4bc5-926f-d3bde6c4413b_isrForCrosstalkSources_997010_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/359315b4-cf2b-4bc5-926f-d3bde6c4413b_isrForCrosstalkSources_997010_61.stderr parsl.dataflow.dflow INFO: Task 1273 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17517d9c9bad4b1a06d6c72aeb2934ed with result from task 1273 parsl.dataflow.dflow DEBUG: Task 1273 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2392d6f-b6f4-4484-bc9a-b8f8c2ea7b69_isrForCrosstalkSources_996996_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2392d6f-b6f4-4484-bc9a-b8f8c2ea7b69_isrForCrosstalkSources_996996_37.stderr parsl.dataflow.dflow INFO: Task 1373 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ae2561f177f5b68bb7ac8d7106286f4 with result from task 1373 parsl.dataflow.dflow DEBUG: Task 1373 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/18687f19-681a-4df7-91dc-6a9f1477b284_isrForCrosstalkSources_996992_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/18687f19-681a-4df7-91dc-6a9f1477b284_isrForCrosstalkSources_996992_11.stderr parsl.dataflow.dflow INFO: Task 1393 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12b7f6b0479ed7fda5cd23858723f535 with result from task 1393 parsl.dataflow.dflow DEBUG: Task 1393 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/f7bc368e-6763-427e-af8a-7974b6938e9c_isrForCrosstalkSources_996991_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/f7bc368e-6763-427e-af8a-7974b6938e9c_isrForCrosstalkSources_996991_32.stderr parsl.dataflow.dflow INFO: Task 1327 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 20268221b7f01d0943afa3c2501dc2e0 with result from task 1327 parsl.dataflow.dflow DEBUG: Task 1327 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/28c8cf10-919f-42f0-a6cb-ed6a39fe897d_isrForCrosstalkSources_997026_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/28c8cf10-919f-42f0-a6cb-ed6a39fe897d_isrForCrosstalkSources_997026_54.stderr parsl.dataflow.dflow INFO: Task 1293 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a0b5e2345aeb89fa9cceb0a7dcf12a7 with result from task 1293 parsl.dataflow.dflow DEBUG: Task 1293 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/24fd5a22-e006-4f4e-b4f8-8cfe3cd4a5c6_isrForCrosstalkSources_997026_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/24fd5a22-e006-4f4e-b4f8-8cfe3cd4a5c6_isrForCrosstalkSources_997026_29.stderr parsl.dataflow.dflow INFO: Task 1252 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6237539a0ce8b4de230d0106578409c with result from task 1252 parsl.dataflow.dflow DEBUG: Task 1252 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/df909d65-8ed8-431e-968b-e9c7fb520613_isrForCrosstalkSources_996962_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/df909d65-8ed8-431e-968b-e9c7fb520613_isrForCrosstalkSources_996962_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:18 proc_lsst.multi:146 [INFO] found job 17075064 in provider checkpoint proc_lsst.multi INFO: found job 17075064 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1391 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf47b680aa11f715f8f370b75ade4fb4 with result from task 1391 parsl.dataflow.dflow DEBUG: Task 1391 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/92384390-7534-4530-ba28-468aa034790f_isrForCrosstalkSources_997029_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/92384390-7534-4530-ba28-468aa034790f_isrForCrosstalkSources_997029_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:18 proc_lsst.multi:146 [INFO] found job 17075065 in provider checkpoint proc_lsst.multi INFO: found job 17075065 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1396 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4890712cdd87eec1965fe1e59aec1bbd with result from task 1396 parsl.dataflow.dflow DEBUG: Task 1396 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6a0d19b4-d024-44a1-acab-50d52ca77dbf_isrForCrosstalkSources_997015_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6a0d19b4-d024-44a1-acab-50d52ca77dbf_isrForCrosstalkSources_997015_25.stderr parsl.dataflow.dflow INFO: Task 1398 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5095625fd8e4a54d07d2d99df62dcc30 with result from task 1398 parsl.dataflow.dflow DEBUG: Task 1398 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/c8159115-b233-40d9-a17d-cce4a6ca9bed_isrForCrosstalkSources_996953_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/c8159115-b233-40d9-a17d-cce4a6ca9bed_isrForCrosstalkSources_996953_61.stderr parsl.dataflow.dflow INFO: Task 1334 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f7bd6ec006cf981282988fb57bfcac1 with result from task 1334 parsl.dataflow.dflow DEBUG: Task 1334 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/240b246c-e75c-45ec-ada5-ebfa7cc7f1d1_isrForCrosstalkSources_997007_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/240b246c-e75c-45ec-ada5-ebfa7cc7f1d1_isrForCrosstalkSources_997007_49.stderr parsl.dataflow.dflow INFO: Task 1335 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0116391dea305913587f4eba65ee22aa with result from task 1335 parsl.dataflow.dflow DEBUG: Task 1335 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d796c913-4885-4562-9910-6643ed86f5f6_isrForCrosstalkSources_996973_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d796c913-4885-4562-9910-6643ed86f5f6_isrForCrosstalkSources_996973_32.stderr parsl.dataflow.dflow INFO: Task 1250 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e895ca3e28aef9ae312f9f465c0a4dcd with result from task 1250 parsl.dataflow.dflow DEBUG: Task 1250 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/85150cea-c614-4bcd-8b25-a3e7719340ab_isrForCrosstalkSources_997017_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/85150cea-c614-4bcd-8b25-a3e7719340ab_isrForCrosstalkSources_997017_14.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:19 proc_lsst.multi:146 [INFO] found job 17075066 in provider checkpoint proc_lsst.multi INFO: found job 17075066 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1324 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a502b070bd0ec8dc55390369e89d7b84 with result from task 1324 parsl.dataflow.dflow DEBUG: Task 1324 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/32321300-18b0-4f2a-bde5-2187c9642783_isrForCrosstalkSources_996976_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/32321300-18b0-4f2a-bde5-2187c9642783_isrForCrosstalkSources_996976_32.stderr parsl.dataflow.dflow INFO: Task 1384 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec062341699cd6566e7ca53024d67146 with result from task 1384 parsl.dataflow.dflow DEBUG: Task 1384 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f487a93c-c73e-473d-a3a2-ab6bf0286866_isrForCrosstalkSources_997017_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f487a93c-c73e-473d-a3a2-ab6bf0286866_isrForCrosstalkSources_997017_6.stderr parsl.dataflow.dflow INFO: Task 1238 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 980f821c69b9480600ad634ae2fdaf07 with result from task 1238 parsl.dataflow.dflow DEBUG: Task 1238 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9e4b4475-fdb1-4d9f-9993-30353bb7b1a9_isrForCrosstalkSources_996986_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9e4b4475-fdb1-4d9f-9993-30353bb7b1a9_isrForCrosstalkSources_996986_49.stderr parsl.dataflow.dflow INFO: Task 1419 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5921bd145e0d21c559a24d914fc5bc82 with result from task 1419 parsl.dataflow.dflow DEBUG: Task 1419 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/db235a98-d7f9-4ba1-9828-aaed65b04635_isrForCrosstalkSources_997004_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/db235a98-d7f9-4ba1-9828-aaed65b04635_isrForCrosstalkSources_997004_20.stderr parsl.dataflow.dflow INFO: Task 1345 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be510ddc1ac34f485df7130bd00d2683 with result from task 1345 parsl.dataflow.dflow DEBUG: Task 1345 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/b814ea2a-52c4-44e4-b0b3-6d4125b87a54_isrForCrosstalkSources_996988_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/b814ea2a-52c4-44e4-b0b3-6d4125b87a54_isrForCrosstalkSources_996988_8.stderr parsl.dataflow.dflow INFO: Task 1256 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51e0d5c60a9677acb8e45b62f5f3b655 with result from task 1256 parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:20 proc_lsst.multi:146 [INFO] found job 17075068 in provider checkpoint proc_lsst.multi INFO: found job 17075068 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 1256 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/93508c79-65d6-41cb-b9d7-3711956710d1_isrForCrosstalkSources_996984_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/93508c79-65d6-41cb-b9d7-3711956710d1_isrForCrosstalkSources_996984_20.stderr parsl.dataflow.dflow INFO: Task 1390 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a687eb077449b49a69ef1723389c352 with result from task 1390 parsl.dataflow.dflow DEBUG: Task 1390 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/0a04c159-0612-4aef-a459-7339456bcb29_isrForCrosstalkSources_996994_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/0a04c159-0612-4aef-a459-7339456bcb29_isrForCrosstalkSources_996994_12.stderr parsl.dataflow.dflow INFO: Task 1344 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3829378929a8850305497eabc04c957e with result from task 1344 parsl.dataflow.dflow DEBUG: Task 1344 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/3384cda8-9b95-4fd7-9c10-d9419b3c22da_isrForCrosstalkSources_996989_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/3384cda8-9b95-4fd7-9c10-d9419b3c22da_isrForCrosstalkSources_996989_14.stderr parsl.dataflow.dflow INFO: Task 1217 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5d37516740c4f621ccfad93358832d8 with result from task 1217 parsl.dataflow.dflow DEBUG: Task 1217 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/e6ea7906-0e50-44e9-b4d4-17a60bfb76cf_isrForCrosstalkSources_997009_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/e6ea7906-0e50-44e9-b4d4-17a60bfb76cf_isrForCrosstalkSources_997009_20.stderr parsl.dataflow.dflow INFO: Task 1369 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ff1dc4abc946757d52ef7cdee7b5bb6 with result from task 1369 parsl.dataflow.dflow DEBUG: Task 1369 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/40c106e9-00db-4ba9-964e-ef716701c4c0_isrForCrosstalkSources_996956_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/40c106e9-00db-4ba9-964e-ef716701c4c0_isrForCrosstalkSources_996956_39.stderr parsl.dataflow.dflow INFO: Task 1360 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a92d79d560e33ba05a977390de2d07d3 with result from task 1360 parsl.dataflow.dflow DEBUG: Task 1360 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/36874233-be3b-45e9-a5bc-70ee8c2872a4_isrForCrosstalkSources_996954_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/36874233-be3b-45e9-a5bc-70ee8c2872a4_isrForCrosstalkSources_996954_34.stderr parsl.dataflow.dflow INFO: Task 1320 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8f6e0446ca3d202b07369a419abe88a with result from task 1320 parsl.dataflow.dflow DEBUG: Task 1320 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/89227b92-74bc-42be-b188-544195de0ab9_isrForCrosstalkSources_997001_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/89227b92-74bc-42be-b188-544195de0ab9_isrForCrosstalkSources_997001_51.stderr parsl.dataflow.dflow INFO: Task 1413 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed7c587bbd45e48804c639b2233e57e2 with result from task 1413 parsl.dataflow.dflow DEBUG: Task 1413 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8f90ac0d-b200-4428-a52a-84f4f26401f6_isrForCrosstalkSources_997017_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8f90ac0d-b200-4428-a52a-84f4f26401f6_isrForCrosstalkSources_997017_16.stderr parsl.dataflow.dflow INFO: Task 1331 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f695125201d8768c168fbb2e0071fd91 with result from task 1331 parsl.dataflow.dflow DEBUG: Task 1331 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5b127dcf-8c94-47a5-a746-4767b03e63ed_isrForCrosstalkSources_996987_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5b127dcf-8c94-47a5-a746-4767b03e63ed_isrForCrosstalkSources_996987_14.stderr parsl.dataflow.dflow INFO: Task 1242 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85fb539be53940b15283474b5dd93e9c with result from task 1242 parsl.dataflow.dflow DEBUG: Task 1242 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/51384153-cd47-4990-802b-03982355d26d_isrForCrosstalkSources_996983_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/51384153-cd47-4990-802b-03982355d26d_isrForCrosstalkSources_996983_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:21 proc_lsst.multi:146 [INFO] found job 17075069 in provider checkpoint proc_lsst.multi INFO: found job 17075069 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1286 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a384d299fd0e9537c4579f6969ba7fc6 with result from task 1286 parsl.dataflow.dflow DEBUG: Task 1286 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/79c87c47-242f-4cfc-b90c-5766dc31b80c_isrForCrosstalkSources_996997_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/79c87c47-242f-4cfc-b90c-5766dc31b80c_isrForCrosstalkSources_996997_45.stderr parsl.dataflow.dflow INFO: Task 1288 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd479a2d4865e77794541468da42c762 with result from task 1288 parsl.dataflow.dflow DEBUG: Task 1288 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/31dbc9b7-d3b7-46f4-b6f4-2a29de68e867_isrForCrosstalkSources_997017_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/31dbc9b7-d3b7-46f4-b6f4-2a29de68e867_isrForCrosstalkSources_997017_25.stderr parsl.dataflow.dflow INFO: Task 1285 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 40fcb8eb47158b61aae7a39c4b3248a7 with result from task 1285 parsl.dataflow.dflow DEBUG: Task 1285 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/137fcb53-4790-45b9-81d3-6554b3bd723f_isrForCrosstalkSources_996967_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/137fcb53-4790-45b9-81d3-6554b3bd723f_isrForCrosstalkSources_996967_34.stderr parsl.dataflow.dflow INFO: Task 1218 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 220cdc5834e335c0c8a8ef7a9eb85895 with result from task 1218 parsl.dataflow.dflow DEBUG: Task 1218 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d31ccfcc-731d-47d2-8838-d9f24c695f8d_isrForCrosstalkSources_996991_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d31ccfcc-731d-47d2-8838-d9f24c695f8d_isrForCrosstalkSources_996991_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:21 proc_lsst.multi:146 [INFO] found job 17075072 in provider checkpoint proc_lsst.multi INFO: found job 17075072 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1329 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4607c7194d52b47910c42ea9c9dd80e with result from task 1329 parsl.dataflow.dflow DEBUG: Task 1329 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a5d4a18f-cb36-4cf0-8d56-b51aeffd397a_isrForCrosstalkSources_996984_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a5d4a18f-cb36-4cf0-8d56-b51aeffd397a_isrForCrosstalkSources_996984_3.stderr parsl.dataflow.dflow INFO: Task 1295 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1b6f33b508b4785cd80005c76ae321fb with result from task 1295 parsl.dataflow.dflow DEBUG: Task 1295 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/aa3d11b2-cd5c-4413-ad56-fde7bf53ff89_isrForCrosstalkSources_997024_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/aa3d11b2-cd5c-4413-ad56-fde7bf53ff89_isrForCrosstalkSources_997024_40.stderr parsl.dataflow.dflow INFO: Task 1338 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0aa75e9ee2c5065a06fb34d41ac609cc with result from task 1338 parsl.dataflow.dflow DEBUG: Task 1338 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/8e52bdbb-a210-40e4-aeb3-3a4a53871a1c_isrForCrosstalkSources_997022_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/8e52bdbb-a210-40e4-aeb3-3a4a53871a1c_isrForCrosstalkSources_997022_20.stderr parsl.dataflow.dflow INFO: Task 1297 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d2d521131c177697235096891a2a2365 with result from task 1297 parsl.dataflow.dflow DEBUG: Task 1297 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/b0f192b6-af69-45a3-b597-71185d20f310_isrForCrosstalkSources_996955_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/b0f192b6-af69-45a3-b597-71185d20f310_isrForCrosstalkSources_996955_19.stderr parsl.dataflow.dflow INFO: Task 1380 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d0327ddba5697bb129f41c85f3cdb9a with result from task 1380 parsl.dataflow.dflow DEBUG: Task 1380 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d72035be-9bc2-43c5-a687-253b271d840b_isrForCrosstalkSources_997027_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d72035be-9bc2-43c5-a687-253b271d840b_isrForCrosstalkSources_997027_32.stderr parsl.dataflow.dflow INFO: Task 1223 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2575469d36389b520d57a5fec4a83c1e with result from task 1223 parsl.dataflow.dflow DEBUG: Task 1223 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6595d0fb-e243-4208-84d1-6b2958fb0e1b_isrForCrosstalkSources_996982_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6595d0fb-e243-4208-84d1-6b2958fb0e1b_isrForCrosstalkSources_996982_51.stderr parsl.dataflow.dflow INFO: Task 1310 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a41b55e65f4c7bf3f7223ec8ad25a0b8 with result from task 1310 parsl.dataflow.dflow DEBUG: Task 1310 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/1e8e2d34-bfed-4bc2-a5d8-2d8a209c4281_isrForCrosstalkSources_996997_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/1e8e2d34-bfed-4bc2-a5d8-2d8a209c4281_isrForCrosstalkSources_996997_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:22 proc_lsst.multi:146 [INFO] found job 17075073 in provider checkpoint proc_lsst.multi INFO: found job 17075073 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1300 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10d48b5f03092af257c58f2c2cf1eb7c with result from task 1300 parsl.dataflow.dflow DEBUG: Task 1300 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/74887274-c16a-40bc-88f2-c481e9671a5f_isrForCrosstalkSources_997033_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/74887274-c16a-40bc-88f2-c481e9671a5f_isrForCrosstalkSources_997033_39.stderr parsl.dataflow.dflow INFO: Task 1374 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 094bcc2c667272b780cf11e37a68b823 with result from task 1374 parsl.dataflow.dflow DEBUG: Task 1374 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c52e06b8-1fab-46ad-aed7-8628d40aecb1_isrForCrosstalkSources_996985_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c52e06b8-1fab-46ad-aed7-8628d40aecb1_isrForCrosstalkSources_996985_19.stderr parsl.dataflow.dflow INFO: Task 1368 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d13139be0681b09373fdd556107895f8 with result from task 1368 parsl.dataflow.dflow DEBUG: Task 1368 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/11631a4e-c027-4d36-91e5-dff9579cb863_isrForCrosstalkSources_997025_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/11631a4e-c027-4d36-91e5-dff9579cb863_isrForCrosstalkSources_997025_13.stderr parsl.dataflow.dflow INFO: Task 1455 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44f96d545d4b16eb276dcdb471cb6015 with result from task 1455 parsl.dataflow.dflow DEBUG: Task 1455 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/8b75221c-7769-481f-bce6-a1afd33c639f_isrForCrosstalkSources_996989_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/8b75221c-7769-481f-bce6-a1afd33c639f_isrForCrosstalkSources_996989_25.stderr parsl.dataflow.dflow INFO: Task 1412 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 833452019e59752abd5a26971b331114 with result from task 1412 parsl.dataflow.dflow DEBUG: Task 1412 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/659bceb8-d19f-4820-9dde-ef368a426240_isrForCrosstalkSources_997033_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/659bceb8-d19f-4820-9dde-ef368a426240_isrForCrosstalkSources_997033_22.stderr parsl.dataflow.dflow INFO: Task 1367 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51c2b4e3c6cdb2c217426729cba7f093 with result from task 1367 parsl.dataflow.dflow DEBUG: Task 1367 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/32c90c0f-6145-460c-8e5c-b2178b84f1a7_isrForCrosstalkSources_996981_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/32c90c0f-6145-460c-8e5c-b2178b84f1a7_isrForCrosstalkSources_996981_14.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1407 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry a8b9063be2f748564882b5f4edbc2048 with result from task 1407 parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1407 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/19479e1f-749a-45b9-b97e-982eaea4f564_isrForCrosstalkSources_996974_13.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/19479e1f-749a-45b9-b97e-982eaea4f564_isrForCrosstalkSources_996974_13.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:22 proc_lsst.multi:146 [INFO] found job 17075074 in provider checkpoint proc_lsst.multi INFO: found job 17075074 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1274 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02646a2f714ea7ca7b23dda4dc58a399 with result from task 1274 parsl.dataflow.dflow DEBUG: Task 1274 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/faec719c-cc80-4d22-99ac-e511c26b367a_isrForCrosstalkSources_997000_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/faec719c-cc80-4d22-99ac-e511c26b367a_isrForCrosstalkSources_997000_23.stderr parsl.dataflow.dflow INFO: Task 1247 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 516f5f35002cc583051ac26804880ab1 with result from task 1247 parsl.dataflow.dflow DEBUG: Task 1247 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/e87b722c-8a66-44ad-a5a1-29f20c9af976_isrForCrosstalkSources_996972_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/e87b722c-8a66-44ad-a5a1-29f20c9af976_isrForCrosstalkSources_996972_1.stderr parsl.dataflow.dflow INFO: Task 1395 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7399bbc347d91027cfb988d743aaaa4b with result from task 1395 parsl.dataflow.dflow DEBUG: Task 1395 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/16ec4af1-2f04-4137-a1cf-fc97ffc91923_isrForCrosstalkSources_996960_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/16ec4af1-2f04-4137-a1cf-fc97ffc91923_isrForCrosstalkSources_996960_14.stderr parsl.dataflow.dflow INFO: Task 1322 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 123dfa8cd7db9cd556e5e3bc7c6f2742 with result from task 1322 parsl.dataflow.dflow DEBUG: Task 1322 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/1a1466af-dc60-4e78-a917-cb22f85dad0e_isrForCrosstalkSources_996982_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/1a1466af-dc60-4e78-a917-cb22f85dad0e_isrForCrosstalkSources_996982_26.stderr parsl.dataflow.dflow INFO: Task 1376 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32992761400341cbdd4f061cd0b68c6c with result from task 1376 parsl.dataflow.dflow DEBUG: Task 1376 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f6ede271-3479-455f-95aa-7d4468b5771d_isrForCrosstalkSources_996955_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f6ede271-3479-455f-95aa-7d4468b5771d_isrForCrosstalkSources_996955_17.stderr parsl.dataflow.dflow INFO: Task 1437 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9005d7ac52ac1628661863f900a7f766 with result from task 1437 parsl.dataflow.dflow DEBUG: Task 1437 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0bd0ed93-3097-4839-bbe7-f3506647e26e_isrForCrosstalkSources_997025_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0bd0ed93-3097-4839-bbe7-f3506647e26e_isrForCrosstalkSources_997025_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:23 proc_lsst.multi:146 [INFO] found job 17075075 in provider checkpoint proc_lsst.multi INFO: found job 17075075 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1410 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad28f06246dc0fb4905abfcea01bfd09 with result from task 1410 parsl.dataflow.dflow DEBUG: Task 1410 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e8f1bf39-b372-4ee9-b232-619e2b46abb8_isrForCrosstalkSources_996985_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e8f1bf39-b372-4ee9-b232-619e2b46abb8_isrForCrosstalkSources_996985_14.stderr parsl.dataflow.dflow INFO: Task 1438 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be027937c1a713ace8b5f166b37ec862 with result from task 1438 parsl.dataflow.dflow DEBUG: Task 1438 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/804181a3-e576-416a-be8c-2fd45bc198d5_isrForCrosstalkSources_997029_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/804181a3-e576-416a-be8c-2fd45bc198d5_isrForCrosstalkSources_997029_4.stderr parsl.dataflow.dflow INFO: Task 1225 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e481c44d9778b2b79314e5196f44177 with result from task 1225 parsl.dataflow.dflow DEBUG: Task 1225 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/9d317245-990f-4eab-bda7-75a8bb0828f9_isrForCrosstalkSources_996996_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/9d317245-990f-4eab-bda7-75a8bb0828f9_isrForCrosstalkSources_996996_36.stderr parsl.dataflow.dflow INFO: Task 1415 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 450e4dcf7c12027164ffdebe479bf1b3 with result from task 1415 parsl.dataflow.dflow DEBUG: Task 1415 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/2de71e17-dc1c-4ee2-888e-34a601be2c4f_isrForCrosstalkSources_996978_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/2de71e17-dc1c-4ee2-888e-34a601be2c4f_isrForCrosstalkSources_996978_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:24 proc_lsst.multi:146 [INFO] found job 17075076 in provider checkpoint proc_lsst.multi INFO: found job 17075076 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1442 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf2c934f9cad433a1dc92feda2eb3efe with result from task 1442 parsl.dataflow.dflow DEBUG: Task 1442 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/8e290043-a8ed-4c9c-b3b5-e568e2f3bf52_isrForCrosstalkSources_996983_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/8e290043-a8ed-4c9c-b3b5-e568e2f3bf52_isrForCrosstalkSources_996983_20.stderr parsl.dataflow.dflow INFO: Task 1305 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c7d686d6e37106e36b93b5d529182f9 with result from task 1305 parsl.dataflow.dflow DEBUG: Task 1305 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/487d083a-e295-45e3-b67f-3e493f2a7155_isrForCrosstalkSources_997028_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/487d083a-e295-45e3-b67f-3e493f2a7155_isrForCrosstalkSources_997028_39.stderr parsl.dataflow.dflow INFO: Task 1304 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50c9f6572e2666edc9d442c0d987a7c9 with result from task 1304 parsl.dataflow.dflow DEBUG: Task 1304 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a3cecc5d-bdc0-4b9f-9847-d4a8251ec350_isrForCrosstalkSources_997033_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a3cecc5d-bdc0-4b9f-9847-d4a8251ec350_isrForCrosstalkSources_997033_42.stderr parsl.dataflow.dflow INFO: Task 1281 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c6297f6f2090bee8c0a03e010cbe7be with result from task 1281 parsl.dataflow.dflow DEBUG: Task 1281 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/94029ffa-dcb1-4d57-a8bd-89112b22b50c_isrForCrosstalkSources_996989_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/94029ffa-dcb1-4d57-a8bd-89112b22b50c_isrForCrosstalkSources_996989_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:24 proc_lsst.multi:146 [INFO] found job 17075078 in provider checkpoint proc_lsst.multi INFO: found job 17075078 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1435 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e5a150098f4b823d2fa7c506315641b with result from task 1435 parsl.dataflow.dflow DEBUG: Task 1435 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/8a9296c4-43c5-487a-b6ca-a96a42646041_isrForCrosstalkSources_997022_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/8a9296c4-43c5-487a-b6ca-a96a42646041_isrForCrosstalkSources_997022_34.stderr parsl.dataflow.dflow INFO: Task 1358 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5958d7105d93f77fcd8c37722810bfa0 with result from task 1358 parsl.dataflow.dflow DEBUG: Task 1358 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/80716ad3-987e-4b76-9c34-441fb48fc620_isrForCrosstalkSources_997019_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/80716ad3-987e-4b76-9c34-441fb48fc620_isrForCrosstalkSources_997019_59.stderr parsl.dataflow.dflow INFO: Task 1321 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df74be6534bac2da1daa1778aa209e13 with result from task 1321 parsl.dataflow.dflow DEBUG: Task 1321 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/5efb51f4-0f75-4bdb-a07e-9ac5afd841e1_isrForCrosstalkSources_996970_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/5efb51f4-0f75-4bdb-a07e-9ac5afd841e1_isrForCrosstalkSources_996970_2.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:25 proc_lsst.multi:146 [INFO] found job 17075080 in provider checkpoint proc_lsst.multi INFO: found job 17075080 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1370 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f68f3b45c88026b6d635bb2516997de9 with result from task 1370 parsl.dataflow.dflow DEBUG: Task 1370 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/58c56853-4cac-492f-9716-18c61b1d5890_isrForCrosstalkSources_997025_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/58c56853-4cac-492f-9716-18c61b1d5890_isrForCrosstalkSources_997025_9.stderr parsl.dataflow.dflow INFO: Task 1447 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5517f37334c2ff05784fc0c60334a8e3 with result from task 1447 parsl.dataflow.dflow DEBUG: Task 1447 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/82a2ff66-0393-4d04-be28-e30052f6bc4e_isrForCrosstalkSources_997006_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/82a2ff66-0393-4d04-be28-e30052f6bc4e_isrForCrosstalkSources_997006_3.stderr parsl.dataflow.dflow INFO: Task 1429 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a56643a067061c495a2e92de404993ad with result from task 1429 parsl.dataflow.dflow DEBUG: Task 1429 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e17bc60f-866c-4ba9-910b-c5b39d414685_isrForCrosstalkSources_996963_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e17bc60f-866c-4ba9-910b-c5b39d414685_isrForCrosstalkSources_996963_42.stderr parsl.dataflow.dflow INFO: Task 1287 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a496e3f857efb87b0066d23b6bcc901d with result from task 1287 parsl.dataflow.dflow DEBUG: Task 1287 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5e203a11-4052-4eb8-8962-132b07342a3e_isrForCrosstalkSources_997010_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5e203a11-4052-4eb8-8962-132b07342a3e_isrForCrosstalkSources_997010_21.stderr parsl.dataflow.dflow INFO: Task 1361 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1d08f3cba28a038b452f9b4998c74c1 with result from task 1361 parsl.dataflow.dflow DEBUG: Task 1361 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/84e05f91-ed6a-444e-b85b-a4b5d9aa7f6b_isrForCrosstalkSources_996999_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/84e05f91-ed6a-444e-b85b-a4b5d9aa7f6b_isrForCrosstalkSources_996999_21.stderr parsl.dataflow.dflow INFO: Task 1440 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7f126f589e487844808c2f26d44f316c with result from task 1440 parsl.dataflow.dflow DEBUG: Task 1440 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5e2b28dd-62d1-403b-bf11-822eb635225c_isrForCrosstalkSources_997033_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5e2b28dd-62d1-403b-bf11-822eb635225c_isrForCrosstalkSources_997033_5.stderr parsl.dataflow.dflow INFO: Task 1339 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d6bab3041861616a11c69d130eda30ed with result from task 1339 parsl.dataflow.dflow DEBUG: Task 1339 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/bab4ab96-dd47-4fe0-8c09-d7577102a235_isrForCrosstalkSources_996967_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/bab4ab96-dd47-4fe0-8c09-d7577102a235_isrForCrosstalkSources_996967_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:26 proc_lsst.multi:146 [INFO] found job 17075081 in provider checkpoint proc_lsst.multi INFO: found job 17075081 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1426 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c835ee6a448f2704ad708594cefd1823 with result from task 1426 parsl.dataflow.dflow DEBUG: Task 1426 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/6ac4d563-708e-4b20-8146-81cceadccace_isrForCrosstalkSources_996981_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/6ac4d563-708e-4b20-8146-81cceadccace_isrForCrosstalkSources_996981_1.stderr parsl.dataflow.dflow INFO: Task 1406 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e715d9f7b58bb2b5c22fbf46f6a0f299 with result from task 1406 parsl.dataflow.dflow DEBUG: Task 1406 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/47818732-8fe1-401b-b600-5cf5bb8fa224_isrForCrosstalkSources_997023_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/47818732-8fe1-401b-b600-5cf5bb8fa224_isrForCrosstalkSources_997023_53.stderr parsl.dataflow.dflow INFO: Task 1386 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b133b696bbdcd7d306fc5a487d5bb946 with result from task 1386 parsl.dataflow.dflow DEBUG: Task 1386 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7a665c5b-174e-4acb-a179-3afa6aae4c1b_isrForCrosstalkSources_997029_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7a665c5b-174e-4acb-a179-3afa6aae4c1b_isrForCrosstalkSources_997029_22.stderr parsl.dataflow.dflow INFO: Task 1387 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4221e7f99ea754c01871e1cd590e8974 with result from task 1387 parsl.dataflow.dflow DEBUG: Task 1387 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/fb0797cb-7a01-4f99-8dd8-cad3e9afa593_isrForCrosstalkSources_996989_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/fb0797cb-7a01-4f99-8dd8-cad3e9afa593_isrForCrosstalkSources_996989_4.stderr parsl.dataflow.dflow INFO: Task 1414 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b069eb03f7cb92a70b70717cb7432ad5 with result from task 1414 parsl.dataflow.dflow DEBUG: Task 1414 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/93565768-9a9c-4bb5-99ca-7f2c1c5d459c_isrForCrosstalkSources_996978_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/93565768-9a9c-4bb5-99ca-7f2c1c5d459c_isrForCrosstalkSources_996978_3.stderr parsl.dataflow.dflow INFO: Task 1303 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08615d3e08bd48bc006ae824a322a421 with result from task 1303 parsl.dataflow.dflow DEBUG: Task 1303 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/457fdc4b-2c2f-41d9-8f64-b18efc179194_isrForCrosstalkSources_996969_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/457fdc4b-2c2f-41d9-8f64-b18efc179194_isrForCrosstalkSources_996969_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:27 proc_lsst.multi:146 [INFO] found job 17075082 in provider checkpoint proc_lsst.multi INFO: found job 17075082 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1446 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 49c899e0a7c7b60f6829270dfbd9e210 with result from task 1446 parsl.dataflow.dflow DEBUG: Task 1446 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/877496fa-5f51-42d4-a303-0050b2e70dc8_isrForCrosstalkSources_997031_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/877496fa-5f51-42d4-a303-0050b2e70dc8_isrForCrosstalkSources_997031_59.stderr parsl.dataflow.dflow INFO: Task 1153 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a34a7570bc4fb063937927d8dd79c83d with result from task 1153 parsl.dataflow.dflow DEBUG: Task 1153 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d51ef99a-7877-4d06-9a3c-ced625a0b46d_isrForCrosstalkSources_996954_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d51ef99a-7877-4d06-9a3c-ced625a0b46d_isrForCrosstalkSources_996954_42.stderr parsl.dataflow.dflow INFO: Task 1409 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 98d29eb13687e55450de6ca164908dbe with result from task 1409 parsl.dataflow.dflow DEBUG: Task 1409 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2280f749-20ee-4d16-b7e2-e99784c2ba7e_isrForCrosstalkSources_996963_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2280f749-20ee-4d16-b7e2-e99784c2ba7e_isrForCrosstalkSources_996963_22.stderr parsl.dataflow.dflow INFO: Task 1232 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e238b7d729220a128dd1813ebfd7f9a7 with result from task 1232 parsl.dataflow.dflow DEBUG: Task 1232 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/58527194-663b-4808-adef-d269525b5e53_isrForCrosstalkSources_996994_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/58527194-663b-4808-adef-d269525b5e53_isrForCrosstalkSources_996994_15.stderr parsl.dataflow.dflow INFO: Task 1296 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5bd5296f4aa2eff4dddde08d5198c13 with result from task 1296 parsl.dataflow.dflow DEBUG: Task 1296 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6d16bd49-5365-4534-aab7-2ee0f515f698_isrForCrosstalkSources_996953_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/6d16bd49-5365-4534-aab7-2ee0f515f698_isrForCrosstalkSources_996953_18.stderr parsl.dataflow.dflow INFO: Task 1272 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dc38f632caa8b32529551a887d0e17c2 with result from task 1272 parsl.dataflow.dflow DEBUG: Task 1272 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f1f4d51a-8d5b-425e-a9dd-83b3bee80c86_isrForCrosstalkSources_996961_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f1f4d51a-8d5b-425e-a9dd-83b3bee80c86_isrForCrosstalkSources_996961_35.stderr parsl.dataflow.dflow INFO: Task 1381 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fed2c0d30bf1f9cf41e11c9d10343f9b with result from task 1381 parsl.dataflow.dflow DEBUG: Task 1381 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/0895b0ec-e4f9-49da-b64a-39e524c853a2_isrForCrosstalkSources_997000_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/0895b0ec-e4f9-49da-b64a-39e524c853a2_isrForCrosstalkSources_997000_30.stderr parsl.dataflow.dflow INFO: Task 1301 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48bd0802a891ec51c4818e8f04b96b74 with result from task 1301 parsl.dataflow.dflow DEBUG: Task 1301 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0dde1fdd-dfea-4d66-8c96-94b2d86c352f_isrForCrosstalkSources_996970_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0dde1fdd-dfea-4d66-8c96-94b2d86c352f_isrForCrosstalkSources_996970_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:28 proc_lsst.multi:146 [INFO] found job 17075083 in provider checkpoint proc_lsst.multi INFO: found job 17075083 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1337 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b46486360d98f262d2a3bbd8ba926139 with result from task 1337 parsl.dataflow.dflow DEBUG: Task 1337 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f62efaf6-69c9-470c-9536-c0199ca768b1_isrForCrosstalkSources_997016_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f62efaf6-69c9-470c-9536-c0199ca768b1_isrForCrosstalkSources_997016_43.stderr parsl.dataflow.dflow INFO: Task 1213 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f67dcf0f6d805576e16bf78ca0c23d29 with result from task 1213 parsl.dataflow.dflow DEBUG: Task 1213 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60f7d0e9-7c12-4892-8db6-e840ace9d97b_isrForCrosstalkSources_996968_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60f7d0e9-7c12-4892-8db6-e840ace9d97b_isrForCrosstalkSources_996968_42.stderr parsl.dataflow.dflow INFO: Task 1463 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13bc7bec38b4d858f1977e465f47febc with result from task 1463 parsl.dataflow.dflow DEBUG: Task 1463 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bc7bc3c0-f50d-476c-a7a2-c566c4e16966_isrForCrosstalkSources_997014_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bc7bc3c0-f50d-476c-a7a2-c566c4e16966_isrForCrosstalkSources_997014_62.stderr parsl.dataflow.dflow INFO: Task 1430 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23bf5f8bbeb72596e100e36f29b10fb4 with result from task 1430 parsl.dataflow.dflow DEBUG: Task 1430 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/49579abe-5e87-42d9-a01f-e9edc10f8d2c_isrForCrosstalkSources_997017_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/49579abe-5e87-42d9-a01f-e9edc10f8d2c_isrForCrosstalkSources_997017_48.stderr parsl.dataflow.dflow INFO: Task 1258 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 40b697ae8ebd4275de470bb595304aa8 with result from task 1258 parsl.dataflow.dflow DEBUG: Task 1258 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0bebb7c0-077b-49fa-b7be-40283d8b4f81_isrForCrosstalkSources_996991_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0bebb7c0-077b-49fa-b7be-40283d8b4f81_isrForCrosstalkSources_996991_16.stderr parsl.dataflow.dflow INFO: Task 1453 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd52390ee9077bd463798d3c9102f3f2 with result from task 1453 parsl.dataflow.dflow DEBUG: Task 1453 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/82612bfa-46af-4e40-a5f6-cf89e396b28d_isrForCrosstalkSources_997019_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/82612bfa-46af-4e40-a5f6-cf89e396b28d_isrForCrosstalkSources_997019_47.stderr parsl.dataflow.dflow INFO: Task 1307 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55b3d26dd5ff371320f635c8c8288c5e with result from task 1307 parsl.dataflow.dflow DEBUG: Task 1307 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/3af9d428-2174-485c-af41-a568d422dd9c_isrForCrosstalkSources_997002_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/3af9d428-2174-485c-af41-a568d422dd9c_isrForCrosstalkSources_997002_39.stderr parsl.dataflow.dflow INFO: Task 1357 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9eed0107cd127161be2c038666c6ac2e with result from task 1357 parsl.dataflow.dflow DEBUG: Task 1357 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c4df4a6b-3d01-4e1d-8b06-d8f9379630cf_isrForCrosstalkSources_996995_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c4df4a6b-3d01-4e1d-8b06-d8f9379630cf_isrForCrosstalkSources_996995_3.stderr parsl.dataflow.dflow INFO: Task 1356 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fb51675de9dbb6b8fa153ac813c4797 with result from task 1356 parsl.dataflow.dflow DEBUG: Task 1356 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/83c0172f-49d6-424a-b70f-a4594a080f74_isrForCrosstalkSources_996971_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/83c0172f-49d6-424a-b70f-a4594a080f74_isrForCrosstalkSources_996971_56.stderr parsl.dataflow.dflow INFO: Task 1289 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e3a6cfaf6b9ad6ee6d5709fcc083258 with result from task 1289 parsl.dataflow.dflow DEBUG: Task 1289 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/de58d4eb-b320-4d0c-9990-d6c899c6377d_isrForCrosstalkSources_996969_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/de58d4eb-b320-4d0c-9990-d6c899c6377d_isrForCrosstalkSources_996969_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:29 proc_lsst.multi:146 [INFO] found job 17075085 in provider checkpoint proc_lsst.multi INFO: found job 17075085 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1214 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 738b6df3fd1238e0f0c597119e386589 with result from task 1214 parsl.dataflow.dflow DEBUG: Task 1214 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c1733764-93ee-4793-899d-f1bd0b68e160_isrForCrosstalkSources_997020_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c1733764-93ee-4793-899d-f1bd0b68e160_isrForCrosstalkSources_997020_62.stderr parsl.dataflow.dflow INFO: Task 1459 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 340bfc370f10f2cc14d998c22e182b4b with result from task 1459 parsl.dataflow.dflow DEBUG: Task 1459 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/07b78570-8e68-483b-a9e8-0d81a6c662fe_isrForCrosstalkSources_996957_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/07b78570-8e68-483b-a9e8-0d81a6c662fe_isrForCrosstalkSources_996957_37.stderr parsl.dataflow.dflow INFO: Task 1248 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec01ede764e9f33ff1de6a8f8029bc2c with result from task 1248 parsl.dataflow.dflow DEBUG: Task 1248 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/98a8b7c6-9ded-4f84-bdba-4482988922d4_isrForCrosstalkSources_996952_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/98a8b7c6-9ded-4f84-bdba-4482988922d4_isrForCrosstalkSources_996952_49.stderr parsl.dataflow.dflow INFO: Task 1431 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7bfdf04c2f6e3d79715411c424a25aa with result from task 1431 parsl.dataflow.dflow DEBUG: Task 1431 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/091f4dac-25c0-4485-8ff1-79d1cbc37fe9_isrForCrosstalkSources_997032_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/091f4dac-25c0-4485-8ff1-79d1cbc37fe9_isrForCrosstalkSources_997032_61.stderr parsl.dataflow.dflow INFO: Task 1239 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a227c7a10afc534cf5d20be585a5445f with result from task 1239 parsl.dataflow.dflow DEBUG: Task 1239 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/515fd2b9-83cc-4fe0-a7b3-f03ba6b09fbb_isrForCrosstalkSources_997011_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/515fd2b9-83cc-4fe0-a7b3-f03ba6b09fbb_isrForCrosstalkSources_997011_37.stderr parsl.dataflow.dflow INFO: Task 1417 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da9baeb60fdf6f71fb0c718b8a0afab3 with result from task 1417 parsl.dataflow.dflow DEBUG: Task 1417 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1b7a2568-6699-4e35-9de2-4449106e31c0_isrForCrosstalkSources_997022_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1b7a2568-6699-4e35-9de2-4449106e31c0_isrForCrosstalkSources_997022_36.stderr parsl.dataflow.dflow INFO: Task 1469 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8eda178aebc06f0d747517e9e7e7516e with result from task 1469 parsl.dataflow.dflow DEBUG: Task 1469 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/8adfd2bb-6df1-41ac-969e-e4456dfc17b8_isrForCrosstalkSources_996966_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/8adfd2bb-6df1-41ac-969e-e4456dfc17b8_isrForCrosstalkSources_996966_42.stderr parsl.dataflow.dflow INFO: Task 1475 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8fd2cf7d8c96533f58083e45f0d2a744 with result from task 1475 parsl.dataflow.dflow DEBUG: Task 1475 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/859b595a-943e-4ce4-ac1c-095491edbe3a_isrForCrosstalkSources_996966_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/859b595a-943e-4ce4-ac1c-095491edbe3a_isrForCrosstalkSources_996966_18.stderr parsl.dataflow.dflow INFO: Task 1470 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8732666ebc8ccd6143f3d9bc105fcfaa with result from task 1470 parsl.dataflow.dflow DEBUG: Task 1470 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/141df55d-41f3-474d-bc36-28ecbc6feacd_isrForCrosstalkSources_997018_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/141df55d-41f3-474d-bc36-28ecbc6feacd_isrForCrosstalkSources_997018_1.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:30 proc_lsst.multi:146 [INFO] found job 17075086 in provider checkpoint proc_lsst.multi INFO: found job 17075086 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1336 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 420b5a9b6adf598c97ab1819a469f8b7 with result from task 1336 parsl.dataflow.dflow DEBUG: Task 1336 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/a21a7632-ca0d-4930-acbc-c61b0882d79a_isrForCrosstalkSources_996975_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/a21a7632-ca0d-4930-acbc-c61b0882d79a_isrForCrosstalkSources_996975_38.stderr parsl.dataflow.dflow INFO: Task 1312 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb0153a7d6dbc77cc0144227d9076ed7 with result from task 1312 parsl.dataflow.dflow DEBUG: Task 1312 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/59488eba-3101-4235-aaa3-92bc69fb0eea_isrForCrosstalkSources_997009_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/59488eba-3101-4235-aaa3-92bc69fb0eea_isrForCrosstalkSources_997009_40.stderr parsl.dataflow.dflow INFO: Task 1319 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d1fcca5fa0f3abf2be7437b5ee358097 with result from task 1319 parsl.dataflow.dflow DEBUG: Task 1319 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f8fc7d37-0b42-4e56-b539-b0723368add4_isrForCrosstalkSources_997019_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f8fc7d37-0b42-4e56-b539-b0723368add4_isrForCrosstalkSources_997019_51.stderr parsl.dataflow.dflow INFO: Task 1314 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f9fc271a547323a47ed351d2e334d15 with result from task 1314 parsl.dataflow.dflow DEBUG: Task 1314 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/5263f56e-2af3-4467-a1ad-95b1f5de7d60_isrForCrosstalkSources_997032_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/5263f56e-2af3-4467-a1ad-95b1f5de7d60_isrForCrosstalkSources_997032_31.stderr parsl.dataflow.dflow INFO: Task 1318 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56006e8cd2a18f272707a92f6ac8748d with result from task 1318 parsl.dataflow.dflow DEBUG: Task 1318 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6dff968f-f8e1-45d2-b423-a31cb7cd7212_isrForCrosstalkSources_996970_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6dff968f-f8e1-45d2-b423-a31cb7cd7212_isrForCrosstalkSources_996970_57.stderr parsl.dataflow.dflow INFO: Task 1317 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1839bc930270c079bb4ec5e876f11c00 with result from task 1317 parsl.dataflow.dflow DEBUG: Task 1317 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/66ee52df-5a24-4781-8b7a-3ca87aa7e49b_isrForCrosstalkSources_996973_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/66ee52df-5a24-4781-8b7a-3ca87aa7e49b_isrForCrosstalkSources_996973_37.stderr parsl.dataflow.dflow INFO: Task 1316 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32ef575d4f8c7e847b3b104aa552a554 with result from task 1316 parsl.dataflow.dflow DEBUG: Task 1316 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/7a537a40-2e7d-4c12-8a64-5d44d2524007_isrForCrosstalkSources_997020_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/7a537a40-2e7d-4c12-8a64-5d44d2524007_isrForCrosstalkSources_997020_38.stderr parsl.dataflow.dflow INFO: Task 1315 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f91688c5ad6120087220b8ad2852fae with result from task 1315 parsl.dataflow.dflow DEBUG: Task 1315 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/de9dabb8-99b6-4574-999c-59c4b4cd74dd_isrForCrosstalkSources_996990_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/de9dabb8-99b6-4574-999c-59c4b4cd74dd_isrForCrosstalkSources_996990_36.stderr parsl.dataflow.dflow INFO: Task 1313 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b123c46006d52ccd7b288850a7a8853 with result from task 1313 parsl.dataflow.dflow DEBUG: Task 1313 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/82be5258-d2f7-490b-9445-715cad7f5833_isrForCrosstalkSources_997017_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/82be5258-d2f7-490b-9445-715cad7f5833_isrForCrosstalkSources_997017_17.stderr parsl.dataflow.dflow INFO: Task 1441 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b344e19ebac188831ad5b815121894d4 with result from task 1441 parsl.dataflow.dflow DEBUG: Task 1441 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/8ee93071-70bb-4821-8003-2144f25e632b_isrForCrosstalkSources_996975_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/8ee93071-70bb-4821-8003-2144f25e632b_isrForCrosstalkSources_996975_11.stderr parsl.dataflow.dflow INFO: Task 1375 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91371c2a9279ddef657d39c8096ffd09 with result from task 1375 parsl.dataflow.dflow DEBUG: Task 1375 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4960308f-934f-4718-837b-90c76a24697d_isrForCrosstalkSources_996980_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4960308f-934f-4718-837b-90c76a24697d_isrForCrosstalkSources_996980_24.stderr parsl.dataflow.dflow INFO: Task 1422 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44594ebf3e84a9f54e8aee6b84f145fa with result from task 1422 parsl.dataflow.dflow DEBUG: Task 1422 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/35f8c91c-bbee-4882-8e87-4cab7ae55f82_isrForCrosstalkSources_996994_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/35f8c91c-bbee-4882-8e87-4cab7ae55f82_isrForCrosstalkSources_996994_52.stderr parsl.dataflow.dflow INFO: Task 1332 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 41f86c4ce569fa76c173cbbf9a7aee4e with result from task 1332 parsl.dataflow.dflow DEBUG: Task 1332 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a21bc2eb-3c8c-4d04-9a8c-8192aa1d38e9_isrForCrosstalkSources_997026_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a21bc2eb-3c8c-4d04-9a8c-8192aa1d38e9_isrForCrosstalkSources_997026_60.stderr parsl.dataflow.dflow INFO: Task 1489 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42dddcf69327b975664e8da93480a861 with result from task 1489 parsl.dataflow.dflow DEBUG: Task 1489 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/7a305445-3390-4e11-b11e-6237dec4b52c_isrForCrosstalkSources_997012_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/7a305445-3390-4e11-b11e-6237dec4b52c_isrForCrosstalkSources_997012_34.stderr parsl.dataflow.dflow INFO: Task 1359 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8325e92a93d7eca86c7daabcea59b402 with result from task 1359 parsl.dataflow.dflow DEBUG: Task 1359 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/63e17bf0-a30e-4b87-a51e-53902bff45a3_isrForCrosstalkSources_997023_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/63e17bf0-a30e-4b87-a51e-53902bff45a3_isrForCrosstalkSources_997023_43.stderr parsl.dataflow.dflow INFO: Task 1389 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf721ce299aae462425042698261c545 with result from task 1389 parsl.dataflow.dflow DEBUG: Task 1389 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9316077c-55b7-4ef1-a82c-0ddba226aaf1_isrForCrosstalkSources_997004_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9316077c-55b7-4ef1-a82c-0ddba226aaf1_isrForCrosstalkSources_997004_30.stderr parsl.dataflow.dflow INFO: Task 1483 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 909eb0b7c39f886f85abbbe5eb13e238 with result from task 1483 parsl.dataflow.dflow DEBUG: Task 1483 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e4e4c221-6240-48a5-ab8d-541ecea86054_isrForCrosstalkSources_997019_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e4e4c221-6240-48a5-ab8d-541ecea86054_isrForCrosstalkSources_997019_20.stderr parsl.dataflow.dflow INFO: Task 1436 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9798a5dc1316cf275b51cd1b89c390c with result from task 1436 parsl.dataflow.dflow DEBUG: Task 1436 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5263eab8-8ae6-4d7e-b268-cad356873830_isrForCrosstalkSources_996988_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5263eab8-8ae6-4d7e-b268-cad356873830_isrForCrosstalkSources_996988_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:31 proc_lsst.multi:146 [INFO] found job 17075088 in provider checkpoint proc_lsst.multi INFO: found job 17075088 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1244 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 74e3a31a6484e5946c7c11e78554f4aa with result from task 1244 parsl.dataflow.dflow DEBUG: Task 1244 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1d294798-9131-4650-9cbb-d2eef4f16c83_isrForCrosstalkSources_996964_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1d294798-9131-4650-9cbb-d2eef4f16c83_isrForCrosstalkSources_996964_5.stderr parsl.dataflow.dflow INFO: Task 1342 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d5f376daff5d30d6662c2668b1df900c with result from task 1342 parsl.dataflow.dflow DEBUG: Task 1342 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/cabb6e59-41d8-4baa-a564-9b831117c418_isrForCrosstalkSources_997002_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/cabb6e59-41d8-4baa-a564-9b831117c418_isrForCrosstalkSources_997002_29.stderr parsl.dataflow.dflow INFO: Task 1402 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d211e517c6b1e8208228299804103fd2 with result from task 1402 parsl.dataflow.dflow DEBUG: Task 1402 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9023dedb-9aae-49ad-8620-55c8757fd573_isrForCrosstalkSources_997004_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9023dedb-9aae-49ad-8620-55c8757fd573_isrForCrosstalkSources_997004_35.stderr parsl.dataflow.dflow INFO: Task 1355 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ea0b89b162ed8140d71461d32bc80a0 with result from task 1355 parsl.dataflow.dflow DEBUG: Task 1355 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/5b123d46-8cee-47aa-b293-1d197ae2647a_isrForCrosstalkSources_997030_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/5b123d46-8cee-47aa-b293-1d197ae2647a_isrForCrosstalkSources_997030_6.stderr parsl.dataflow.dflow INFO: Task 1377 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d14810ea04fcc512e50cfb23e5d9612 with result from task 1377 parsl.dataflow.dflow DEBUG: Task 1377 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5cfdbcda-8edd-4a5d-bc2f-342849c14f56_isrForCrosstalkSources_996979_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5cfdbcda-8edd-4a5d-bc2f-342849c14f56_isrForCrosstalkSources_996979_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:32 proc_lsst.multi:146 [INFO] found job 17075090 in provider checkpoint proc_lsst.multi INFO: found job 17075090 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1411 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4a6bdc43f1cd6833a37f16c4129ed5f with result from task 1411 parsl.dataflow.dflow DEBUG: Task 1411 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/e2f4f58f-ddb3-4fd1-8f80-64282396a47d_isrForCrosstalkSources_997009_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/e2f4f58f-ddb3-4fd1-8f80-64282396a47d_isrForCrosstalkSources_997009_16.stderr parsl.dataflow.dflow INFO: Task 1487 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5aa7c960693c0ef04e0b3026374a2ced with result from task 1487 parsl.dataflow.dflow DEBUG: Task 1487 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/8c6aa64d-233c-4e65-b200-df3ffad5b27f_isrForCrosstalkSources_996986_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/8c6aa64d-233c-4e65-b200-df3ffad5b27f_isrForCrosstalkSources_996986_1.stderr parsl.dataflow.dflow INFO: Task 1323 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cbc723a87cc0baaf5b88896da84ba927 with result from task 1323 parsl.dataflow.dflow DEBUG: Task 1323 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/65923a1a-921f-4ad4-979e-6342213b4a20_isrForCrosstalkSources_996996_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/65923a1a-921f-4ad4-979e-6342213b4a20_isrForCrosstalkSources_996996_48.stderr parsl.dataflow.dflow INFO: Task 1444 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c108ddaa3b83768406f9102c21577ad with result from task 1444 parsl.dataflow.dflow DEBUG: Task 1444 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/46c978cf-bbd3-4948-8a7d-453ea6ee5a12_isrForCrosstalkSources_996955_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/46c978cf-bbd3-4948-8a7d-453ea6ee5a12_isrForCrosstalkSources_996955_61.stderr parsl.dataflow.dflow INFO: Task 1271 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81d72838d59e1ec23c17761900b44f51 with result from task 1271 parsl.dataflow.dflow DEBUG: Task 1271 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/344423a0-0b05-45be-8fb9-12c8daed71b5_isrForCrosstalkSources_997026_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/344423a0-0b05-45be-8fb9-12c8daed71b5_isrForCrosstalkSources_997026_41.stderr parsl.dataflow.dflow INFO: Task 1298 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a11d22e480eb97dbdfd5e2dcf46ef7a with result from task 1298 parsl.dataflow.dflow DEBUG: Task 1298 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/226bdfac-d83e-4f11-aa1f-63196d9ea7c4_isrForCrosstalkSources_997014_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/226bdfac-d83e-4f11-aa1f-63196d9ea7c4_isrForCrosstalkSources_997014_16.stderr parsl.dataflow.dflow INFO: Task 1385 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eedc12e150076d72b79db2cf5735b9c0 with result from task 1385 parsl.dataflow.dflow DEBUG: Task 1385 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/2987827c-de3f-4d8e-8ddc-a928c9b03613_isrForCrosstalkSources_996986_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/2987827c-de3f-4d8e-8ddc-a928c9b03613_isrForCrosstalkSources_996986_21.stderr parsl.dataflow.dflow INFO: Task 1439 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 87a2119cfc3916a36c8cb88bfea1a5cd with result from task 1439 parsl.dataflow.dflow DEBUG: Task 1439 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a4087502-7e6b-4736-aee4-3c0f28dcea6f_isrForCrosstalkSources_997013_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a4087502-7e6b-4736-aee4-3c0f28dcea6f_isrForCrosstalkSources_997013_60.stderr parsl.dataflow.dflow INFO: Task 1366 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11d69a19704a79aaecc65f4dc84ead66 with result from task 1366 parsl.dataflow.dflow DEBUG: Task 1366 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/043cb00f-9507-4053-a34c-c2c8244c1f71_isrForCrosstalkSources_997008_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/043cb00f-9507-4053-a34c-c2c8244c1f71_isrForCrosstalkSources_997008_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:33 proc_lsst.multi:146 [INFO] found job 17075091 in provider checkpoint proc_lsst.multi INFO: found job 17075091 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1275 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 177b187c22023222a6b3a4096c71dca4 with result from task 1275 parsl.dataflow.dflow DEBUG: Task 1275 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/834d848f-cea3-496f-86e4-c4b80a22543b_isrForCrosstalkSources_996980_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/834d848f-cea3-496f-86e4-c4b80a22543b_isrForCrosstalkSources_996980_41.stderr parsl.dataflow.dflow INFO: Task 1383 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 80ae07328fc5c8b5a84df4cd8fc272b9 with result from task 1383 parsl.dataflow.dflow DEBUG: Task 1383 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/3f585083-a96d-4660-b8a2-6259452b6f77_isrForCrosstalkSources_997023_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/3f585083-a96d-4660-b8a2-6259452b6f77_isrForCrosstalkSources_997023_47.stderr parsl.dataflow.dflow INFO: Task 1224 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a45f199ae56e7a30122faa1afba898eb with result from task 1224 parsl.dataflow.dflow DEBUG: Task 1224 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/73fd87e0-0866-489e-9f0c-f41444ecd967_isrForCrosstalkSources_997009_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/73fd87e0-0866-489e-9f0c-f41444ecd967_isrForCrosstalkSources_997009_41.stderr parsl.dataflow.dflow INFO: Task 1233 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6fe6b05f48582155aa7e6ede5a35620a with result from task 1233 parsl.dataflow.dflow DEBUG: Task 1233 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c98af136-f874-44e5-86b8-eb79913df754_isrForCrosstalkSources_996969_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c98af136-f874-44e5-86b8-eb79913df754_isrForCrosstalkSources_996969_41.stderr parsl.dataflow.dflow INFO: Task 1522 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31cab6b218c23d97141c89f0bc3d49ae with result from task 1522 parsl.dataflow.dflow DEBUG: Task 1522 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5657cb6f-173a-4bc3-849c-c07816cc48ae_isrForCrosstalkSources_996981_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5657cb6f-173a-4bc3-849c-c07816cc48ae_isrForCrosstalkSources_996981_30.stderr parsl.dataflow.dflow INFO: Task 1299 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cace9d0912d354e3be04ecffdb77292b with result from task 1299 parsl.dataflow.dflow DEBUG: Task 1299 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/df6670c5-4cf0-4b4b-b988-708afcd945c9_isrForCrosstalkSources_996955_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/df6670c5-4cf0-4b4b-b988-708afcd945c9_isrForCrosstalkSources_996955_37.stderr parsl.dataflow.dflow INFO: Task 1400 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 20913d264f337e2fd7a9b723c288a12b with result from task 1400 parsl.dataflow.dflow DEBUG: Task 1400 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/25688a2f-dc36-45c6-9aae-fd5be5e174b7_isrForCrosstalkSources_996955_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/25688a2f-dc36-45c6-9aae-fd5be5e174b7_isrForCrosstalkSources_996955_55.stderr parsl.dataflow.dflow INFO: Task 1505 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c355d0df88d7db872634d81f6b1b501 with result from task 1505 parsl.dataflow.dflow DEBUG: Task 1505 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/4285129d-563b-436d-b7d5-807a746b8aea_isrForCrosstalkSources_996964_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/4285129d-563b-436d-b7d5-807a746b8aea_isrForCrosstalkSources_996964_57.stderr parsl.dataflow.dflow INFO: Task 1456 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 808ab62a3f501854713d1baecd85fdbb with result from task 1456 parsl.dataflow.dflow DEBUG: Task 1456 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c09a8288-8766-4664-b39d-e5819faf716f_isrForCrosstalkSources_996993_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c09a8288-8766-4664-b39d-e5819faf716f_isrForCrosstalkSources_996993_29.stderr parsl.dataflow.dflow INFO: Task 1291 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82a8ead1bf4f4a4236c4c9cd35f4e819 with result from task 1291 parsl.dataflow.dflow DEBUG: Task 1291 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e41f1800-c981-4603-a5ab-000e68026fcb_isrForCrosstalkSources_997033_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e41f1800-c981-4603-a5ab-000e68026fcb_isrForCrosstalkSources_997033_31.stderr parsl.dataflow.dflow INFO: Task 1236 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 28874969eaf0442206a847922a6eb091 with result from task 1236 parsl.dataflow.dflow DEBUG: Task 1236 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6ef02598-c8ba-46e5-88ad-93c51a5fe772_isrForCrosstalkSources_997017_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6ef02598-c8ba-46e5-88ad-93c51a5fe772_isrForCrosstalkSources_997017_57.stderr parsl.dataflow.dflow INFO: Task 1418 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e5e66541d34a44b06411fc0d8ed06abd with result from task 1418 parsl.dataflow.dflow DEBUG: Task 1418 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7fce1d81-5833-42f4-9c1a-f7d5589c0bb0_isrForCrosstalkSources_997026_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7fce1d81-5833-42f4-9c1a-f7d5589c0bb0_isrForCrosstalkSources_997026_3.stderr parsl.dataflow.dflow INFO: Task 1434 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 243886579a82c663a71dc25cd5e10a23 with result from task 1434 parsl.dataflow.dflow DEBUG: Task 1434 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/bdde0b14-ff4a-46af-8d7c-286ced3e6e81_isrForCrosstalkSources_996991_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/bdde0b14-ff4a-46af-8d7c-286ced3e6e81_isrForCrosstalkSources_996991_49.stderr parsl.dataflow.dflow INFO: Task 1445 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e2a340ab802d62285df871fb8922964 with result from task 1445 parsl.dataflow.dflow DEBUG: Task 1445 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/06ad926e-1cc3-4c30-b565-4b18252b42fb_isrForCrosstalkSources_997005_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/06ad926e-1cc3-4c30-b565-4b18252b42fb_isrForCrosstalkSources_997005_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:35 proc_lsst.multi:146 [INFO] found job 17075092 in provider checkpoint proc_lsst.multi INFO: found job 17075092 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1421 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f139e8db0b7e944d66112f460528927 with result from task 1421 parsl.dataflow.dflow DEBUG: Task 1421 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/56ee6034-3834-408d-b5c1-b849fe471f02_isrForCrosstalkSources_997006_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/56ee6034-3834-408d-b5c1-b849fe471f02_isrForCrosstalkSources_997006_29.stderr parsl.dataflow.dflow INFO: Task 1452 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d391cc12b4d8d548d6dd818e1941385 with result from task 1452 parsl.dataflow.dflow DEBUG: Task 1452 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/9f36cc29-7cd2-4d10-a648-dc4d2a8a3c00_isrForCrosstalkSources_996954_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/9f36cc29-7cd2-4d10-a648-dc4d2a8a3c00_isrForCrosstalkSources_996954_60.stderr parsl.dataflow.dflow INFO: Task 1401 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7d113270175c86603ce153e0a6bcb6b with result from task 1401 parsl.dataflow.dflow DEBUG: Task 1401 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5b85ef51-20f6-4dd0-b149-e1ba89f3345b_isrForCrosstalkSources_997018_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5b85ef51-20f6-4dd0-b149-e1ba89f3345b_isrForCrosstalkSources_997018_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:35 proc_lsst.multi:146 [INFO] found job 17075093 in provider checkpoint proc_lsst.multi INFO: found job 17075093 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1451 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65639748ee3712b4a899e89078e714b7 with result from task 1451 parsl.dataflow.dflow DEBUG: Task 1451 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9cacc2a1-3ff6-4767-b9bb-a9467da2e947_isrForCrosstalkSources_997008_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9cacc2a1-3ff6-4767-b9bb-a9467da2e947_isrForCrosstalkSources_997008_29.stderr parsl.dataflow.dflow INFO: Task 1556 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2b3218a771cc47b283521cc896a599d with result from task 1556 parsl.dataflow.dflow DEBUG: Task 1556 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/3c12f4c2-e42e-4519-8d25-1ea9ca44feb2_isrForCrosstalkSources_996960_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/3c12f4c2-e42e-4519-8d25-1ea9ca44feb2_isrForCrosstalkSources_996960_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:36 proc_lsst.multi:146 [INFO] found job 17075094 in provider checkpoint proc_lsst.multi INFO: found job 17075094 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1408 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 67ca993e9f5be421661fbe536a1f0a92 with result from task 1408 parsl.dataflow.dflow DEBUG: Task 1408 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/33f5933e-4847-4e1a-916e-31293186b258_isrForCrosstalkSources_996960_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/33f5933e-4847-4e1a-916e-31293186b258_isrForCrosstalkSources_996960_48.stderr parsl.dataflow.dflow INFO: Task 1343 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73f1ca1cc45d287762f0ee3009e74355 with result from task 1343 parsl.dataflow.dflow DEBUG: Task 1343 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/66560b0d-ae21-4c61-9c71-a2403f733e7c_isrForCrosstalkSources_996963_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/66560b0d-ae21-4c61-9c71-a2403f733e7c_isrForCrosstalkSources_996963_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:37 proc_lsst.multi:146 [INFO] found job 17075095 in provider checkpoint proc_lsst.multi INFO: found job 17075095 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1394 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb1c55b6e98df2c652929bf301005575 with result from task 1394 parsl.dataflow.dflow DEBUG: Task 1394 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/414194ed-bf69-4b74-8e9a-9c978e6ac727_isrForCrosstalkSources_997016_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/414194ed-bf69-4b74-8e9a-9c978e6ac727_isrForCrosstalkSources_997016_50.stderr parsl.dataflow.dflow INFO: Task 1432 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd050651c87c2dd48cb37bcde65323b5 with result from task 1432 parsl.dataflow.dflow DEBUG: Task 1432 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/cadc95bd-fab6-418c-b7a8-8599161e1785_isrForCrosstalkSources_996971_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/cadc95bd-fab6-418c-b7a8-8599161e1785_isrForCrosstalkSources_996971_26.stderr parsl.dataflow.dflow INFO: Task 1467 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 749aed52bbd609196bf417f5c89dcff1 with result from task 1467 parsl.dataflow.dflow DEBUG: Task 1467 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ab822ff8-a111-46f8-9f18-5b5224cb6d32_isrForCrosstalkSources_997000_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ab822ff8-a111-46f8-9f18-5b5224cb6d32_isrForCrosstalkSources_997000_25.stderr parsl.dataflow.dflow INFO: Task 1427 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b484f3655306f7d956439e1fd0736f3 with result from task 1427 parsl.dataflow.dflow DEBUG: Task 1427 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/0db53437-4385-4bdf-ae9c-8f339b70c4fb_isrForCrosstalkSources_996976_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/0db53437-4385-4bdf-ae9c-8f339b70c4fb_isrForCrosstalkSources_996976_27.stderr parsl.dataflow.dflow INFO: Task 1450 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b14f3708bd704a8193719db6268d4f23 with result from task 1450 parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow DEBUG: Task 1450 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/575832a9-bb14-42b6-9079-b757a8050eda_isrForCrosstalkSources_997005_26.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/575832a9-bb14-42b6-9079-b757a8050eda_isrForCrosstalkSources_997005_26.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:37 proc_lsst.multi:146 [INFO] found job 17075096 in provider checkpoint proc_lsst.multi INFO: found job 17075096 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1566 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9cb31e73013f0051ef8528efa11fbf63 with result from task 1566 parsl.dataflow.dflow DEBUG: Task 1566 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/9f96640a-7bf3-43ba-939d-e5a8344789cf_isrForCrosstalkSources_996981_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/9f96640a-7bf3-43ba-939d-e5a8344789cf_isrForCrosstalkSources_996981_62.stderr parsl.dataflow.dflow INFO: Task 1404 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7eecd3dfd62df61413f306b7aff9be53 with result from task 1404 parsl.dataflow.dflow DEBUG: Task 1404 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/eed05226-eeb0-4c76-985e-85e72a143f1a_isrForCrosstalkSources_996995_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/eed05226-eeb0-4c76-985e-85e72a143f1a_isrForCrosstalkSources_996995_20.stderr parsl.dataflow.dflow INFO: Task 1220 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39ac8370274e50e5a57973b8859f1978 with result from task 1220 parsl.dataflow.dflow DEBUG: Task 1220 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/2a995f59-aecb-43e8-a77e-e18f4935ff0e_isrForCrosstalkSources_996985_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/2a995f59-aecb-43e8-a77e-e18f4935ff0e_isrForCrosstalkSources_996985_58.stderr parsl.dataflow.dflow INFO: Task 1425 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 79c1486178072dc75487091d756bcefd with result from task 1425 parsl.dataflow.dflow DEBUG: Task 1425 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d8648b19-7986-4921-b48b-fbe28e395a88_isrForCrosstalkSources_996971_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d8648b19-7986-4921-b48b-fbe28e395a88_isrForCrosstalkSources_996971_14.stderr parsl.dataflow.dflow INFO: Task 1354 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 519f3b1fdc9e20c25d8556c7cdeecdcb with result from task 1354 parsl.dataflow.dflow DEBUG: Task 1354 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/df396c51-8137-49c2-a500-3615095cd399_isrForCrosstalkSources_996967_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/df396c51-8137-49c2-a500-3615095cd399_isrForCrosstalkSources_996967_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:38 proc_lsst.multi:146 [INFO] found job 17075097 in provider checkpoint proc_lsst.multi INFO: found job 17075097 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1462 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4497b19203990a053cf2a76fa21444e8 with result from task 1462 parsl.dataflow.dflow DEBUG: Task 1462 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/b1ca0f5d-c56b-473b-a7af-77d0ce9aeade_isrForCrosstalkSources_996987_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/b1ca0f5d-c56b-473b-a7af-77d0ce9aeade_isrForCrosstalkSources_996987_39.stderr parsl.dataflow.dflow INFO: Task 1341 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1945fa391eb0ad5b82e5be5c41409060 with result from task 1341 parsl.dataflow.dflow DEBUG: Task 1341 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/32769c93-990a-48ac-81c8-52e1e5795c0c_isrForCrosstalkSources_996954_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/32769c93-990a-48ac-81c8-52e1e5795c0c_isrForCrosstalkSources_996954_57.stderr parsl.dataflow.dflow INFO: Task 1480 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf9021cec93157d93e172cdec4b2d03f with result from task 1480 parsl.dataflow.dflow DEBUG: Task 1480 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a647783f-84a7-4d8a-b567-bc6aecf8b923_isrForCrosstalkSources_997013_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a647783f-84a7-4d8a-b567-bc6aecf8b923_isrForCrosstalkSources_997013_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1548 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry e902971f6fe01106089fdbd117f1bc40 with result from task 1548 parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1548 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/aba9cee1-3124-495a-8bc7-c85e95ebdc66_isrForCrosstalkSources_996971_10.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/aba9cee1-3124-495a-8bc7-c85e95ebdc66_isrForCrosstalkSources_996971_10.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:38 proc_lsst.multi:146 [INFO] found job 17075098 in provider checkpoint proc_lsst.multi INFO: found job 17075098 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1403 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57005397032bc6b326cf6100da6e7196 with result from task 1403 parsl.dataflow.dflow DEBUG: Task 1403 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2801cc1a-59a4-4bea-a96b-fe7c3049c3a9_isrForCrosstalkSources_996963_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2801cc1a-59a4-4bea-a96b-fe7c3049c3a9_isrForCrosstalkSources_996963_16.stderr parsl.dataflow.dflow INFO: Task 1448 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 878a40127a6cadbfb4b4bfc995d5ef86 with result from task 1448 parsl.dataflow.dflow DEBUG: Task 1448 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4b75651e-11a4-4610-bb1f-acb6d77af405_isrForCrosstalkSources_997020_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4b75651e-11a4-4610-bb1f-acb6d77af405_isrForCrosstalkSources_997020_33.stderr parsl.dataflow.dflow INFO: Task 1423 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73d6b589220411d1aa524db24bddf671 with result from task 1423 parsl.dataflow.dflow DEBUG: Task 1423 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d5900cfa-5589-4615-bc59-2c4d93e172c4_isrForCrosstalkSources_996954_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d5900cfa-5589-4615-bc59-2c4d93e172c4_isrForCrosstalkSources_996954_23.stderr parsl.dataflow.dflow INFO: Task 1460 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1cc1301c334c27dceb7cd56b75c472e1 with result from task 1460 parsl.dataflow.dflow DEBUG: Task 1460 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8eda186b-bccc-49f4-a8ea-ea27d9c35516_isrForCrosstalkSources_996960_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8eda186b-bccc-49f4-a8ea-ea27d9c35516_isrForCrosstalkSources_996960_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1471 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry e71bc14905838240bd57868e82916061 with result from task 1471 parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1471 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/fa482bd4-364e-4e34-945e-15163e67b36b_isrForCrosstalkSources_997016_25.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/fa482bd4-364e-4e34-945e-15163e67b36b_isrForCrosstalkSources_997016_25.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1458 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 7c2cc1030def1b7e37f26b719922148f with result from task 1458 parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1458 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/68b04fa4-fa45-41d2-90c7-764fab9d962c_isrForCrosstalkSources_997033_51.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/68b04fa4-fa45-41d2-90c7-764fab9d962c_isrForCrosstalkSources_997033_51.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:39 proc_lsst.multi:146 [INFO] found job 17075099 in provider checkpoint proc_lsst.multi INFO: found job 17075099 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1433 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 06e76a83e6ba685a0f6c1a3515fe6e06 with result from task 1433 parsl.dataflow.dflow DEBUG: Task 1433 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c3e097b8-19df-4131-b342-e8aa1af89a6d_isrForCrosstalkSources_997014_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c3e097b8-19df-4131-b342-e8aa1af89a6d_isrForCrosstalkSources_997014_45.stderr parsl.dataflow.dflow INFO: Task 1538 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 45e05fec8789406860d2e43ed0cb8993 with result from task 1538 parsl.dataflow.dflow DEBUG: Task 1538 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ccc7758c-7f1c-4216-a99c-d3179c44964d_isrForCrosstalkSources_996992_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ccc7758c-7f1c-4216-a99c-d3179c44964d_isrForCrosstalkSources_996992_25.stderr parsl.dataflow.dflow INFO: Task 1378 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7619ad3eca4e0aa7c5ac0898aca9faaa with result from task 1378 parsl.dataflow.dflow DEBUG: Task 1378 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/63ab8952-1687-426e-87f4-e478bdc90ece_isrForCrosstalkSources_997027_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/63ab8952-1687-426e-87f4-e478bdc90ece_isrForCrosstalkSources_997027_54.stderr parsl.dataflow.dflow INFO: Task 1500 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27ca982883ade4fb5bee014681959470 with result from task 1500 parsl.dataflow.dflow DEBUG: Task 1500 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f42eb58c-d8b5-488e-bfb2-605b5153619e_isrForCrosstalkSources_996960_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f42eb58c-d8b5-488e-bfb2-605b5153619e_isrForCrosstalkSources_996960_46.stderr parsl.dataflow.dflow INFO: Task 1485 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 337b894c9f6b8d71ab99aa205acf52ff with result from task 1485 parsl.dataflow.dflow DEBUG: Task 1485 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/085e3c5d-987f-4fec-b57e-fece8ac22659_isrForCrosstalkSources_997023_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/085e3c5d-987f-4fec-b57e-fece8ac22659_isrForCrosstalkSources_997023_33.stderr parsl.dataflow.dflow INFO: Task 1477 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32767ce342b63d7d06270c27ed1829a9 with result from task 1477 parsl.dataflow.dflow DEBUG: Task 1477 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f640112d-8bd8-4b0e-b865-bef040b94cdb_isrForCrosstalkSources_996986_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f640112d-8bd8-4b0e-b865-bef040b94cdb_isrForCrosstalkSources_996986_26.stderr parsl.dataflow.dflow INFO: Task 1333 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fca170e3340f2cd7f5d82e04b9f0471e with result from task 1333 parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:39 proc_lsst.multi:146 [INFO] found job 17075100 in provider checkpoint proc_lsst.multi INFO: found job 17075100 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 1333 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0a52b2c0-9b70-4f4b-b9b1-f4d7611326dd_isrForCrosstalkSources_996953_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0a52b2c0-9b70-4f4b-b9b1-f4d7611326dd_isrForCrosstalkSources_996953_55.stderr parsl.dataflow.dflow INFO: Task 1405 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff7cd3a91c7b44b6bb207de8e65e972f with result from task 1405 parsl.dataflow.dflow DEBUG: Task 1405 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/822a4c9f-d790-47e1-9a17-346730c563e2_isrForCrosstalkSources_996972_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/822a4c9f-d790-47e1-9a17-346730c563e2_isrForCrosstalkSources_996972_61.stderr parsl.dataflow.dflow INFO: Task 1486 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry afb40e504b943ca5345f8930525b956b with result from task 1486 parsl.dataflow.dflow DEBUG: Task 1486 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/9abbef3f-917f-4d8d-902a-1c308264b93b_isrForCrosstalkSources_996962_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/9abbef3f-917f-4d8d-902a-1c308264b93b_isrForCrosstalkSources_996962_8.stderr parsl.dataflow.dflow INFO: Task 1552 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 127fbe4a180b11fac81eacba8306c977 with result from task 1552 parsl.dataflow.dflow DEBUG: Task 1552 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9af38b87-ef5a-41ba-99bc-8e06967d997e_isrForCrosstalkSources_996964_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9af38b87-ef5a-41ba-99bc-8e06967d997e_isrForCrosstalkSources_996964_19.stderr parsl.dataflow.dflow INFO: Task 1497 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 09322ca5330a5859cb5811b323bc5c62 with result from task 1497 parsl.dataflow.dflow DEBUG: Task 1497 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/d42dd075-c70e-4639-a89a-088c530b740f_isrForCrosstalkSources_997029_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/d42dd075-c70e-4639-a89a-088c530b740f_isrForCrosstalkSources_997029_25.stderr parsl.dataflow.dflow INFO: Task 1473 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f5f28c652a32d72f0642ecd8b616c8b with result from task 1473 parsl.dataflow.dflow DEBUG: Task 1473 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/bab4ce2f-9a0d-45c4-9c1b-b584de2f770f_isrForCrosstalkSources_996983_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/bab4ce2f-9a0d-45c4-9c1b-b584de2f770f_isrForCrosstalkSources_996983_13.stderr parsl.dataflow.dflow INFO: Task 1584 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eae92035dab637759d6f82f20fdca0bb with result from task 1584 parsl.dataflow.dflow DEBUG: Task 1584 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/50a8c756-3764-4376-a642-43932e938c82_isrForCrosstalkSources_996955_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/50a8c756-3764-4376-a642-43932e938c82_isrForCrosstalkSources_996955_33.stderr parsl.dataflow.dflow INFO: Task 1457 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8265ff696fd469fafb401c668e60b19 with result from task 1457 parsl.dataflow.dflow DEBUG: Task 1457 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/238de23f-fab5-4f4d-bd17-a07859b0aa58_isrForCrosstalkSources_996963_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/238de23f-fab5-4f4d-bd17-a07859b0aa58_isrForCrosstalkSources_996963_10.stderr parsl.dataflow.dflow INFO: Task 1589 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a3501ca32f038a6b7a287507a5f3d96e with result from task 1589 parsl.dataflow.dflow DEBUG: Task 1589 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/03b43be3-3892-4c93-a14d-1cbb878dc3be_isrForCrosstalkSources_996958_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/03b43be3-3892-4c93-a14d-1cbb878dc3be_isrForCrosstalkSources_996958_27.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:40 proc_lsst.multi:146 [INFO] found job 17075101 in provider checkpoint proc_lsst.multi INFO: found job 17075101 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1492 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca25ab544dcca7a23d7ca38a9f1fa4c6 with result from task 1492 parsl.dataflow.dflow DEBUG: Task 1492 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/dc252747-9ac5-4548-9124-537252dffb72_isrForCrosstalkSources_996970_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/dc252747-9ac5-4548-9124-537252dffb72_isrForCrosstalkSources_996970_25.stderr parsl.dataflow.dflow INFO: Task 1511 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4bd8b3ed97aeddbb11a9d07c40283a0 with result from task 1511 parsl.dataflow.dflow DEBUG: Task 1511 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9fe0f193-ce39-4f8e-9b8e-8e8808f529ee_isrForCrosstalkSources_996958_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9fe0f193-ce39-4f8e-9b8e-8e8808f529ee_isrForCrosstalkSources_996958_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:41 proc_lsst.multi:146 [INFO] found job 17075102 in provider checkpoint proc_lsst.multi INFO: found job 17075102 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1558 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 66aa61bb657e778aee07e2decb6e756a with result from task 1558 parsl.dataflow.dflow DEBUG: Task 1558 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/48fe998a-f70e-4afd-80c8-5947c5e84f98_isrForCrosstalkSources_997011_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/48fe998a-f70e-4afd-80c8-5947c5e84f98_isrForCrosstalkSources_997011_40.stderr parsl.dataflow.dflow INFO: Task 1472 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9882c6ebbd8c1fdb477ef48aa0c97d6 with result from task 1472 parsl.dataflow.dflow DEBUG: Task 1472 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/9502aa45-0bc2-4e84-a682-55f811923112_isrForCrosstalkSources_996990_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/9502aa45-0bc2-4e84-a682-55f811923112_isrForCrosstalkSources_996990_29.stderr parsl.dataflow.dflow INFO: Task 1588 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 654c112a5d8142e342b54bbd3c5c8387 with result from task 1588 parsl.dataflow.dflow DEBUG: Task 1588 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/f1abf74e-8431-49d6-928d-90e5555b595f_isrForCrosstalkSources_997013_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/f1abf74e-8431-49d6-928d-90e5555b595f_isrForCrosstalkSources_997013_14.stderr parsl.dataflow.dflow INFO: Task 1535 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c302e82781b9b63d545c1a4ba773db81 with result from task 1535 parsl.dataflow.dflow DEBUG: Task 1535 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/1a949cdb-7765-45b9-b1b8-7a47246c7942_isrForCrosstalkSources_996955_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/1a949cdb-7765-45b9-b1b8-7a47246c7942_isrForCrosstalkSources_996955_51.stderr parsl.dataflow.dflow INFO: Task 1399 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62efe12f7f23f7ae81c8fd667a9d0b22 with result from task 1399 parsl.dataflow.dflow DEBUG: Task 1399 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/18c14827-03f2-436e-b7b8-0a7691298774_isrForCrosstalkSources_997020_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/18c14827-03f2-436e-b7b8-0a7691298774_isrForCrosstalkSources_997020_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075055 PD 17075056 PD 17075057 PD 17075058 PD 17075059 PD 17075060 PD 17075061 PD 17075062 PD 17075063 PD 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 1569 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry aa8d5c253c1e3a4fed30f167f783007d with result from task 1569 parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1569 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/67ae528d-8c27-4193-9a91-2bc9ef3b1b28_isrForCrosstalkSources_996982_11.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:41 proc_lsst.multi:146 [INFO] found job 17075103 in provider checkpoint proc_lsst.multi INFO: found job 17075103 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Standard error for task 1569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/67ae528d-8c27-4193-9a91-2bc9ef3b1b28_isrForCrosstalkSources_996982_11.stderr parsl.dataflow.dflow INFO: Task 1496 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31b4d92dba3e8c3d6da3220090680de0 with result from task 1496 parsl.dataflow.dflow DEBUG: Task 1496 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/22dd9694-1925-4ea5-a14a-e07e27f4b0d4_isrForCrosstalkSources_997003_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/22dd9694-1925-4ea5-a14a-e07e27f4b0d4_isrForCrosstalkSources_997003_56.stderr parsl.dataflow.dflow INFO: Task 1586 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bef6f82c7a3a560ed27f3bfac9979f9e with result from task 1586 parsl.dataflow.dflow DEBUG: Task 1586 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/16e1f6a5-993f-443a-93fb-a0f04af78673_isrForCrosstalkSources_997023_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/16e1f6a5-993f-443a-93fb-a0f04af78673_isrForCrosstalkSources_997023_23.stderr parsl.dataflow.dflow INFO: Task 1608 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dbb1a06cd1cec34404322dc0366deebd with result from task 1608 parsl.dataflow.dflow DEBUG: Task 1608 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e875d7e8-898a-430d-b621-e1ec7398102b_isrForCrosstalkSources_997026_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e875d7e8-898a-430d-b621-e1ec7398102b_isrForCrosstalkSources_997026_25.stderr parsl.dataflow.dflow INFO: Task 1542 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a3e513e2c42db53b0747ddf2f336eb86 with result from task 1542 parsl.dataflow.dflow DEBUG: Task 1542 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d156b8b6-22a9-42d0-9900-460364d29096_isrForCrosstalkSources_996990_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d156b8b6-22a9-42d0-9900-460364d29096_isrForCrosstalkSources_996990_40.stderr parsl.dataflow.dflow INFO: Task 1585 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e647b666619e0fb99395233894bfba69 with result from task 1585 parsl.dataflow.dflow DEBUG: Task 1585 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/4b937ebf-3c37-4360-8b25-62ef78ac8d35_isrForCrosstalkSources_996960_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/4b937ebf-3c37-4360-8b25-62ef78ac8d35_isrForCrosstalkSources_996960_29.stderr parsl.dataflow.dflow INFO: Task 1591 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 265dd09546851ae0fe44fcff00bd164a with result from task 1591 parsl.dataflow.dflow DEBUG: Task 1591 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c9e40796-4ea2-41c6-be5d-6b8b955f99f8_isrForCrosstalkSources_996955_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/c9e40796-4ea2-41c6-be5d-6b8b955f99f8_isrForCrosstalkSources_996955_13.stderr parsl.dataflow.dflow INFO: Task 1617 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 71ddc945807d3ac77f21e08e75c5b676 with result from task 1617 parsl.dataflow.dflow DEBUG: Task 1617 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/8e01f1aa-c888-4bbf-8eff-3ab77ae6caf1_isrForCrosstalkSources_996969_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/8e01f1aa-c888-4bbf-8eff-3ab77ae6caf1_isrForCrosstalkSources_996969_25.stderr parsl.dataflow.dflow INFO: Task 1587 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9aecd14404d43ba1830d6362eabe2ba0 with result from task 1587 parsl.dataflow.dflow DEBUG: Task 1587 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/8aaadfff-c125-4f05-bedc-7019926c2fff_isrForCrosstalkSources_996968_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/8aaadfff-c125-4f05-bedc-7019926c2fff_isrForCrosstalkSources_996968_22.stderr parsl.dataflow.dflow INFO: Task 1595 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc6a9594ae1de64f44cde1494c09e43f with result from task 1595 parsl.dataflow.dflow DEBUG: Task 1595 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a7ca9ef0-4dee-422e-95e4-953acf7196cb_isrForCrosstalkSources_997033_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a7ca9ef0-4dee-422e-95e4-953acf7196cb_isrForCrosstalkSources_997033_19.stderr parsl.dataflow.dflow INFO: Task 1537 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5c92b327554143137d359e50fe750a8 with result from task 1537 parsl.dataflow.dflow DEBUG: Task 1537 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/97238b31-7495-4f7d-872a-809417f42b1d_isrForCrosstalkSources_997028_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/97238b31-7495-4f7d-872a-809417f42b1d_isrForCrosstalkSources_997028_21.stderr parsl.dataflow.dflow INFO: Task 1424 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 427a81c43234651c4fd2add5a9fd5887 with result from task 1424 parsl.dataflow.dflow DEBUG: Task 1424 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/5c0be663-c3df-46da-85c1-0dd94d6decdd_isrForCrosstalkSources_996966_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/5c0be663-c3df-46da-85c1-0dd94d6decdd_isrForCrosstalkSources_996966_55.stderr parsl.dataflow.dflow INFO: Task 1590 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9eefb937cfab32255c8a4002caef428f with result from task 1590 parsl.dataflow.dflow DEBUG: Task 1590 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/babdcd8a-c4c3-4ac6-af1c-e488732051f8_isrForCrosstalkSources_997005_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/babdcd8a-c4c3-4ac6-af1c-e488732051f8_isrForCrosstalkSources_997005_8.stderr parsl.dataflow.dflow INFO: Task 1428 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f542d40f51ff254b4815a66fa6d1a17 with result from task 1428 parsl.dataflow.dflow DEBUG: Task 1428 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0e6f4d16-643d-4a3e-9fa3-6b013649d20d_isrForCrosstalkSources_996973_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0e6f4d16-643d-4a3e-9fa3-6b013649d20d_isrForCrosstalkSources_996973_49.stderr parsl.dataflow.dflow INFO: Task 1575 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b99c8ee4dd8c96c62acdc7bde6abd81 with result from task 1575 parsl.dataflow.dflow DEBUG: Task 1575 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/34c22ff2-4c41-40f9-8190-3ac4730ca050_isrForCrosstalkSources_997026_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/34c22ff2-4c41-40f9-8190-3ac4730ca050_isrForCrosstalkSources_997026_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:44 proc_lsst.multi:146 [INFO] found job 17075109 in provider checkpoint proc_lsst.multi INFO: found job 17075109 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1603 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9600bde5d9786c3fd78248358953418 with result from task 1603 parsl.dataflow.dflow DEBUG: Task 1603 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/01bd91ef-6c22-4f30-9ecd-a37d3b7daf18_isrForCrosstalkSources_997012_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/01bd91ef-6c22-4f30-9ecd-a37d3b7daf18_isrForCrosstalkSources_997012_56.stderr parsl.dataflow.dflow INFO: Task 1550 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 05b9d1004f74e8999571dc6a624d40d7 with result from task 1550 parsl.dataflow.dflow DEBUG: Task 1550 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/60c30f39-44c7-48c6-9fc1-f0f0b4e5e121_isrForCrosstalkSources_996996_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/60c30f39-44c7-48c6-9fc1-f0f0b4e5e121_isrForCrosstalkSources_996996_8.stderr parsl.dataflow.dflow INFO: Task 1578 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9f4b9c03b0e4b150d66b393a400f11ed with result from task 1578 parsl.dataflow.dflow DEBUG: Task 1578 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/f6b9e0e9-c1e4-42ef-9283-3e16d0fd1072_isrForCrosstalkSources_996976_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/f6b9e0e9-c1e4-42ef-9283-3e16d0fd1072_isrForCrosstalkSources_996976_20.stderr parsl.dataflow.dflow INFO: Task 1582 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e982bfef72b6fbd0f84042caab089164 with result from task 1582 parsl.dataflow.dflow DEBUG: Task 1582 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/3224f670-f29e-44b7-85c7-484e108e827c_isrForCrosstalkSources_997009_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/3224f670-f29e-44b7-85c7-484e108e827c_isrForCrosstalkSources_997009_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1613 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry bf7bb336d43e91aadc8d23f315cc989f with result from task 1613 parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1613 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/af5ebe60-181e-4c74-b60b-5242a29dc514_isrForCrosstalkSources_996971_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/af5ebe60-181e-4c74-b60b-5242a29dc514_isrForCrosstalkSources_996971_33.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1454 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry bfb454714d1c3c277c3c2568ad80c59f with result from task 1454 parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1454 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2692bd89-11ba-48f8-9472-1cd715245cec_isrForCrosstalkSources_996995_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2692bd89-11ba-48f8-9472-1cd715245cec_isrForCrosstalkSources_996995_52.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1611 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry bef5fdfa41323bfc3523b93f5907854c with result from task 1611 parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1611 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 1611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a48b467b-3e7d-41e1-949e-da0d520f19ab_isrForCrosstalkSources_997011_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a48b467b-3e7d-41e1-949e-da0d520f19ab_isrForCrosstalkSources_997011_4.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:44 proc_lsst.multi:146 [INFO] found job 17075110 in provider checkpoint proc_lsst.multi INFO: found job 17075110 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1527 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4dfa10f372cb74f401091ade9b367528 with result from task 1527 parsl.dataflow.dflow DEBUG: Task 1527 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b3adb660-ffed-4c68-8d3d-ec5a3b554389_isrForCrosstalkSources_997004_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b3adb660-ffed-4c68-8d3d-ec5a3b554389_isrForCrosstalkSources_997004_4.stderr parsl.dataflow.dflow INFO: Task 1512 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51069c415f667bc763f415f17b0ff6b4 with result from task 1512 parsl.dataflow.dflow DEBUG: Task 1512 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4b396fcf-c1dd-42ab-a5f2-ecadabe3111e_isrForCrosstalkSources_997033_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4b396fcf-c1dd-42ab-a5f2-ecadabe3111e_isrForCrosstalkSources_997033_18.stderr parsl.dataflow.dflow INFO: Task 1547 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a3aacb8b94076bb5280790a681a3732 with result from task 1547 parsl.dataflow.dflow DEBUG: Task 1547 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4dbf54c2-079c-46b0-aa78-828a5b34f1ca_isrForCrosstalkSources_996980_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/4dbf54c2-079c-46b0-aa78-828a5b34f1ca_isrForCrosstalkSources_996980_17.stderr parsl.dataflow.dflow INFO: Task 1503 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8fc59aa15861887142b7f905aec1e6c0 with result from task 1503 parsl.dataflow.dflow DEBUG: Task 1503 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/883fb164-105c-4169-993a-d3fc84d29af6_isrForCrosstalkSources_996991_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/883fb164-105c-4169-993a-d3fc84d29af6_isrForCrosstalkSources_996991_51.stderr parsl.dataflow.dflow INFO: Task 1464 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e859ef9aa5d28521d75dc44574fce99 with result from task 1464 parsl.dataflow.dflow DEBUG: Task 1464 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/d7df6361-4ed5-4a7e-aea0-e743b3a050e6_isrForCrosstalkSources_997012_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/d7df6361-4ed5-4a7e-aea0-e743b3a050e6_isrForCrosstalkSources_997012_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:45 proc_lsst.multi:146 [INFO] found job 17075111 in provider checkpoint proc_lsst.multi INFO: found job 17075111 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1449 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf1d3faf47516e0e8de6a183341615e9 with result from task 1449 parsl.dataflow.dflow DEBUG: Task 1449 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/20b3ea58-0211-4f49-b973-2ac7f3e7aa4e_isrForCrosstalkSources_996954_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/20b3ea58-0211-4f49-b973-2ac7f3e7aa4e_isrForCrosstalkSources_996954_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:46 proc_lsst.multi:146 [INFO] found job 17075112 in provider checkpoint proc_lsst.multi INFO: found job 17075112 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1553 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb08dd00f849dcf1bd3ab6ec6f3da751 with result from task 1553 parsl.dataflow.dflow DEBUG: Task 1553 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7296a330-1653-4095-b881-402fed49a2b3_isrForCrosstalkSources_997027_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7296a330-1653-4095-b881-402fed49a2b3_isrForCrosstalkSources_997027_22.stderr parsl.dataflow.dflow INFO: Task 1416 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d54c84c345104f90bdcbefaa3a74f7a with result from task 1416 parsl.dataflow.dflow DEBUG: Task 1416 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/5c0124ad-68d4-4a50-8c53-972c0cb5d0d4_isrForCrosstalkSources_997008_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/5c0124ad-68d4-4a50-8c53-972c0cb5d0d4_isrForCrosstalkSources_997008_62.stderr parsl.dataflow.dflow INFO: Task 1596 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 948708923ae2b1a81f09f41a66550772 with result from task 1596 parsl.dataflow.dflow DEBUG: Task 1596 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7c30c948-99ab-4287-b74c-5307b159940d_isrForCrosstalkSources_996971_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7c30c948-99ab-4287-b74c-5307b159940d_isrForCrosstalkSources_996971_55.stderr parsl.dataflow.dflow INFO: Task 1543 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 69e95688da02372fd14e0a3cff36011f with result from task 1543 parsl.dataflow.dflow DEBUG: Task 1543 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/17eb8961-712f-4370-b3a8-6885751a0bc7_isrForCrosstalkSources_997010_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/17eb8961-712f-4370-b3a8-6885751a0bc7_isrForCrosstalkSources_997010_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:46 proc_lsst.multi:146 [INFO] found job 17075113 in provider checkpoint proc_lsst.multi INFO: found job 17075113 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1545 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1591e9ab9f01eaf81805b99cb2333b10 with result from task 1545 parsl.dataflow.dflow DEBUG: Task 1545 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/93a47611-db30-47c8-9ce2-d4fdb6bb68f2_isrForCrosstalkSources_996972_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/93a47611-db30-47c8-9ce2-d4fdb6bb68f2_isrForCrosstalkSources_996972_16.stderr parsl.dataflow.dflow INFO: Task 1568 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 410ccd8a9cd860ceab59805d3756887f with result from task 1568 parsl.dataflow.dflow DEBUG: Task 1568 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cc5783b1-6607-4bee-b7dc-fafa8d564835_isrForCrosstalkSources_997015_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cc5783b1-6607-4bee-b7dc-fafa8d564835_isrForCrosstalkSources_997015_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:47 proc_lsst.multi:146 [INFO] found job 17075114 in provider checkpoint proc_lsst.multi INFO: found job 17075114 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1580 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0185c843a42f5970189fc63d8463c48f with result from task 1580 parsl.dataflow.dflow DEBUG: Task 1580 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/dba5bb04-70e3-4b26-ad72-463bf5974ec6_isrForCrosstalkSources_997008_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/dba5bb04-70e3-4b26-ad72-463bf5974ec6_isrForCrosstalkSources_997008_1.stderr parsl.dataflow.dflow INFO: Task 1599 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e4106f200a0dc5ee8442a6a09c3eb47 with result from task 1599 parsl.dataflow.dflow DEBUG: Task 1599 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/bf663ffa-b048-4f73-b990-3e49dd35acb6_isrForCrosstalkSources_996987_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/bf663ffa-b048-4f73-b990-3e49dd35acb6_isrForCrosstalkSources_996987_42.stderr parsl.dataflow.dflow INFO: Task 1607 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry feff27cd1a36a3b5b8140c218663621d with result from task 1607 parsl.dataflow.dflow DEBUG: Task 1607 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/c56ba9d5-f516-480c-bd25-47c0c325682d_isrForCrosstalkSources_997031_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/c56ba9d5-f516-480c-bd25-47c0c325682d_isrForCrosstalkSources_997031_19.stderr parsl.dataflow.dflow INFO: Task 1581 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9bfc549fd8a44dd192f8a9185f1d1f10 with result from task 1581 parsl.dataflow.dflow DEBUG: Task 1581 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7285f41b-f367-4648-a71c-e6eb354175df_isrForCrosstalkSources_997015_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7285f41b-f367-4648-a71c-e6eb354175df_isrForCrosstalkSources_997015_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:48 proc_lsst.multi:146 [INFO] found job 17075115 in provider checkpoint proc_lsst.multi INFO: found job 17075115 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1616 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8eac9856d1a14f9c120631f34d062639 with result from task 1616 parsl.dataflow.dflow DEBUG: Task 1616 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/3816b224-72b8-4009-8f82-3275ffeea8b6_isrForCrosstalkSources_996965_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/3816b224-72b8-4009-8f82-3275ffeea8b6_isrForCrosstalkSources_996965_24.stderr parsl.dataflow.dflow INFO: Task 1540 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5abe094ff11c74045bc4a82dd2055e68 with result from task 1540 parsl.dataflow.dflow DEBUG: Task 1540 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d19de0a3-65cc-46c6-a29d-e4c88f8cf0d3_isrForCrosstalkSources_996981_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d19de0a3-65cc-46c6-a29d-e4c88f8cf0d3_isrForCrosstalkSources_996981_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:48 proc_lsst.multi:146 [INFO] found job 17075116 in provider checkpoint proc_lsst.multi INFO: found job 17075116 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1536 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0770464da6894da510b1f213050fd32f with result from task 1536 parsl.dataflow.dflow DEBUG: Task 1536 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/321be0e8-acab-4d5c-8fb7-6f6b44023c2a_isrForCrosstalkSources_996999_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/321be0e8-acab-4d5c-8fb7-6f6b44023c2a_isrForCrosstalkSources_996999_52.stderr parsl.dataflow.dflow INFO: Task 1372 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 74f8079f09249597010249d33e1d569d with result from task 1372 parsl.dataflow.dflow DEBUG: Task 1372 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4bfab800-d624-4aef-9776-38f9e611cab3_isrForCrosstalkSources_997021_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4bfab800-d624-4aef-9776-38f9e611cab3_isrForCrosstalkSources_997021_58.stderr parsl.dataflow.dflow INFO: Task 1493 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 686d2a7bee110a3bebe495945f222d06 with result from task 1493 parsl.dataflow.dflow DEBUG: Task 1493 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/65963caf-d448-41c9-a4b7-29672731979e_isrForCrosstalkSources_997006_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/65963caf-d448-41c9-a4b7-29672731979e_isrForCrosstalkSources_997006_7.stderr parsl.dataflow.dflow INFO: Task 1559 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d3c5218a4bed04a1e8de158d239c3bc with result from task 1559 parsl.dataflow.dflow DEBUG: Task 1559 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/3900dbd7-e0d0-4703-844f-33bd350b6f6e_isrForCrosstalkSources_997016_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/3900dbd7-e0d0-4703-844f-33bd350b6f6e_isrForCrosstalkSources_997016_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:49 proc_lsst.multi:146 [INFO] found job 17075117 in provider checkpoint proc_lsst.multi INFO: found job 17075117 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1555 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f72ee2868443c5685a8c89469ceaa715 with result from task 1555 parsl.dataflow.dflow DEBUG: Task 1555 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/53993f1f-7aa3-44a2-b430-7ca849750263_isrForCrosstalkSources_996967_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/53993f1f-7aa3-44a2-b430-7ca849750263_isrForCrosstalkSources_996967_8.stderr parsl.dataflow.dflow INFO: Task 1614 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94a8c386493608c679f97f98d5426d34 with result from task 1614 parsl.dataflow.dflow DEBUG: Task 1614 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/a4bf2e22-cd8d-4cd3-87a8-f2be728c423a_isrForCrosstalkSources_997003_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/a4bf2e22-cd8d-4cd3-87a8-f2be728c423a_isrForCrosstalkSources_997003_45.stderr parsl.dataflow.dflow INFO: Task 1622 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 952281a98f0527848293d4ecc9ba6c23 with result from task 1622 parsl.dataflow.dflow DEBUG: Task 1622 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/315e9df4-9e9b-41cf-966b-9da910761a9c_isrForCrosstalkSources_997013_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/315e9df4-9e9b-41cf-966b-9da910761a9c_isrForCrosstalkSources_997013_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:50 proc_lsst.multi:146 [INFO] found job 17075118 in provider checkpoint proc_lsst.multi INFO: found job 17075118 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1600 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23e0dbcdb9a6c2d4af87aab15babda9d with result from task 1600 parsl.dataflow.dflow DEBUG: Task 1600 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b729d635-f3a7-4e89-8ec1-ecfb3711b40d_isrForCrosstalkSources_997014_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b729d635-f3a7-4e89-8ec1-ecfb3711b40d_isrForCrosstalkSources_997014_17.stderr parsl.dataflow.dflow INFO: Task 1516 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ba3f5ef1bc48daf230782e79ba68dd7 with result from task 1516 parsl.dataflow.dflow DEBUG: Task 1516 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e7f1f3fc-f164-4c6e-a69c-8158ab6642f7_isrForCrosstalkSources_997028_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e7f1f3fc-f164-4c6e-a69c-8158ab6642f7_isrForCrosstalkSources_997028_11.stderr parsl.dataflow.dflow INFO: Task 1495 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89273111cc62e1ab36880d03d633efce with result from task 1495 parsl.dataflow.dflow DEBUG: Task 1495 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/97f80376-cb70-42c7-bc52-4130d3278fa5_isrForCrosstalkSources_997007_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/97f80376-cb70-42c7-bc52-4130d3278fa5_isrForCrosstalkSources_997007_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:50 proc_lsst.multi:146 [INFO] found job 17075121 in provider checkpoint proc_lsst.multi INFO: found job 17075121 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1490 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b7bd26232f3f38b15920b34a314ca49 with result from task 1490 parsl.dataflow.dflow DEBUG: Task 1490 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/498826c6-ad2c-4dcc-9b29-b2301f0b376f_isrForCrosstalkSources_996983_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/498826c6-ad2c-4dcc-9b29-b2301f0b376f_isrForCrosstalkSources_996983_28.stderr parsl.dataflow.dflow INFO: Task 1524 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10ac69014bf1520beb16458ffdd2e570 with result from task 1524 parsl.dataflow.dflow DEBUG: Task 1524 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c496dd36-8dcf-40c2-bc41-298231aca09b_isrForCrosstalkSources_997016_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c496dd36-8dcf-40c2-bc41-298231aca09b_isrForCrosstalkSources_997016_38.stderr parsl.dataflow.dflow INFO: Task 1474 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 934095b8a9f2f79992f36c27b48c2df7 with result from task 1474 parsl.dataflow.dflow DEBUG: Task 1474 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a53f53ef-c6c5-4bb7-a15d-e032a9f8478d_isrForCrosstalkSources_996952_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a53f53ef-c6c5-4bb7-a15d-e032a9f8478d_isrForCrosstalkSources_996952_62.stderr parsl.dataflow.dflow INFO: Task 1528 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a1ddd1969ae20081b411ec82e78ae8c with result from task 1528 parsl.dataflow.dflow DEBUG: Task 1528 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/cfee817f-d45a-462a-9078-c13d4163f574_isrForCrosstalkSources_996981_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/cfee817f-d45a-462a-9078-c13d4163f574_isrForCrosstalkSources_996981_35.stderr parsl.dataflow.dflow INFO: Task 1572 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c9d33efb2c2a3757a547915aa96f73b with result from task 1572 parsl.dataflow.dflow DEBUG: Task 1572 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/10a653c3-7cb9-4a2a-aa2b-88876f74d6bc_isrForCrosstalkSources_997011_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/10a653c3-7cb9-4a2a-aa2b-88876f74d6bc_isrForCrosstalkSources_997011_35.stderr parsl.dataflow.dflow INFO: Task 1565 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a8e541b2fd46f35fe078b0f08b1907a with result from task 1565 parsl.dataflow.dflow DEBUG: Task 1565 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ef82f429-f0c1-400e-a850-740f7a502707_isrForCrosstalkSources_997021_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ef82f429-f0c1-400e-a850-740f7a502707_isrForCrosstalkSources_997021_21.stderr parsl.dataflow.dflow INFO: Task 1461 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 125507546757ef2ffec96d26caa92fb8 with result from task 1461 parsl.dataflow.dflow DEBUG: Task 1461 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9e06c4bd-e478-4f6f-97e4-18168a27a93e_isrForCrosstalkSources_996986_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/9e06c4bd-e478-4f6f-97e4-18168a27a93e_isrForCrosstalkSources_996986_53.stderr parsl.dataflow.dflow INFO: Task 1598 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26933d1ba5e6bdae3197ceb0b427a0c3 with result from task 1598 parsl.dataflow.dflow DEBUG: Task 1598 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/dd6dcfdc-cf4e-4b4c-8634-1fc390b94de2_isrForCrosstalkSources_997002_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/dd6dcfdc-cf4e-4b4c-8634-1fc390b94de2_isrForCrosstalkSources_997002_55.stderr parsl.dataflow.dflow INFO: Task 1633 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 003c4c8bbe7858d4a26067c00d6ea209 with result from task 1633 parsl.dataflow.dflow DEBUG: Task 1633 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/5bea4852-f4e3-4f43-b6a0-cfe8655c2f3b_isrForCrosstalkSources_997002_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/5bea4852-f4e3-4f43-b6a0-cfe8655c2f3b_isrForCrosstalkSources_997002_31.stderr parsl.dataflow.dflow INFO: Task 1641 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b986805adb9c60da3382308c582becb with result from task 1641 parsl.dataflow.dflow DEBUG: Task 1641 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/3e18b4ab-d886-491d-9e64-0cfab1a2e09a_isrForCrosstalkSources_997004_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/3e18b4ab-d886-491d-9e64-0cfab1a2e09a_isrForCrosstalkSources_997004_48.stderr parsl.dataflow.dflow INFO: Task 1484 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e66ae865b008c52f96a909aecdf8c9de with result from task 1484 parsl.dataflow.dflow DEBUG: Task 1484 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/679b074c-5daf-4ede-858e-0ba2a22a2eaf_isrForCrosstalkSources_997002_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/679b074c-5daf-4ede-858e-0ba2a22a2eaf_isrForCrosstalkSources_997002_11.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:51 proc_lsst.multi:146 [INFO] found job 17075122 in provider checkpoint proc_lsst.multi INFO: found job 17075122 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1604 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e45b8d3b7dd25dce7519237b8df9d06f with result from task 1604 parsl.dataflow.dflow DEBUG: Task 1604 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/5ed1126f-07d6-47c7-9b4f-424458ccea50_isrForCrosstalkSources_997020_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/5ed1126f-07d6-47c7-9b4f-424458ccea50_isrForCrosstalkSources_997020_15.stderr parsl.dataflow.dflow INFO: Task 1551 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 897c8e9105061f05dee99b7521511505 with result from task 1551 parsl.dataflow.dflow DEBUG: Task 1551 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8e083bb0-9571-442a-a1ab-f8f5a65885fc_isrForCrosstalkSources_997017_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8e083bb0-9571-442a-a1ab-f8f5a65885fc_isrForCrosstalkSources_997017_37.stderr parsl.dataflow.dflow INFO: Task 1574 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a7f8cd333e7d691bd57050987d9cd116 with result from task 1574 parsl.dataflow.dflow DEBUG: Task 1574 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/9d8077e9-786d-4a65-bb57-9f149422d4c8_isrForCrosstalkSources_997012_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/9d8077e9-786d-4a65-bb57-9f149422d4c8_isrForCrosstalkSources_997012_31.stderr parsl.dataflow.dflow INFO: Task 1605 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c611ca677d2da60d8a9ffafdedd055c7 with result from task 1605 parsl.dataflow.dflow DEBUG: Task 1605 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/74a90412-7cb6-47d3-a69c-7847c05669f4_isrForCrosstalkSources_997016_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/74a90412-7cb6-47d3-a69c-7847c05669f4_isrForCrosstalkSources_997016_36.stderr parsl.dataflow.dflow INFO: Task 1629 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b1a0eb0fc0db5e4d6f7c6491deb77b7 with result from task 1629 parsl.dataflow.dflow DEBUG: Task 1629 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/455fb03a-3d89-4a67-aa49-54c962b95618_isrForCrosstalkSources_997005_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/455fb03a-3d89-4a67-aa49-54c962b95618_isrForCrosstalkSources_997005_5.stderr parsl.dataflow.dflow INFO: Task 1597 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63d93f58f69596fce709a97229d00186 with result from task 1597 parsl.dataflow.dflow DEBUG: Task 1597 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/e9811915-206a-42bf-911e-3764d7b21245_isrForCrosstalkSources_997027_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/e9811915-206a-42bf-911e-3764d7b21245_isrForCrosstalkSources_997027_61.stderr parsl.dataflow.dflow INFO: Task 1630 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 453647e33e868bc421d0727b2b883a54 with result from task 1630 parsl.dataflow.dflow DEBUG: Task 1630 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4964578a-a35a-4c15-b1e1-00a9cacac8a7_isrForCrosstalkSources_996954_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4964578a-a35a-4c15-b1e1-00a9cacac8a7_isrForCrosstalkSources_996954_45.stderr parsl.dataflow.dflow INFO: Task 1668 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f4679c930f4ab87ecf6325b382e5abe with result from task 1668 parsl.dataflow.dflow DEBUG: Task 1668 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6cb1b78a-cc63-48ba-a0d9-f3d5c241646c_isrForCrosstalkSources_996973_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6cb1b78a-cc63-48ba-a0d9-f3d5c241646c_isrForCrosstalkSources_996973_20.stderr parsl.dataflow.dflow INFO: Task 1502 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b87f1736521f009daa0db331bac531f7 with result from task 1502 parsl.dataflow.dflow DEBUG: Task 1502 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/78de333d-ce91-4903-8431-b013b1ef5e29_isrForCrosstalkSources_996955_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/78de333d-ce91-4903-8431-b013b1ef5e29_isrForCrosstalkSources_996955_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1619 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1da0f188a376a071e0aa6f163870ecef with result from task 1619 parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:53 proc_lsst.multi:146 [INFO] found job 17075124 in provider checkpoint proc_lsst.multi INFO: found job 17075124 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 1619 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e143eece-9394-4641-9818-cc73cd9576d8_isrForCrosstalkSources_996984_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e143eece-9394-4641-9818-cc73cd9576d8_isrForCrosstalkSources_996984_7.stderr parsl.dataflow.dflow INFO: Task 1615 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db37331e03295cc14b26a12100673f69 with result from task 1615 parsl.dataflow.dflow DEBUG: Task 1615 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/8c0f21f1-1e61-4d6e-871a-e73fe78a3d78_isrForCrosstalkSources_997030_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/8c0f21f1-1e61-4d6e-871a-e73fe78a3d78_isrForCrosstalkSources_997030_3.stderr parsl.dataflow.dflow INFO: Task 1523 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53889aec963af2e6441843f567a85e2a with result from task 1523 parsl.dataflow.dflow DEBUG: Task 1523 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/6c68607b-2fc0-4ba1-8869-3c2ee49a49ba_isrForCrosstalkSources_996978_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/6c68607b-2fc0-4ba1-8869-3c2ee49a49ba_isrForCrosstalkSources_996978_44.stderr parsl.dataflow.dflow INFO: Task 1567 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b58d6268309cdc351254b348089f296e with result from task 1567 parsl.dataflow.dflow DEBUG: Task 1567 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/468790a7-792d-47df-aae0-e431983b68c6_isrForCrosstalkSources_997008_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/468790a7-792d-47df-aae0-e431983b68c6_isrForCrosstalkSources_997008_60.stderr parsl.dataflow.dflow INFO: Task 1494 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 220c4193615a10f1847c41e5428a0dba with result from task 1494 parsl.dataflow.dflow DEBUG: Task 1494 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/2fe348e2-4856-4a79-ba04-8651fa0e522d_isrForCrosstalkSources_997008_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/2fe348e2-4856-4a79-ba04-8651fa0e522d_isrForCrosstalkSources_997008_28.stderr parsl.dataflow.dflow INFO: Task 1683 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea1c7ce020e2a61451727edd0adb45cf with result from task 1683 parsl.dataflow.dflow DEBUG: Task 1683 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/6bcc8dde-8eeb-480b-9438-75383f256a38_isrForCrosstalkSources_997021_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/6bcc8dde-8eeb-480b-9438-75383f256a38_isrForCrosstalkSources_997021_32.stderr parsl.dataflow.dflow INFO: Task 1517 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d74386a05f0bcccd92275ee02e0e7f84 with result from task 1517 parsl.dataflow.dflow DEBUG: Task 1517 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b71d70af-a38f-4ae2-96a4-7965cf354f6f_isrForCrosstalkSources_996997_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b71d70af-a38f-4ae2-96a4-7965cf354f6f_isrForCrosstalkSources_996997_11.stderr parsl.dataflow.dflow INFO: Task 1443 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a397295d804baf82f60a76cd7c3b042d with result from task 1443 parsl.dataflow.dflow DEBUG: Task 1443 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/5fe2c676-5a2c-4d9c-8c42-5a6b9e8bb768_isrForCrosstalkSources_996989_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/5fe2c676-5a2c-4d9c-8c42-5a6b9e8bb768_isrForCrosstalkSources_996989_61.stderr parsl.dataflow.dflow INFO: Task 1510 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1c096f4253d4092b754033ccfae57a7 with result from task 1510 parsl.dataflow.dflow DEBUG: Task 1510 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/220601d8-b795-46c1-a86a-7a43e5bff073_isrForCrosstalkSources_997002_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/220601d8-b795-46c1-a86a-7a43e5bff073_isrForCrosstalkSources_997002_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:54 proc_lsst.multi:146 [INFO] found job 17075127 in provider checkpoint proc_lsst.multi INFO: found job 17075127 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1515 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b00c67363f3c777f6e065e2abb4a6366 with result from task 1515 parsl.dataflow.dflow DEBUG: Task 1515 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c1d454a6-b8c0-46a3-bd33-9f503cd07154_isrForCrosstalkSources_997020_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c1d454a6-b8c0-46a3-bd33-9f503cd07154_isrForCrosstalkSources_997020_41.stderr parsl.dataflow.dflow INFO: Task 1561 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3bfa7f73ee3355f814cb7c8f689f61f7 with result from task 1561 parsl.dataflow.dflow DEBUG: Task 1561 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/e435dd66-2a59-4a24-93d2-b978de87564c_isrForCrosstalkSources_997004_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/e435dd66-2a59-4a24-93d2-b978de87564c_isrForCrosstalkSources_997004_41.stderr parsl.dataflow.dflow INFO: Task 1544 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac60a1800630d5d4a9ad2d071dbc5e40 with result from task 1544 parsl.dataflow.dflow DEBUG: Task 1544 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/3811c446-afa0-43ae-ba11-88e4be8a89d6_isrForCrosstalkSources_997008_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/3811c446-afa0-43ae-ba11-88e4be8a89d6_isrForCrosstalkSources_997008_18.stderr parsl.dataflow.dflow INFO: Task 1618 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 934d5d6526b3411ea016dadae2a8f610 with result from task 1618 parsl.dataflow.dflow DEBUG: Task 1618 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5ee7713a-d4ea-4bfd-a47a-964f31d04913_isrForCrosstalkSources_996954_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5ee7713a-d4ea-4bfd-a47a-964f31d04913_isrForCrosstalkSources_996954_10.stderr parsl.dataflow.dflow INFO: Task 1546 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19ba207b9afe8e417b53fb7395dc3198 with result from task 1546 parsl.dataflow.dflow DEBUG: Task 1546 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/8151e5cf-f633-42a2-a29f-2a1ed7381226_isrForCrosstalkSources_996965_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/8151e5cf-f633-42a2-a29f-2a1ed7381226_isrForCrosstalkSources_996965_53.stderr parsl.dataflow.dflow INFO: Task 1684 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 340461f418ec653a6ffbe10edda8da15 with result from task 1684 parsl.dataflow.dflow DEBUG: Task 1684 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/cbf41ff3-c23c-4f5a-9c9d-4fa928f72fb2_isrForCrosstalkSources_997019_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/cbf41ff3-c23c-4f5a-9c9d-4fa928f72fb2_isrForCrosstalkSources_997019_1.stderr parsl.dataflow.dflow INFO: Task 1351 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da553d95e50ff5bf96000d73924b9826 with result from task 1351 parsl.dataflow.dflow DEBUG: Task 1351 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4f1f41e1-f1e9-4c1e-bfe7-3011c987c0c8_isrForCrosstalkSources_996959_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4f1f41e1-f1e9-4c1e-bfe7-3011c987c0c8_isrForCrosstalkSources_996959_11.stderr parsl.dataflow.dflow INFO: Task 1350 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c97915f2f45c024f4fac416b45c4534 with result from task 1350 parsl.dataflow.dflow DEBUG: Task 1350 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/256136dd-a48d-44a3-8da9-5b1909b97c9d_isrForCrosstalkSources_996967_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/256136dd-a48d-44a3-8da9-5b1909b97c9d_isrForCrosstalkSources_996967_26.stderr parsl.dataflow.dflow INFO: Task 1347 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55fe375909be34a3e85fd20f498b1d8e with result from task 1347 parsl.dataflow.dflow DEBUG: Task 1347 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/cd8857a7-8aa7-41a1-b2d3-f764e8851472_isrForCrosstalkSources_996960_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/cd8857a7-8aa7-41a1-b2d3-f764e8851472_isrForCrosstalkSources_996960_52.stderr parsl.dataflow.dflow INFO: Task 1346 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec53e2838f05ee31001a72cc3b38055e with result from task 1346 parsl.dataflow.dflow DEBUG: Task 1346 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/25293d16-91fd-4d4c-baf8-dbdb94feb743_isrForCrosstalkSources_996983_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/25293d16-91fd-4d4c-baf8-dbdb94feb743_isrForCrosstalkSources_996983_9.stderr parsl.dataflow.dflow INFO: Task 1348 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 237293d1f36a18f0f25ea1da8cee5cc5 with result from task 1348 parsl.dataflow.dflow DEBUG: Task 1348 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/89e4ecd4-9bd8-45a6-85cd-44233c76f46a_isrForCrosstalkSources_997029_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/89e4ecd4-9bd8-45a6-85cd-44233c76f46a_isrForCrosstalkSources_997029_1.stderr parsl.dataflow.dflow INFO: Task 1352 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3396a26583fb5acee48583661489960e with result from task 1352 parsl.dataflow.dflow DEBUG: Task 1352 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/6443bc6e-8ec1-4dc2-be77-dc03fae7b3ad_isrForCrosstalkSources_997019_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/6443bc6e-8ec1-4dc2-be77-dc03fae7b3ad_isrForCrosstalkSources_997019_61.stderr parsl.dataflow.dflow INFO: Task 1349 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 043383cd307a4810ce23a2be9322d487 with result from task 1349 parsl.dataflow.dflow DEBUG: Task 1349 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/6a86c5e7-98d3-426b-bcdc-b5ed53688e4a_isrForCrosstalkSources_997026_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/6a86c5e7-98d3-426b-bcdc-b5ed53688e4a_isrForCrosstalkSources_997026_38.stderr parsl.dataflow.dflow INFO: Task 1353 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ea9d9de9a8b85ad1f7251af80a60c01 with result from task 1353 parsl.dataflow.dflow DEBUG: Task 1353 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/82e6cc5e-8ddf-4a36-956f-6545f3d0d008_isrForCrosstalkSources_997032_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/82e6cc5e-8ddf-4a36-956f-6545f3d0d008_isrForCrosstalkSources_997032_43.stderr parsl.dataflow.dflow INFO: Task 1678 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa89d477aea802ba7d008421a89647b5 with result from task 1678 parsl.dataflow.dflow DEBUG: Task 1678 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/22e51beb-c8fd-46bd-8654-108a8677749c_isrForCrosstalkSources_996970_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/22e51beb-c8fd-46bd-8654-108a8677749c_isrForCrosstalkSources_996970_17.stderr parsl.dataflow.dflow INFO: Task 1554 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75f46894bda3569358559ec2df282be4 with result from task 1554 parsl.dataflow.dflow DEBUG: Task 1554 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/3d4546bb-a1c9-45ab-bc38-5e926f677a1d_isrForCrosstalkSources_996962_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/3d4546bb-a1c9-45ab-bc38-5e926f677a1d_isrForCrosstalkSources_996962_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:55 proc_lsst.multi:146 [INFO] found job 17075128 in provider checkpoint proc_lsst.multi INFO: found job 17075128 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1594 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a8f8855a72fe9fde7b38b3993784def with result from task 1594 parsl.dataflow.dflow DEBUG: Task 1594 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0b014c23-ef61-4170-a2e8-912968de9ce1_isrForCrosstalkSources_996991_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0b014c23-ef61-4170-a2e8-912968de9ce1_isrForCrosstalkSources_996991_38.stderr parsl.dataflow.dflow INFO: Task 1526 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c5e3486c353ef4857c1f2730cb7b24d with result from task 1526 parsl.dataflow.dflow DEBUG: Task 1526 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/b242fe10-067a-435f-891a-2790a3f90bc4_isrForCrosstalkSources_997010_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/b242fe10-067a-435f-891a-2790a3f90bc4_isrForCrosstalkSources_997010_41.stderr parsl.dataflow.dflow INFO: Task 1680 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 039e44b4fd6786ee4a9132666dc339e2 with result from task 1680 parsl.dataflow.dflow DEBUG: Task 1680 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/98437ad4-b927-438f-b50c-e20677340dfc_isrForCrosstalkSources_996961_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/98437ad4-b927-438f-b50c-e20677340dfc_isrForCrosstalkSources_996961_39.stderr parsl.dataflow.dflow INFO: Task 1479 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19bbaf1e3be743f7a02b617ae6b3124a with result from task 1479 parsl.dataflow.dflow DEBUG: Task 1479 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4e6458bd-8b01-42da-b302-2dd80ab28eb2_isrForCrosstalkSources_997033_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4e6458bd-8b01-42da-b302-2dd80ab28eb2_isrForCrosstalkSources_997033_15.stderr parsl.dataflow.dflow INFO: Task 1562 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc77f8bfc0ec4d1ee2af5279153ab281 with result from task 1562 parsl.dataflow.dflow DEBUG: Task 1562 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/e2adafdd-ddba-4052-a601-b5bbbfeaa4db_isrForCrosstalkSources_997027_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/e2adafdd-ddba-4052-a601-b5bbbfeaa4db_isrForCrosstalkSources_997027_49.stderr parsl.dataflow.dflow INFO: Task 1576 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fe5246450f34bc7b30cba0da7783604 with result from task 1576 parsl.dataflow.dflow DEBUG: Task 1576 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/bcd792d9-a55b-4247-8bde-90b789b415bf_isrForCrosstalkSources_997001_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/bcd792d9-a55b-4247-8bde-90b789b415bf_isrForCrosstalkSources_997001_30.stderr parsl.dataflow.dflow INFO: Task 1592 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a8c777204cbff8a551b839f06c13c69 with result from task 1592 parsl.dataflow.dflow DEBUG: Task 1592 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/92bbfd61-7ec0-49fa-9652-e12675d9586d_isrForCrosstalkSources_996990_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/92bbfd61-7ec0-49fa-9652-e12675d9586d_isrForCrosstalkSources_996990_22.stderr parsl.dataflow.dflow INFO: Task 1686 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4bfe0242d0fb095cd19a27f87c1e5efa with result from task 1686 parsl.dataflow.dflow DEBUG: Task 1686 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/61a59458-51c6-45d4-a240-cf1975b5fc0c_isrForCrosstalkSources_996983_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/61a59458-51c6-45d4-a240-cf1975b5fc0c_isrForCrosstalkSources_996983_18.stderr parsl.dataflow.dflow INFO: Task 1632 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 058332e4fafeae593b966d8a593fb20c with result from task 1632 parsl.dataflow.dflow DEBUG: Task 1632 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/31fd8b32-87b2-4428-9041-5c8c79c10815_isrForCrosstalkSources_996973_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/31fd8b32-87b2-4428-9041-5c8c79c10815_isrForCrosstalkSources_996973_14.stderr parsl.dataflow.dflow INFO: Task 1531 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 14c7a52c01d3fdb26cdefdfae99d9875 with result from task 1531 parsl.dataflow.dflow DEBUG: Task 1531 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/271b2f55-4281-46c6-9060-dec1c8cbf0cd_isrForCrosstalkSources_996972_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/271b2f55-4281-46c6-9060-dec1c8cbf0cd_isrForCrosstalkSources_996972_52.stderr parsl.dataflow.dflow INFO: Task 1609 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94ce2abb51194a1fb72addc5af1bcc4d with result from task 1609 parsl.dataflow.dflow DEBUG: Task 1609 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/dd25be28-b1ba-4477-b85b-47e03b5f04ea_isrForCrosstalkSources_996973_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/dd25be28-b1ba-4477-b85b-47e03b5f04ea_isrForCrosstalkSources_996973_21.stderr parsl.dataflow.dflow INFO: Task 1508 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 044a4dd6ae8bc49e57dc0331657eb41c with result from task 1508 parsl.dataflow.dflow DEBUG: Task 1508 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/612a3ca7-5e6b-4379-86dc-1626b2178a9c_isrForCrosstalkSources_996985_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/612a3ca7-5e6b-4379-86dc-1626b2178a9c_isrForCrosstalkSources_996985_7.stderr parsl.dataflow.dflow INFO: Task 1482 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4287ac668171ab145f786434e1e5a7d4 with result from task 1482 parsl.dataflow.dflow DEBUG: Task 1482 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/cffa0df5-8126-4621-9b61-f83919bbf476_isrForCrosstalkSources_996972_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/cffa0df5-8126-4621-9b61-f83919bbf476_isrForCrosstalkSources_996972_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:56 proc_lsst.multi:146 [INFO] found job 17075129 in provider checkpoint proc_lsst.multi INFO: found job 17075129 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1499 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21da6a01c2bd438064f2140253032549 with result from task 1499 parsl.dataflow.dflow DEBUG: Task 1499 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5c619962-58fd-4a36-ba6f-9d92a229547c_isrForCrosstalkSources_996991_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5c619962-58fd-4a36-ba6f-9d92a229547c_isrForCrosstalkSources_996991_43.stderr parsl.dataflow.dflow INFO: Task 1465 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4cf7670220d3412984a824c36a86afc2 with result from task 1465 parsl.dataflow.dflow DEBUG: Task 1465 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/60687ebc-5ce1-4411-b6ea-651b43960473_isrForCrosstalkSources_996976_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/60687ebc-5ce1-4411-b6ea-651b43960473_isrForCrosstalkSources_996976_28.stderr parsl.dataflow.dflow INFO: Task 1675 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1cd8c2e95fdb899a5bcb5dc0ebaf6c4b with result from task 1675 parsl.dataflow.dflow DEBUG: Task 1675 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/dac05893-42da-4a57-b8ad-ebcd4b12577c_isrForCrosstalkSources_996994_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/dac05893-42da-4a57-b8ad-ebcd4b12577c_isrForCrosstalkSources_996994_58.stderr parsl.dataflow.dflow INFO: Task 1541 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 327b9fe2df0e279ae77194d7556f5d9a with result from task 1541 parsl.dataflow.dflow DEBUG: Task 1541 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/e27b060b-d7ca-413b-9e6a-5f38d5661d27_isrForCrosstalkSources_996988_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/e27b060b-d7ca-413b-9e6a-5f38d5661d27_isrForCrosstalkSources_996988_36.stderr parsl.dataflow.dflow INFO: Task 1519 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc45015dd62a56ecc8056f9d54ebc87a with result from task 1519 parsl.dataflow.dflow DEBUG: Task 1519 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4ec7c032-2423-4f8c-86b0-e66cdbd1aa42_isrForCrosstalkSources_996987_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4ec7c032-2423-4f8c-86b0-e66cdbd1aa42_isrForCrosstalkSources_996987_59.stderr parsl.dataflow.dflow INFO: Task 1631 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba27ec4e09ef8e5d3d85ab73b92221ac with result from task 1631 parsl.dataflow.dflow DEBUG: Task 1631 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/2bd22fe3-4a33-4d14-880d-e88ca92bc0b9_isrForCrosstalkSources_997009_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/2bd22fe3-4a33-4d14-880d-e88ca92bc0b9_isrForCrosstalkSources_997009_55.stderr parsl.dataflow.dflow INFO: Task 1521 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d487ba2d495dced271dadee9dcad5949 with result from task 1521 parsl.dataflow.dflow DEBUG: Task 1521 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9ecc5a08-a3bc-419e-8923-20c920fd99ac_isrForCrosstalkSources_996971_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9ecc5a08-a3bc-419e-8923-20c920fd99ac_isrForCrosstalkSources_996971_30.stderr parsl.dataflow.dflow INFO: Task 1610 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cdba0a9893bdcfe8900cd13bd1fe22cc with result from task 1610 parsl.dataflow.dflow DEBUG: Task 1610 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/f0c714ae-5b4a-4462-866e-a6de713c0590_isrForCrosstalkSources_997022_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/f0c714ae-5b4a-4462-866e-a6de713c0590_isrForCrosstalkSources_997022_12.stderr parsl.dataflow.dflow INFO: Task 1690 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ed0bc702a4c7011441fa887a3941453 with result from task 1690 parsl.dataflow.dflow DEBUG: Task 1690 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2c6a3aac-93b8-4cf6-aabf-2ae10f634324_isrForCrosstalkSources_996969_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2c6a3aac-93b8-4cf6-aabf-2ae10f634324_isrForCrosstalkSources_996969_58.stderr parsl.dataflow.dflow INFO: Task 1670 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8ac065e0f2b0723b9c8a10e876ebeaf with result from task 1670 parsl.dataflow.dflow DEBUG: Task 1670 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/b63862d9-4959-4147-ad3c-ea2c05479ff9_isrForCrosstalkSources_997000_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/b63862d9-4959-4147-ad3c-ea2c05479ff9_isrForCrosstalkSources_997000_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:57 proc_lsst.multi:146 [INFO] found job 17075131 in provider checkpoint proc_lsst.multi INFO: found job 17075131 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1692 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8733450461e969cc5ae35840ab5e0724 with result from task 1692 parsl.dataflow.dflow DEBUG: Task 1692 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8a1a37e7-7043-43e1-ae16-11df66258bc0_isrForCrosstalkSources_996987_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8a1a37e7-7043-43e1-ae16-11df66258bc0_isrForCrosstalkSources_996987_2.stderr parsl.dataflow.dflow INFO: Task 1420 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d9c4771e09c21ed5c586e4e5236b023 with result from task 1420 parsl.dataflow.dflow DEBUG: Task 1420 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ebb7c666-a18b-4203-9f77-9da2e8301326_isrForCrosstalkSources_996986_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ebb7c666-a18b-4203-9f77-9da2e8301326_isrForCrosstalkSources_996986_52.stderr parsl.dataflow.dflow INFO: Task 1549 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 193f96c7f492ae44e11ea03e62bb55ca with result from task 1549 parsl.dataflow.dflow DEBUG: Task 1549 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/b08586b8-d960-4e08-a6e1-05a6179eb6f9_isrForCrosstalkSources_997019_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/b08586b8-d960-4e08-a6e1-05a6179eb6f9_isrForCrosstalkSources_997019_57.stderr parsl.dataflow.dflow INFO: Task 1488 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2120924a7db01a6810a1cdc8049a0b63 with result from task 1488 parsl.dataflow.dflow DEBUG: Task 1488 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9c070e48-49c2-4c60-8a54-86a5057b42ac_isrForCrosstalkSources_997008_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9c070e48-49c2-4c60-8a54-86a5057b42ac_isrForCrosstalkSources_997008_54.stderr parsl.dataflow.dflow INFO: Task 1466 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3a6713910102f057b5d05b2b288a3bf with result from task 1466 parsl.dataflow.dflow DEBUG: Task 1466 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ce9c5f99-8541-4044-9e74-ea6c19afa2c1_isrForCrosstalkSources_996992_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ce9c5f99-8541-4044-9e74-ea6c19afa2c1_isrForCrosstalkSources_996992_3.stderr parsl.dataflow.dflow INFO: Task 1649 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 303813636b5461844b7f2aeafffce737 with result from task 1649 parsl.dataflow.dflow DEBUG: Task 1649 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b6c25168-e813-44ec-b227-51948e9b0231_isrForCrosstalkSources_996992_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b6c25168-e813-44ec-b227-51948e9b0231_isrForCrosstalkSources_996992_31.stderr parsl.dataflow.dflow INFO: Task 1606 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da81b1e6e05fb94a318c550a40a7de96 with result from task 1606 parsl.dataflow.dflow DEBUG: Task 1606 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/95b45f0d-38d3-43ff-93d2-ff3e8c61088f_isrForCrosstalkSources_997025_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/95b45f0d-38d3-43ff-93d2-ff3e8c61088f_isrForCrosstalkSources_997025_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:58 proc_lsst.multi:146 [INFO] found job 17075132 in provider checkpoint proc_lsst.multi INFO: found job 17075132 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1583 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27e548bb79eaf103a1ebf46d9b4c74e9 with result from task 1583 parsl.dataflow.dflow DEBUG: Task 1583 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/5793d7dd-81a7-43f2-945a-78f3cb17889d_isrForCrosstalkSources_996975_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/5793d7dd-81a7-43f2-945a-78f3cb17889d_isrForCrosstalkSources_996975_31.stderr parsl.dataflow.dflow INFO: Task 1625 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85bc9fc59be21b5cb22fb09783660dc5 with result from task 1625 parsl.dataflow.dflow DEBUG: Task 1625 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/f3b27d9d-fc5b-4d50-9f6b-2c7586f0be90_isrForCrosstalkSources_996983_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/f3b27d9d-fc5b-4d50-9f6b-2c7586f0be90_isrForCrosstalkSources_996983_42.stderr parsl.dataflow.dflow INFO: Task 1708 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1286f16bb8155342c997d39f83083e09 with result from task 1708 parsl.dataflow.dflow DEBUG: Task 1708 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6196fbff-c913-4962-94e8-731815216018_isrForCrosstalkSources_997003_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6196fbff-c913-4962-94e8-731815216018_isrForCrosstalkSources_997003_20.stderr parsl.dataflow.dflow INFO: Task 1534 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 840f74ad5c4584299b2b01971ea82ee8 with result from task 1534 parsl.dataflow.dflow DEBUG: Task 1534 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6bb6fe36-71a2-48d7-9888-85aa96d9603c_isrForCrosstalkSources_997005_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6bb6fe36-71a2-48d7-9888-85aa96d9603c_isrForCrosstalkSources_997005_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:54:59 proc_lsst.multi:146 [INFO] found job 17075133 in provider checkpoint proc_lsst.multi INFO: found job 17075133 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1563 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de09325355183b7c070c257eec9e562d with result from task 1563 parsl.dataflow.dflow DEBUG: Task 1563 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/29122da0-7dc6-4e24-9d26-76d4794b3684_isrForCrosstalkSources_996971_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/29122da0-7dc6-4e24-9d26-76d4794b3684_isrForCrosstalkSources_996971_44.stderr parsl.dataflow.dflow INFO: Task 1481 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d0942cce8670882e8b3a5edcdbeb85a with result from task 1481 parsl.dataflow.dflow DEBUG: Task 1481 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/c1e35aa5-9e54-4994-804d-e41e5c0258cf_isrForCrosstalkSources_996984_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/c1e35aa5-9e54-4994-804d-e41e5c0258cf_isrForCrosstalkSources_996984_43.stderr parsl.dataflow.dflow INFO: Task 1696 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b3088a96421b36d558643924b6c789cb with result from task 1696 parsl.dataflow.dflow DEBUG: Task 1696 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/136e5b05-366c-4737-a2cc-6d2dc62bf13e_isrForCrosstalkSources_996980_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/136e5b05-366c-4737-a2cc-6d2dc62bf13e_isrForCrosstalkSources_996980_31.stderr parsl.dataflow.dflow INFO: Task 1509 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 724a40f85f293d693f15551d1674b59c with result from task 1509 parsl.dataflow.dflow DEBUG: Task 1509 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/7887f3d5-b59c-459a-923d-2a2889bff98e_isrForCrosstalkSources_996987_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/7887f3d5-b59c-459a-923d-2a2889bff98e_isrForCrosstalkSources_996987_53.stderr parsl.dataflow.dflow INFO: Task 1564 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab6cc377acaabf1cba376e83aa2a2212 with result from task 1564 parsl.dataflow.dflow DEBUG: Task 1564 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0079eb0a-d9ce-49c4-aa21-e89a0cecd990_isrForCrosstalkSources_997010_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0079eb0a-d9ce-49c4-aa21-e89a0cecd990_isrForCrosstalkSources_997010_29.stderr parsl.dataflow.dflow INFO: Task 1507 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 946e1ab95fb5bab3db3595e0fc79e19f with result from task 1507 parsl.dataflow.dflow DEBUG: Task 1507 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/83f6a31f-5a73-4f85-8fca-04d1f5b8aa2d_isrForCrosstalkSources_996955_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/83f6a31f-5a73-4f85-8fca-04d1f5b8aa2d_isrForCrosstalkSources_996955_57.stderr parsl.dataflow.dflow INFO: Task 1577 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8fac901427a692b62bda40de65c3e463 with result from task 1577 parsl.dataflow.dflow DEBUG: Task 1577 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a9459040-d0d5-41b4-b099-4fab03b41a5c_isrForCrosstalkSources_996996_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a9459040-d0d5-41b4-b099-4fab03b41a5c_isrForCrosstalkSources_996996_49.stderr parsl.dataflow.dflow INFO: Task 1504 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04e964ba407941235a8cf11cf988539b with result from task 1504 parsl.dataflow.dflow DEBUG: Task 1504 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dd8aff54-7c8c-45bb-a051-a01d73a37851_isrForCrosstalkSources_997001_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dd8aff54-7c8c-45bb-a051-a01d73a37851_isrForCrosstalkSources_997001_43.stderr parsl.dataflow.dflow INFO: Task 1623 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93f54e062526e11cbf8157d279ec51fc with result from task 1623 parsl.dataflow.dflow DEBUG: Task 1623 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/25357b8a-d47d-4070-ade7-5ea411c6a599_isrForCrosstalkSources_996965_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/25357b8a-d47d-4070-ade7-5ea411c6a599_isrForCrosstalkSources_996965_43.stderr parsl.dataflow.dflow INFO: Task 1624 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f856dd9d2469c8a5339a80140a4b3b07 with result from task 1624 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:00 proc_lsst.multi:146 [INFO] found job 17075134 in provider checkpoint proc_lsst.multi INFO: found job 17075134 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 1624 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6148973a-0263-4ecb-b13f-bd4c014232d3_isrForCrosstalkSources_997017_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/6148973a-0263-4ecb-b13f-bd4c014232d3_isrForCrosstalkSources_997017_31.stderr parsl.dataflow.dflow INFO: Task 1513 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e2a5fd765808667a4271c3bda3269bd with result from task 1513 parsl.dataflow.dflow DEBUG: Task 1513 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/baff17c7-ee9a-4916-a827-ea1c16de5daf_isrForCrosstalkSources_996953_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/baff17c7-ee9a-4916-a827-ea1c16de5daf_isrForCrosstalkSources_996953_50.stderr parsl.dataflow.dflow INFO: Task 1533 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 35c5e302b542e7a37104240765a24979 with result from task 1533 parsl.dataflow.dflow DEBUG: Task 1533 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/cfc544f9-1a9e-47b2-980b-eeba3de2e94a_isrForCrosstalkSources_996966_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/cfc544f9-1a9e-47b2-980b-eeba3de2e94a_isrForCrosstalkSources_996966_57.stderr parsl.dataflow.dflow INFO: Task 1579 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8482fd2a76b058114f7d4626096cb4b6 with result from task 1579 parsl.dataflow.dflow DEBUG: Task 1579 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1186c2d8-8727-48fa-829f-16fc3bbbe05a_isrForCrosstalkSources_997000_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1186c2d8-8727-48fa-829f-16fc3bbbe05a_isrForCrosstalkSources_997000_35.stderr parsl.dataflow.dflow INFO: Task 1506 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a74d75ab1229fd41201cfecc03f6305 with result from task 1506 parsl.dataflow.dflow DEBUG: Task 1506 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/65138cc8-3ee6-4c68-b338-c4b8e377c316_isrForCrosstalkSources_996990_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/65138cc8-3ee6-4c68-b338-c4b8e377c316_isrForCrosstalkSources_996990_54.stderr parsl.dataflow.dflow INFO: Task 1612 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1295295b9bcbbbea637d71d0f7183261 with result from task 1612 parsl.dataflow.dflow DEBUG: Task 1612 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/57add925-48ba-4c39-92b3-7a8eadc94700_isrForCrosstalkSources_996987_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/57add925-48ba-4c39-92b3-7a8eadc94700_isrForCrosstalkSources_996987_62.stderr parsl.dataflow.dflow INFO: Task 1532 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef42ed858502f986e9c8c963ddc85a85 with result from task 1532 parsl.dataflow.dflow DEBUG: Task 1532 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/84ad607d-58dc-40a6-a7a7-61f07279a1b5_isrForCrosstalkSources_996954_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/84ad607d-58dc-40a6-a7a7-61f07279a1b5_isrForCrosstalkSources_996954_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:01 proc_lsst.multi:146 [INFO] found job 17075135 in provider checkpoint proc_lsst.multi INFO: found job 17075135 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1628 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b697a203244cc5fabdca8dadb04c6a74 with result from task 1628 parsl.dataflow.dflow DEBUG: Task 1628 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/4bc7eae2-467b-4cf0-a4a0-a4621c4ec1cc_isrForCrosstalkSources_996984_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/4bc7eae2-467b-4cf0-a4a0-a4621c4ec1cc_isrForCrosstalkSources_996984_47.stderr parsl.dataflow.dflow INFO: Task 1498 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0426d2cf195c11b68ed0b492d2270f98 with result from task 1498 parsl.dataflow.dflow DEBUG: Task 1498 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c9245c44-1c69-4896-a046-9917a5e0e15a_isrForCrosstalkSources_997033_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c9245c44-1c69-4896-a046-9917a5e0e15a_isrForCrosstalkSources_997033_60.stderr parsl.dataflow.dflow INFO: Task 1724 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 69248eed4143628a8707610211894d89 with result from task 1724 parsl.dataflow.dflow DEBUG: Task 1724 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2bd80e4b-a25a-44a5-bdda-0ba14b6a065d_isrForCrosstalkSources_996970_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2bd80e4b-a25a-44a5-bdda-0ba14b6a065d_isrForCrosstalkSources_996970_35.stderr parsl.dataflow.dflow INFO: Task 1530 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dc5063d92c51737a4c7f8665d9e491b5 with result from task 1530 parsl.dataflow.dflow DEBUG: Task 1530 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/5c488b99-3be3-434d-a158-825c71e60c80_isrForCrosstalkSources_996982_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/5c488b99-3be3-434d-a158-825c71e60c80_isrForCrosstalkSources_996982_55.stderr parsl.dataflow.dflow INFO: Task 1601 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2de7566001eb115a765c89f95b78ca2 with result from task 1601 parsl.dataflow.dflow DEBUG: Task 1601 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/085a5886-7843-4a66-bead-b9769171d559_isrForCrosstalkSources_996977_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/085a5886-7843-4a66-bead-b9769171d559_isrForCrosstalkSources_996977_47.stderr parsl.dataflow.dflow INFO: Task 1697 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c447c546a9ad34e17c90d10247cc0639 with result from task 1697 parsl.dataflow.dflow DEBUG: Task 1697 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/1505a9fb-efa7-4914-aeba-fd170a1d6263_isrForCrosstalkSources_996973_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/1505a9fb-efa7-4914-aeba-fd170a1d6263_isrForCrosstalkSources_996973_35.stderr parsl.dataflow.dflow INFO: Task 1736 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 561dcc08b163130d12bed4515ae68101 with result from task 1736 parsl.dataflow.dflow DEBUG: Task 1736 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/439a29ce-aba1-4dba-8fc2-4a564505be02_isrForCrosstalkSources_997002_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/439a29ce-aba1-4dba-8fc2-4a564505be02_isrForCrosstalkSources_997002_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:01 proc_lsst.multi:146 [INFO] found job 17075136 in provider checkpoint proc_lsst.multi INFO: found job 17075136 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1478 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ceb6dcc52d1260299b60ef11b26eb5a9 with result from task 1478 parsl.dataflow.dflow DEBUG: Task 1478 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d8cebc9a-63ee-4871-bebd-001d96c2e1d0_isrForCrosstalkSources_997010_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/d8cebc9a-63ee-4871-bebd-001d96c2e1d0_isrForCrosstalkSources_997010_54.stderr parsl.dataflow.dflow INFO: Task 1525 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01bd16447baa958b97146764639c7e3a with result from task 1525 parsl.dataflow.dflow DEBUG: Task 1525 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/12dde2f6-5aa8-4f02-ad5e-617c6ab3aed1_isrForCrosstalkSources_996983_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/12dde2f6-5aa8-4f02-ad5e-617c6ab3aed1_isrForCrosstalkSources_996983_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:02 proc_lsst.multi:146 [INFO] found job 17075137 in provider checkpoint proc_lsst.multi INFO: found job 17075137 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1571 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f34469379aeeec6b4a40ddb22ac917a4 with result from task 1571 parsl.dataflow.dflow DEBUG: Task 1571 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ae23a244-7d79-4082-8bdb-1c78bb3d23c4_isrForCrosstalkSources_997030_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ae23a244-7d79-4082-8bdb-1c78bb3d23c4_isrForCrosstalkSources_997030_60.stderr parsl.dataflow.dflow INFO: Task 1529 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33b84512abadd16542df0432c6b7727b with result from task 1529 parsl.dataflow.dflow DEBUG: Task 1529 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/bacff6c0-fcb1-48c4-b6b1-3e4bfb77e0a9_isrForCrosstalkSources_997023_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/bacff6c0-fcb1-48c4-b6b1-3e4bfb77e0a9_isrForCrosstalkSources_997023_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:03 proc_lsst.multi:146 [INFO] found job 17075138 in provider checkpoint proc_lsst.multi INFO: found job 17075138 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1735 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry efe9fd87822789875b3b6b3ebb3c7085 with result from task 1735 parsl.dataflow.dflow DEBUG: Task 1735 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ecdf1f1b-9271-4ab6-97c9-bf8780e055e2_isrForCrosstalkSources_996972_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ecdf1f1b-9271-4ab6-97c9-bf8780e055e2_isrForCrosstalkSources_996972_43.stderr parsl.dataflow.dflow INFO: Task 1713 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9168173fa45d2b938a6a8c790249591 with result from task 1713 parsl.dataflow.dflow DEBUG: Task 1713 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60775b99-e0c3-4420-b918-8c77fa290a3c_isrForCrosstalkSources_996968_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60775b99-e0c3-4420-b918-8c77fa290a3c_isrForCrosstalkSources_996968_9.stderr parsl.dataflow.dflow INFO: Task 1742 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3963a1211e02be501229cbce1d7443d2 with result from task 1742 parsl.dataflow.dflow DEBUG: Task 1742 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/30e9c844-bbb8-459a-b488-248d89617f19_isrForCrosstalkSources_997018_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/30e9c844-bbb8-459a-b488-248d89617f19_isrForCrosstalkSources_997018_44.stderr parsl.dataflow.dflow INFO: Task 1620 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b400e5a343442417652bf401de3f1eb3 with result from task 1620 parsl.dataflow.dflow DEBUG: Task 1620 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/de6f77b6-b585-4a5f-84d7-d6cd3140d5ec_isrForCrosstalkSources_996966_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/de6f77b6-b585-4a5f-84d7-d6cd3140d5ec_isrForCrosstalkSources_996966_14.stderr parsl.dataflow.dflow INFO: Task 1573 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0284d1037441312f822a3f04775ceb6c with result from task 1573 parsl.dataflow.dflow DEBUG: Task 1573 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4b8b4ee5-ee39-43ce-b546-c1231481098e_isrForCrosstalkSources_996998_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4b8b4ee5-ee39-43ce-b546-c1231481098e_isrForCrosstalkSources_996998_61.stderr parsl.dataflow.dflow INFO: Task 1560 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b3dc5c5f7a7f540592dfd6b56eb8d93 with result from task 1560 parsl.dataflow.dflow DEBUG: Task 1560 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e4e13630-f58c-4566-bec1-f9f9cdbd7c39_isrForCrosstalkSources_996956_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e4e13630-f58c-4566-bec1-f9f9cdbd7c39_isrForCrosstalkSources_996956_60.stderr parsl.dataflow.dflow INFO: Task 1662 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d73162e34d64999fdc23f29df5248a05 with result from task 1662 parsl.dataflow.dflow DEBUG: Task 1662 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/86c0c54c-b93d-4816-9020-9fab717e6971_isrForCrosstalkSources_997002_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/86c0c54c-b93d-4816-9020-9fab717e6971_isrForCrosstalkSources_997002_14.stderr parsl.dataflow.dflow INFO: Task 1714 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03d40c46c19b2806dd6dfdfdd41aa0b6 with result from task 1714 parsl.dataflow.dflow DEBUG: Task 1714 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/fde039f1-2006-4569-af95-297e721e7fd1_isrForCrosstalkSources_997000_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/fde039f1-2006-4569-af95-297e721e7fd1_isrForCrosstalkSources_997000_43.stderr parsl.dataflow.dflow INFO: Task 1501 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2feaf32f5302929aa4cab029a3aed39b with result from task 1501 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1501 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:04 proc_lsst.multi:146 [INFO] found job 17075139 in provider checkpoint proc_lsst.multi INFO: found job 17075139 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/898a29d3-9b7b-406f-aa69-1c677bdca2b8_isrForCrosstalkSources_997018_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/898a29d3-9b7b-406f-aa69-1c677bdca2b8_isrForCrosstalkSources_997018_52.stderr parsl.dataflow.dflow INFO: Task 1746 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e0de1ac5ea8a251f961dc92044273d3 with result from task 1746 parsl.dataflow.dflow DEBUG: Task 1746 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9387c9e6-9c08-4ed0-a258-f576ffae3570_isrForCrosstalkSources_996988_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9387c9e6-9c08-4ed0-a258-f576ffae3570_isrForCrosstalkSources_996988_62.stderr parsl.dataflow.dflow INFO: Task 1753 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ad2a269d4cc4a9ba986017eeb895371 with result from task 1753 parsl.dataflow.dflow DEBUG: Task 1753 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/86e53c76-ac6c-4e53-90d5-a0617cc6b984_isrForCrosstalkSources_997009_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/86e53c76-ac6c-4e53-90d5-a0617cc6b984_isrForCrosstalkSources_997009_24.stderr parsl.dataflow.dflow INFO: Task 1712 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 828a15e7c165adc404f7cd9827bce581 with result from task 1712 parsl.dataflow.dflow DEBUG: Task 1712 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a4b8b12e-25ce-421c-8846-737f8c1f9a46_isrForCrosstalkSources_997002_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a4b8b12e-25ce-421c-8846-737f8c1f9a46_isrForCrosstalkSources_997002_6.stderr parsl.dataflow.dflow INFO: Task 1764 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 789844f324c7ac3649e6484b0f20dff2 with result from task 1764 parsl.dataflow.dflow DEBUG: Task 1764 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f12b2c49-e199-48fc-95ba-4bbafb45646d_isrForCrosstalkSources_997026_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f12b2c49-e199-48fc-95ba-4bbafb45646d_isrForCrosstalkSources_997026_20.stderr parsl.dataflow.dflow INFO: Task 1715 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 920d9e11be58c3e91591d40d0f612b5d with result from task 1715 parsl.dataflow.dflow DEBUG: Task 1715 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c0e9d19c-71de-45fc-96ac-219547d0fb4b_isrForCrosstalkSources_996960_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/c0e9d19c-71de-45fc-96ac-219547d0fb4b_isrForCrosstalkSources_996960_58.stderr parsl.dataflow.dflow INFO: Task 1476 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1af31adb7ddea1b630e1aecef47d55a9 with result from task 1476 parsl.dataflow.dflow DEBUG: Task 1476 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/793a4c3b-3dff-4c9b-a436-d5dde2c6c08f_isrForCrosstalkSources_996989_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/793a4c3b-3dff-4c9b-a436-d5dde2c6c08f_isrForCrosstalkSources_996989_54.stderr parsl.dataflow.dflow INFO: Task 1468 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f9bff68533bcb2ea9d35d8cc003ac50 with result from task 1468 parsl.dataflow.dflow DEBUG: Task 1468 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/294e7cc6-b5bf-4583-8371-91c5e8bbf90f_isrForCrosstalkSources_996999_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/294e7cc6-b5bf-4583-8371-91c5e8bbf90f_isrForCrosstalkSources_996999_53.stderr parsl.dataflow.dflow INFO: Task 1644 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 60edd8e82f8d5d983ce8b6ddf0a6335b with result from task 1644 parsl.dataflow.dflow DEBUG: Task 1644 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/642497be-7a33-4a9d-aed1-8b2fa05d54a0_isrForCrosstalkSources_997011_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/642497be-7a33-4a9d-aed1-8b2fa05d54a0_isrForCrosstalkSources_997011_27.stderr parsl.dataflow.dflow INFO: Task 1763 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d4c749f01c42518701a32606e6b030c with result from task 1763 parsl.dataflow.dflow DEBUG: Task 1763 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/dfc67618-b543-4173-988f-20f5d60d20b4_isrForCrosstalkSources_996978_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/dfc67618-b543-4173-988f-20f5d60d20b4_isrForCrosstalkSources_996978_32.stderr parsl.dataflow.dflow INFO: Task 1737 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d531923f7c2fe178313c03df97afc0b with result from task 1737 parsl.dataflow.dflow DEBUG: Task 1737 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1fd822fb-c7c6-4457-b051-02051af48927_isrForCrosstalkSources_997022_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1fd822fb-c7c6-4457-b051-02051af48927_isrForCrosstalkSources_997022_61.stderr parsl.dataflow.dflow INFO: Task 1602 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70a4059dbba552ec380796f9e1caa7ff with result from task 1602 parsl.dataflow.dflow DEBUG: Task 1602 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/41775b54-5980-4dcc-80b0-8b1ffddf3a25_isrForCrosstalkSources_997013_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/41775b54-5980-4dcc-80b0-8b1ffddf3a25_isrForCrosstalkSources_997013_59.stderr parsl.dataflow.dflow INFO: Task 1748 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4cdfe18c40cc8ec721951544ff8090a with result from task 1748 parsl.dataflow.dflow DEBUG: Task 1748 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/65b4af3b-f785-4bbd-a36c-6659b362ce2c_isrForCrosstalkSources_997006_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/65b4af3b-f785-4bbd-a36c-6659b362ce2c_isrForCrosstalkSources_997006_58.stderr parsl.dataflow.dflow INFO: Task 1514 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 891789bac009b061be58f99e73764bb9 with result from task 1514 parsl.dataflow.dflow DEBUG: Task 1514 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/95626e66-9fd5-4650-ab98-288c2cc673ee_isrForCrosstalkSources_996977_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/95626e66-9fd5-4650-ab98-288c2cc673ee_isrForCrosstalkSources_996977_57.stderr parsl.dataflow.dflow INFO: Task 1539 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6a586d1c0e20ae2e239c61f4ec83d0c with result from task 1539 parsl.dataflow.dflow DEBUG: Task 1539 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/e09f547f-bda9-4480-b965-d518fd6102fd_isrForCrosstalkSources_996982_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/e09f547f-bda9-4480-b965-d518fd6102fd_isrForCrosstalkSources_996982_54.stderr parsl.dataflow.dflow INFO: Task 1762 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 656797d7f2c46de98d1083cc285203f7 with result from task 1762 parsl.dataflow.dflow DEBUG: Task 1762 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/9a5a38b0-d7dc-4ea8-b363-cd326c7f371b_isrForCrosstalkSources_997029_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/9a5a38b0-d7dc-4ea8-b363-cd326c7f371b_isrForCrosstalkSources_997029_13.stderr parsl.dataflow.dflow INFO: Task 1788 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e44fc6c3d1587e71b4e3a3a39356193 with result from task 1788 parsl.dataflow.dflow DEBUG: Task 1788 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/983bd7ae-d370-4c6f-bd4a-c0844676c972_isrForCrosstalkSources_997013_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/983bd7ae-d370-4c6f-bd4a-c0844676c972_isrForCrosstalkSources_997013_19.stderr parsl.dataflow.dflow INFO: Task 1491 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 922e5cbd2e059086cac133e74b2d1096 with result from task 1491 parsl.dataflow.dflow DEBUG: Task 1491 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/32e5cf89-826a-40fd-984b-4fb023f09df1_isrForCrosstalkSources_997024_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/32e5cf89-826a-40fd-984b-4fb023f09df1_isrForCrosstalkSources_997024_59.stderr parsl.dataflow.dflow INFO: Task 1651 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8c1e76eba45cb1cb98847d71db70090 with result from task 1651 parsl.dataflow.dflow DEBUG: Task 1651 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e94f31a2-1652-4e3e-8062-5e763cfe37a8_isrForCrosstalkSources_996993_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e94f31a2-1652-4e3e-8062-5e763cfe37a8_isrForCrosstalkSources_996993_46.stderr parsl.dataflow.dflow INFO: Task 1768 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11bc7f782c3013b83089e29e0634d83c with result from task 1768 parsl.dataflow.dflow DEBUG: Task 1768 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8c16230b-4c05-4b22-9416-2bcc395bd36c_isrForCrosstalkSources_996960_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8c16230b-4c05-4b22-9416-2bcc395bd36c_isrForCrosstalkSources_996960_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:05 proc_lsst.multi:146 [INFO] found job 17075140 in provider checkpoint proc_lsst.multi INFO: found job 17075140 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1518 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry efd0d6bb7ef314abdb0181144302827a with result from task 1518 parsl.dataflow.dflow DEBUG: Task 1518 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0d7da39f-ccbe-47f3-9e43-2d18da246717_isrForCrosstalkSources_996983_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0d7da39f-ccbe-47f3-9e43-2d18da246717_isrForCrosstalkSources_996983_62.stderr parsl.dataflow.dflow INFO: Task 1767 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7303f8715e22a602f07fe4cf0a8b19e1 with result from task 1767 parsl.dataflow.dflow DEBUG: Task 1767 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/25ed6db6-f8c5-401d-82b0-daac52482ba8_isrForCrosstalkSources_996964_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/25ed6db6-f8c5-401d-82b0-daac52482ba8_isrForCrosstalkSources_996964_14.stderr parsl.dataflow.dflow INFO: Task 1765 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d0b0e4338f61a440787b06f4b4e8adcb with result from task 1765 parsl.dataflow.dflow DEBUG: Task 1765 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e893d531-6ab0-4d54-abd6-fd869e3cc1b9_isrForCrosstalkSources_997021_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e893d531-6ab0-4d54-abd6-fd869e3cc1b9_isrForCrosstalkSources_997021_56.stderr parsl.dataflow.dflow INFO: Task 1775 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3e440a9fbe798f3d544472c4c340def with result from task 1775 parsl.dataflow.dflow DEBUG: Task 1775 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7d18e1b6-df43-4a11-955e-38864971dc14_isrForCrosstalkSources_996986_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7d18e1b6-df43-4a11-955e-38864971dc14_isrForCrosstalkSources_996986_39.stderr parsl.dataflow.dflow INFO: Task 1769 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1778c9a5e6b027669931eb070b7a962f with result from task 1769 parsl.dataflow.dflow DEBUG: Task 1769 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/ea9dc2ba-7a25-40f5-b167-81095769a4bc_isrForCrosstalkSources_996965_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/ea9dc2ba-7a25-40f5-b167-81095769a4bc_isrForCrosstalkSources_996965_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1770 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 5b39b7ac1df8bda967067fc6f0dc0af5 with result from task 1770 parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1770 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8068d567-8da1-403f-8445-448481792a91_isrForCrosstalkSources_997013_54.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 1770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8068d567-8da1-403f-8445-448481792a91_isrForCrosstalkSources_997013_54.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:06 proc_lsst.multi:146 [INFO] found job 17075141 in provider checkpoint proc_lsst.multi INFO: found job 17075141 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1693 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5525c0381f99334364eec2831f1aea77 with result from task 1693 parsl.dataflow.dflow DEBUG: Task 1693 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5daf36cc-4977-4bf4-881d-5a47fac94b3a_isrForCrosstalkSources_997033_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5daf36cc-4977-4bf4-881d-5a47fac94b3a_isrForCrosstalkSources_997033_9.stderr parsl.dataflow.dflow INFO: Task 1643 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c44cdd4f9079b55396c75c45e22f3561 with result from task 1643 parsl.dataflow.dflow DEBUG: Task 1643 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/53c17f98-0b53-4bfc-ba99-232010f149a3_isrForCrosstalkSources_997006_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/53c17f98-0b53-4bfc-ba99-232010f149a3_isrForCrosstalkSources_997006_56.stderr parsl.dataflow.dflow INFO: Task 1621 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00de6fbb3ad888c1dd32735181858abd with result from task 1621 parsl.dataflow.dflow DEBUG: Task 1621 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c24d7cd0-20c9-4ce3-9f7f-fa31826541d4_isrForCrosstalkSources_996985_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/c24d7cd0-20c9-4ce3-9f7f-fa31826541d4_isrForCrosstalkSources_996985_62.stderr parsl.dataflow.dflow INFO: Task 1593 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.memoization DEBUG: Storing app cache entry 77da7b0d9ea386f7f2e521b1b9407a78 with result from task 1593 parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1593 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e94627ee-9450-4b02-956f-c64c4d0e5019_isrForCrosstalkSources_996985_54.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e94627ee-9450-4b02-956f-c64c4d0e5019_isrForCrosstalkSources_996985_54.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1766 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 74b3d06697ddeeeb70ea794ee6557bed with result from task 1766 parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1766 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bbf2808d-9a3f-49ea-a513-9c4098c35d19_isrForCrosstalkSources_996992_35.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bbf2808d-9a3f-49ea-a513-9c4098c35d19_isrForCrosstalkSources_996992_35.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:06 proc_lsst.multi:146 [INFO] found job 17075142 in provider checkpoint proc_lsst.multi INFO: found job 17075142 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1812 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f1ca4e4489c3985f37ceb95f7d63fc1e with result from task 1812 parsl.dataflow.dflow DEBUG: Task 1812 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/65a47739-011e-4b3d-92fa-207934f5d3b5_isrForCrosstalkSources_997013_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/65a47739-011e-4b3d-92fa-207934f5d3b5_isrForCrosstalkSources_997013_24.stderr parsl.dataflow.dflow INFO: Task 1771 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 20ec6f84893fd3aec5d43fbaa59e0683 with result from task 1771 parsl.dataflow.dflow DEBUG: Task 1771 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/20cde41d-aabe-4fee-9a8d-9f142170ad33_isrForCrosstalkSources_996984_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/20cde41d-aabe-4fee-9a8d-9f142170ad33_isrForCrosstalkSources_996984_33.stderr parsl.dataflow.dflow INFO: Task 1677 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc83b869f9fd04020fd55fed63411917 with result from task 1677 parsl.dataflow.dflow DEBUG: Task 1677 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/10071cd0-cda5-497f-9bfa-5e7fa5815e7b_isrForCrosstalkSources_996989_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/10071cd0-cda5-497f-9bfa-5e7fa5815e7b_isrForCrosstalkSources_996989_20.stderr parsl.dataflow.dflow INFO: Task 1658 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73668641246169bc86985c2719a53fa8 with result from task 1658 parsl.dataflow.dflow DEBUG: Task 1658 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d0c54afe-c1c3-420d-b70a-0444c5ae9996_isrForCrosstalkSources_997004_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d0c54afe-c1c3-420d-b70a-0444c5ae9996_isrForCrosstalkSources_997004_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:07 proc_lsst.multi:146 [INFO] found job 17075143 in provider checkpoint proc_lsst.multi INFO: found job 17075143 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1720 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f63315ea1689442ad0c25bdd1a18896d with result from task 1720 parsl.dataflow.dflow DEBUG: Task 1720 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f19c488f-34aa-44f0-8067-dd8ccd0d03b9_isrForCrosstalkSources_996966_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f19c488f-34aa-44f0-8067-dd8ccd0d03b9_isrForCrosstalkSources_996966_1.stderr parsl.dataflow.dflow INFO: Task 1813 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c5b88ebcdb4a3554cc506d7bb66208a with result from task 1813 parsl.dataflow.dflow DEBUG: Task 1813 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/e3fde26a-c4a8-4f26-8281-dc324acb3452_isrForCrosstalkSources_996962_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/e3fde26a-c4a8-4f26-8281-dc324acb3452_isrForCrosstalkSources_996962_33.stderr parsl.dataflow.dflow INFO: Task 1520 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 879edcef9df0b28781bceb340d0e555c with result from task 1520 parsl.dataflow.dflow DEBUG: Task 1520 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/238e5c4f-7234-451a-81e8-57c5762f7de2_isrForCrosstalkSources_997029_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/238e5c4f-7234-451a-81e8-57c5762f7de2_isrForCrosstalkSources_997029_52.stderr parsl.dataflow.dflow INFO: Task 1657 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d5b59c189caecd56f9c9b3748bdd75cf with result from task 1657 parsl.dataflow.dflow DEBUG: Task 1657 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/85557699-e752-4b90-b806-5c78ba0bcd8c_isrForCrosstalkSources_996994_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/85557699-e752-4b90-b806-5c78ba0bcd8c_isrForCrosstalkSources_996994_2.stderr parsl.dataflow.dflow INFO: Task 1842 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2a81e2077f5094ffd8d29537d1da2801 with result from task 1842 parsl.dataflow.dflow DEBUG: Task 1842 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/659f08a5-2414-405e-a4e7-cba49ca3ab9a_isrForCrosstalkSources_996957_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/659f08a5-2414-405e-a4e7-cba49ca3ab9a_isrForCrosstalkSources_996957_46.stderr parsl.dataflow.dflow INFO: Task 1801 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1bf500d84283ba650accc6d14c5b46ec with result from task 1801 parsl.dataflow.dflow DEBUG: Task 1801 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/178e3b30-ac17-4b97-a000-6b13e79550be_isrForCrosstalkSources_997005_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/178e3b30-ac17-4b97-a000-6b13e79550be_isrForCrosstalkSources_997005_27.stderr parsl.dataflow.dflow INFO: Task 1843 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abc3894afb4b2ea6cd7021cf2a0065e5 with result from task 1843 parsl.dataflow.dflow DEBUG: Task 1843 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1efe983d-15e4-4314-bca8-80cabe07960c_isrForCrosstalkSources_997027_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1efe983d-15e4-4314-bca8-80cabe07960c_isrForCrosstalkSources_997027_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:08 proc_lsst.multi:146 [INFO] found job 17075144 in provider checkpoint proc_lsst.multi INFO: found job 17075144 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1718 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b21b20d8fae3e56a8d8d6f0d6b0969a with result from task 1718 parsl.dataflow.dflow DEBUG: Task 1718 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/43edbd5b-bfdb-4d82-a86b-6c64d348bf49_isrForCrosstalkSources_996988_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/43edbd5b-bfdb-4d82-a86b-6c64d348bf49_isrForCrosstalkSources_996988_28.stderr parsl.dataflow.dflow INFO: Task 1703 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 72286d106a95bb320c63a7abbe50f1f0 with result from task 1703 parsl.dataflow.dflow DEBUG: Task 1703 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/f50c5322-a8ef-4f37-b890-e57f932cb136_isrForCrosstalkSources_997007_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/f50c5322-a8ef-4f37-b890-e57f932cb136_isrForCrosstalkSources_997007_33.stderr parsl.dataflow.dflow INFO: Task 1699 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0148cd39b8791ed0e709201d7143fb1e with result from task 1699 parsl.dataflow.dflow DEBUG: Task 1699 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/69c0bd03-6988-47ad-b107-486dabaeed89_isrForCrosstalkSources_997001_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/69c0bd03-6988-47ad-b107-486dabaeed89_isrForCrosstalkSources_997001_8.stderr parsl.dataflow.dflow INFO: Task 1811 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26d36c5a3ef14bfaff19f2f978baaae8 with result from task 1811 parsl.dataflow.dflow DEBUG: Task 1811 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/aee9cbff-20a1-45b8-8904-32c9e5b1ffca_isrForCrosstalkSources_996992_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/aee9cbff-20a1-45b8-8904-32c9e5b1ffca_isrForCrosstalkSources_996992_13.stderr parsl.dataflow.dflow INFO: Task 1761 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f67b7bcd60c65015f2e1ed920bc7cea with result from task 1761 parsl.dataflow.dflow DEBUG: Task 1761 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/1999cf19-36ba-46db-ae8b-eab251f2f591_isrForCrosstalkSources_996980_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/1999cf19-36ba-46db-ae8b-eab251f2f591_isrForCrosstalkSources_996980_51.stderr parsl.dataflow.dflow INFO: Task 1661 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50a3d79e7bc3431394ad251d08784b1a with result from task 1661 parsl.dataflow.dflow DEBUG: Task 1661 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e09c2bb2-c1bf-4425-87c9-c3765f9a88e2_isrForCrosstalkSources_997011_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e09c2bb2-c1bf-4425-87c9-c3765f9a88e2_isrForCrosstalkSources_997011_20.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:08 proc_lsst.multi:146 [INFO] found job 17075145 in provider checkpoint proc_lsst.multi INFO: found job 17075145 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1773 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 411b135e7d9af9ea24ac8506047bf9a6 with result from task 1773 parsl.dataflow.dflow DEBUG: Task 1773 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7e817931-e103-4866-af12-7f5a8bb6e963_isrForCrosstalkSources_997006_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7e817931-e103-4866-af12-7f5a8bb6e963_isrForCrosstalkSources_997006_38.stderr parsl.dataflow.dflow INFO: Task 1698 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f67f9182325cc98539d02f53177d905c with result from task 1698 parsl.dataflow.dflow DEBUG: Task 1698 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0fd7a8eb-e057-4c87-ad1c-5c010dc8c6cc_isrForCrosstalkSources_997010_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0fd7a8eb-e057-4c87-ad1c-5c010dc8c6cc_isrForCrosstalkSources_997010_26.stderr parsl.dataflow.dflow INFO: Task 1800 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8593807dc5989565bd7008eb8ac19c57 with result from task 1800 parsl.dataflow.dflow DEBUG: Task 1800 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/81da5f35-7794-4ef3-bb5d-5b92fec235bd_isrForCrosstalkSources_996964_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/81da5f35-7794-4ef3-bb5d-5b92fec235bd_isrForCrosstalkSources_996964_13.stderr parsl.dataflow.dflow INFO: Task 1642 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c91e2ff132dd33ae325f928b0e746f3 with result from task 1642 parsl.dataflow.dflow DEBUG: Task 1642 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6ddf105f-a82b-487f-983f-d5b1fdcfe584_isrForCrosstalkSources_996971_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6ddf105f-a82b-487f-983f-d5b1fdcfe584_isrForCrosstalkSources_996971_32.stderr parsl.dataflow.dflow INFO: Task 1776 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a54cfaf37878a9ba4df77f8051f57769 with result from task 1776 parsl.dataflow.dflow DEBUG: Task 1776 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/387a27c2-41da-4830-b142-c1b375f76b3a_isrForCrosstalkSources_996998_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/387a27c2-41da-4830-b142-c1b375f76b3a_isrForCrosstalkSources_996998_27.stderr parsl.dataflow.dflow INFO: Task 1636 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa153dedf5d3321aba918a5698f22b3c with result from task 1636 parsl.dataflow.dflow DEBUG: Task 1636 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/cb400988-aaca-4f02-a39f-1b0cec073132_isrForCrosstalkSources_996967_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/cb400988-aaca-4f02-a39f-1b0cec073132_isrForCrosstalkSources_996967_9.stderr parsl.dataflow.dflow INFO: Task 1844 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94b26ba2627d14d628d1f156b4f65a71 with result from task 1844 parsl.dataflow.dflow DEBUG: Task 1844 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/9c7898c0-8e5d-4cdf-a0f5-d4929fe9c79a_isrForCrosstalkSources_996982_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/9c7898c0-8e5d-4cdf-a0f5-d4929fe9c79a_isrForCrosstalkSources_996982_16.stderr parsl.dataflow.dflow INFO: Task 1732 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6211d29f8d7831e024a774af1b126a7d with result from task 1732 parsl.dataflow.dflow DEBUG: Task 1732 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e3ba8461-7129-4134-8392-bfb27b20a1ec_isrForCrosstalkSources_996963_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e3ba8461-7129-4134-8392-bfb27b20a1ec_isrForCrosstalkSources_996963_26.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:09 proc_lsst.multi:146 [INFO] found job 17075146 in provider checkpoint proc_lsst.multi INFO: found job 17075146 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1709 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fc1626f47c4270eb78ef798c89bbcfe with result from task 1709 parsl.dataflow.dflow DEBUG: Task 1709 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f6a1e8ff-1342-488a-9ec0-7008449d66b5_isrForCrosstalkSources_996987_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f6a1e8ff-1342-488a-9ec0-7008449d66b5_isrForCrosstalkSources_996987_20.stderr parsl.dataflow.dflow INFO: Task 1691 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2dc2aca583b2b962c9f0d9d638549a6c with result from task 1691 parsl.dataflow.dflow DEBUG: Task 1691 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/868c3dd5-1ed2-4fb0-83b2-0b1488edad42_isrForCrosstalkSources_996986_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/868c3dd5-1ed2-4fb0-83b2-0b1488edad42_isrForCrosstalkSources_996986_20.stderr parsl.dataflow.dflow INFO: Task 1650 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0afb6b992ec1b811fdbb1b5d91265e73 with result from task 1650 parsl.dataflow.dflow DEBUG: Task 1650 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2021634d-95a7-4520-9254-0a8f0c3580c9_isrForCrosstalkSources_997002_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2021634d-95a7-4520-9254-0a8f0c3580c9_isrForCrosstalkSources_997002_1.stderr parsl.dataflow.dflow INFO: Task 1857 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fe67cf2b6100115ee6b569fd3b1533b with result from task 1857 parsl.dataflow.dflow DEBUG: Task 1857 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bc0fc8af-48b5-4e6a-9892-8d62ec5dbb90_isrForCrosstalkSources_996992_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bc0fc8af-48b5-4e6a-9892-8d62ec5dbb90_isrForCrosstalkSources_996992_19.stderr parsl.dataflow.dflow INFO: Task 1701 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad6d5be867e6fdf5657dfcd2e9afd5c5 with result from task 1701 parsl.dataflow.dflow DEBUG: Task 1701 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/617445d2-f68d-47a0-af82-edcac6f96e01_isrForCrosstalkSources_996962_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/617445d2-f68d-47a0-af82-edcac6f96e01_isrForCrosstalkSources_996962_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:10 proc_lsst.multi:146 [INFO] found job 17075147 in provider checkpoint proc_lsst.multi INFO: found job 17075147 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1721 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a620c32886aed45dfb4e2046915cc8be with result from task 1721 parsl.dataflow.dflow DEBUG: Task 1721 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/60501077-3067-4734-8481-726ba65af1c4_isrForCrosstalkSources_997009_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/60501077-3067-4734-8481-726ba65af1c4_isrForCrosstalkSources_997009_14.stderr parsl.dataflow.dflow INFO: Task 1669 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00cacf57a3ab763002f2d69c6cee9d43 with result from task 1669 parsl.dataflow.dflow DEBUG: Task 1669 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f6410546-fa7c-4105-bb4a-c189f334a1f6_isrForCrosstalkSources_996957_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f6410546-fa7c-4105-bb4a-c189f334a1f6_isrForCrosstalkSources_996957_13.stderr parsl.dataflow.dflow INFO: Task 1676 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 743754766cefcd22d39d936f96300b69 with result from task 1676 parsl.dataflow.dflow DEBUG: Task 1676 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0b952525-7cf5-47d0-8a6d-d4c1099d6bee_isrForCrosstalkSources_996982_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0b952525-7cf5-47d0-8a6d-d4c1099d6bee_isrForCrosstalkSources_996982_7.stderr parsl.dataflow.dflow INFO: Task 1772 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae661d3be4e694686b24cae040342a0d with result from task 1772 parsl.dataflow.dflow DEBUG: Task 1772 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/6e0830a0-8260-4a65-888c-fd3862105969_isrForCrosstalkSources_996989_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/6e0830a0-8260-4a65-888c-fd3862105969_isrForCrosstalkSources_996989_26.stderr parsl.dataflow.dflow INFO: Task 1663 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46619a4c37d53f35e3c9d811f837b74a with result from task 1663 parsl.dataflow.dflow DEBUG: Task 1663 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/7409b6cb-ea34-487f-8271-ae9eb835662f_isrForCrosstalkSources_996999_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/7409b6cb-ea34-487f-8271-ae9eb835662f_isrForCrosstalkSources_996999_3.stderr parsl.dataflow.dflow INFO: Task 1717 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6dfea09c1692f13f03b5454162061da4 with result from task 1717 parsl.dataflow.dflow DEBUG: Task 1717 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/5b773f22-9fa4-4304-a5e4-e877d4f91329_isrForCrosstalkSources_996952_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/5b773f22-9fa4-4304-a5e4-e877d4f91329_isrForCrosstalkSources_996952_27.stderr parsl.dataflow.dflow INFO: Task 1810 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04d7f10ceddce129351922b203bc59df with result from task 1810 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow DEBUG: Task 1810 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/71271bdf-4e0b-42fd-ae58-5fa2453c95ad_isrForCrosstalkSources_996956_32.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/71271bdf-4e0b-42fd-ae58-5fa2453c95ad_isrForCrosstalkSources_996956_32.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 1673 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 10eb014f8ad01829e4314f54c5cda391 with result from task 1673 parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1673 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/c59202b6-b756-43ed-9bc0-2b86587c0acd_isrForCrosstalkSources_996958_7.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/c59202b6-b756-43ed-9bc0-2b86587c0acd_isrForCrosstalkSources_996958_7.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:10 proc_lsst.multi:146 [INFO] found job 17075148 in provider checkpoint proc_lsst.multi INFO: found job 17075148 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1856 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 245c6c906361f1ec9436c05ac48957db with result from task 1856 parsl.dataflow.dflow DEBUG: Task 1856 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4cf0accd-97a7-4adf-92f4-eae505a45803_isrForCrosstalkSources_996959_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4cf0accd-97a7-4adf-92f4-eae505a45803_isrForCrosstalkSources_996959_23.stderr parsl.dataflow.dflow INFO: Task 1757 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 71f6e7da597fef7bbbd84058a9bee4a6 with result from task 1757 parsl.dataflow.dflow DEBUG: Task 1757 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a484ff53-92cc-4d29-9d10-a0e598ccc2e5_isrForCrosstalkSources_996983_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a484ff53-92cc-4d29-9d10-a0e598ccc2e5_isrForCrosstalkSources_996983_19.stderr parsl.dataflow.dflow INFO: Task 1760 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea94c9a2af60a75791a0a046ad2ba3ea with result from task 1760 parsl.dataflow.dflow DEBUG: Task 1760 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/98aca171-3a4c-434c-919e-f64be98591e5_isrForCrosstalkSources_997005_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/98aca171-3a4c-434c-919e-f64be98591e5_isrForCrosstalkSources_997005_20.stderr parsl.dataflow.dflow INFO: Task 1570 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f833e47b3c01935f3de6fb2f51d9315f with result from task 1570 parsl.dataflow.dflow DEBUG: Task 1570 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/460cab89-dcd3-4a4c-b3f4-56c3debed576_isrForCrosstalkSources_996959_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/460cab89-dcd3-4a4c-b3f4-56c3debed576_isrForCrosstalkSources_996959_60.stderr parsl.dataflow.dflow INFO: Task 1638 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 304d73605115156950eb37e0e0c37258 with result from task 1638 parsl.dataflow.dflow DEBUG: Task 1638 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f0d85de7-b3c1-449c-be99-0afc6c218461_isrForCrosstalkSources_996986_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f0d85de7-b3c1-449c-be99-0afc6c218461_isrForCrosstalkSources_996986_15.stderr parsl.dataflow.dflow INFO: Task 1906 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b2a297183c1e81f26cce8a0613b261c2 with result from task 1906 parsl.dataflow.dflow DEBUG: Task 1906 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e52251dc-e63e-4222-acf1-dc81a4749f1e_isrForCrosstalkSources_997023_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e52251dc-e63e-4222-acf1-dc81a4749f1e_isrForCrosstalkSources_997023_13.stderr parsl.dataflow.dflow INFO: Task 1707 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4825b491f5477f023cf8ebd41da808bb with result from task 1707 parsl.dataflow.dflow DEBUG: Task 1707 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/bcdd8641-90b4-4df7-a767-4d19dca63b58_isrForCrosstalkSources_996953_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/bcdd8641-90b4-4df7-a767-4d19dca63b58_isrForCrosstalkSources_996953_34.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:11 proc_lsst.multi:146 [INFO] found job 17075149 in provider checkpoint proc_lsst.multi INFO: found job 17075149 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1716 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f973fd7e151d717d7ba7f0de94f9ada with result from task 1716 parsl.dataflow.dflow DEBUG: Task 1716 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/888a7e3c-7e52-4fba-bab0-03a977952d6e_isrForCrosstalkSources_996960_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/888a7e3c-7e52-4fba-bab0-03a977952d6e_isrForCrosstalkSources_996960_40.stderr parsl.dataflow.dflow INFO: Task 1731 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 959bfb52256fcc79e844b1103ed3b6cb with result from task 1731 parsl.dataflow.dflow DEBUG: Task 1731 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/2591a22e-5455-428c-85b8-d1aad9ba975a_isrForCrosstalkSources_997023_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/2591a22e-5455-428c-85b8-d1aad9ba975a_isrForCrosstalkSources_997023_15.stderr parsl.dataflow.dflow INFO: Task 1672 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c11b3ca8f4d408339f3c75092ff73c84 with result from task 1672 parsl.dataflow.dflow DEBUG: Task 1672 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ca771207-41f3-47f6-bf4a-3d8f4457ea83_isrForCrosstalkSources_996988_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ca771207-41f3-47f6-bf4a-3d8f4457ea83_isrForCrosstalkSources_996988_17.stderr parsl.dataflow.dflow INFO: Task 1727 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b86ac6b419eece5f702cc99c7de2d40b with result from task 1727 parsl.dataflow.dflow DEBUG: Task 1727 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a29fdc8d-4b04-4712-8388-ae0f19c672f1_isrForCrosstalkSources_997011_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a29fdc8d-4b04-4712-8388-ae0f19c672f1_isrForCrosstalkSources_997011_1.stderr parsl.dataflow.dflow INFO: Task 1740 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f0fba61a07feea919b04a8feda12fb21 with result from task 1740 parsl.dataflow.dflow DEBUG: Task 1740 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/de31690c-0bc5-483d-9524-bcf7fde969d1_isrForCrosstalkSources_996962_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/de31690c-0bc5-483d-9524-bcf7fde969d1_isrForCrosstalkSources_996962_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:11 proc_lsst.multi:146 [INFO] found job 17075150 in provider checkpoint proc_lsst.multi INFO: found job 17075150 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1752 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4aca1c79580546cd83b8e87c19e0fb38 with result from task 1752 parsl.dataflow.dflow DEBUG: Task 1752 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/48437d3c-db4c-4f6e-90df-17651f15b0bb_isrForCrosstalkSources_996990_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/48437d3c-db4c-4f6e-90df-17651f15b0bb_isrForCrosstalkSources_996990_8.stderr parsl.dataflow.dflow INFO: Task 1728 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e56b808b2ebfbb90c6bb49e676b8db2 with result from task 1728 parsl.dataflow.dflow DEBUG: Task 1728 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/dba2d302-ce67-4a28-90ba-e8c002fe5a39_isrForCrosstalkSources_997003_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/dba2d302-ce67-4a28-90ba-e8c002fe5a39_isrForCrosstalkSources_997003_5.stderr parsl.dataflow.dflow INFO: Task 1892 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry afdcd56c1a69b7645b780a52d71284e2 with result from task 1892 parsl.dataflow.dflow DEBUG: Task 1892 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/7b8109a3-8aab-400c-843a-78e27796af39_isrForCrosstalkSources_996995_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/7b8109a3-8aab-400c-843a-78e27796af39_isrForCrosstalkSources_996995_9.stderr parsl.dataflow.dflow INFO: Task 1685 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07b03b23866d40410fbc6340764c7de0 with result from task 1685 parsl.dataflow.dflow DEBUG: Task 1685 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/827942e0-744e-4dd9-b65e-233dd6fc3bda_isrForCrosstalkSources_997003_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/827942e0-744e-4dd9-b65e-233dd6fc3bda_isrForCrosstalkSources_997003_22.stderr parsl.dataflow.dflow INFO: Task 1747 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38338cc82b0a30d6c8267ec36f89735f with result from task 1747 parsl.dataflow.dflow DEBUG: Task 1747 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/3e4220b5-c000-48c3-810a-57581ae665f9_isrForCrosstalkSources_997000_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/3e4220b5-c000-48c3-810a-57581ae665f9_isrForCrosstalkSources_997000_34.stderr parsl.dataflow.dflow INFO: Task 1655 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 240081eb5021da0e43f9240acdd1f7c0 with result from task 1655 parsl.dataflow.dflow DEBUG: Task 1655 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/51d0b279-59e5-47be-a230-b9da48f7ebf7_isrForCrosstalkSources_996989_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/51d0b279-59e5-47be-a230-b9da48f7ebf7_isrForCrosstalkSources_996989_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:12 proc_lsst.multi:146 [INFO] found job 17075151 in provider checkpoint proc_lsst.multi INFO: found job 17075151 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1860 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1cd557b67ab0fb452217074089f0818 with result from task 1860 parsl.dataflow.dflow DEBUG: Task 1860 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/171893d7-1f1e-444d-9781-b79ab8450cca_isrForCrosstalkSources_996970_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/171893d7-1f1e-444d-9781-b79ab8450cca_isrForCrosstalkSources_996970_45.stderr parsl.dataflow.dflow INFO: Task 1750 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2403a9a51ef6f00794b165d88b4b6954 with result from task 1750 parsl.dataflow.dflow DEBUG: Task 1750 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/77df117b-0fbe-4aa4-8d5b-bc13d108d6fd_isrForCrosstalkSources_997006_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/77df117b-0fbe-4aa4-8d5b-bc13d108d6fd_isrForCrosstalkSources_997006_39.stderr parsl.dataflow.dflow INFO: Task 1704 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b21d32809735205f4ed3e8a987272fc6 with result from task 1704 parsl.dataflow.dflow DEBUG: Task 1704 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/219b0437-8060-4484-8b21-d3a2f4758707_isrForCrosstalkSources_997001_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/219b0437-8060-4484-8b21-d3a2f4758707_isrForCrosstalkSources_997001_21.stderr parsl.dataflow.dflow INFO: Task 1648 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2bce0ed17b7ffaab2fdf18965f1f3939 with result from task 1648 parsl.dataflow.dflow DEBUG: Task 1648 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ec0d97a8-8ece-4a3c-a0b4-f8e1eec1d79e_isrForCrosstalkSources_997021_4.stdout parsl.dataflow.dflow INFO: Standard error for task 1648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ec0d97a8-8ece-4a3c-a0b4-f8e1eec1d79e_isrForCrosstalkSources_997021_4.stderr parsl.dataflow.dflow INFO: Task 1777 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8821bd520b3f7b4c8291d70ab2ec523b with result from task 1777 parsl.dataflow.dflow DEBUG: Task 1777 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/94e02588-d9fc-4809-ae4b-0d5914dbec5c_isrForCrosstalkSources_997008_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/94e02588-d9fc-4809-ae4b-0d5914dbec5c_isrForCrosstalkSources_997008_49.stderr parsl.dataflow.dflow INFO: Task 1910 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f1d1bb4af524d89585264082d27427da with result from task 1910 parsl.dataflow.dflow DEBUG: Task 1910 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/6ba26f90-ad6b-4706-92b5-2710854af85c_isrForCrosstalkSources_997004_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/6ba26f90-ad6b-4706-92b5-2710854af85c_isrForCrosstalkSources_997004_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:13 proc_lsst.multi:146 [INFO] found job 17075152 in provider checkpoint proc_lsst.multi INFO: found job 17075152 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1861 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d6cce721df07dbd082aad7a48050c03 with result from task 1861 parsl.dataflow.dflow DEBUG: Task 1861 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/fb22ec23-0354-456f-a0f5-e7bd243b0383_isrForCrosstalkSources_997031_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/fb22ec23-0354-456f-a0f5-e7bd243b0383_isrForCrosstalkSources_997031_46.stderr parsl.dataflow.dflow INFO: Task 1899 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2f5ee112ff4a3105e3a9b2ef9affaee with result from task 1899 parsl.dataflow.dflow DEBUG: Task 1899 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/26769602-fd21-49b8-890d-1251602be253_isrForCrosstalkSources_997016_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/26769602-fd21-49b8-890d-1251602be253_isrForCrosstalkSources_997016_27.stderr parsl.dataflow.dflow INFO: Task 1915 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82b1d5886f4ec359b174e0e47c12102c with result from task 1915 parsl.dataflow.dflow DEBUG: Task 1915 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3e72f7e8-2527-45b9-bd2e-9256568982ed_isrForCrosstalkSources_996968_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3e72f7e8-2527-45b9-bd2e-9256568982ed_isrForCrosstalkSources_996968_21.stderr parsl.dataflow.dflow INFO: Task 1887 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c8d8725a959e478214a739a77c90ad6 with result from task 1887 parsl.dataflow.dflow DEBUG: Task 1887 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/296c4755-5f0d-4a35-9628-e5dc63ed5dfb_isrForCrosstalkSources_996987_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/296c4755-5f0d-4a35-9628-e5dc63ed5dfb_isrForCrosstalkSources_996987_41.stderr parsl.dataflow.dflow INFO: Task 1888 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4899e1078081c6af603d9f92503a389c with result from task 1888 parsl.dataflow.dflow DEBUG: Task 1888 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5ed3c35d-88a2-4dad-b84a-6706159fc22f_isrForCrosstalkSources_997033_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/5ed3c35d-88a2-4dad-b84a-6706159fc22f_isrForCrosstalkSources_997033_21.stderr parsl.dataflow.dflow INFO: Task 1866 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe3f23184172c8299d99f50f454019bf with result from task 1866 parsl.dataflow.dflow DEBUG: Task 1866 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/751fed7d-5280-4cdb-878c-fc2efd124851_isrForCrosstalkSources_997001_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/751fed7d-5280-4cdb-878c-fc2efd124851_isrForCrosstalkSources_997001_16.stderr parsl.dataflow.dflow INFO: Task 1710 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b0a6db3ec670014f1238d70ec0ec1ae with result from task 1710 parsl.dataflow.dflow DEBUG: Task 1710 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6d968f59-63c3-4099-89c5-48a217dba668_isrForCrosstalkSources_997003_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6d968f59-63c3-4099-89c5-48a217dba668_isrForCrosstalkSources_997003_30.stderr parsl.dataflow.dflow INFO: Task 1901 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1bbd76f93d3c8f02a8dc0b951f4032a0 with result from task 1901 parsl.dataflow.dflow DEBUG: Task 1901 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c90e1972-4325-4a9e-b4a6-dadde2fddb21_isrForCrosstalkSources_996961_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c90e1972-4325-4a9e-b4a6-dadde2fddb21_isrForCrosstalkSources_996961_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:13 proc_lsst.multi:146 [INFO] found job 17075153 in provider checkpoint proc_lsst.multi INFO: found job 17075153 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1666 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dab3e804cce00d41b3d7b6f2c16a1308 with result from task 1666 parsl.dataflow.dflow DEBUG: Task 1666 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6e0630c-bf8d-4c44-8df0-8409da2cc7ff_isrForCrosstalkSources_996976_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6e0630c-bf8d-4c44-8df0-8409da2cc7ff_isrForCrosstalkSources_996976_1.stderr parsl.dataflow.dflow INFO: Task 1739 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7990a6a0537e5e82a1ef42548d76b391 with result from task 1739 parsl.dataflow.dflow DEBUG: Task 1739 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/960f70ea-f6a2-4efb-9a61-aba8f585b3f2_isrForCrosstalkSources_996980_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/960f70ea-f6a2-4efb-9a61-aba8f585b3f2_isrForCrosstalkSources_996980_7.stderr parsl.dataflow.dflow INFO: Task 1694 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd280a6ea39b026b34406efc3e3f82d7 with result from task 1694 parsl.dataflow.dflow DEBUG: Task 1694 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/401b7390-e5d6-4a9f-b580-029018efd5ed_isrForCrosstalkSources_996971_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/401b7390-e5d6-4a9f-b580-029018efd5ed_isrForCrosstalkSources_996971_3.stderr parsl.dataflow.dflow INFO: Task 1730 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ff0218a6fb2140193ca17f4758a4593 with result from task 1730 parsl.dataflow.dflow DEBUG: Task 1730 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/24858623-9d18-489a-b201-fe1e4a3fe6e8_isrForCrosstalkSources_996965_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/24858623-9d18-489a-b201-fe1e4a3fe6e8_isrForCrosstalkSources_996965_18.stderr parsl.dataflow.dflow INFO: Task 1557 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8159c862ab8c9acaa104faec2c3d294 with result from task 1557 parsl.dataflow.dflow DEBUG: Task 1557 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7dc5849a-fe38-4565-b946-e400c1417c43_isrForCrosstalkSources_996984_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7dc5849a-fe38-4565-b946-e400c1417c43_isrForCrosstalkSources_996984_6.stderr parsl.dataflow.dflow INFO: Task 1858 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fdb93575081dce81bbb4884926ae3b4e with result from task 1858 parsl.dataflow.dflow DEBUG: Task 1858 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d2a3e477-18b7-4555-9ff0-124296260439_isrForCrosstalkSources_997009_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d2a3e477-18b7-4555-9ff0-124296260439_isrForCrosstalkSources_997009_18.stderr parsl.dataflow.dflow INFO: Task 1687 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 810cadc6c141031e27f7a86eaab37d00 with result from task 1687 parsl.dataflow.dflow DEBUG: Task 1687 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/77dfd0bc-4731-4a32-97b9-8911b297591b_isrForCrosstalkSources_997007_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/77dfd0bc-4731-4a32-97b9-8911b297591b_isrForCrosstalkSources_997007_58.stderr parsl.dataflow.dflow INFO: Task 1734 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 130ee89069c811738f44f0db7e2a3c45 with result from task 1734 parsl.dataflow.dflow DEBUG: Task 1734 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/8cb01edf-2f3a-4e6d-8fcf-f0dd811c3841_isrForCrosstalkSources_996999_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/8cb01edf-2f3a-4e6d-8fcf-f0dd811c3841_isrForCrosstalkSources_996999_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:14 proc_lsst.multi:146 [INFO] found job 17075154 in provider checkpoint proc_lsst.multi INFO: found job 17075154 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1865 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da5727b3977bab1702e7b6f0773c92f1 with result from task 1865 parsl.dataflow.dflow DEBUG: Task 1865 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/ecd9f573-951e-47b3-ad11-3ac83772716f_isrForCrosstalkSources_997033_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/ecd9f573-951e-47b3-ad11-3ac83772716f_isrForCrosstalkSources_997033_49.stderr parsl.dataflow.dflow INFO: Task 1741 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d74bbef760d9c4314aefd289536a09ab with result from task 1741 parsl.dataflow.dflow DEBUG: Task 1741 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/faee15b9-9c04-452b-abb9-123132c1025d_isrForCrosstalkSources_996970_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/faee15b9-9c04-452b-abb9-123132c1025d_isrForCrosstalkSources_996970_19.stderr parsl.dataflow.dflow INFO: Task 1799 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c45ae8ba887332b057b63fb023d088e with result from task 1799 parsl.dataflow.dflow DEBUG: Task 1799 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/ecde18dd-4109-43a9-bcdc-97c055548fee_isrForCrosstalkSources_996953_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/ecde18dd-4109-43a9-bcdc-97c055548fee_isrForCrosstalkSources_996953_33.stderr parsl.dataflow.dflow INFO: Task 1674 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab5550f2bec136715370568fae5172bd with result from task 1674 parsl.dataflow.dflow DEBUG: Task 1674 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b14db87a-0509-4744-8f93-f4120e0ef01d_isrForCrosstalkSources_997016_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b14db87a-0509-4744-8f93-f4120e0ef01d_isrForCrosstalkSources_997016_51.stderr parsl.dataflow.dflow INFO: Task 1654 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff94a1cd60143e19e19a61130b2265f3 with result from task 1654 parsl.dataflow.dflow DEBUG: Task 1654 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/2f324fbd-cc58-40c0-8b53-38c76319a3e7_isrForCrosstalkSources_997032_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/2f324fbd-cc58-40c0-8b53-38c76319a3e7_isrForCrosstalkSources_997032_18.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:15 proc_lsst.multi:146 [INFO] found job 17075155 in provider checkpoint proc_lsst.multi INFO: found job 17075155 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1647 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8c673778396a5dfb702eda57e64195f with result from task 1647 parsl.dataflow.dflow DEBUG: Task 1647 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/14a30738-9525-4585-8443-cc5efb8a7d8c_isrForCrosstalkSources_996977_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/14a30738-9525-4585-8443-cc5efb8a7d8c_isrForCrosstalkSources_996977_30.stderr parsl.dataflow.dflow INFO: Task 1754 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 472c56406b03f931094ba32f41546fd2 with result from task 1754 parsl.dataflow.dflow DEBUG: Task 1754 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0bc11908-d7e6-4afd-9acd-051053d35456_isrForCrosstalkSources_997023_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0bc11908-d7e6-4afd-9acd-051053d35456_isrForCrosstalkSources_997023_10.stderr parsl.dataflow.dflow INFO: Task 1751 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d0a539db8fb0a436a0e3d0c987fc462b with result from task 1751 parsl.dataflow.dflow DEBUG: Task 1751 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/659ed4e0-b11f-4b0c-90ee-715677a653c4_isrForCrosstalkSources_996977_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/659ed4e0-b11f-4b0c-90ee-715677a653c4_isrForCrosstalkSources_996977_49.stderr parsl.dataflow.dflow INFO: Task 1756 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94d3e94c4b5a42a4818c246eb48909e1 with result from task 1756 parsl.dataflow.dflow DEBUG: Task 1756 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/407086a5-7278-4c06-a738-aea816904c12_isrForCrosstalkSources_996976_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/407086a5-7278-4c06-a738-aea816904c12_isrForCrosstalkSources_996976_42.stderr parsl.dataflow.dflow INFO: Task 1778 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cedc8670103f6bdfabc1235fcefd48ea with result from task 1778 parsl.dataflow.dflow DEBUG: Task 1778 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/f336555e-c4bb-4daf-b2f8-fdc1b60dbf20_isrForCrosstalkSources_997022_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/f336555e-c4bb-4daf-b2f8-fdc1b60dbf20_isrForCrosstalkSources_997022_13.stderr parsl.dataflow.dflow INFO: Task 1815 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe9838de99b96e9f4d501a401fdd03dd with result from task 1815 parsl.dataflow.dflow DEBUG: Task 1815 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e02b1e02-ad7e-43a3-b53b-fa25bc04906f_isrForCrosstalkSources_997016_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e02b1e02-ad7e-43a3-b53b-fa25bc04906f_isrForCrosstalkSources_997016_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:16 proc_lsst.multi:146 [INFO] found job 17075157 in provider checkpoint proc_lsst.multi INFO: found job 17075157 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1916 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f0dd3b14b07a0f446949af3c44319e0 with result from task 1916 parsl.dataflow.dflow DEBUG: Task 1916 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ad1504e0-b499-4584-af4e-3d6b0be0047d_isrForCrosstalkSources_996975_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ad1504e0-b499-4584-af4e-3d6b0be0047d_isrForCrosstalkSources_996975_44.stderr parsl.dataflow.dflow INFO: Task 1931 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d28ff653411408ce36791917b892d5d5 with result from task 1931 parsl.dataflow.dflow DEBUG: Task 1931 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6cfd5861-c76d-460c-946e-9fc6a3d10094_isrForCrosstalkSources_996974_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6cfd5861-c76d-460c-946e-9fc6a3d10094_isrForCrosstalkSources_996974_32.stderr parsl.dataflow.dflow INFO: Task 1897 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cec9f6993d85a15420b29ee5e3c42137 with result from task 1897 parsl.dataflow.dflow DEBUG: Task 1897 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/6e61471f-a0c7-445c-949e-9910f3358cef_isrForCrosstalkSources_997008_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/6e61471f-a0c7-445c-949e-9910f3358cef_isrForCrosstalkSources_997008_52.stderr parsl.dataflow.dflow INFO: Task 1679 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a870fb8534b300639d9a7e7f50c542b7 with result from task 1679 parsl.dataflow.dflow DEBUG: Task 1679 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/da0a31e4-aec7-484f-afa1-f17ffa6de7e2_isrForCrosstalkSources_997029_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/da0a31e4-aec7-484f-afa1-f17ffa6de7e2_isrForCrosstalkSources_997029_46.stderr parsl.dataflow.dflow INFO: Task 1890 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b14cd942b10e016767a1e07bc81bb6fb with result from task 1890 parsl.dataflow.dflow DEBUG: Task 1890 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/758810b4-1616-49ea-ba79-c4b32a4f6047_isrForCrosstalkSources_997027_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/758810b4-1616-49ea-ba79-c4b32a4f6047_isrForCrosstalkSources_997027_23.stderr parsl.dataflow.dflow INFO: Task 1814 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf8d97eb34cd0fc9168d819cef5ca07a with result from task 1814 parsl.dataflow.dflow DEBUG: Task 1814 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7e873933-9df4-4fd3-b05c-2e2ea791af05_isrForCrosstalkSources_997029_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7e873933-9df4-4fd3-b05c-2e2ea791af05_isrForCrosstalkSources_997029_40.stderr parsl.dataflow.dflow INFO: Task 1729 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c1f120f87b91c6384fcf8138fa15000 with result from task 1729 parsl.dataflow.dflow DEBUG: Task 1729 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/414aa1a4-9c73-4992-9405-705591d1df11_isrForCrosstalkSources_996966_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/414aa1a4-9c73-4992-9405-705591d1df11_isrForCrosstalkSources_996966_6.stderr parsl.dataflow.dflow INFO: Task 1671 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 116f560cdb507ef30d4ae5d5846e1c23 with result from task 1671 parsl.dataflow.dflow DEBUG: Task 1671 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bb504754-cec2-4c0e-838d-19078410c9c5_isrForCrosstalkSources_997027_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bb504754-cec2-4c0e-838d-19078410c9c5_isrForCrosstalkSources_997027_16.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:17 proc_lsst.multi:146 [INFO] found job 17075159 in provider checkpoint proc_lsst.multi INFO: found job 17075159 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1876 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3ca8d936d614cd45d5e4be87b482aff with result from task 1876 parsl.dataflow.dflow DEBUG: Task 1876 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7a040c32-a320-4d90-9f02-f768172642cd_isrForCrosstalkSources_997011_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7a040c32-a320-4d90-9f02-f768172642cd_isrForCrosstalkSources_997011_51.stderr parsl.dataflow.dflow INFO: Task 1705 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry feb120c90417abc933e5942712016c23 with result from task 1705 parsl.dataflow.dflow DEBUG: Task 1705 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/99ee6a54-85ef-4cc3-9e5a-1b04b71bc4fa_isrForCrosstalkSources_996968_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/99ee6a54-85ef-4cc3-9e5a-1b04b71bc4fa_isrForCrosstalkSources_996968_46.stderr parsl.dataflow.dflow INFO: Task 1902 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 71bd0eed1556d63671d395a9ef22ccb7 with result from task 1902 parsl.dataflow.dflow DEBUG: Task 1902 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/368aeeb0-270a-4129-83fd-62a797ad26f2_isrForCrosstalkSources_997027_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/368aeeb0-270a-4129-83fd-62a797ad26f2_isrForCrosstalkSources_997027_40.stderr parsl.dataflow.dflow INFO: Task 1900 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 87e898ba6a481f52ef94c47a803137f0 with result from task 1900 parsl.dataflow.dflow DEBUG: Task 1900 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/09344bdc-b576-4f52-8b28-f3a2d8b7b58c_isrForCrosstalkSources_996974_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/09344bdc-b576-4f52-8b28-f3a2d8b7b58c_isrForCrosstalkSources_996974_35.stderr parsl.dataflow.dflow INFO: Task 1930 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c94e6a33469a06f0ee781a4f84cb437 with result from task 1930 parsl.dataflow.dflow DEBUG: Task 1930 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/2215079b-7c23-4a7d-b83c-d365139b5d1b_isrForCrosstalkSources_996979_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/2215079b-7c23-4a7d-b83c-d365139b5d1b_isrForCrosstalkSources_996979_5.stderr parsl.dataflow.dflow INFO: Task 1864 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d3f1d8df22dfcc394dacff407ddb0b4 with result from task 1864 parsl.dataflow.dflow DEBUG: Task 1864 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/79f84e48-d077-4155-abd4-044ac7ec2474_isrForCrosstalkSources_996952_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/79f84e48-d077-4155-abd4-044ac7ec2474_isrForCrosstalkSources_996952_55.stderr parsl.dataflow.dflow INFO: Task 1700 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a5742787c08220004da6d319b6a59fc with result from task 1700 parsl.dataflow.dflow DEBUG: Task 1700 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/94344a04-5805-4c3c-b079-be9ce463c908_isrForCrosstalkSources_996999_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/94344a04-5805-4c3c-b079-be9ce463c908_isrForCrosstalkSources_996999_13.stderr parsl.dataflow.dflow INFO: Task 1706 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 370b7fe770f97bc0e4296022b581dddf with result from task 1706 parsl.dataflow.dflow DEBUG: Task 1706 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/20b45e25-2b0e-44b9-962f-82b19b72856d_isrForCrosstalkSources_996960_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/20b45e25-2b0e-44b9-962f-82b19b72856d_isrForCrosstalkSources_996960_50.stderr parsl.dataflow.dflow INFO: Task 1937 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f2864ca242bf8a75dbb57d105fe73e94 with result from task 1937 parsl.dataflow.dflow DEBUG: Task 1937 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/099a5e55-d343-4f4d-9094-1761d28a36c6_isrForCrosstalkSources_996979_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/099a5e55-d343-4f4d-9094-1761d28a36c6_isrForCrosstalkSources_996979_35.stderr parsl.dataflow.dflow INFO: Task 1862 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac02d6f5d6229126fc5eaa213a6bb00b with result from task 1862 parsl.dataflow.dflow DEBUG: Task 1862 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/721f558d-83a8-4139-8ff7-998b3bc7a556_isrForCrosstalkSources_996986_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/721f558d-83a8-4139-8ff7-998b3bc7a556_isrForCrosstalkSources_996986_17.stderr parsl.dataflow.dflow INFO: Task 1904 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 678dd7898fc8e87cdf1e1571c4cd264d with result from task 1904 parsl.dataflow.dflow DEBUG: Task 1904 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/171564fe-81c9-41a8-8144-a4c20e692319_isrForCrosstalkSources_996971_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/171564fe-81c9-41a8-8144-a4c20e692319_isrForCrosstalkSources_996971_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:17 proc_lsst.multi:146 [INFO] found job 17075160 in provider checkpoint proc_lsst.multi INFO: found job 17075160 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1893 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23a86297c40662935f85cd424371cf29 with result from task 1893 parsl.dataflow.dflow DEBUG: Task 1893 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/bab0e260-dba4-4abb-a58f-ada1c00859b8_isrForCrosstalkSources_997016_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/bab0e260-dba4-4abb-a58f-ada1c00859b8_isrForCrosstalkSources_997016_11.stderr parsl.dataflow.dflow INFO: Task 1634 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9e63eaf8c4e3c7f818ca1435a851555 with result from task 1634 parsl.dataflow.dflow DEBUG: Task 1634 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b03b64c6-5088-4b17-b505-9cb5b148508e_isrForCrosstalkSources_997007_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b03b64c6-5088-4b17-b505-9cb5b148508e_isrForCrosstalkSources_997007_56.stderr parsl.dataflow.dflow INFO: Task 1743 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50fbdb7e96a8b3e057f68525aa763207 with result from task 1743 parsl.dataflow.dflow DEBUG: Task 1743 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/5a140e14-68b7-4d96-ae33-fd0349c43f3d_isrForCrosstalkSources_996992_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/5a140e14-68b7-4d96-ae33-fd0349c43f3d_isrForCrosstalkSources_996992_39.stderr parsl.dataflow.dflow INFO: Task 1926 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb32600500668adb12a2ed4486a57edf with result from task 1926 parsl.dataflow.dflow DEBUG: Task 1926 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/cf111af4-7b60-4ede-b3f9-dc07a221b9c6_isrForCrosstalkSources_997028_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/cf111af4-7b60-4ede-b3f9-dc07a221b9c6_isrForCrosstalkSources_997028_59.stderr parsl.dataflow.dflow INFO: Task 1924 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f37e6c25dfd37fb1ef6313b5da94c59 with result from task 1924 parsl.dataflow.dflow DEBUG: Task 1924 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cad7590b-277b-4491-98a2-e2468fc58c28_isrForCrosstalkSources_996988_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cad7590b-277b-4491-98a2-e2468fc58c28_isrForCrosstalkSources_996988_50.stderr parsl.dataflow.dflow INFO: Task 1945 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c45be3b6ef304a5faf8cbb6adea0fe41 with result from task 1945 parsl.dataflow.dflow DEBUG: Task 1945 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/3bd540ce-ff6d-49b7-ad02-14958449e397_isrForCrosstalkSources_997024_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/3bd540ce-ff6d-49b7-ad02-14958449e397_isrForCrosstalkSources_997024_6.stderr parsl.dataflow.dflow INFO: Task 1929 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a96ed8f3fef590472d91d67317b776f with result from task 1929 parsl.dataflow.dflow DEBUG: Task 1929 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/bb1996e1-01ce-42ae-956b-10d0cf0f9b66_isrForCrosstalkSources_996993_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/bb1996e1-01ce-42ae-956b-10d0cf0f9b66_isrForCrosstalkSources_996993_28.stderr parsl.dataflow.dflow INFO: Task 1719 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f2d7b62a02c401d9b346995db9e888d4 with result from task 1719 parsl.dataflow.dflow DEBUG: Task 1719 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e6c71203-ee7b-4836-bb6f-381b4ade6989_isrForCrosstalkSources_997008_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e6c71203-ee7b-4836-bb6f-381b4ade6989_isrForCrosstalkSources_997008_12.stderr parsl.dataflow.dflow INFO: Task 1711 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8b1364c869ab457a7f4feb2452d7ccf with result from task 1711 parsl.dataflow.dflow DEBUG: Task 1711 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/3ddf3d63-c391-4019-b516-6a5a9cfbaa75_isrForCrosstalkSources_997011_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/3ddf3d63-c391-4019-b516-6a5a9cfbaa75_isrForCrosstalkSources_997011_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:19 proc_lsst.multi:146 [INFO] found job 17075161 in provider checkpoint proc_lsst.multi INFO: found job 17075161 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1797 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59fb7258e36f4ebcf38679c8b174b4c8 with result from task 1797 parsl.dataflow.dflow DEBUG: Task 1797 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6c96910-0077-4c04-8bd2-37abd3c0cd1a_isrForCrosstalkSources_996976_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6c96910-0077-4c04-8bd2-37abd3c0cd1a_isrForCrosstalkSources_996976_35.stderr parsl.dataflow.dflow INFO: Task 1946 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2cca58b3854115a5a58ad8c143b47876 with result from task 1946 parsl.dataflow.dflow DEBUG: Task 1946 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/75e088b5-1c0d-4dae-8b83-2a8d623cc3d4_isrForCrosstalkSources_997024_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/75e088b5-1c0d-4dae-8b83-2a8d623cc3d4_isrForCrosstalkSources_997024_13.stderr parsl.dataflow.dflow INFO: Task 1955 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7e568ea28f76de218c6e55189f40eb1 with result from task 1955 parsl.dataflow.dflow DEBUG: Task 1955 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/05a33231-f2a8-4c84-b779-871b992ecd92_isrForCrosstalkSources_997032_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/05a33231-f2a8-4c84-b779-871b992ecd92_isrForCrosstalkSources_997032_30.stderr parsl.dataflow.dflow INFO: Task 1855 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3eb57eccbb1d134346e66f96bf713561 with result from task 1855 parsl.dataflow.dflow DEBUG: Task 1855 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/30379af6-8fd9-4498-82f5-89d01b2c0a3d_isrForCrosstalkSources_996991_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/30379af6-8fd9-4498-82f5-89d01b2c0a3d_isrForCrosstalkSources_996991_61.stderr parsl.dataflow.dflow INFO: Task 1953 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51abb4779e106073afafeb7fbafd99d3 with result from task 1953 parsl.dataflow.dflow DEBUG: Task 1953 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/001b3917-9f06-4bfa-9350-68fd68f4d1c6_isrForCrosstalkSources_997002_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/001b3917-9f06-4bfa-9350-68fd68f4d1c6_isrForCrosstalkSources_997002_50.stderr parsl.dataflow.dflow INFO: Task 1878 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 412de130da7c03a66e2f9ecc5804cb01 with result from task 1878 parsl.dataflow.dflow DEBUG: Task 1878 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4ecaf6c7-98a9-4b96-a6d5-8ba044bad2b5_isrForCrosstalkSources_996997_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4ecaf6c7-98a9-4b96-a6d5-8ba044bad2b5_isrForCrosstalkSources_996997_26.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:19 proc_lsst.multi:146 [INFO] found job 17075162 in provider checkpoint proc_lsst.multi INFO: found job 17075162 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1938 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7f90912f207f623341879ee75ea3939 with result from task 1938 parsl.dataflow.dflow DEBUG: Task 1938 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/3bfd3b52-ed10-46bf-812d-df43927e7b2d_isrForCrosstalkSources_996956_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/3bfd3b52-ed10-46bf-812d-df43927e7b2d_isrForCrosstalkSources_996956_22.stderr parsl.dataflow.dflow INFO: Task 1925 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6e6104f85df31788440990b9a9f8acc7 with result from task 1925 parsl.dataflow.dflow DEBUG: Task 1925 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/146cbfc7-c37a-4642-88fe-8988ece87ed4_isrForCrosstalkSources_997018_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/146cbfc7-c37a-4642-88fe-8988ece87ed4_isrForCrosstalkSources_997018_35.stderr parsl.dataflow.dflow INFO: Task 1895 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cedbd2191bae9a85eb009b249857084a with result from task 1895 parsl.dataflow.dflow DEBUG: Task 1895 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/990a8800-4d91-43e1-b6d3-db942f2190a2_isrForCrosstalkSources_996978_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/990a8800-4d91-43e1-b6d3-db942f2190a2_isrForCrosstalkSources_996978_37.stderr parsl.dataflow.dflow INFO: Task 1627 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa9328cc48cd63e470513ac66ff70b98 with result from task 1627 parsl.dataflow.dflow DEBUG: Task 1627 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b11a649c-0a17-4544-9600-1fe04d7d7f13_isrForCrosstalkSources_996979_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b11a649c-0a17-4544-9600-1fe04d7d7f13_isrForCrosstalkSources_996979_10.stderr parsl.dataflow.dflow INFO: Task 1911 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce07bacabbf84ac11c3abaccfb80661f with result from task 1911 parsl.dataflow.dflow DEBUG: Task 1911 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/09da4a60-f491-46a1-a53a-a0fee2e77baa_isrForCrosstalkSources_997013_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/09da4a60-f491-46a1-a53a-a0fee2e77baa_isrForCrosstalkSources_997013_6.stderr parsl.dataflow.dflow INFO: Task 1923 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b35bc40b4b965bb0b50f6285a779bd50 with result from task 1923 parsl.dataflow.dflow DEBUG: Task 1923 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/503e2989-b8af-432b-aab7-abab7c37176f_isrForCrosstalkSources_996994_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/503e2989-b8af-432b-aab7-abab7c37176f_isrForCrosstalkSources_996994_11.stderr parsl.dataflow.dflow INFO: Task 1944 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa7e1daa9f01b565a15e36594504bb5f with result from task 1944 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:20 proc_lsst.multi:146 [INFO] found job 17075163 in provider checkpoint proc_lsst.multi INFO: found job 17075163 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 1944 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6cfbe654-cc11-4e5e-a904-afdd392ebe52_isrForCrosstalkSources_997015_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6cfbe654-cc11-4e5e-a904-afdd392ebe52_isrForCrosstalkSources_997015_50.stderr parsl.dataflow.dflow INFO: Task 1939 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b26b65170baf7a37cb1aa4c5ca1038c with result from task 1939 parsl.dataflow.dflow DEBUG: Task 1939 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/57cdf6cc-998c-462e-8b79-f60399feffaf_isrForCrosstalkSources_997021_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/57cdf6cc-998c-462e-8b79-f60399feffaf_isrForCrosstalkSources_997021_53.stderr parsl.dataflow.dflow INFO: Task 1652 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e90beed6a707f191e3e73a362b54daa with result from task 1652 parsl.dataflow.dflow DEBUG: Task 1652 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/4f639df0-6fde-498d-87a6-0fc45266dd61_isrForCrosstalkSources_996978_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/4f639df0-6fde-498d-87a6-0fc45266dd61_isrForCrosstalkSources_996978_43.stderr parsl.dataflow.dflow INFO: Task 1936 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b2657bf9244d6051f51d1cf62aa82f2 with result from task 1936 parsl.dataflow.dflow DEBUG: Task 1936 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/18d25c7d-0e89-44af-8c1a-c00468e78ba6_isrForCrosstalkSources_996963_9.stdout parsl.dataflow.dflow INFO: Standard error for task 1936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/18d25c7d-0e89-44af-8c1a-c00468e78ba6_isrForCrosstalkSources_996963_9.stderr parsl.dataflow.dflow INFO: Task 1640 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa551310c2a7f718947f4d42b309d1a9 with result from task 1640 parsl.dataflow.dflow DEBUG: Task 1640 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/4f57e350-6256-4b6d-aef0-5ba06ae44e0a_isrForCrosstalkSources_996982_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/4f57e350-6256-4b6d-aef0-5ba06ae44e0a_isrForCrosstalkSources_996982_6.stderr parsl.dataflow.dflow INFO: Task 1933 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86a0f8944cb9f58ebf8ed70e852eae1f with result from task 1933 parsl.dataflow.dflow DEBUG: Task 1933 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/7a02722f-aa62-45bd-b3e5-ad1e1748030e_isrForCrosstalkSources_996966_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/7a02722f-aa62-45bd-b3e5-ad1e1748030e_isrForCrosstalkSources_996966_41.stderr parsl.dataflow.dflow INFO: Task 1639 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c5bc1d3047f0271d625cfc458492f1e1 with result from task 1639 parsl.dataflow.dflow DEBUG: Task 1639 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4e7d6628-47e8-4e29-978b-b5a4f95d3824_isrForCrosstalkSources_996999_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4e7d6628-47e8-4e29-978b-b5a4f95d3824_isrForCrosstalkSources_996999_30.stderr parsl.dataflow.dflow INFO: Task 1918 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6faf264598dcd1bababdb53099f6cb93 with result from task 1918 parsl.dataflow.dflow DEBUG: Task 1918 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/24417ec7-de99-497f-8c7d-af09cd40e4d2_isrForCrosstalkSources_996987_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/24417ec7-de99-497f-8c7d-af09cd40e4d2_isrForCrosstalkSources_996987_58.stderr parsl.dataflow.dflow INFO: Task 1935 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ecdc9375d15e680fc76b770788f2226 with result from task 1935 parsl.dataflow.dflow DEBUG: Task 1935 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/96583a66-fe7b-44fe-8ad4-ecf578c5890f_isrForCrosstalkSources_996976_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/96583a66-fe7b-44fe-8ad4-ecf578c5890f_isrForCrosstalkSources_996976_50.stderr parsl.dataflow.dflow INFO: Task 1867 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e52e9819bbcf61e8eb48ae70eb62ac7 with result from task 1867 parsl.dataflow.dflow DEBUG: Task 1867 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ecc31ca8-81f2-4015-ad7a-3b6fbfbcae62_isrForCrosstalkSources_996962_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ecc31ca8-81f2-4015-ad7a-3b6fbfbcae62_isrForCrosstalkSources_996962_57.stderr parsl.dataflow.dflow INFO: Task 1759 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2498aff8e50a8cc01507766ba3d7719d with result from task 1759 parsl.dataflow.dflow DEBUG: Task 1759 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/c9014848-788d-46f0-b2d3-df714f8374a6_isrForCrosstalkSources_997002_15.stdout parsl.dataflow.dflow INFO: Standard error for task 1759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/c9014848-788d-46f0-b2d3-df714f8374a6_isrForCrosstalkSources_997002_15.stderr parsl.dataflow.dflow INFO: Task 1665 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c89f98c20e432952797c0408d3927ff with result from task 1665 parsl.dataflow.dflow DEBUG: Task 1665 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/f88632fc-f20b-4f5b-9de4-26efd8a62f30_isrForCrosstalkSources_996992_18.stdout parsl.dataflow.dflow INFO: Standard error for task 1665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/f88632fc-f20b-4f5b-9de4-26efd8a62f30_isrForCrosstalkSources_996992_18.stderr parsl.dataflow.dflow INFO: Task 1914 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab2067bcd30738447856379e57314c9d with result from task 1914 parsl.dataflow.dflow DEBUG: Task 1914 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/6a8f20bb-8189-4644-b306-89772e23fe85_isrForCrosstalkSources_996993_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/6a8f20bb-8189-4644-b306-89772e23fe85_isrForCrosstalkSources_996993_48.stderr parsl.dataflow.dflow INFO: Task 1917 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6a5a84c5a02b08828d149b144b035e6 with result from task 1917 parsl.dataflow.dflow DEBUG: Task 1917 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f33b7348-7908-404f-ad59-6aee4c382e18_isrForCrosstalkSources_996978_51.stdout parsl.dataflow.dflow INFO: Standard error for task 1917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f33b7348-7908-404f-ad59-6aee4c382e18_isrForCrosstalkSources_996978_51.stderr parsl.dataflow.dflow INFO: Task 1825 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fdba8ae85f5edc57146ae9dd2ba387d3 with result from task 1825 parsl.dataflow.dflow DEBUG: Task 1825 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fc5be51f-df12-4092-be5d-9aa02e23293f_isrForCrosstalkSources_997001_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fc5be51f-df12-4092-be5d-9aa02e23293f_isrForCrosstalkSources_997001_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:21 proc_lsst.multi:146 [INFO] found job 17075166 in provider checkpoint proc_lsst.multi INFO: found job 17075166 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1695 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd93cf02a40ce9befbf900ee241eda87 with result from task 1695 parsl.dataflow.dflow DEBUG: Task 1695 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/960843c4-ae9b-494e-924d-350e2c05e060_isrForCrosstalkSources_996952_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/960843c4-ae9b-494e-924d-350e2c05e060_isrForCrosstalkSources_996952_22.stderr parsl.dataflow.dflow INFO: Task 1689 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 514ebc57ca9f35c47038d50e164c722a with result from task 1689 parsl.dataflow.dflow DEBUG: Task 1689 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b80bd3e9-dfc1-4018-91e2-ec78622229e4_isrForCrosstalkSources_996997_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b80bd3e9-dfc1-4018-91e2-ec78622229e4_isrForCrosstalkSources_996997_5.stderr parsl.dataflow.dflow INFO: Task 1905 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e020020ce4429b2eb057401cef833d1 with result from task 1905 parsl.dataflow.dflow DEBUG: Task 1905 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/53916fb4-1abb-4bd8-93a9-699da9f9518f_isrForCrosstalkSources_997032_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/53916fb4-1abb-4bd8-93a9-699da9f9518f_isrForCrosstalkSources_997032_25.stderr parsl.dataflow.dflow INFO: Task 1749 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 72456bc4f69dade24c588000ecc8e5e3 with result from task 1749 parsl.dataflow.dflow DEBUG: Task 1749 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/6893e18b-44ff-44f6-b51c-784f158c0467_isrForCrosstalkSources_996956_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/6893e18b-44ff-44f6-b51c-784f158c0467_isrForCrosstalkSources_996956_30.stderr parsl.dataflow.dflow INFO: Task 1745 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4225165a5c55110bcb3ff56f19d2fdf6 with result from task 1745 parsl.dataflow.dflow DEBUG: Task 1745 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/da97f656-a547-48d3-a5ef-031dcaade546_isrForCrosstalkSources_997009_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/da97f656-a547-48d3-a5ef-031dcaade546_isrForCrosstalkSources_997009_42.stderr parsl.dataflow.dflow INFO: Task 1758 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f5ba248e4a856ad47ddf879e901d3d9 with result from task 1758 parsl.dataflow.dflow DEBUG: Task 1758 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ee87434a-984f-4e28-8695-61ba9fe8badd_isrForCrosstalkSources_996986_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ee87434a-984f-4e28-8695-61ba9fe8badd_isrForCrosstalkSources_996986_44.stderr parsl.dataflow.dflow INFO: Task 1896 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e339f6d74aa5e13778aa46f1b614f2e8 with result from task 1896 parsl.dataflow.dflow DEBUG: Task 1896 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9827589b-66c9-43ff-8cd8-b289be770fbe_isrForCrosstalkSources_996997_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/9827589b-66c9-43ff-8cd8-b289be770fbe_isrForCrosstalkSources_996997_35.stderr parsl.dataflow.dflow INFO: Task 1927 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 412b807e6174df0840d54f451cd0847d with result from task 1927 parsl.dataflow.dflow DEBUG: Task 1927 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0ca4b4a2-4c3e-410f-860c-6d7b4b2da874_isrForCrosstalkSources_996997_8.stdout parsl.dataflow.dflow INFO: Standard error for task 1927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0ca4b4a2-4c3e-410f-860c-6d7b4b2da874_isrForCrosstalkSources_996997_8.stderr parsl.dataflow.dflow INFO: Task 1725 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 68d345baba49ed4f203e72c7d50aaa49 with result from task 1725 parsl.dataflow.dflow DEBUG: Task 1725 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ee6408b0-fa59-4ad1-a11b-f6ec10f554cf_isrForCrosstalkSources_997005_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ee6408b0-fa59-4ad1-a11b-f6ec10f554cf_isrForCrosstalkSources_997005_52.stderr parsl.dataflow.dflow INFO: Task 1907 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e742092c76f3deaf7b5fbc30b3c8c113 with result from task 1907 parsl.dataflow.dflow DEBUG: Task 1907 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4cc0b197-05bc-43b4-9a82-2b434f8412ba_isrForCrosstalkSources_997018_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4cc0b197-05bc-43b4-9a82-2b434f8412ba_isrForCrosstalkSources_997018_57.stderr parsl.dataflow.dflow INFO: Task 1877 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2269c74382b4534fb3a52baa88d33da1 with result from task 1877 parsl.dataflow.dflow DEBUG: Task 1877 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/7d8e637a-f91d-4fd7-8645-36f37fc0b09b_isrForCrosstalkSources_996959_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/7d8e637a-f91d-4fd7-8645-36f37fc0b09b_isrForCrosstalkSources_996959_58.stderr parsl.dataflow.dflow INFO: Task 1889 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef443df86f77613611ebb4563070c054 with result from task 1889 parsl.dataflow.dflow DEBUG: Task 1889 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/392dfb93-d7a7-414b-9c78-a9e44948254c_isrForCrosstalkSources_996956_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/392dfb93-d7a7-414b-9c78-a9e44948254c_isrForCrosstalkSources_996956_61.stderr parsl.dataflow.dflow INFO: Task 1702 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c64ef61de642b17d82c7bf497d43862c with result from task 1702 parsl.dataflow.dflow DEBUG: Task 1702 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ee77bfcb-f4b6-4970-9572-44aba1a05f6b_isrForCrosstalkSources_996990_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ee77bfcb-f4b6-4970-9572-44aba1a05f6b_isrForCrosstalkSources_996990_1.stderr parsl.dataflow.dflow INFO: Task 1637 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ebc29c4c134027bda9cdf521850f08f1 with result from task 1637 parsl.dataflow.dflow DEBUG: Task 1637 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/4754eb95-3137-4190-8dc4-cb5db3681ceb_isrForCrosstalkSources_996991_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/4754eb95-3137-4190-8dc4-cb5db3681ceb_isrForCrosstalkSources_996991_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:22 proc_lsst.multi:146 [INFO] found job 17075167 in provider checkpoint proc_lsst.multi INFO: found job 17075167 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1859 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2fc7caac1d878b79dce593066151f61 with result from task 1859 parsl.dataflow.dflow DEBUG: Task 1859 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1f4cc7cb-4189-46d2-a555-287f0053f87e_isrForCrosstalkSources_997004_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1f4cc7cb-4189-46d2-a555-287f0053f87e_isrForCrosstalkSources_997004_53.stderr parsl.dataflow.dflow INFO: Task 1898 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7810beefabc5966c36e2a22a2f6e3ff7 with result from task 1898 parsl.dataflow.dflow DEBUG: Task 1898 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/38c5f7b0-f513-4e81-ac6a-a1fc2478bbe4_isrForCrosstalkSources_996964_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/38c5f7b0-f513-4e81-ac6a-a1fc2478bbe4_isrForCrosstalkSources_996964_17.stderr parsl.dataflow.dflow INFO: Task 1733 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4dfcf4c0e920f25d20d2419bb340319 with result from task 1733 parsl.dataflow.dflow DEBUG: Task 1733 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1925ca27-b4f8-465b-bc34-eb6f19d3f8e6_isrForCrosstalkSources_997000_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1925ca27-b4f8-465b-bc34-eb6f19d3f8e6_isrForCrosstalkSources_997000_38.stderr parsl.dataflow.dflow INFO: Task 1919 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce9b68522175b6993b9c4649cd14cfa0 with result from task 1919 parsl.dataflow.dflow DEBUG: Task 1919 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9189a285-2adf-4650-ac23-371b93a67e31_isrForCrosstalkSources_996958_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9189a285-2adf-4650-ac23-371b93a67e31_isrForCrosstalkSources_996958_59.stderr parsl.dataflow.dflow INFO: Task 1738 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe38395108e12adc726651ac2a2240f6 with result from task 1738 parsl.dataflow.dflow DEBUG: Task 1738 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/3c8477df-e281-4e35-8787-9d137012ba34_isrForCrosstalkSources_996953_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/3c8477df-e281-4e35-8787-9d137012ba34_isrForCrosstalkSources_996953_60.stderr parsl.dataflow.dflow INFO: Task 1968 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 369316592fbe9e96d183b6d09d23d8b7 with result from task 1968 parsl.dataflow.dflow DEBUG: Task 1968 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9b3757e7-f376-404e-a6f4-3552b2baaf62_isrForCrosstalkSources_996960_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9b3757e7-f376-404e-a6f4-3552b2baaf62_isrForCrosstalkSources_996960_17.stderr parsl.dataflow.dflow INFO: Task 1664 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31a1343c98361466601451bc0788a746 with result from task 1664 parsl.dataflow.dflow DEBUG: Task 1664 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/481e0680-9c91-4108-99e6-80db8c9a21f5_isrForCrosstalkSources_996962_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/481e0680-9c91-4108-99e6-80db8c9a21f5_isrForCrosstalkSources_996962_44.stderr parsl.dataflow.dflow INFO: Task 1645 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c44c4291783a81a654978a3b96374d52 with result from task 1645 parsl.dataflow.dflow DEBUG: Task 1645 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/5600fcb8-8e30-44fb-860c-8b122d1701b6_isrForCrosstalkSources_996990_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/5600fcb8-8e30-44fb-860c-8b122d1701b6_isrForCrosstalkSources_996990_57.stderr parsl.dataflow.dflow INFO: Task 1973 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7e74d82b0624817093dcfd1b75be132 with result from task 1973 parsl.dataflow.dflow DEBUG: Task 1973 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ca504218-7396-4930-9ece-ca947907685d_isrForCrosstalkSources_996980_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ca504218-7396-4930-9ece-ca947907685d_isrForCrosstalkSources_996980_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:22 proc_lsst.multi:146 [INFO] found job 17075168 in provider checkpoint proc_lsst.multi INFO: found job 17075168 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1908 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0a8be92334ba2bb37ded9eda4e1aa14 with result from task 1908 parsl.dataflow.dflow DEBUG: Task 1908 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f18b5bec-22f7-4c30-9ef3-fecd32474ebd_isrForCrosstalkSources_996975_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f18b5bec-22f7-4c30-9ef3-fecd32474ebd_isrForCrosstalkSources_996975_62.stderr parsl.dataflow.dflow INFO: Task 1913 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27b9f1ebbbc83b8662be2e6d50876c00 with result from task 1913 parsl.dataflow.dflow DEBUG: Task 1913 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/16525e9d-6ec8-4fa7-bf32-9afbe246b379_isrForCrosstalkSources_996975_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/16525e9d-6ec8-4fa7-bf32-9afbe246b379_isrForCrosstalkSources_996975_58.stderr parsl.dataflow.dflow INFO: Task 1774 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85cdae7df5a3f5382c2c4b949e3ba2ee with result from task 1774 parsl.dataflow.dflow DEBUG: Task 1774 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1044d54b-74d4-41d5-87e0-09f24a0c9e08_isrForCrosstalkSources_997027_59.stdout parsl.dataflow.dflow INFO: Standard error for task 1774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1044d54b-74d4-41d5-87e0-09f24a0c9e08_isrForCrosstalkSources_997027_59.stderr parsl.dataflow.dflow INFO: Task 1688 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2468d98fec2f1b413bd2683a403411ca with result from task 1688 parsl.dataflow.dflow DEBUG: Task 1688 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/8c11e034-a27b-40c0-a18b-f071a0f0a4ae_isrForCrosstalkSources_997001_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/8c11e034-a27b-40c0-a18b-f071a0f0a4ae_isrForCrosstalkSources_997001_38.stderr parsl.dataflow.dflow INFO: Task 1646 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bcde8b6e964d168ee3680d7748e9167c with result from task 1646 parsl.dataflow.dflow DEBUG: Task 1646 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/793f86ef-4e9d-437c-b137-901967300db0_isrForCrosstalkSources_997019_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/793f86ef-4e9d-437c-b137-901967300db0_isrForCrosstalkSources_997019_49.stderr parsl.dataflow.dflow INFO: Task 1656 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03991dbf6dd815a9757bb5ec25a51fed with result from task 1656 parsl.dataflow.dflow DEBUG: Task 1656 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/48833159-ea8f-43b6-aed5-b35a110d8c60_isrForCrosstalkSources_997030_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/48833159-ea8f-43b6-aed5-b35a110d8c60_isrForCrosstalkSources_997030_55.stderr parsl.dataflow.dflow INFO: Task 2014 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e5dd0e74bb23467cc3cd05db70db09de with result from task 2014 parsl.dataflow.dflow DEBUG: Task 2014 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/440d0992-faa3-4d2e-9c18-ae9de223496f_isrForCrosstalkSources_997002_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/440d0992-faa3-4d2e-9c18-ae9de223496f_isrForCrosstalkSources_997002_27.stderr parsl.dataflow.dflow INFO: Task 1653 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a74a69ed2ed5c21bcd1ed09e8ad0126 with result from task 1653 parsl.dataflow.dflow DEBUG: Task 1653 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/a0ec4fbf-d4e2-41f4-b7ad-340c95071a22_isrForCrosstalkSources_997009_35.stdout parsl.dataflow.dflow INFO: Standard error for task 1653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/a0ec4fbf-d4e2-41f4-b7ad-340c95071a22_isrForCrosstalkSources_997009_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:23 proc_lsst.multi:146 [INFO] found job 17075171 in provider checkpoint proc_lsst.multi INFO: found job 17075171 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1659 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53ac8c37d8b3eaeb34fc95b1ed240796 with result from task 1659 parsl.dataflow.dflow DEBUG: Task 1659 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1ba1b0f4-39a1-4d37-a2b7-45ff5717c076_isrForCrosstalkSources_997022_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1ba1b0f4-39a1-4d37-a2b7-45ff5717c076_isrForCrosstalkSources_997022_2.stderr parsl.dataflow.dflow INFO: Task 1755 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c3d272bf29788e67625a8dfe559491a with result from task 1755 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:24 proc_lsst.multi:146 [INFO] found job 17075172 in provider checkpoint proc_lsst.multi INFO: found job 17075172 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 1755 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/14daf1c9-e969-4d42-87f7-20c7d2cc423b_isrForCrosstalkSources_997012_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/14daf1c9-e969-4d42-87f7-20c7d2cc423b_isrForCrosstalkSources_997012_61.stderr parsl.dataflow.dflow INFO: Task 1660 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d5c9772b1430aa09721655f5d2bd2c5 with result from task 1660 parsl.dataflow.dflow DEBUG: Task 1660 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3791e36f-9328-4aab-a2f9-c1af2b4f54d0_isrForCrosstalkSources_997033_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3791e36f-9328-4aab-a2f9-c1af2b4f54d0_isrForCrosstalkSources_997033_55.stderr parsl.dataflow.dflow INFO: Task 1940 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc5b52f909fb16aa4eb8239ca05c11e3 with result from task 1940 parsl.dataflow.dflow DEBUG: Task 1940 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a9a0a067-3249-4105-904d-97ce975ea8ec_isrForCrosstalkSources_996995_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a9a0a067-3249-4105-904d-97ce975ea8ec_isrForCrosstalkSources_996995_40.stderr parsl.dataflow.dflow INFO: Task 2009 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97cf43957487ef7481b154047b003c48 with result from task 2009 parsl.dataflow.dflow DEBUG: Task 2009 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6fe2051a-24e2-46cc-8494-5ac49a517c56_isrForCrosstalkSources_997023_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6fe2051a-24e2-46cc-8494-5ac49a517c56_isrForCrosstalkSources_997023_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:24 proc_lsst.multi:146 [INFO] found job 17075173 in provider checkpoint proc_lsst.multi INFO: found job 17075173 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1626 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f35f83094f38b54bfa21ea6737aef41 with result from task 1626 parsl.dataflow.dflow DEBUG: Task 1626 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ca164872-4693-4c12-a0fe-d3791246c892_isrForCrosstalkSources_996998_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ca164872-4693-4c12-a0fe-d3791246c892_isrForCrosstalkSources_996998_3.stderr parsl.dataflow.dflow INFO: Task 1977 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c87e7e3dfeb13d4b029e00296e453bb with result from task 1977 parsl.dataflow.dflow DEBUG: Task 1977 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/718f4033-221f-4e85-b833-652bdfaebdd3_isrForCrosstalkSources_997021_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/718f4033-221f-4e85-b833-652bdfaebdd3_isrForCrosstalkSources_997021_17.stderr parsl.dataflow.dflow INFO: Task 1744 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8649cd199ef764c058a011656bdd8dd2 with result from task 1744 parsl.dataflow.dflow DEBUG: Task 1744 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/89518e0a-1cca-49fa-aa54-46d08bc5dc1a_isrForCrosstalkSources_996982_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/89518e0a-1cca-49fa-aa54-46d08bc5dc1a_isrForCrosstalkSources_996982_53.stderr parsl.dataflow.dflow INFO: Task 1798 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec2c2927905d1db5fce75dca304159f6 with result from task 1798 parsl.dataflow.dflow DEBUG: Task 1798 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ac135897-d8bc-48b0-aa6b-d43b397a67be_isrForCrosstalkSources_996954_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ac135897-d8bc-48b0-aa6b-d43b397a67be_isrForCrosstalkSources_996954_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:25 proc_lsst.multi:146 [INFO] found job 17075179 in provider checkpoint proc_lsst.multi INFO: found job 17075179 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2034 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02bc3c09c89fe4b07a537a70fcb97062 with result from task 2034 parsl.dataflow.dflow DEBUG: Task 2034 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ad9cc232-d4e5-4d2e-baef-a726008549c5_isrForCrosstalkSources_996989_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ad9cc232-d4e5-4d2e-baef-a726008549c5_isrForCrosstalkSources_996989_3.stderr parsl.dataflow.dflow INFO: Task 1982 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83ec8922cbf718450aea77a67ffa17a6 with result from task 1982 parsl.dataflow.dflow DEBUG: Task 1982 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/1bfa2bd7-850e-4695-a792-44bbb3dc7d8f_isrForCrosstalkSources_996995_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/1bfa2bd7-850e-4695-a792-44bbb3dc7d8f_isrForCrosstalkSources_996995_47.stderr parsl.dataflow.dflow INFO: Task 1779 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry feef489ef20151adbce016db2769001a with result from task 1779 parsl.dataflow.dflow DEBUG: Task 1779 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d9e3ac89-7cea-4e72-a6cb-008001fbb912_isrForCrosstalkSources_996958_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d9e3ac89-7cea-4e72-a6cb-008001fbb912_isrForCrosstalkSources_996958_50.stderr parsl.dataflow.dflow INFO: Task 1681 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 35d187688611f76c5004436d83112bcb with result from task 1681 parsl.dataflow.dflow DEBUG: Task 1681 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/472ea70f-3e7e-4cd7-a544-590e84a643d2_isrForCrosstalkSources_996968_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/472ea70f-3e7e-4cd7-a544-590e84a643d2_isrForCrosstalkSources_996968_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:26 proc_lsst.multi:146 [INFO] found job 17075180 in provider checkpoint proc_lsst.multi INFO: found job 17075180 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2046 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9453beb3557c96aedb84bfaa30288096 with result from task 2046 parsl.dataflow.dflow DEBUG: Task 2046 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/f5bc0dbc-73fd-43ca-b0fa-a7e816089556_isrForCrosstalkSources_996979_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/f5bc0dbc-73fd-43ca-b0fa-a7e816089556_isrForCrosstalkSources_996979_14.stderr parsl.dataflow.dflow INFO: Task 1723 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ed555fe47a16f0cf479f67fd3f1809f with result from task 1723 parsl.dataflow.dflow DEBUG: Task 1723 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/a1d91784-a046-4760-8000-c3d1d2a17063_isrForCrosstalkSources_997005_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/a1d91784-a046-4760-8000-c3d1d2a17063_isrForCrosstalkSources_997005_29.stderr parsl.dataflow.dflow INFO: Task 1891 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25bc6dcd448624e6e0a099e52825959b with result from task 1891 parsl.dataflow.dflow DEBUG: Task 1891 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e1f128fa-f4c5-456f-b8ab-209c199c8405_isrForCrosstalkSources_997005_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e1f128fa-f4c5-456f-b8ab-209c199c8405_isrForCrosstalkSources_997005_43.stderr parsl.dataflow.dflow INFO: Task 1667 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d989d134e475186dc84bad615379d901 with result from task 1667 parsl.dataflow.dflow DEBUG: Task 1667 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/d92ffb1a-644d-444a-b294-ebdc2ccb87cd_isrForCrosstalkSources_997020_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/d92ffb1a-644d-444a-b294-ebdc2ccb87cd_isrForCrosstalkSources_997020_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:26 proc_lsst.multi:146 [INFO] found job 17075181 in provider checkpoint proc_lsst.multi INFO: found job 17075181 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1635 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 239902bdfa9edc9783cbad6f87b10735 with result from task 1635 parsl.dataflow.dflow DEBUG: Task 1635 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/85d09345-5669-4bd8-b619-01392e824a92_isrForCrosstalkSources_997028_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/85d09345-5669-4bd8-b619-01392e824a92_isrForCrosstalkSources_997028_49.stderr parsl.dataflow.dflow INFO: Task 2007 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56921a7418921fb494d83a8fd546cc16 with result from task 2007 parsl.dataflow.dflow DEBUG: Task 2007 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/28c71d6f-9bdf-46b9-8a8e-9940fc8afcf0_isrForCrosstalkSources_997014_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/28c71d6f-9bdf-46b9-8a8e-9940fc8afcf0_isrForCrosstalkSources_997014_10.stderr parsl.dataflow.dflow INFO: Task 1956 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry acf536ad9b7d0de5784d05589e013b96 with result from task 1956 parsl.dataflow.dflow DEBUG: Task 1956 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/450cef44-2e6c-47ff-890c-5f4964f5f1ac_isrForCrosstalkSources_996952_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/450cef44-2e6c-47ff-890c-5f4964f5f1ac_isrForCrosstalkSources_996952_52.stderr parsl.dataflow.dflow INFO: Task 2050 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 43f0411ae986017c74e0fd56ebe76917 with result from task 2050 parsl.dataflow.dflow DEBUG: Task 2050 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/5fb398ef-80c5-4711-9419-59623d23fb71_isrForCrosstalkSources_996973_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/5fb398ef-80c5-4711-9419-59623d23fb71_isrForCrosstalkSources_996973_5.stderr parsl.dataflow.dflow INFO: Task 2023 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8fc1ae8422cf94a570a72248db3c0630 with result from task 2023 parsl.dataflow.dflow DEBUG: Task 2023 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b320fdad-5e9f-4e5e-b030-baa43ede4010_isrForCrosstalkSources_997033_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b320fdad-5e9f-4e5e-b030-baa43ede4010_isrForCrosstalkSources_997033_45.stderr parsl.dataflow.dflow INFO: Task 1981 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96625f56e787f3d03f9cf1ddd1fe3afc with result from task 1981 parsl.dataflow.dflow DEBUG: Task 1981 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/34e1915a-1c0c-4de1-9c4e-982fdd44431b_isrForCrosstalkSources_996969_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/34e1915a-1c0c-4de1-9c4e-982fdd44431b_isrForCrosstalkSources_996969_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:27 proc_lsst.multi:146 [INFO] found job 17075182 in provider checkpoint proc_lsst.multi INFO: found job 17075182 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1726 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de92f561fb389cb0a6d7b2aaf1a9c8d0 with result from task 1726 parsl.dataflow.dflow DEBUG: Task 1726 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/d7c16d33-24eb-46d6-b7fc-d215083874b9_isrForCrosstalkSources_996976_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/d7c16d33-24eb-46d6-b7fc-d215083874b9_isrForCrosstalkSources_996976_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2056 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 330861c64a068b8dc9aca2c23ce3d175 with result from task 2056 parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:27 proc_lsst.multi:146 [INFO] found job 17075183 in provider checkpoint proc_lsst.multi INFO: found job 17075183 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2056 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/eee86dbe-fd25-4582-97f8-b397ed9087ec_isrForCrosstalkSources_997028_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/eee86dbe-fd25-4582-97f8-b397ed9087ec_isrForCrosstalkSources_997028_34.stderr parsl.dataflow.dflow INFO: Task 2057 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3a232e3abbfbbac3b265d1ff6bf3183 with result from task 2057 parsl.dataflow.dflow DEBUG: Task 2057 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/b454fc62-57e0-4c08-80cb-78443a9353d0_isrForCrosstalkSources_997027_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/b454fc62-57e0-4c08-80cb-78443a9353d0_isrForCrosstalkSources_997027_4.stderr parsl.dataflow.dflow INFO: Task 1722 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b44032b9c247cbc2f61193d64a968eab with result from task 1722 parsl.dataflow.dflow DEBUG: Task 1722 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/7b80fd65-3ede-460a-a919-756b70e37b1a_isrForCrosstalkSources_997016_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/7b80fd65-3ede-460a-a919-756b70e37b1a_isrForCrosstalkSources_997016_61.stderr parsl.dataflow.dflow INFO: Task 2059 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5357026e9fdd707352be40ec2d31a481 with result from task 2059 parsl.dataflow.dflow DEBUG: Task 2059 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/218633cb-c11e-434e-a4c2-7c11230619cd_isrForCrosstalkSources_997013_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/218633cb-c11e-434e-a4c2-7c11230619cd_isrForCrosstalkSources_997013_1.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:28 proc_lsst.multi:146 [INFO] found job 17075185 in provider checkpoint proc_lsst.multi INFO: found job 17075185 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1954 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 137f4ff2fe158187a59ffd079a517c76 with result from task 1954 parsl.dataflow.dflow DEBUG: Task 1954 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3ee71b28-0e42-4ddc-b99b-651e22d4c2ad_isrForCrosstalkSources_996959_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3ee71b28-0e42-4ddc-b99b-651e22d4c2ad_isrForCrosstalkSources_996959_27.stderr parsl.dataflow.dflow INFO: Task 1682 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 92afca080574bd7b224941d2ea5c278d with result from task 1682 parsl.dataflow.dflow DEBUG: Task 1682 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f55fe125-2a6b-48ff-9761-69dc1c94e26c_isrForCrosstalkSources_996986_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f55fe125-2a6b-48ff-9761-69dc1c94e26c_isrForCrosstalkSources_996986_58.stderr parsl.dataflow.dflow INFO: Task 1988 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12c776fa337bf52afe4619f84e31011f with result from task 1988 parsl.dataflow.dflow DEBUG: Task 1988 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/678e5ff5-f461-4645-bc18-c470d58a0023_isrForCrosstalkSources_997018_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/678e5ff5-f461-4645-bc18-c470d58a0023_isrForCrosstalkSources_997018_5.stderr parsl.dataflow.dflow INFO: Task 1983 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 74ced5920d50b78cd4ec1815a9d67d3f with result from task 1983 parsl.dataflow.dflow DEBUG: Task 1983 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/fed76d96-ec3f-4a71-8b9e-f42020855e8a_isrForCrosstalkSources_996957_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/fed76d96-ec3f-4a71-8b9e-f42020855e8a_isrForCrosstalkSources_996957_26.stderr parsl.dataflow.dflow INFO: Task 1824 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7dc9620a28693be9aff06a9180933ed with result from task 1824 parsl.dataflow.dflow DEBUG: Task 1824 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d6a21b4c-856a-4b78-931b-9ea25bb1d219_isrForCrosstalkSources_996982_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d6a21b4c-856a-4b78-931b-9ea25bb1d219_isrForCrosstalkSources_996982_29.stderr parsl.dataflow.dflow INFO: Task 1921 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83f3dad69f46b9e88229534ce10f9524 with result from task 1921 parsl.dataflow.dflow DEBUG: Task 1921 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/790f4585-2811-4947-9d50-2c9e0c176682_isrForCrosstalkSources_996962_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/790f4585-2811-4947-9d50-2c9e0c176682_isrForCrosstalkSources_996962_13.stderr parsl.dataflow.dflow INFO: Task 2065 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e0698783132ac91b8d13f4268925648 with result from task 2065 parsl.dataflow.dflow DEBUG: Task 2065 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/b550d3aa-2d81-4e2a-a907-46809332c2dc_isrForCrosstalkSources_997009_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/b550d3aa-2d81-4e2a-a907-46809332c2dc_isrForCrosstalkSources_997009_10.stderr parsl.dataflow.dflow INFO: Task 1894 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8893318748153d266135ef93b3d1dc2a with result from task 1894 parsl.dataflow.dflow DEBUG: Task 1894 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/54bf96cf-750e-4721-95cc-840aeb33f6aa_isrForCrosstalkSources_996989_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/54bf96cf-750e-4721-95cc-840aeb33f6aa_isrForCrosstalkSources_996989_28.stderr parsl.dataflow.dflow INFO: Task 1846 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88685816f54f03ed87cfa5fcabd5f6be with result from task 1846 parsl.dataflow.dflow DEBUG: Task 1846 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0ce80222-7095-4a9b-b68e-6b6da0cc59f8_isrForCrosstalkSources_996953_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0ce80222-7095-4a9b-b68e-6b6da0cc59f8_isrForCrosstalkSources_996953_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:28 proc_lsst.multi:146 [INFO] found job 17075187 in provider checkpoint proc_lsst.multi INFO: found job 17075187 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1863 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb069741deedf3ad5513b6a26689591a with result from task 1863 parsl.dataflow.dflow DEBUG: Task 1863 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a9cd22d3-11bd-4a7f-8495-a95f2f695b1a_isrForCrosstalkSources_997032_41.stdout parsl.dataflow.dflow INFO: Standard error for task 1863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a9cd22d3-11bd-4a7f-8495-a95f2f695b1a_isrForCrosstalkSources_997032_41.stderr parsl.dataflow.dflow INFO: Task 2036 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6e21aec0c9b076de518876c9244dd064 with result from task 2036 parsl.dataflow.dflow DEBUG: Task 2036 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/8f6b3902-0227-4162-a8bc-16966b48a832_isrForCrosstalkSources_997006_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/8f6b3902-0227-4162-a8bc-16966b48a832_isrForCrosstalkSources_997006_19.stderr parsl.dataflow.dflow INFO: Task 2058 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a9b3d264efbdd623808ebd822b52e2e with result from task 2058 parsl.dataflow.dflow DEBUG: Task 2058 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e04429c3-07e5-4e0f-9a42-0eeb22b9c391_isrForCrosstalkSources_997010_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e04429c3-07e5-4e0f-9a42-0eeb22b9c391_isrForCrosstalkSources_997010_47.stderr parsl.dataflow.dflow INFO: Task 2005 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry afa0ef468a229abcb4482bfdd42fae12 with result from task 2005 parsl.dataflow.dflow DEBUG: Task 2005 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/64612e54-3f42-4e26-9787-3c6e56a24176_isrForCrosstalkSources_997028_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/64612e54-3f42-4e26-9787-3c6e56a24176_isrForCrosstalkSources_997028_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:29 proc_lsst.multi:146 [INFO] found job 17075188 in provider checkpoint proc_lsst.multi INFO: found job 17075188 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2038 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c5b4694e7a37cb86995680ac59a61e4 with result from task 2038 parsl.dataflow.dflow DEBUG: Task 2038 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c6a00ffa-44f7-4ede-b498-3a50a4e2bc5b_isrForCrosstalkSources_997000_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/c6a00ffa-44f7-4ede-b498-3a50a4e2bc5b_isrForCrosstalkSources_997000_8.stderr parsl.dataflow.dflow INFO: Task 2020 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ba14ac1bd2c0f1eb7b2f0311e3dc025 with result from task 2020 parsl.dataflow.dflow DEBUG: Task 2020 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/73070103-ceb2-4a44-988b-59385aee60bb_isrForCrosstalkSources_997000_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/73070103-ceb2-4a44-988b-59385aee60bb_isrForCrosstalkSources_997000_27.stderr parsl.dataflow.dflow INFO: Task 2064 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3174533675ccbedfb7734c6f2e5a3acf with result from task 2064 parsl.dataflow.dflow DEBUG: Task 2064 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cd071930-647a-43ae-bfb7-4f77bb3916b3_isrForCrosstalkSources_996987_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cd071930-647a-43ae-bfb7-4f77bb3916b3_isrForCrosstalkSources_996987_54.stderr parsl.dataflow.dflow INFO: Task 1960 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31b2ff3d4885e550d018d0c834444990 with result from task 1960 parsl.dataflow.dflow DEBUG: Task 1960 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/61d41d83-e8b7-4ee6-82ee-d437db62ba82_isrForCrosstalkSources_996973_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/61d41d83-e8b7-4ee6-82ee-d437db62ba82_isrForCrosstalkSources_996973_40.stderr parsl.dataflow.dflow INFO: Task 1943 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5822d8621f599d3ab4f62a154f11945b with result from task 1943 parsl.dataflow.dflow DEBUG: Task 1943 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9b3e5fec-1228-492f-a120-fdb946a467bc_isrForCrosstalkSources_996968_26.stdout parsl.dataflow.dflow INFO: Standard error for task 1943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9b3e5fec-1228-492f-a120-fdb946a467bc_isrForCrosstalkSources_996968_26.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:29 proc_lsst.multi:146 [INFO] found job 17075190 in provider checkpoint proc_lsst.multi INFO: found job 17075190 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2017 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d1bc242cf71a13db66aa8d5e581184d9 with result from task 2017 parsl.dataflow.dflow DEBUG: Task 2017 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/56b151ce-6c0e-40e9-9c92-eff1ad752b97_isrForCrosstalkSources_997017_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/56b151ce-6c0e-40e9-9c92-eff1ad752b97_isrForCrosstalkSources_997017_47.stderr parsl.dataflow.dflow INFO: Task 2021 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e254aca02f91199a114e33d23608b435 with result from task 2021 parsl.dataflow.dflow DEBUG: Task 2021 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/1998069b-d873-4442-bd2c-427bbc3210fe_isrForCrosstalkSources_996965_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/1998069b-d873-4442-bd2c-427bbc3210fe_isrForCrosstalkSources_996965_17.stderr parsl.dataflow.dflow INFO: Task 1952 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4a83dfcccac4a6a55becc1408f5a1fe with result from task 1952 parsl.dataflow.dflow DEBUG: Task 1952 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/c9ac5096-a490-4731-b552-b20e4f7f77c8_isrForCrosstalkSources_996981_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/c9ac5096-a490-4731-b552-b20e4f7f77c8_isrForCrosstalkSources_996981_25.stderr parsl.dataflow.dflow INFO: Task 2025 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb6bc82bd1cb8f262161e16b364ee917 with result from task 2025 parsl.dataflow.dflow DEBUG: Task 2025 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a938a158-000e-448f-a018-3cedf6080e97_isrForCrosstalkSources_997008_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a938a158-000e-448f-a018-3cedf6080e97_isrForCrosstalkSources_997008_26.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:30 proc_lsst.multi:146 [INFO] found job 17075191 in provider checkpoint proc_lsst.multi INFO: found job 17075191 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2066 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d00d832fb30e09e34b3d09d45e94d539 with result from task 2066 parsl.dataflow.dflow DEBUG: Task 2066 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a13e2926-0edc-4419-bfd9-625e6b0f136a_isrForCrosstalkSources_996965_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a13e2926-0edc-4419-bfd9-625e6b0f136a_isrForCrosstalkSources_996965_44.stderr parsl.dataflow.dflow INFO: Task 2011 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7663d37b78276cb5c33440bcaa30ae89 with result from task 2011 parsl.dataflow.dflow DEBUG: Task 2011 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/8de5960e-799f-4e7b-aa06-38b149d73eea_isrForCrosstalkSources_996991_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/8de5960e-799f-4e7b-aa06-38b149d73eea_isrForCrosstalkSources_996991_26.stderr parsl.dataflow.dflow INFO: Task 2033 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2cbacf4802f6c95e61adabeac4cd660 with result from task 2033 parsl.dataflow.dflow DEBUG: Task 2033 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/da3f8c0d-73a7-497c-9b9a-109a5b28e740_isrForCrosstalkSources_996970_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/da3f8c0d-73a7-497c-9b9a-109a5b28e740_isrForCrosstalkSources_996970_16.stderr parsl.dataflow.dflow INFO: Task 2030 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31ab6667996e5775e2f912ad94ffa4d0 with result from task 2030 parsl.dataflow.dflow DEBUG: Task 2030 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2b5492c8-578f-4de5-906a-a3b469e8440d_isrForCrosstalkSources_996956_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2b5492c8-578f-4de5-906a-a3b469e8440d_isrForCrosstalkSources_996956_41.stderr parsl.dataflow.dflow INFO: Task 2012 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aad29a4233eed83819742fadba309aec with result from task 2012 parsl.dataflow.dflow DEBUG: Task 2012 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/83a84137-71ef-4510-9bca-a08ab10b0a49_isrForCrosstalkSources_997030_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/83a84137-71ef-4510-9bca-a08ab10b0a49_isrForCrosstalkSources_997030_32.stderr parsl.dataflow.dflow INFO: Task 1998 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2dc0aff9e0ea7d9924e90e424849cfbe with result from task 1998 parsl.dataflow.dflow DEBUG: Task 1998 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0a824ce0-f8b5-42b7-be73-383357e39ccf_isrForCrosstalkSources_996956_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0a824ce0-f8b5-42b7-be73-383357e39ccf_isrForCrosstalkSources_996956_17.stderr parsl.dataflow.dflow INFO: Task 1991 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a3441c6b81a487cc52db8ef59fbe9e2 with result from task 1991 parsl.dataflow.dflow DEBUG: Task 1991 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/868e4092-be79-4164-9873-16f4f7d2272b_isrForCrosstalkSources_996993_13.stdout parsl.dataflow.dflow INFO: Standard error for task 1991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/868e4092-be79-4164-9873-16f4f7d2272b_isrForCrosstalkSources_996993_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:31 proc_lsst.multi:146 [INFO] found job 17075197 in provider checkpoint proc_lsst.multi INFO: found job 17075197 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:31 proc_lsst.multi:146 [INFO] found job 17075200 in provider checkpoint proc_lsst.multi INFO: found job 17075200 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2052 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e63589d48f9088d9b93c478d0795f2b2 with result from task 2052 parsl.dataflow.dflow DEBUG: Task 2052 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/4a939833-bc18-4c4f-8fb8-ac724c515195_isrForCrosstalkSources_996996_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/4a939833-bc18-4c4f-8fb8-ac724c515195_isrForCrosstalkSources_996996_20.stderr parsl.dataflow.dflow INFO: Task 1951 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd0666a55de3adb7d8bdab191046db08 with result from task 1951 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:32 proc_lsst.multi:146 [INFO] found job 17075201 in provider checkpoint proc_lsst.multi INFO: found job 17075201 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 1951 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/29e928a1-cee3-4574-9f98-533ac6e205d8_isrForCrosstalkSources_997009_17.stdout parsl.dataflow.dflow INFO: Standard error for task 1951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/29e928a1-cee3-4574-9f98-533ac6e205d8_isrForCrosstalkSources_997009_17.stderr parsl.dataflow.dflow INFO: Task 1948 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7cd25e6722483550588da0ff14dde1a with result from task 1948 parsl.dataflow.dflow DEBUG: Task 1948 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7a182a1c-e1d6-48e4-82de-affc1d80ea2a_isrForCrosstalkSources_996953_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7a182a1c-e1d6-48e4-82de-affc1d80ea2a_isrForCrosstalkSources_996953_40.stderr parsl.dataflow.dflow INFO: Task 2072 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3ac063dcafc5fa14f207acc9dbfcc547 with result from task 2072 parsl.dataflow.dflow DEBUG: Task 2072 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/10b72434-a514-455b-b5ab-eb13ba488999_isrForCrosstalkSources_997002_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/10b72434-a514-455b-b5ab-eb13ba488999_isrForCrosstalkSources_997002_19.stderr parsl.dataflow.dflow INFO: Task 2006 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f48ed7edf2375fadf72c436007a0c77 with result from task 2006 parsl.dataflow.dflow DEBUG: Task 2006 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/6b019dee-bd9a-499a-a477-fb99c02de5d4_isrForCrosstalkSources_997018_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/6b019dee-bd9a-499a-a477-fb99c02de5d4_isrForCrosstalkSources_997018_15.stderr parsl.dataflow.dflow INFO: Task 2055 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f89427ca32e0635c9619db59be83cf92 with result from task 2055 parsl.dataflow.dflow DEBUG: Task 2055 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6852f9d9-4afc-43f5-ad87-5dc81b271465_isrForCrosstalkSources_996970_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6852f9d9-4afc-43f5-ad87-5dc81b271465_isrForCrosstalkSources_996970_27.stderr parsl.dataflow.dflow INFO: Task 2016 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 484aa73b174a7ac75f4797e6aa8670fa with result from task 2016 parsl.dataflow.dflow DEBUG: Task 2016 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2deb948d-d35d-4f98-8d23-325ecd54b629_isrForCrosstalkSources_997005_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2deb948d-d35d-4f98-8d23-325ecd54b629_isrForCrosstalkSources_997005_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:32 proc_lsst.multi:146 [INFO] found job 17075202 in provider checkpoint proc_lsst.multi INFO: found job 17075202 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1974 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8a28c1749d241c1da0480022a1739b1 with result from task 1974 parsl.dataflow.dflow DEBUG: Task 1974 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a0a798cf-81f9-4ad3-92f9-924c07cadb52_isrForCrosstalkSources_996972_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a0a798cf-81f9-4ad3-92f9-924c07cadb52_isrForCrosstalkSources_996972_39.stderr parsl.dataflow.dflow INFO: Task 2063 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 842334b6e02bb9c58d658653747b9f6c with result from task 2063 parsl.dataflow.dflow DEBUG: Task 2063 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c2a76dbf-7a60-441c-a6e4-d564ec97133b_isrForCrosstalkSources_997006_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c2a76dbf-7a60-441c-a6e4-d564ec97133b_isrForCrosstalkSources_997006_11.stderr parsl.dataflow.dflow INFO: Task 1950 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a05c12340dc887cbcd86684b969417a with result from task 1950 parsl.dataflow.dflow DEBUG: Task 1950 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4abe25be-86dc-4ce3-9921-4b467aa7ca56_isrForCrosstalkSources_997030_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4abe25be-86dc-4ce3-9921-4b467aa7ca56_isrForCrosstalkSources_997030_22.stderr parsl.dataflow.dflow INFO: Task 2075 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 497f6d109cc1c61709f5cf538cc5e7ae with result from task 2075 parsl.dataflow.dflow DEBUG: Task 2075 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/1327477d-d419-48e2-aa30-e0008fb2ed6a_isrForCrosstalkSources_996965_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/1327477d-d419-48e2-aa30-e0008fb2ed6a_isrForCrosstalkSources_996965_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:33 proc_lsst.multi:146 [INFO] found job 17075203 in provider checkpoint proc_lsst.multi INFO: found job 17075203 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2087 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0fb2d66e328df5325c33c5c3c918b631 with result from task 2087 parsl.dataflow.dflow DEBUG: Task 2087 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7cef2bcc-c5e5-4e26-9c0e-63d783b3c368_isrForCrosstalkSources_997013_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7cef2bcc-c5e5-4e26-9c0e-63d783b3c368_isrForCrosstalkSources_997013_27.stderr parsl.dataflow.dflow INFO: Task 2074 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d869d55433a23b01b88f1698a88c5630 with result from task 2074 parsl.dataflow.dflow DEBUG: Task 2074 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/9d2c7896-0c38-4e1f-97ca-c242b52e4359_isrForCrosstalkSources_996987_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/9d2c7896-0c38-4e1f-97ca-c242b52e4359_isrForCrosstalkSources_996987_19.stderr parsl.dataflow.dflow INFO: Task 2092 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44b04ab420f4b2d92efe8e42a3643377 with result from task 2092 parsl.dataflow.dflow DEBUG: Task 2092 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1573376a-060e-4212-9ab8-42171479245a_isrForCrosstalkSources_997003_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1573376a-060e-4212-9ab8-42171479245a_isrForCrosstalkSources_997003_34.stderr parsl.dataflow.dflow INFO: Task 1959 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12fe04c81d27829e58092196b69c4dfb with result from task 1959 parsl.dataflow.dflow DEBUG: Task 1959 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/42d0b89d-2c2f-4eae-8b8d-9a4a29babc40_isrForCrosstalkSources_996961_23.stdout parsl.dataflow.dflow INFO: Standard error for task 1959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/42d0b89d-2c2f-4eae-8b8d-9a4a29babc40_isrForCrosstalkSources_996961_23.stderr parsl.dataflow.dflow INFO: Task 2124 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 64ff4a005831b3ca70de02093967506d with result from task 2124 parsl.dataflow.dflow DEBUG: Task 2124 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e5a40d57-7657-490a-8316-21f365596d4d_isrForCrosstalkSources_996989_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e5a40d57-7657-490a-8316-21f365596d4d_isrForCrosstalkSources_996989_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:33 proc_lsst.multi:146 [INFO] found job 17075204 in provider checkpoint proc_lsst.multi INFO: found job 17075204 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1975 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0106cba6a393b10afcbd2c8563e3d5f7 with result from task 1975 parsl.dataflow.dflow DEBUG: Task 1975 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bed4f0fe-1b8c-447d-ac89-5267b5690dfb_isrForCrosstalkSources_997027_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bed4f0fe-1b8c-447d-ac89-5267b5690dfb_isrForCrosstalkSources_997027_14.stderr parsl.dataflow.dflow INFO: Task 2104 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e3086ce81b47ef20837a5fdc6f0b491 with result from task 2104 parsl.dataflow.dflow DEBUG: Task 2104 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b74af192-0fa8-4ac2-ab5c-29b91db5deff_isrForCrosstalkSources_996992_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b74af192-0fa8-4ac2-ab5c-29b91db5deff_isrForCrosstalkSources_996992_32.stderr parsl.dataflow.dflow INFO: Task 1970 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a10c55c366eb43596d1aea7501969232 with result from task 1970 parsl.dataflow.dflow DEBUG: Task 1970 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/1c5f1024-71bc-487d-9dee-7dc0a86f6e15_isrForCrosstalkSources_997015_45.stdout parsl.dataflow.dflow INFO: Standard error for task 1970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/1c5f1024-71bc-487d-9dee-7dc0a86f6e15_isrForCrosstalkSources_997015_45.stderr parsl.dataflow.dflow INFO: Task 2098 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec6b590730c48d3cca754c013d5d45f1 with result from task 2098 parsl.dataflow.dflow DEBUG: Task 2098 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/b372cf59-1da0-49a4-9bf4-1ded02d4e7d3_isrForCrosstalkSources_996968_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/b372cf59-1da0-49a4-9bf4-1ded02d4e7d3_isrForCrosstalkSources_996968_20.stderr parsl.dataflow.dflow INFO: Task 2010 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af67f84a3cfaefd97a4e6546cc861271 with result from task 2010 parsl.dataflow.dflow DEBUG: Task 2010 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/88cc38d7-98c0-4ef6-a905-3ce904433aff_isrForCrosstalkSources_997012_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/88cc38d7-98c0-4ef6-a905-3ce904433aff_isrForCrosstalkSources_997012_30.stderr parsl.dataflow.dflow INFO: Task 1978 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 184383a0320823f658c4dc2d375acbd2 with result from task 1978 parsl.dataflow.dflow DEBUG: Task 1978 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/752535a3-569d-470c-b4d5-06c286baf99f_isrForCrosstalkSources_996976_16.stdout parsl.dataflow.dflow INFO: Standard error for task 1978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/752535a3-569d-470c-b4d5-06c286baf99f_isrForCrosstalkSources_996976_16.stderr parsl.dataflow.dflow INFO: Task 2031 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d157e8e41c57b8687ebe2e542a038268 with result from task 2031 parsl.dataflow.dflow DEBUG: Task 2031 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/91448bb0-f229-4da6-bbbb-5dfe5a59c5f2_isrForCrosstalkSources_996986_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/91448bb0-f229-4da6-bbbb-5dfe5a59c5f2_isrForCrosstalkSources_996986_14.stderr parsl.dataflow.dflow INFO: Task 2002 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1278335129227cf2c7d15e9db1c0eab with result from task 2002 parsl.dataflow.dflow DEBUG: Task 2002 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/465a3f94-e236-4360-855e-065dc1c397a5_isrForCrosstalkSources_996988_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/465a3f94-e236-4360-855e-065dc1c397a5_isrForCrosstalkSources_996988_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:34 proc_lsst.multi:146 [INFO] found job 17075205 in provider checkpoint proc_lsst.multi INFO: found job 17075205 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1985 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 913db9cc7ec0c1b9196a58d6528e5394 with result from task 1985 parsl.dataflow.dflow DEBUG: Task 1985 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b668db7a-690b-4077-9d4e-d8b2debf2e7c_isrForCrosstalkSources_996992_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/b668db7a-690b-4077-9d4e-d8b2debf2e7c_isrForCrosstalkSources_996992_40.stderr parsl.dataflow.dflow INFO: Task 1999 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc7c2b5e71f43126e93fcb75ac7821e3 with result from task 1999 parsl.dataflow.dflow DEBUG: Task 1999 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9ca305cd-194f-49f4-84dd-53ef2358e2d8_isrForCrosstalkSources_997018_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9ca305cd-194f-49f4-84dd-53ef2358e2d8_isrForCrosstalkSources_997018_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:34 proc_lsst.multi:146 [INFO] found job 17075206 in provider checkpoint proc_lsst.multi INFO: found job 17075206 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1964 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83efd2d4140f48b149aaf711029c54c7 with result from task 1964 parsl.dataflow.dflow DEBUG: Task 1964 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c19ba78a-34cf-42ee-a030-f4fc05f01a52_isrForCrosstalkSources_997016_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c19ba78a-34cf-42ee-a030-f4fc05f01a52_isrForCrosstalkSources_997016_42.stderr parsl.dataflow.dflow INFO: Task 2103 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4046446424005210e33ddde2dd4d72dd with result from task 2103 parsl.dataflow.dflow DEBUG: Task 2103 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/035d01b3-c4cd-41e6-95b7-ccd13be9a608_isrForCrosstalkSources_996959_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/035d01b3-c4cd-41e6-95b7-ccd13be9a608_isrForCrosstalkSources_996959_39.stderr parsl.dataflow.dflow INFO: Task 2083 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e21e93226d5c127c68ac4d80a113bdd5 with result from task 2083 parsl.dataflow.dflow DEBUG: Task 2083 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b51a0ab0-f2f1-448d-965d-6748457e1016_isrForCrosstalkSources_997013_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b51a0ab0-f2f1-448d-965d-6748457e1016_isrForCrosstalkSources_997013_12.stderr parsl.dataflow.dflow INFO: Task 1928 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fece7fa9233e69a6fad1ff15334714e with result from task 1928 parsl.dataflow.dflow DEBUG: Task 1928 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/566ebdd4-ce5a-4528-9866-287f2aa1079e_isrForCrosstalkSources_996973_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/566ebdd4-ce5a-4528-9866-287f2aa1079e_isrForCrosstalkSources_996973_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2070 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry e8ae9f32dfa0f1bb756360e99f1f6c26 with result from task 2070 parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:35 proc_lsst.multi:146 [INFO] found job 17075207 in provider checkpoint proc_lsst.multi INFO: found job 17075207 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2070 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/b23787fd-6c36-4b54-ae4b-89bea30fd99e_isrForCrosstalkSources_997029_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/b23787fd-6c36-4b54-ae4b-89bea30fd99e_isrForCrosstalkSources_997029_14.stderr parsl.dataflow.dflow INFO: Task 1838 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f69cae33e50e03af0e29b985a2c5d68d with result from task 1838 parsl.dataflow.dflow DEBUG: Task 1838 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9e06a674-19c9-480b-930e-c6507ad86e10_isrForCrosstalkSources_997030_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9e06a674-19c9-480b-930e-c6507ad86e10_isrForCrosstalkSources_997030_34.stderr parsl.dataflow.dflow INFO: Task 1837 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7715fde651d85f0be014ef28cc3b20d with result from task 1837 parsl.dataflow.dflow DEBUG: Task 1837 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/792f8fd2-a2e3-4017-bb0d-bb939d3d2da6_isrForCrosstalkSources_997031_27.stdout parsl.dataflow.dflow INFO: Standard error for task 1837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/792f8fd2-a2e3-4017-bb0d-bb939d3d2da6_isrForCrosstalkSources_997031_27.stderr parsl.dataflow.dflow INFO: Task 1836 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ebe9179ce5d61aae37a27111cc9665c with result from task 1836 parsl.dataflow.dflow DEBUG: Task 1836 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/627de1a7-5ed8-43fd-a6d0-89a15bf08e73_isrForCrosstalkSources_996985_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/627de1a7-5ed8-43fd-a6d0-89a15bf08e73_isrForCrosstalkSources_996985_33.stderr parsl.dataflow.dflow INFO: Task 1841 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d86b03e317a95cc0415904231fa4d1a0 with result from task 1841 parsl.dataflow.dflow DEBUG: Task 1841 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c9dab5ab-4a6a-419d-ba02-e36aa1d9a17d_isrForCrosstalkSources_997012_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c9dab5ab-4a6a-419d-ba02-e36aa1d9a17d_isrForCrosstalkSources_997012_25.stderr parsl.dataflow.dflow INFO: Task 1949 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58d8b9d225485e1273defab5562632e4 with result from task 1949 parsl.dataflow.dflow DEBUG: Task 1949 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/76d8428f-59d9-4e8c-9e2e-2204c0573cf3_isrForCrosstalkSources_996961_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/76d8428f-59d9-4e8c-9e2e-2204c0573cf3_isrForCrosstalkSources_996961_44.stderr parsl.dataflow.dflow INFO: Task 2053 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b294313b8e58fe6cecc951bc8a0e4c8a with result from task 2053 parsl.dataflow.dflow DEBUG: Task 2053 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60aa17cb-d23e-46fe-8e28-de1aebc07c1a_isrForCrosstalkSources_996968_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/60aa17cb-d23e-46fe-8e28-de1aebc07c1a_isrForCrosstalkSources_996968_12.stderr parsl.dataflow.dflow INFO: Task 2077 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1190e6ec21f46989d45a9f6b13f1a7df with result from task 2077 parsl.dataflow.dflow DEBUG: Task 2077 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/99582163-b280-48cf-addc-28e931ecedee_isrForCrosstalkSources_997030_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/99582163-b280-48cf-addc-28e931ecedee_isrForCrosstalkSources_997030_45.stderr parsl.dataflow.dflow INFO: Task 2081 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7abe43952bd589d80b9c6d1526416f6c with result from task 2081 parsl.dataflow.dflow DEBUG: Task 2081 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a3ea756-2f16-4cde-8bcf-9e404a264d8d_isrForCrosstalkSources_996994_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a3ea756-2f16-4cde-8bcf-9e404a264d8d_isrForCrosstalkSources_996994_7.stderr parsl.dataflow.dflow INFO: Task 2073 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f6c58dc59b79db053538824668256663 with result from task 2073 parsl.dataflow.dflow DEBUG: Task 2073 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/11ba2719-16e3-4062-aeb9-03e0196db163_isrForCrosstalkSources_996966_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/11ba2719-16e3-4062-aeb9-03e0196db163_isrForCrosstalkSources_996966_54.stderr parsl.dataflow.dflow INFO: Task 2049 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6bb1d60b5fe040bca8a0e6832b442f99 with result from task 2049 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:35 proc_lsst.multi:146 [INFO] found job 17075208 in provider checkpoint proc_lsst.multi INFO: found job 17075208 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2049 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/353e2760-3496-41f6-9aa3-ead0cf21a449_isrForCrosstalkSources_996960_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/353e2760-3496-41f6-9aa3-ead0cf21a449_isrForCrosstalkSources_996960_9.stderr parsl.dataflow.dflow INFO: Task 1979 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7abd7593787f140f26892e95ed4106a0 with result from task 1979 parsl.dataflow.dflow DEBUG: Task 1979 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4ab92762-2f00-4d69-82f2-52932f3f46c6_isrForCrosstalkSources_996954_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4ab92762-2f00-4d69-82f2-52932f3f46c6_isrForCrosstalkSources_996954_22.stderr parsl.dataflow.dflow INFO: Task 2062 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6250a43dc709a2fd9ce614f20fe9be4a with result from task 2062 parsl.dataflow.dflow DEBUG: Task 2062 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a384147c-573c-4972-99c2-db853ef76891_isrForCrosstalkSources_997018_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a384147c-573c-4972-99c2-db853ef76891_isrForCrosstalkSources_997018_41.stderr parsl.dataflow.dflow INFO: Task 1986 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bdb174bf7fe78c9500c1766ba12dbb30 with result from task 1986 parsl.dataflow.dflow DEBUG: Task 1986 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fcaff9e1-37fb-40f5-a371-8b2b74e92a7b_isrForCrosstalkSources_997008_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fcaff9e1-37fb-40f5-a371-8b2b74e92a7b_isrForCrosstalkSources_997008_7.stderr parsl.dataflow.dflow INFO: Task 2129 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c059a48516dd682f92537c4017ec2d5 with result from task 2129 parsl.dataflow.dflow DEBUG: Task 2129 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/0455eab6-59e6-478c-ab4c-e14d90e17990_isrForCrosstalkSources_996969_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/0455eab6-59e6-478c-ab4c-e14d90e17990_isrForCrosstalkSources_996969_19.stderr parsl.dataflow.dflow INFO: Task 2111 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27170c60d23c6f4b657a1fb53aa18aa1 with result from task 2111 parsl.dataflow.dflow DEBUG: Task 2111 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/adc6ffbf-0757-4042-b90d-009831bb3333_isrForCrosstalkSources_996957_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/adc6ffbf-0757-4042-b90d-009831bb3333_isrForCrosstalkSources_996957_54.stderr parsl.dataflow.dflow INFO: Task 2096 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 948bf9c131124dba1c0f18963a9f0017 with result from task 2096 parsl.dataflow.dflow DEBUG: Task 2096 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/e53e8533-bcc2-48b0-8be9-7682d862ef60_isrForCrosstalkSources_996981_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/e53e8533-bcc2-48b0-8be9-7682d862ef60_isrForCrosstalkSources_996981_19.stderr parsl.dataflow.dflow INFO: Task 2067 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f0d2c2fcc9d07250c87c871aeb0d32cc with result from task 2067 parsl.dataflow.dflow DEBUG: Task 2067 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/1e913048-fceb-44a0-9c7b-baaa901d0f54_isrForCrosstalkSources_996996_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/1e913048-fceb-44a0-9c7b-baaa901d0f54_isrForCrosstalkSources_996996_6.stderr parsl.dataflow.dflow INFO: Task 2069 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 06aa34e5f2ed84850317ade270d26389 with result from task 2069 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:36 proc_lsst.multi:146 [INFO] found job 17075209 in provider checkpoint proc_lsst.multi INFO: found job 17075209 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2069 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8039e204-d190-4f94-9ec6-8c975b98b7ab_isrForCrosstalkSources_996984_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8039e204-d190-4f94-9ec6-8c975b98b7ab_isrForCrosstalkSources_996984_39.stderr parsl.dataflow.dflow INFO: Task 1942 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90243d312824d085324613974348893f with result from task 1942 parsl.dataflow.dflow DEBUG: Task 1942 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8f6118b7-3635-4387-a9be-051a69f294f9_isrForCrosstalkSources_997017_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8f6118b7-3635-4387-a9be-051a69f294f9_isrForCrosstalkSources_997017_10.stderr parsl.dataflow.dflow INFO: Task 2113 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7098313913473cccd5965e830ed3036 with result from task 2113 parsl.dataflow.dflow DEBUG: Task 2113 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/687b9fea-c9cf-46f0-9f2b-60366f337de8_isrForCrosstalkSources_996955_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/687b9fea-c9cf-46f0-9f2b-60366f337de8_isrForCrosstalkSources_996955_38.stderr parsl.dataflow.dflow INFO: Task 1934 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a7766d4bbdd31cd49ba135eedfdaae0e with result from task 1934 parsl.dataflow.dflow DEBUG: Task 1934 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/0e6e877d-221d-4f03-9465-337c186cb900_isrForCrosstalkSources_996992_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/0e6e877d-221d-4f03-9465-337c186cb900_isrForCrosstalkSources_996992_46.stderr parsl.dataflow.dflow INFO: Task 2147 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 641d8d80a7a3b751f084016672f409ea with result from task 2147 parsl.dataflow.dflow DEBUG: Task 2147 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/88141355-6069-485d-a329-395cdf2f7356_isrForCrosstalkSources_996973_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/88141355-6069-485d-a329-395cdf2f7356_isrForCrosstalkSources_996973_13.stderr parsl.dataflow.dflow INFO: Task 2082 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b567174062ff18dc446d6bff07b7f6e6 with result from task 2082 parsl.dataflow.dflow DEBUG: Task 2082 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8746edce-d51a-4ea9-a99c-87ed349dbf3f_isrForCrosstalkSources_997018_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8746edce-d51a-4ea9-a99c-87ed349dbf3f_isrForCrosstalkSources_997018_9.stderr parsl.dataflow.dflow INFO: Task 2118 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 68b0df7921f4c5f786ba8a8e799ff372 with result from task 2118 parsl.dataflow.dflow DEBUG: Task 2118 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cc660a0c-c3cd-4945-be5f-79cee0696b44_isrForCrosstalkSources_996983_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cc660a0c-c3cd-4945-be5f-79cee0696b44_isrForCrosstalkSources_996983_48.stderr parsl.dataflow.dflow INFO: Task 2084 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fa4df54ed0ffb3adb1064722832474c with result from task 2084 parsl.dataflow.dflow DEBUG: Task 2084 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e9977d60-d4b1-4019-b174-ae2ca8cab3ad_isrForCrosstalkSources_996997_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e9977d60-d4b1-4019-b174-ae2ca8cab3ad_isrForCrosstalkSources_996997_19.stderr parsl.dataflow.dflow INFO: Task 2015 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b681a4a617e1fe711fcea1396289fcc with result from task 2015 parsl.dataflow.dflow DEBUG: Task 2015 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9f825bd4-3d5d-4ce9-9a17-97b2c559cb80_isrForCrosstalkSources_996960_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9f825bd4-3d5d-4ce9-9a17-97b2c559cb80_isrForCrosstalkSources_996960_22.stderr parsl.dataflow.dflow INFO: Task 2108 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b170325de9bb7ae35aa0d3d0386a27c3 with result from task 2108 parsl.dataflow.dflow DEBUG: Task 2108 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c5963900-7749-4a1e-997f-15784dc26432_isrForCrosstalkSources_996962_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c5963900-7749-4a1e-997f-15784dc26432_isrForCrosstalkSources_996962_38.stderr parsl.dataflow.dflow INFO: Task 2106 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d4c1bae2ac45425478ef71b48868cd3 with result from task 2106 parsl.dataflow.dflow DEBUG: Task 2106 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/840482be-b198-4b32-b82f-852c54f7db90_isrForCrosstalkSources_996953_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/840482be-b198-4b32-b82f-852c54f7db90_isrForCrosstalkSources_996953_9.stderr parsl.dataflow.dflow INFO: Task 1873 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c0762b6fef6db8778dc526f3f9bb6f3 with result from task 1873 parsl.dataflow.dflow DEBUG: Task 1873 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0a3558b9-1eae-4c56-a4b0-dc51718fccb3_isrForCrosstalkSources_996981_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0a3558b9-1eae-4c56-a4b0-dc51718fccb3_isrForCrosstalkSources_996981_55.stderr parsl.dataflow.dflow INFO: Task 1874 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b346e646d90fd568a265223f830a8152 with result from task 1874 parsl.dataflow.dflow DEBUG: Task 1874 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/deb0f44e-33d0-4950-891a-f27b14a764b2_isrForCrosstalkSources_997017_44.stdout parsl.dataflow.dflow INFO: Standard error for task 1874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/deb0f44e-33d0-4950-891a-f27b14a764b2_isrForCrosstalkSources_997017_44.stderr parsl.dataflow.dflow INFO: Task 1870 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b991d3905a0133db50f252688662e15 with result from task 1870 parsl.dataflow.dflow DEBUG: Task 1870 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9d51e1dd-a95e-42f4-aefa-79befa309790_isrForCrosstalkSources_996958_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9d51e1dd-a95e-42f4-aefa-79befa309790_isrForCrosstalkSources_996958_12.stderr parsl.dataflow.dflow INFO: Task 1875 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb01f6fe0a711046801d847f1f446699 with result from task 1875 parsl.dataflow.dflow DEBUG: Task 1875 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/85773acf-bed2-4166-a80c-6f3b67ffa419_isrForCrosstalkSources_996959_33.stdout parsl.dataflow.dflow INFO: Standard error for task 1875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/85773acf-bed2-4166-a80c-6f3b67ffa419_isrForCrosstalkSources_996959_33.stderr parsl.dataflow.dflow INFO: Task 1868 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 076734f5cbb85619492fbb7fa36d750a with result from task 1868 parsl.dataflow.dflow DEBUG: Task 1868 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/94ee4584-93e2-4799-b758-846602fbf260_isrForCrosstalkSources_996960_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/94ee4584-93e2-4799-b758-846602fbf260_isrForCrosstalkSources_996960_21.stderr parsl.dataflow.dflow INFO: Task 1871 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d0fa0fb9e896cdce14b8f0292ee9dfb with result from task 1871 parsl.dataflow.dflow DEBUG: Task 1871 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/2fe8a882-d1e6-429e-8b8e-0f5214bc1a61_isrForCrosstalkSources_997019_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/2fe8a882-d1e6-429e-8b8e-0f5214bc1a61_isrForCrosstalkSources_997019_58.stderr parsl.dataflow.dflow INFO: Task 1872 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b4665150deadbf33b546d45ef2743e3 with result from task 1872 parsl.dataflow.dflow DEBUG: Task 1872 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4e00d835-d395-4b33-9391-5de0e77755b7_isrForCrosstalkSources_996997_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4e00d835-d395-4b33-9391-5de0e77755b7_isrForCrosstalkSources_996997_55.stderr parsl.dataflow.dflow INFO: Task 1869 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 52b86bf7291d3f55c7370f5a9f90c690 with result from task 1869 parsl.dataflow.dflow DEBUG: Task 1869 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/566c529a-0d85-433c-bd11-87d717a2955a_isrForCrosstalkSources_997028_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/566c529a-0d85-433c-bd11-87d717a2955a_isrForCrosstalkSources_997028_36.stderr parsl.dataflow.dflow INFO: Task 1909 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f4d82376edcf17bcf44e36ae2ef6245 with result from task 1909 parsl.dataflow.dflow DEBUG: Task 1909 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/af5dfca7-6735-4125-99c9-d6852d122741_isrForCrosstalkSources_997020_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/af5dfca7-6735-4125-99c9-d6852d122741_isrForCrosstalkSources_997020_10.stderr parsl.dataflow.dflow INFO: Task 1980 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32998d0852cd91bfdd387c01d5d85db2 with result from task 1980 parsl.dataflow.dflow DEBUG: Task 1980 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/2718c44f-ef80-41bd-9ef0-87f4260e2430_isrForCrosstalkSources_997001_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/2718c44f-ef80-41bd-9ef0-87f4260e2430_isrForCrosstalkSources_997001_42.stderr parsl.dataflow.dflow INFO: Task 1995 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f10c80772442614fcde013e0866e7fbc with result from task 1995 parsl.dataflow.dflow DEBUG: Task 1995 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/aa5eb64a-14f7-4772-af2c-e91a23f1f6bb_isrForCrosstalkSources_996968_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/aa5eb64a-14f7-4772-af2c-e91a23f1f6bb_isrForCrosstalkSources_996968_62.stderr parsl.dataflow.dflow INFO: Task 2079 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f78b7aeee8be5f3f222e38710f96f019 with result from task 2079 parsl.dataflow.dflow DEBUG: Task 2079 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b9d01926-e52d-4aa7-98fd-01866023df37_isrForCrosstalkSources_997033_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b9d01926-e52d-4aa7-98fd-01866023df37_isrForCrosstalkSources_997033_35.stderr parsl.dataflow.dflow INFO: Task 2068 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13b34a27fd342a8e937c103ace46c721 with result from task 2068 parsl.dataflow.dflow DEBUG: Task 2068 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eeab50e-82f6-4a78-a076-1a9ed8ca60a3_isrForCrosstalkSources_996985_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eeab50e-82f6-4a78-a076-1a9ed8ca60a3_isrForCrosstalkSources_996985_12.stderr parsl.dataflow.dflow INFO: Task 1903 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f9b654c1863ff30fe5beea635500845a with result from task 1903 parsl.dataflow.dflow DEBUG: Task 1903 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d91ddd3d-0fee-4fa0-8d9f-4c23384b331f_isrForCrosstalkSources_996992_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d91ddd3d-0fee-4fa0-8d9f-4c23384b331f_isrForCrosstalkSources_996992_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:37 proc_lsst.multi:146 [INFO] found job 17075210 in provider checkpoint proc_lsst.multi INFO: found job 17075210 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2109 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry daa3f03fad58f7ed895739f5133bf1e1 with result from task 2109 parsl.dataflow.dflow DEBUG: Task 2109 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c05b2e8f-88d8-4d43-81ad-25862f71cb14_isrForCrosstalkSources_997003_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c05b2e8f-88d8-4d43-81ad-25862f71cb14_isrForCrosstalkSources_997003_27.stderr parsl.dataflow.dflow INFO: Task 1994 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1cce42d492bd803d1abc27e2196c04c3 with result from task 1994 parsl.dataflow.dflow DEBUG: Task 1994 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/193c5a8c-766a-43ed-bf31-89d16b321307_isrForCrosstalkSources_996979_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/193c5a8c-766a-43ed-bf31-89d16b321307_isrForCrosstalkSources_996979_55.stderr parsl.dataflow.dflow INFO: Task 2013 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84cc2515caac8825d95331f5ca53a450 with result from task 2013 parsl.dataflow.dflow DEBUG: Task 2013 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a27721c2-6dda-43e8-8538-72ec876d9383_isrForCrosstalkSources_996965_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/a27721c2-6dda-43e8-8538-72ec876d9383_isrForCrosstalkSources_996965_6.stderr parsl.dataflow.dflow INFO: Task 2027 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f0c8a1103f61858242ff741959a6ed64 with result from task 2027 parsl.dataflow.dflow DEBUG: Task 2027 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/17084782-3885-4c1b-a4db-68b0e1ef5260_isrForCrosstalkSources_996977_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/17084782-3885-4c1b-a4db-68b0e1ef5260_isrForCrosstalkSources_996977_12.stderr parsl.dataflow.dflow INFO: Task 2133 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f87661a5ddb0791ac334fadf6deda1c with result from task 2133 parsl.dataflow.dflow DEBUG: Task 2133 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9be69c0e-f115-4132-884f-5e0007942c53_isrForCrosstalkSources_996988_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9be69c0e-f115-4132-884f-5e0007942c53_isrForCrosstalkSources_996988_39.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:38 proc_lsst.multi:146 [INFO] found job 17075211 in provider checkpoint proc_lsst.multi INFO: found job 17075211 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1941 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc95c8d2b2d14bd6ce8f2407d271330f with result from task 1941 parsl.dataflow.dflow DEBUG: Task 1941 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e5998e2e-45e9-4868-b84d-f3de17996dc8_isrForCrosstalkSources_996954_6.stdout parsl.dataflow.dflow INFO: Standard error for task 1941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/e5998e2e-45e9-4868-b84d-f3de17996dc8_isrForCrosstalkSources_996954_6.stderr parsl.dataflow.dflow INFO: Task 1920 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c045ef6fbfd2bfc6380f388447750d1f with result from task 1920 parsl.dataflow.dflow DEBUG: Task 1920 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a3f2812a-6433-4927-ad0c-5918373e6774_isrForCrosstalkSources_997032_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a3f2812a-6433-4927-ad0c-5918373e6774_isrForCrosstalkSources_997032_11.stderr parsl.dataflow.dflow INFO: Task 1792 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 69da28f3ee739e2c8b55992c1292c081 with result from task 1792 parsl.dataflow.dflow DEBUG: Task 1792 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/366d2304-f930-4c5c-83ba-de1d7500ca54_isrForCrosstalkSources_996981_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/366d2304-f930-4c5c-83ba-de1d7500ca54_isrForCrosstalkSources_996981_56.stderr parsl.dataflow.dflow INFO: Task 1795 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0fe60dd3ff256b3e41ca0523e2946973 with result from task 1795 parsl.dataflow.dflow DEBUG: Task 1795 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/976acb37-ec98-47b0-8ff8-d273fe65efb0_isrForCrosstalkSources_997025_39.stdout parsl.dataflow.dflow INFO: Standard error for task 1795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/976acb37-ec98-47b0-8ff8-d273fe65efb0_isrForCrosstalkSources_997025_39.stderr parsl.dataflow.dflow INFO: Task 1796 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 79cdcd0fc1e72f03023eaed83b79a744 with result from task 1796 parsl.dataflow.dflow DEBUG: Task 1796 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/524fa598-d155-4572-9ac9-8cfa4d7b01bd_isrForCrosstalkSources_996998_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/524fa598-d155-4572-9ac9-8cfa4d7b01bd_isrForCrosstalkSources_996998_56.stderr parsl.dataflow.dflow INFO: Task 1794 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2546cb888278cdb560a5e70d830a205 with result from task 1794 parsl.dataflow.dflow DEBUG: Task 1794 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/35c282d3-fb7c-44d4-8090-dd86832086a5_isrForCrosstalkSources_996990_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/35c282d3-fb7c-44d4-8090-dd86832086a5_isrForCrosstalkSources_996990_34.stderr parsl.dataflow.dflow INFO: Task 1791 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f00a4db55fbaa701f4c976e6b32e3336 with result from task 1791 parsl.dataflow.dflow DEBUG: Task 1791 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/6d737b6d-f7e7-45aa-bd4c-b2bbec06ffbf_isrForCrosstalkSources_996995_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/6d737b6d-f7e7-45aa-bd4c-b2bbec06ffbf_isrForCrosstalkSources_996995_21.stderr parsl.dataflow.dflow INFO: Task 1789 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9f518364bc23e3ba0883ba5ebaf0cf51 with result from task 1789 parsl.dataflow.dflow DEBUG: Task 1789 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/3b537229-b774-4434-a0be-c226d9798b64_isrForCrosstalkSources_997004_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/3b537229-b774-4434-a0be-c226d9798b64_isrForCrosstalkSources_997004_31.stderr parsl.dataflow.dflow INFO: Task 2159 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2cd04693147b965aebe12bbf48e6bda with result from task 2159 parsl.dataflow.dflow DEBUG: Task 2159 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/fbb35f0e-b7fb-4392-89c7-165b843f99d1_isrForCrosstalkSources_996969_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/fbb35f0e-b7fb-4392-89c7-165b843f99d1_isrForCrosstalkSources_996969_39.stderr parsl.dataflow.dflow INFO: Task 2041 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db7e09c0e8a07109b580571072a0e9bd with result from task 2041 parsl.dataflow.dflow DEBUG: Task 2041 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/13f528fc-3c1c-4896-9499-8ed6ef9c6eea_isrForCrosstalkSources_997013_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/13f528fc-3c1c-4896-9499-8ed6ef9c6eea_isrForCrosstalkSources_997013_5.stderr parsl.dataflow.dflow INFO: Task 1947 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9f0cd342f2e25429575235693fd2fbee with result from task 1947 parsl.dataflow.dflow DEBUG: Task 1947 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ddb04963-3e6b-46c9-a9fe-fa3ac925fae2_isrForCrosstalkSources_996976_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ddb04963-3e6b-46c9-a9fe-fa3ac925fae2_isrForCrosstalkSources_996976_11.stderr parsl.dataflow.dflow INFO: Task 1957 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d198cff70135819bce2627aeaaa9674 with result from task 1957 parsl.dataflow.dflow DEBUG: Task 1957 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6acee158-4e74-483d-9dd1-a0943f666fb2_isrForCrosstalkSources_996974_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6acee158-4e74-483d-9dd1-a0943f666fb2_isrForCrosstalkSources_996974_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:39 proc_lsst.multi:146 [INFO] found job 17075212 in provider checkpoint proc_lsst.multi INFO: found job 17075212 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2122 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03abaeeec3e9f6c46ee4b7d90c53aece with result from task 2122 parsl.dataflow.dflow DEBUG: Task 2122 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9cc69a81-4dba-4a6c-9848-bd418fe76240_isrForCrosstalkSources_996971_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9cc69a81-4dba-4a6c-9848-bd418fe76240_isrForCrosstalkSources_996971_8.stderr parsl.dataflow.dflow INFO: Task 1990 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d3f85e917539dc16c39f03d88c2c78ab with result from task 1990 parsl.dataflow.dflow DEBUG: Task 1990 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ebccccfd-0442-4ac7-b25d-913bd7005bf5_isrForCrosstalkSources_997020_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ebccccfd-0442-4ac7-b25d-913bd7005bf5_isrForCrosstalkSources_997020_50.stderr parsl.dataflow.dflow INFO: Task 2140 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd1eafdbd78b4a4b707f7ac23e67ed37 with result from task 2140 parsl.dataflow.dflow DEBUG: Task 2140 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6e46ca47-035e-41a9-a516-5f0214dca8c1_isrForCrosstalkSources_996971_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6e46ca47-035e-41a9-a516-5f0214dca8c1_isrForCrosstalkSources_996971_61.stderr parsl.dataflow.dflow INFO: Task 2144 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8542f617280ffcd58ade0d0a4169c15a with result from task 2144 parsl.dataflow.dflow DEBUG: Task 2144 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/f78cea55-81f2-4cf5-b04c-d449daf0ce52_isrForCrosstalkSources_996969_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/f78cea55-81f2-4cf5-b04c-d449daf0ce52_isrForCrosstalkSources_996969_24.stderr parsl.dataflow.dflow INFO: Task 2145 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed84be43cad7a36a0e27f248754f6081 with result from task 2145 parsl.dataflow.dflow DEBUG: Task 2145 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f2141cb5-2b6e-47f5-9663-840f60117665_isrForCrosstalkSources_997017_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f2141cb5-2b6e-47f5-9663-840f60117665_isrForCrosstalkSources_997017_59.stderr parsl.dataflow.dflow INFO: Task 2019 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 40651453cdba9e7b17e3513768b0e945 with result from task 2019 parsl.dataflow.dflow DEBUG: Task 2019 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a1a139f7-58e1-41e9-a903-f8df45522387_isrForCrosstalkSources_996952_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a1a139f7-58e1-41e9-a903-f8df45522387_isrForCrosstalkSources_996952_24.stderr parsl.dataflow.dflow INFO: Task 2045 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42f72061c48af96505dcf772d6b5ee6a with result from task 2045 parsl.dataflow.dflow DEBUG: Task 2045 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d193d9a6-7623-401b-b08a-3092f29d32d2_isrForCrosstalkSources_996998_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d193d9a6-7623-401b-b08a-3092f29d32d2_isrForCrosstalkSources_996998_58.stderr parsl.dataflow.dflow INFO: Task 2163 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fd553b98a34634bfbd507235a152d0d with result from task 2163 parsl.dataflow.dflow DEBUG: Task 2163 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/919faa1c-2320-4919-b929-6b05ad40edc0_isrForCrosstalkSources_996991_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/919faa1c-2320-4919-b929-6b05ad40edc0_isrForCrosstalkSources_996991_34.stderr parsl.dataflow.dflow INFO: Task 2028 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dbfdbcb18f2f529694c546ddfae84e67 with result from task 2028 parsl.dataflow.dflow DEBUG: Task 2028 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a64fa012-2af4-4f36-aac4-170554613965_isrForCrosstalkSources_997011_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a64fa012-2af4-4f36-aac4-170554613965_isrForCrosstalkSources_997011_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:40 proc_lsst.multi:146 [INFO] found job 17075213 in provider checkpoint proc_lsst.multi INFO: found job 17075213 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1996 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 088a5ab471f89d5dcf6119a26a14f1b0 with result from task 1996 parsl.dataflow.dflow DEBUG: Task 1996 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a7d458bf-faf6-434a-ab20-133cf4c29809_isrForCrosstalkSources_996956_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a7d458bf-faf6-434a-ab20-133cf4c29809_isrForCrosstalkSources_996956_7.stderr parsl.dataflow.dflow INFO: Task 2093 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d14703ac5232efb3cbd1934c74808a92 with result from task 2093 parsl.dataflow.dflow DEBUG: Task 2093 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/95c9bcc8-636d-4aec-8f5d-9b230ef15d6b_isrForCrosstalkSources_996963_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/95c9bcc8-636d-4aec-8f5d-9b230ef15d6b_isrForCrosstalkSources_996963_51.stderr parsl.dataflow.dflow INFO: Task 2008 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ecea3137d8ef70eacfb5237a2bc0d16 with result from task 2008 parsl.dataflow.dflow DEBUG: Task 2008 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/71b6728f-63ad-42e8-ade5-5cc0b8a39b6b_isrForCrosstalkSources_996986_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/71b6728f-63ad-42e8-ade5-5cc0b8a39b6b_isrForCrosstalkSources_996986_29.stderr parsl.dataflow.dflow INFO: Task 1962 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a84f9bc9bc7ecb972871ab1c769b949b with result from task 1962 parsl.dataflow.dflow DEBUG: Task 1962 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e500c71e-fd51-43af-adbf-dc81fec59429_isrForCrosstalkSources_997031_61.stdout parsl.dataflow.dflow INFO: Standard error for task 1962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e500c71e-fd51-43af-adbf-dc81fec59429_isrForCrosstalkSources_997031_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:40 proc_lsst.multi:146 [INFO] found job 17075214 in provider checkpoint proc_lsst.multi INFO: found job 17075214 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1793 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 813f534f6e8ddc3bb0377c52db0c94e4 with result from task 1793 parsl.dataflow.dflow DEBUG: Task 1793 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/95d70b1b-a951-4641-b18c-87468f1bfb18_isrForCrosstalkSources_996994_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/95d70b1b-a951-4641-b18c-87468f1bfb18_isrForCrosstalkSources_996994_42.stderr parsl.dataflow.dflow INFO: Task 2127 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c366d2022d349bff334a80df55e6bef with result from task 2127 parsl.dataflow.dflow DEBUG: Task 2127 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b34a2079-9be2-4e6b-b9ef-65945d874266_isrForCrosstalkSources_996998_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b34a2079-9be2-4e6b-b9ef-65945d874266_isrForCrosstalkSources_996998_26.stderr parsl.dataflow.dflow INFO: Task 2047 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dbfdb5b60795b849d0849412ec3148c4 with result from task 2047 parsl.dataflow.dflow DEBUG: Task 2047 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5132cef3-414c-4319-8e87-5c852b259b12_isrForCrosstalkSources_997003_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5132cef3-414c-4319-8e87-5c852b259b12_isrForCrosstalkSources_997003_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:41 proc_lsst.multi:146 [INFO] found job 17075216 in provider checkpoint proc_lsst.multi INFO: found job 17075216 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1839 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3ad1eb1a23c91d69daffd3ff6b041e4c with result from task 1839 parsl.dataflow.dflow DEBUG: Task 1839 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/202a2949-7366-48de-82f1-0b4e909fe798_isrForCrosstalkSources_996974_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/202a2949-7366-48de-82f1-0b4e909fe798_isrForCrosstalkSources_996974_50.stderr parsl.dataflow.dflow INFO: Task 2170 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1224a3d3a04e3f908caad7650dad9f61 with result from task 2170 parsl.dataflow.dflow DEBUG: Task 2170 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/b2a69ac7-ec16-47f2-9f89-23000a43f722_isrForCrosstalkSources_997026_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/b2a69ac7-ec16-47f2-9f89-23000a43f722_isrForCrosstalkSources_997026_1.stderr parsl.dataflow.dflow INFO: Task 2022 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 800f50dc1f46fdf48e15d6055985545b with result from task 2022 parsl.dataflow.dflow DEBUG: Task 2022 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d94f2960-4f57-440d-b166-204de7091285_isrForCrosstalkSources_996992_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d94f2960-4f57-440d-b166-204de7091285_isrForCrosstalkSources_996992_33.stderr parsl.dataflow.dflow INFO: Task 2071 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12566178ff19fcb54cd880819713f080 with result from task 2071 parsl.dataflow.dflow DEBUG: Task 2071 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f0a3fec3-3e87-4a7f-bb88-89159e796040_isrForCrosstalkSources_996957_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f0a3fec3-3e87-4a7f-bb88-89159e796040_isrForCrosstalkSources_996957_25.stderr parsl.dataflow.dflow INFO: Task 1932 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1c9c3295c461aca36fed72b2498ec4e with result from task 1932 parsl.dataflow.dflow DEBUG: Task 1932 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/72731792-bebb-4402-a625-44476d6690e7_isrForCrosstalkSources_996997_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/72731792-bebb-4402-a625-44476d6690e7_isrForCrosstalkSources_996997_10.stderr parsl.dataflow.dflow INFO: Task 2117 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5841ee83905c5d049c47496d77bae99b with result from task 2117 parsl.dataflow.dflow DEBUG: Task 2117 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/8754712c-4e41-4500-8697-6a2bee5c8d79_isrForCrosstalkSources_997031_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/8754712c-4e41-4500-8697-6a2bee5c8d79_isrForCrosstalkSources_997031_32.stderr parsl.dataflow.dflow INFO: Task 1790 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4dee72b023711911a38373c5ca297f5 with result from task 1790 parsl.dataflow.dflow DEBUG: Task 1790 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/dfdff0b5-b6f8-41c7-b61d-d81458141ead_isrForCrosstalkSources_997007_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/dfdff0b5-b6f8-41c7-b61d-d81458141ead_isrForCrosstalkSources_997007_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:42 proc_lsst.multi:146 [INFO] found job 17075217 in provider checkpoint proc_lsst.multi INFO: found job 17075217 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2029 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df3c7f326624abb1bfc23c4e4eeda598 with result from task 2029 parsl.dataflow.dflow DEBUG: Task 2029 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/42109cf8-4f6b-4470-89f7-4d2f9d2d0fb0_isrForCrosstalkSources_996972_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/42109cf8-4f6b-4470-89f7-4d2f9d2d0fb0_isrForCrosstalkSources_996972_3.stderr parsl.dataflow.dflow INFO: Task 1967 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2147889e83da841d7e394aaa858d0867 with result from task 1967 parsl.dataflow.dflow DEBUG: Task 1967 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/190f9e86-9beb-41f9-8797-cd8ba495a22b_isrForCrosstalkSources_997004_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/190f9e86-9beb-41f9-8797-cd8ba495a22b_isrForCrosstalkSources_997004_47.stderr parsl.dataflow.dflow INFO: Task 1922 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 307edeaf1936a4983b9fa840e219eb3b with result from task 1922 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:42 proc_lsst.multi:146 [INFO] found job 17075218 in provider checkpoint proc_lsst.multi INFO: found job 17075218 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 1922 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/a177a73c-111a-4229-9d63-1d3be60d1f6a_isrForCrosstalkSources_997009_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/a177a73c-111a-4229-9d63-1d3be60d1f6a_isrForCrosstalkSources_997009_1.stderr parsl.dataflow.dflow INFO: Task 1971 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 664b3793ce57babceb206c2d6b430aa3 with result from task 1971 parsl.dataflow.dflow DEBUG: Task 1971 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ec172865-72b8-4da3-9875-46c50c9a44b6_isrForCrosstalkSources_997001_34.stdout parsl.dataflow.dflow INFO: Standard error for task 1971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/ec172865-72b8-4da3-9875-46c50c9a44b6_isrForCrosstalkSources_997001_34.stderr parsl.dataflow.dflow INFO: Task 1835 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry efcb24f4c8fc73783941d4e14479854e with result from task 1835 parsl.dataflow.dflow DEBUG: Task 1835 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f3267e7d-eb0b-4206-a687-d96a90441dfd_isrForCrosstalkSources_996960_43.stdout parsl.dataflow.dflow INFO: Standard error for task 1835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f3267e7d-eb0b-4206-a687-d96a90441dfd_isrForCrosstalkSources_996960_43.stderr parsl.dataflow.dflow INFO: Task 2116 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ff92624180d1434c28fd67164ccbbb8 with result from task 2116 parsl.dataflow.dflow DEBUG: Task 2116 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/312e381f-27df-410a-bd43-a05eb854ad3a_isrForCrosstalkSources_996972_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/312e381f-27df-410a-bd43-a05eb854ad3a_isrForCrosstalkSources_996972_32.stderr parsl.dataflow.dflow INFO: Task 1840 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0272ceb417a5374f29530ae048e06d3a with result from task 1840 parsl.dataflow.dflow DEBUG: Task 1840 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4ce1da69-01bf-4f0d-9a16-7dbf9053b61b_isrForCrosstalkSources_997030_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4ce1da69-01bf-4f0d-9a16-7dbf9053b61b_isrForCrosstalkSources_997030_48.stderr parsl.dataflow.dflow INFO: Task 2043 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4a8ac1af4443de5a0cf4f7c8d67ec01 with result from task 2043 parsl.dataflow.dflow DEBUG: Task 2043 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/2e981cdf-025c-4b87-afea-749a81705787_isrForCrosstalkSources_997000_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/2e981cdf-025c-4b87-afea-749a81705787_isrForCrosstalkSources_997000_32.stderr parsl.dataflow.dflow INFO: Task 2158 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7d19dd99f55eb16a5fa1816eff3b971 with result from task 2158 parsl.dataflow.dflow DEBUG: Task 2158 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/de047ed7-2711-494f-905a-f7b2537f6bef_isrForCrosstalkSources_996958_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/de047ed7-2711-494f-905a-f7b2537f6bef_isrForCrosstalkSources_996958_56.stderr parsl.dataflow.dflow INFO: Task 2138 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5dd9d7e371bde867b2614fff63b1b2ce with result from task 2138 parsl.dataflow.dflow DEBUG: Task 2138 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d0d2a3a2-b9dc-4636-b1b7-e3015dab8433_isrForCrosstalkSources_997025_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d0d2a3a2-b9dc-4636-b1b7-e3015dab8433_isrForCrosstalkSources_997025_50.stderr parsl.dataflow.dflow INFO: Task 2187 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e5051203a406fb4e4e174d4e20e2834 with result from task 2187 parsl.dataflow.dflow DEBUG: Task 2187 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/04be391b-083b-4419-9add-585067b01671_isrForCrosstalkSources_997015_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/04be391b-083b-4419-9add-585067b01671_isrForCrosstalkSources_997015_36.stderr parsl.dataflow.dflow INFO: Task 2141 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d33bbeec8e2b42893e9937ed3365d1ea with result from task 2141 parsl.dataflow.dflow DEBUG: Task 2141 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/3d4da5ba-f534-4b7b-ab09-cc35d56dda45_isrForCrosstalkSources_997020_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/3d4da5ba-f534-4b7b-ab09-cc35d56dda45_isrForCrosstalkSources_997020_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:43 proc_lsst.multi:146 [INFO] found job 17075219 in provider checkpoint proc_lsst.multi INFO: found job 17075219 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2090 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d1897e7019d13a8dea5e10ef0766fce with result from task 2090 parsl.dataflow.dflow DEBUG: Task 2090 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/a5ec7064-e6fe-4373-ba81-7c7badb81923_isrForCrosstalkSources_997003_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/a5ec7064-e6fe-4373-ba81-7c7badb81923_isrForCrosstalkSources_997003_14.stderr parsl.dataflow.dflow INFO: Task 1854 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1556eed7e90a72f577684746201d15d9 with result from task 1854 parsl.dataflow.dflow DEBUG: Task 1854 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c6373780-921d-4b4c-956f-585d5e24546f_isrForCrosstalkSources_997016_56.stdout parsl.dataflow.dflow INFO: Standard error for task 1854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c6373780-921d-4b4c-956f-585d5e24546f_isrForCrosstalkSources_997016_56.stderr parsl.dataflow.dflow INFO: Task 1853 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c98514a897f0202bd037b803c2dda79e with result from task 1853 parsl.dataflow.dflow DEBUG: Task 1853 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/08dc15f0-e66d-40ae-96f0-1ed8bbb82981_isrForCrosstalkSources_997010_31.stdout parsl.dataflow.dflow INFO: Standard error for task 1853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/08dc15f0-e66d-40ae-96f0-1ed8bbb82981_isrForCrosstalkSources_997010_31.stderr parsl.dataflow.dflow INFO: Task 1849 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46ca145f7e515d52b56d8a78ba6e5deb with result from task 1849 parsl.dataflow.dflow DEBUG: Task 1849 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b10d63fb-f296-4370-a822-ee2d9f3b1338_isrForCrosstalkSources_996999_7.stdout parsl.dataflow.dflow INFO: Standard error for task 1849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b10d63fb-f296-4370-a822-ee2d9f3b1338_isrForCrosstalkSources_996999_7.stderr parsl.dataflow.dflow INFO: Task 1850 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 76ec24e8f64343e6f83e8a3f65e4e1bb with result from task 1850 parsl.dataflow.dflow DEBUG: Task 1850 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/9b548712-e53e-4ac3-8691-1209ae89d8a8_isrForCrosstalkSources_997025_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/9b548712-e53e-4ac3-8691-1209ae89d8a8_isrForCrosstalkSources_997025_24.stderr parsl.dataflow.dflow INFO: Task 1852 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23fbaa24e23a5a51a602f3d5ee44f695 with result from task 1852 parsl.dataflow.dflow DEBUG: Task 1852 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/ffd0f37c-8572-4daa-9c8b-042b823cc15e_isrForCrosstalkSources_997025_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/ffd0f37c-8572-4daa-9c8b-042b823cc15e_isrForCrosstalkSources_997025_52.stderr parsl.dataflow.dflow INFO: Task 1848 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 71e626c1a46b7933be9b81565b9df8e3 with result from task 1848 parsl.dataflow.dflow DEBUG: Task 1848 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ee36ed81-1522-45f0-b0fb-33a957a7ff26_isrForCrosstalkSources_996982_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ee36ed81-1522-45f0-b0fb-33a957a7ff26_isrForCrosstalkSources_996982_30.stderr parsl.dataflow.dflow INFO: Task 1847 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd0eee8caaa4000c08f925b9b95053ad with result from task 1847 parsl.dataflow.dflow DEBUG: Task 1847 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/cd8930a3-172c-456b-af0f-dd18a05c0351_isrForCrosstalkSources_997022_32.stdout parsl.dataflow.dflow INFO: Standard error for task 1847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/cd8930a3-172c-456b-af0f-dd18a05c0351_isrForCrosstalkSources_997022_32.stderr parsl.dataflow.dflow INFO: Task 1851 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d3b22408712a3eb374e749a8cf1d5e47 with result from task 1851 parsl.dataflow.dflow DEBUG: Task 1851 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/d3b01399-6d96-4adc-bcc0-b11b11a357d6_isrForCrosstalkSources_996967_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/d3b01399-6d96-4adc-bcc0-b11b11a357d6_isrForCrosstalkSources_996967_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:43 proc_lsst.multi:146 [INFO] found job 17075220 in provider checkpoint proc_lsst.multi INFO: found job 17075220 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1834 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 110aec82af3a30799bc3713412d9a657 with result from task 1834 parsl.dataflow.dflow DEBUG: Task 1834 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fd92eb1e-036f-4629-ae38-91a23def7ce2_isrForCrosstalkSources_996968_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fd92eb1e-036f-4629-ae38-91a23def7ce2_isrForCrosstalkSources_996968_48.stderr parsl.dataflow.dflow INFO: Task 1972 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d0257ff195850e58ce55e707609a3b0 with result from task 1972 parsl.dataflow.dflow DEBUG: Task 1972 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1788145a-2b51-4699-b25c-b6c605547d3b_isrForCrosstalkSources_997027_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1788145a-2b51-4699-b25c-b6c605547d3b_isrForCrosstalkSources_997027_29.stderr parsl.dataflow.dflow INFO: Task 2189 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 34d067f2ca95ae4db0a384adadd7e427 with result from task 2189 parsl.dataflow.dflow DEBUG: Task 2189 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6607ac2a-330c-4a33-a6da-84955a25658d_isrForCrosstalkSources_997030_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6607ac2a-330c-4a33-a6da-84955a25658d_isrForCrosstalkSources_997030_38.stderr parsl.dataflow.dflow INFO: Task 2035 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd7aeda87dec5ea4500f835867380609 with result from task 2035 parsl.dataflow.dflow DEBUG: Task 2035 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/12a3432e-8a58-4a55-85ef-2fe2e396e0ba_isrForCrosstalkSources_997009_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/12a3432e-8a58-4a55-85ef-2fe2e396e0ba_isrForCrosstalkSources_997009_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:44 proc_lsst.multi:146 [INFO] found job 17075221 in provider checkpoint proc_lsst.multi INFO: found job 17075221 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2192 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b6df7469be09575c22b17416b64dc78 with result from task 2192 parsl.dataflow.dflow DEBUG: Task 2192 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cf266be9-c504-4c36-9208-dce6d35ed572_isrForCrosstalkSources_996983_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/cf266be9-c504-4c36-9208-dce6d35ed572_isrForCrosstalkSources_996983_10.stderr parsl.dataflow.dflow INFO: Task 2004 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85400d56556fa945051842604ddb4532 with result from task 2004 parsl.dataflow.dflow DEBUG: Task 2004 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/dfb18199-4480-4698-9583-83b6f98afdc5_isrForCrosstalkSources_996952_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/dfb18199-4480-4698-9583-83b6f98afdc5_isrForCrosstalkSources_996952_21.stderr parsl.dataflow.dflow INFO: Task 2196 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 913e5374030d1b87cc57b5047815a6f9 with result from task 2196 parsl.dataflow.dflow DEBUG: Task 2196 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/8e0fe56e-f477-4eeb-9f47-a00bf9c5bfec_isrForCrosstalkSources_997024_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/8e0fe56e-f477-4eeb-9f47-a00bf9c5bfec_isrForCrosstalkSources_997024_25.stderr parsl.dataflow.dflow INFO: Task 1997 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f350147799400c103a11dba9471d6c6 with result from task 1997 parsl.dataflow.dflow DEBUG: Task 1997 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9e091ec3-ba7e-48eb-9930-d3afc669eb08_isrForCrosstalkSources_997000_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9e091ec3-ba7e-48eb-9930-d3afc669eb08_isrForCrosstalkSources_997000_60.stderr parsl.dataflow.dflow INFO: Task 2188 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a286e3211ff3cacd0803fb8d7c45a4fc with result from task 2188 parsl.dataflow.dflow DEBUG: Task 2188 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2b610c61-0b56-4562-8f26-ba65a5bc8c24_isrForCrosstalkSources_996963_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2b610c61-0b56-4562-8f26-ba65a5bc8c24_isrForCrosstalkSources_996963_33.stderr parsl.dataflow.dflow INFO: Task 2114 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e412d9f364a37541098c0bd2db1c636f with result from task 2114 parsl.dataflow.dflow DEBUG: Task 2114 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/14baf94a-4f96-482d-8526-5647d45ff41d_isrForCrosstalkSources_997012_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/14baf94a-4f96-482d-8526-5647d45ff41d_isrForCrosstalkSources_997012_27.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:45 proc_lsst.multi:146 [INFO] found job 17075222 in provider checkpoint proc_lsst.multi INFO: found job 17075222 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2248 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5006a6696a27e478e756965ebec3db77 with result from task 2248 parsl.dataflow.dflow DEBUG: Task 2248 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/38107634-15e5-49c8-8c4b-6f98d49aea5b_isrForCrosstalkSources_997030_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/38107634-15e5-49c8-8c4b-6f98d49aea5b_isrForCrosstalkSources_997030_41.stderr parsl.dataflow.dflow INFO: Task 1912 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f290412e002bdda28b6269bbf0d2449 with result from task 1912 parsl.dataflow.dflow DEBUG: Task 1912 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/826af4f3-7443-41fa-893c-bb048dc120c5_isrForCrosstalkSources_996993_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/826af4f3-7443-41fa-893c-bb048dc120c5_isrForCrosstalkSources_996993_57.stderr parsl.dataflow.dflow INFO: Task 2097 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39a1648d09cd27f02cc766950b819c0d with result from task 2097 parsl.dataflow.dflow DEBUG: Task 2097 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/31d2f3c8-86cd-403c-a460-dec538353331_isrForCrosstalkSources_996993_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/31d2f3c8-86cd-403c-a460-dec538353331_isrForCrosstalkSources_996993_27.stderr parsl.dataflow.dflow INFO: Task 2185 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 24f9e48ce9f8d32b3e036236cebc5123 with result from task 2185 parsl.dataflow.dflow DEBUG: Task 2185 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/9dd4f557-6afc-4b9b-9e65-64e0eedbda30_isrForCrosstalkSources_997026_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/9dd4f557-6afc-4b9b-9e65-64e0eedbda30_isrForCrosstalkSources_997026_56.stderr parsl.dataflow.dflow INFO: Task 1958 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33eed3bd556cce33befd7787acf684be with result from task 1958 parsl.dataflow.dflow DEBUG: Task 1958 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/51712fe1-f8a5-4b4c-9ec5-c0b34468574b_isrForCrosstalkSources_997012_48.stdout parsl.dataflow.dflow INFO: Standard error for task 1958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/51712fe1-f8a5-4b4c-9ec5-c0b34468574b_isrForCrosstalkSources_997012_48.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:46 proc_lsst.multi:146 [INFO] found job 17075223 in provider checkpoint proc_lsst.multi INFO: found job 17075223 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2215 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 884dcfc4796cc082bebae855d9d10be5 with result from task 2215 parsl.dataflow.dflow DEBUG: Task 2215 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/79ce3c28-f7aa-4dfb-827e-d5b0570bc7be_isrForCrosstalkSources_996997_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/79ce3c28-f7aa-4dfb-827e-d5b0570bc7be_isrForCrosstalkSources_996997_51.stderr parsl.dataflow.dflow INFO: Task 2216 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 904f8cafe06f6fcfb3ce4b6b0a206ba3 with result from task 2216 parsl.dataflow.dflow DEBUG: Task 2216 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1be9b606-3223-4120-a79e-21633901aead_isrForCrosstalkSources_997019_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1be9b606-3223-4120-a79e-21633901aead_isrForCrosstalkSources_997019_4.stderr parsl.dataflow.dflow INFO: Task 2060 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04477160549c1a30f9aa502983d43093 with result from task 2060 parsl.dataflow.dflow DEBUG: Task 2060 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/60a64a85-d167-43db-8382-694d17781557_isrForCrosstalkSources_997008_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/60a64a85-d167-43db-8382-694d17781557_isrForCrosstalkSources_997008_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:47 proc_lsst.multi:146 [INFO] found job 17075224 in provider checkpoint proc_lsst.multi INFO: found job 17075224 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2130 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6763ae7ea6baa7ed010a0a85683184fe with result from task 2130 parsl.dataflow.dflow DEBUG: Task 2130 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b32e5af8-b67f-4643-a7a2-8ae1b65703ac_isrForCrosstalkSources_996963_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b32e5af8-b67f-4643-a7a2-8ae1b65703ac_isrForCrosstalkSources_996963_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:47 proc_lsst.multi:146 [INFO] found job 17075225 in provider checkpoint proc_lsst.multi INFO: found job 17075225 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2254 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 60fed9f850fdcdef257d2c48408bc30b with result from task 2254 parsl.dataflow.dflow DEBUG: Task 2254 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/6ae5303a-657c-46f0-9d54-07b369308139_isrForCrosstalkSources_997026_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/6ae5303a-657c-46f0-9d54-07b369308139_isrForCrosstalkSources_997026_47.stderr parsl.dataflow.dflow INFO: Task 2221 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c31e0735aca35d7ac2ebbe8218812478 with result from task 2221 parsl.dataflow.dflow DEBUG: Task 2221 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/306c81fd-9d4f-4d2b-b504-9f3c04704e83_isrForCrosstalkSources_997020_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/306c81fd-9d4f-4d2b-b504-9f3c04704e83_isrForCrosstalkSources_997020_60.stderr parsl.dataflow.dflow INFO: Task 2214 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51ba607fb759688021b3cfdd9ebaaafa with result from task 2214 parsl.dataflow.dflow DEBUG: Task 2214 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/4718103f-3f0b-4cec-9e7d-237a52035040_isrForCrosstalkSources_996975_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/4718103f-3f0b-4cec-9e7d-237a52035040_isrForCrosstalkSources_996975_6.stderr parsl.dataflow.dflow INFO: Task 2167 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3939ed4942f20f21b8d616b633ba94e7 with result from task 2167 parsl.dataflow.dflow DEBUG: Task 2167 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/a53c51b2-d912-4136-802b-6ff8eb53acfb_isrForCrosstalkSources_997005_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/a53c51b2-d912-4136-802b-6ff8eb53acfb_isrForCrosstalkSources_997005_39.stderr parsl.dataflow.dflow INFO: Task 2032 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26cfeff72b5ea721757f8bde04440e68 with result from task 2032 parsl.dataflow.dflow DEBUG: Task 2032 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9f9667b8-aa36-4527-b431-d5d8870c89f1_isrForCrosstalkSources_997023_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9f9667b8-aa36-4527-b431-d5d8870c89f1_isrForCrosstalkSources_997023_34.stderr parsl.dataflow.dflow INFO: Task 2115 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 042ba42001f3f3ae8cfc4ed7d118902d with result from task 2115 parsl.dataflow.dflow DEBUG: Task 2115 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8cb23279-0ca3-45bc-b96a-0ce5fbe30793_isrForCrosstalkSources_997018_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8cb23279-0ca3-45bc-b96a-0ce5fbe30793_isrForCrosstalkSources_997018_40.stderr parsl.dataflow.dflow INFO: Task 1829 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea7111eb069d99c36bb37d52ff972dc4 with result from task 1829 parsl.dataflow.dflow DEBUG: Task 1829 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8523c039-71db-47e6-ad12-f18b90cbeef7_isrForCrosstalkSources_996963_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8523c039-71db-47e6-ad12-f18b90cbeef7_isrForCrosstalkSources_996963_40.stderr parsl.dataflow.dflow INFO: Task 1826 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58daffeaee3476712954a7f4a7e1aed8 with result from task 1826 parsl.dataflow.dflow DEBUG: Task 1826 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fc5844cd-3043-4794-b568-0dfdb3a77149_isrForCrosstalkSources_996961_2.stdout parsl.dataflow.dflow INFO: Standard error for task 1826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fc5844cd-3043-4794-b568-0dfdb3a77149_isrForCrosstalkSources_996961_2.stderr parsl.dataflow.dflow INFO: Task 1828 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry acf7f2af9cf071c29890226f4ce0e09c with result from task 1828 parsl.dataflow.dflow DEBUG: Task 1828 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/8c114687-ce02-4bd0-8b0a-2d1b67b2cfa0_isrForCrosstalkSources_997002_40.stdout parsl.dataflow.dflow INFO: Standard error for task 1828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/8c114687-ce02-4bd0-8b0a-2d1b67b2cfa0_isrForCrosstalkSources_997002_40.stderr parsl.dataflow.dflow INFO: Task 1831 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eae7551bc7f6b20f3a63c76f5ce710a4 with result from task 1831 parsl.dataflow.dflow DEBUG: Task 1831 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/af95b851-b00f-4320-86ca-3dba64f1152b_isrForCrosstalkSources_996984_37.stdout parsl.dataflow.dflow INFO: Standard error for task 1831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/af95b851-b00f-4320-86ca-3dba64f1152b_isrForCrosstalkSources_996984_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:48 proc_lsst.multi:146 [INFO] found job 17075226 in provider checkpoint proc_lsst.multi INFO: found job 17075226 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1827 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f709bf1317dc3d91ce6c32ea3a59907e with result from task 1827 parsl.dataflow.dflow DEBUG: Task 1827 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9898abcb-de8e-4e5d-a32d-1ace028abb33_isrForCrosstalkSources_997022_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9898abcb-de8e-4e5d-a32d-1ace028abb33_isrForCrosstalkSources_997022_1.stderr parsl.dataflow.dflow INFO: Task 1832 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7df1aae886796c115d56618b404cce1b with result from task 1832 parsl.dataflow.dflow DEBUG: Task 1832 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3cae46ab-7460-4246-8c79-120314ff9c1c_isrForCrosstalkSources_996968_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3cae46ab-7460-4246-8c79-120314ff9c1c_isrForCrosstalkSources_996968_30.stderr parsl.dataflow.dflow INFO: Task 2230 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a6614dc8ff742f28ea11491016038ea with result from task 2230 parsl.dataflow.dflow DEBUG: Task 2230 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b7bfecac-8df2-48bb-a746-db1b7fc6f22e_isrForCrosstalkSources_996963_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b7bfecac-8df2-48bb-a746-db1b7fc6f22e_isrForCrosstalkSources_996963_38.stderr parsl.dataflow.dflow INFO: Task 2262 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e548b8db897a4aaa705e63d2af5786a with result from task 2262 parsl.dataflow.dflow DEBUG: Task 2262 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/9638cdc2-c83c-40d1-a2a6-fd267d00b25f_isrForCrosstalkSources_997020_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/9638cdc2-c83c-40d1-a2a6-fd267d00b25f_isrForCrosstalkSources_997020_26.stderr parsl.dataflow.dflow INFO: Task 2048 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.memoization DEBUG: Storing app cache entry 81dcdf0ee21d157c086c1ad4f09e1fdb with result from task 2048 parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2048 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/20f8338b-1db2-40c1-9854-f69c50d62de0_isrForCrosstalkSources_996963_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/20f8338b-1db2-40c1-9854-f69c50d62de0_isrForCrosstalkSources_996963_45.stderr parsl.dataflow.dflow INFO: Task 2094 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1dc9d94003dfa8def7b5f1d99d3738a4 with result from task 2094 parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:49 proc_lsst.multi:146 [INFO] found job 17075227 in provider checkpoint proc_lsst.multi INFO: found job 17075227 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2094 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/0b70d21c-3acc-4e74-8df0-309501504165_isrForCrosstalkSources_997033_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/0b70d21c-3acc-4e74-8df0-309501504165_isrForCrosstalkSources_997033_13.stderr parsl.dataflow.dflow INFO: Task 2207 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 22c25543a617a5f455a743cc0146ad5e with result from task 2207 parsl.dataflow.dflow DEBUG: Task 2207 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5067265b-cb64-49e0-9da0-563c24396884_isrForCrosstalkSources_997015_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/5067265b-cb64-49e0-9da0-563c24396884_isrForCrosstalkSources_997015_34.stderr parsl.dataflow.dflow INFO: Task 2150 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bfa987d8f436f56981b446427d74c9aa with result from task 2150 parsl.dataflow.dflow DEBUG: Task 2150 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/03c7466f-09b1-4bc2-a9a7-a13b72f0585f_isrForCrosstalkSources_997007_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/03c7466f-09b1-4bc2-a9a7-a13b72f0585f_isrForCrosstalkSources_997007_21.stderr parsl.dataflow.dflow INFO: Task 2232 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7721bae2141e789f97f9b08b44972c7e with result from task 2232 parsl.dataflow.dflow DEBUG: Task 2232 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/fd5c429c-6a61-45ea-8d72-cb8a0b1708aa_isrForCrosstalkSources_997006_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/fd5c429c-6a61-45ea-8d72-cb8a0b1708aa_isrForCrosstalkSources_997006_25.stderr parsl.dataflow.dflow INFO: Task 2154 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e47d45efe1c627ce1f976d0b27e6e1e with result from task 2154 parsl.dataflow.dflow DEBUG: Task 2154 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/36c00b91-5bb1-41e1-94f7-981bf7d4276b_isrForCrosstalkSources_997023_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/36c00b91-5bb1-41e1-94f7-981bf7d4276b_isrForCrosstalkSources_997023_11.stderr parsl.dataflow.dflow INFO: Task 2173 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27f01a7adcd4c57b1bc3eca8d975ed6e with result from task 2173 parsl.dataflow.dflow DEBUG: Task 2173 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e179da26-36b4-4129-8094-1502fa442d88_isrForCrosstalkSources_996970_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e179da26-36b4-4129-8094-1502fa442d88_isrForCrosstalkSources_996970_21.stderr parsl.dataflow.dflow INFO: Task 2139 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 989c3cdda2d06578e4f32d16c6c3cda4 with result from task 2139 parsl.dataflow.dflow DEBUG: Task 2139 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a7b195f9-3bfb-45bc-89b2-0a2297e6080e_isrForCrosstalkSources_997007_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a7b195f9-3bfb-45bc-89b2-0a2297e6080e_isrForCrosstalkSources_997007_46.stderr parsl.dataflow.dflow INFO: Task 2247 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82187c5f9e869a814cf396755d4e00af with result from task 2247 parsl.dataflow.dflow DEBUG: Task 2247 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/610e36a4-80fd-468b-8bc1-b86c68f10419_isrForCrosstalkSources_996998_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/610e36a4-80fd-468b-8bc1-b86c68f10419_isrForCrosstalkSources_996998_47.stderr parsl.dataflow.dflow INFO: Task 2128 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b10a64c2fb0330060bf8b5a5701f420f with result from task 2128 parsl.dataflow.dflow DEBUG: Task 2128 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/65dcf141-e3f9-46f0-9f37-70190fe67e54_isrForCrosstalkSources_997007_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/65dcf141-e3f9-46f0-9f37-70190fe67e54_isrForCrosstalkSources_997007_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:50 proc_lsst.multi:146 [INFO] found job 17075228 in provider checkpoint proc_lsst.multi INFO: found job 17075228 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2095 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c162afbe2ca6d333915b311cf4524161 with result from task 2095 parsl.dataflow.dflow DEBUG: Task 2095 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/17de7cc2-7b51-4e23-91e1-927b45e9c3f9_isrForCrosstalkSources_997028_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/17de7cc2-7b51-4e23-91e1-927b45e9c3f9_isrForCrosstalkSources_997028_45.stderr parsl.dataflow.dflow INFO: Task 1845 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ff19eddce2e3d59d1f99ee4cf6d883c with result from task 1845 parsl.dataflow.dflow DEBUG: Task 1845 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/4e31aec4-8fb2-4e72-844d-441c97ef3b1a_isrForCrosstalkSources_996955_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/4e31aec4-8fb2-4e72-844d-441c97ef3b1a_isrForCrosstalkSources_996955_54.stderr parsl.dataflow.dflow INFO: Task 2001 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d62b3e5f1c25f875e761eafe2e518a1f with result from task 2001 parsl.dataflow.dflow DEBUG: Task 2001 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5a592af6-1f72-41af-b956-5266a290a51f_isrForCrosstalkSources_997004_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/5a592af6-1f72-41af-b956-5266a290a51f_isrForCrosstalkSources_997004_59.stderr parsl.dataflow.dflow INFO: Task 2125 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33b29cd51d2457dfa758e0e31e6c7a0a with result from task 2125 parsl.dataflow.dflow DEBUG: Task 2125 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/79c027be-4d97-4ca3-bfdd-059fecd950a9_isrForCrosstalkSources_996964_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/79c027be-4d97-4ca3-bfdd-059fecd950a9_isrForCrosstalkSources_996964_24.stderr parsl.dataflow.dflow INFO: Task 1987 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96e9c1b1f0b3d1e17e623015217030b0 with result from task 1987 parsl.dataflow.dflow DEBUG: Task 1987 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dc2d39b7-9d3d-490c-b64a-8f52666769cc_isrForCrosstalkSources_996971_12.stdout parsl.dataflow.dflow INFO: Standard error for task 1987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dc2d39b7-9d3d-490c-b64a-8f52666769cc_isrForCrosstalkSources_996971_12.stderr parsl.dataflow.dflow INFO: Task 2229 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 247a3f7edd181d918d7db084c7138091 with result from task 2229 parsl.dataflow.dflow DEBUG: Task 2229 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/17073cc0-b66b-4d26-acf4-9b0d4684cef3_isrForCrosstalkSources_996999_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/17073cc0-b66b-4d26-acf4-9b0d4684cef3_isrForCrosstalkSources_996999_12.stderr parsl.dataflow.dflow INFO: Task 2112 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2428998caeec4cd6d028d2b094ebd9f7 with result from task 2112 parsl.dataflow.dflow DEBUG: Task 2112 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a5cd789e-00a3-42c8-926a-741dfe6e1f0a_isrForCrosstalkSources_997027_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a5cd789e-00a3-42c8-926a-741dfe6e1f0a_isrForCrosstalkSources_997027_7.stderr parsl.dataflow.dflow INFO: Task 2274 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3018d0b28f3d1cced6a41f15c67f37d4 with result from task 2274 parsl.dataflow.dflow DEBUG: Task 2274 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0338a735-2141-45f4-9981-af42d67d89a4_isrForCrosstalkSources_996958_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0338a735-2141-45f4-9981-af42d67d89a4_isrForCrosstalkSources_996958_13.stderr parsl.dataflow.dflow INFO: Task 1830 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33abb696e209b1e06986d7eb82a67cd7 with result from task 1830 parsl.dataflow.dflow DEBUG: Task 1830 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/2aaf9b69-227a-4fa4-a4ce-0f62c3ad3128_isrForCrosstalkSources_997025_28.stdout parsl.dataflow.dflow INFO: Standard error for task 1830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/2aaf9b69-227a-4fa4-a4ce-0f62c3ad3128_isrForCrosstalkSources_997025_28.stderr parsl.dataflow.dflow INFO: Task 1992 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b8551c02e391e362d30824077de6904 with result from task 1992 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 1992 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 1992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/60efebc1-2ea2-4043-a56e-31fce78b303e_isrForCrosstalkSources_996952_59.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 1992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/60efebc1-2ea2-4043-a56e-31fce78b303e_isrForCrosstalkSources_996952_59.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:51 proc_lsst.multi:146 [INFO] found job 17075229 in provider checkpoint proc_lsst.multi INFO: found job 17075229 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2110 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50dfecb1ae57672a8831052dd208a0b7 with result from task 2110 parsl.dataflow.dflow DEBUG: Task 2110 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/b24fc94b-9752-4e70-b288-06fad4f6c1c1_isrForCrosstalkSources_996990_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/b24fc94b-9752-4e70-b288-06fad4f6c1c1_isrForCrosstalkSources_996990_56.stderr parsl.dataflow.dflow INFO: Task 2211 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3df1915586ab1209c86eed7acdb0bdf5 with result from task 2211 parsl.dataflow.dflow DEBUG: Task 2211 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a77fbafb-e6d1-45cf-849c-20bdc9570903_isrForCrosstalkSources_996996_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a77fbafb-e6d1-45cf-849c-20bdc9570903_isrForCrosstalkSources_996996_17.stderr parsl.dataflow.dflow INFO: Task 2172 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e914ef080b2cdd9991c0f5d0043f783a with result from task 2172 parsl.dataflow.dflow DEBUG: Task 2172 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6034ec1f-aac4-4ab8-b157-5e3a1236e11b_isrForCrosstalkSources_996957_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6034ec1f-aac4-4ab8-b157-5e3a1236e11b_isrForCrosstalkSources_996957_31.stderr parsl.dataflow.dflow INFO: Task 2044 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a699849893e1634b0d16617f75d5282e with result from task 2044 parsl.dataflow.dflow DEBUG: Task 2044 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/deaa0f57-a09e-4205-abaf-e56809b2237c_isrForCrosstalkSources_996999_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/deaa0f57-a09e-4205-abaf-e56809b2237c_isrForCrosstalkSources_996999_54.stderr parsl.dataflow.dflow INFO: Task 1993 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cff85a686582e736caba44e1b4a1c139 with result from task 1993 parsl.dataflow.dflow DEBUG: Task 1993 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/ad4a0f24-9912-4ba9-8e18-efd2c1f8d501_isrForCrosstalkSources_996967_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/ad4a0f24-9912-4ba9-8e18-efd2c1f8d501_isrForCrosstalkSources_996967_29.stderr parsl.dataflow.dflow INFO: Task 2263 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af686d86609a0e8c344852ad6e902fec with result from task 2263 parsl.dataflow.dflow DEBUG: Task 2263 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/772ae935-52f3-4095-9185-44be9cdfb449_isrForCrosstalkSources_996963_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/772ae935-52f3-4095-9185-44be9cdfb449_isrForCrosstalkSources_996963_50.stderr parsl.dataflow.dflow INFO: Task 2018 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 847cbba6ee76340b7f08afd44ffcab93 with result from task 2018 parsl.dataflow.dflow DEBUG: Task 2018 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/14722342-3f8c-440a-bd55-7b46770612a1_isrForCrosstalkSources_997001_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/14722342-3f8c-440a-bd55-7b46770612a1_isrForCrosstalkSources_997001_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:51 proc_lsst.multi:146 [INFO] found job 17075230 in provider checkpoint proc_lsst.multi INFO: found job 17075230 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2142 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11e54406481ef0f5989f10ed2eda25c8 with result from task 2142 parsl.dataflow.dflow DEBUG: Task 2142 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/f37e1de5-8b18-429c-8085-b34b5a8c2113_isrForCrosstalkSources_997029_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/f37e1de5-8b18-429c-8085-b34b5a8c2113_isrForCrosstalkSources_997029_10.stderr parsl.dataflow.dflow INFO: Task 2088 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83a29af64b0835f8604757aec5fe2841 with result from task 2088 parsl.dataflow.dflow DEBUG: Task 2088 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8c2df2c9-d3a3-4f68-be8c-9c292b9c3962_isrForCrosstalkSources_997018_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/8c2df2c9-d3a3-4f68-be8c-9c292b9c3962_isrForCrosstalkSources_997018_61.stderr parsl.dataflow.dflow INFO: Task 2289 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aabb5458773965c8f715fd6d7314948f with result from task 2289 parsl.dataflow.dflow DEBUG: Task 2289 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/e3af2b1b-e718-4d3e-8736-ecf47241cf8a_isrForCrosstalkSources_997015_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/e3af2b1b-e718-4d3e-8736-ecf47241cf8a_isrForCrosstalkSources_997015_32.stderr parsl.dataflow.dflow INFO: Task 2258 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86f2a91f76c55928c61ee58083e4d212 with result from task 2258 parsl.dataflow.dflow DEBUG: Task 2258 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d375b4c0-859e-437b-abe6-2a112e8c1cd5_isrForCrosstalkSources_996995_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d375b4c0-859e-437b-abe6-2a112e8c1cd5_isrForCrosstalkSources_996995_14.stderr parsl.dataflow.dflow INFO: Task 2105 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df598dd034375c7fdf9661c31cf4bd73 with result from task 2105 parsl.dataflow.dflow DEBUG: Task 2105 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/77633662-67ee-41cf-b187-fdd63a3a26a8_isrForCrosstalkSources_996985_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/77633662-67ee-41cf-b187-fdd63a3a26a8_isrForCrosstalkSources_996985_5.stderr parsl.dataflow.dflow INFO: Task 2210 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd4cf7a5ffbee33da9a301798ecb3e0f with result from task 2210 parsl.dataflow.dflow DEBUG: Task 2210 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/df5b9bab-6ddf-45ed-b8d2-96ad3bd287fc_isrForCrosstalkSources_996963_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/df5b9bab-6ddf-45ed-b8d2-96ad3bd287fc_isrForCrosstalkSources_996963_29.stderr parsl.dataflow.dflow INFO: Task 2270 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a70d1162f5cb0bf319445524e2c976b9 with result from task 2270 parsl.dataflow.dflow DEBUG: Task 2270 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c28096fc-d850-437e-9c4b-851f34369d19_isrForCrosstalkSources_996975_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c28096fc-d850-437e-9c4b-851f34369d19_isrForCrosstalkSources_996975_10.stderr parsl.dataflow.dflow INFO: Task 2222 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e97a56f7c386cda6219b8c2c48e22e3 with result from task 2222 parsl.dataflow.dflow DEBUG: Task 2222 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/13193e84-3894-4f9d-98fb-6432f1c33afb_isrForCrosstalkSources_997004_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/13193e84-3894-4f9d-98fb-6432f1c33afb_isrForCrosstalkSources_997004_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:52 proc_lsst.multi:146 [INFO] found job 17075231 in provider checkpoint proc_lsst.multi INFO: found job 17075231 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1882 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7076040cdf96b1d9ee0898d2930e77c2 with result from task 1882 parsl.dataflow.dflow DEBUG: Task 1882 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/386fbc25-d6fc-4bd2-b884-170b6ca850ec_isrForCrosstalkSources_996983_46.stdout parsl.dataflow.dflow INFO: Standard error for task 1882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/386fbc25-d6fc-4bd2-b884-170b6ca850ec_isrForCrosstalkSources_996983_46.stderr parsl.dataflow.dflow INFO: Task 1880 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75f9778a2329cf1069887ef44bb1dcd9 with result from task 1880 parsl.dataflow.dflow DEBUG: Task 1880 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ca1295f8-a353-4766-ba56-e06ace7ef419_isrForCrosstalkSources_997005_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ca1295f8-a353-4766-ba56-e06ace7ef419_isrForCrosstalkSources_997005_25.stderr parsl.dataflow.dflow INFO: Task 1886 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abaccdb05bc9b705ce74d5bea1088c2e with result from task 1886 parsl.dataflow.dflow DEBUG: Task 1886 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a29ca2f4-2c7b-4136-b637-b2a7b89ca5ce_isrForCrosstalkSources_996991_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a29ca2f4-2c7b-4136-b637-b2a7b89ca5ce_isrForCrosstalkSources_996991_14.stderr parsl.dataflow.dflow INFO: Task 1883 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3ba393a7dc33328364d1a11b9ac11459 with result from task 1883 parsl.dataflow.dflow DEBUG: Task 1883 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/14fe085c-47de-4956-a0ac-1261f8d464dc_isrForCrosstalkSources_996984_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/14fe085c-47de-4956-a0ac-1261f8d464dc_isrForCrosstalkSources_996984_29.stderr parsl.dataflow.dflow INFO: Task 2269 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0917ce5c10056246ef37c992de0fee9 with result from task 2269 parsl.dataflow.dflow DEBUG: Task 2269 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/7af39a1c-9c87-4404-a5de-0016975e18f9_isrForCrosstalkSources_997000_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/7af39a1c-9c87-4404-a5de-0016975e18f9_isrForCrosstalkSources_997000_29.stderr parsl.dataflow.dflow INFO: Task 2209 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff44083b784ad50f6bd120598c8db4ed with result from task 2209 parsl.dataflow.dflow DEBUG: Task 2209 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/36a33ced-531a-458d-9ba0-4c4d6a348b26_isrForCrosstalkSources_997015_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/36a33ced-531a-458d-9ba0-4c4d6a348b26_isrForCrosstalkSources_997015_2.stderr parsl.dataflow.dflow INFO: Task 2271 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dda5bd703ead9cc9a0aad47371d88361 with result from task 2271 parsl.dataflow.dflow DEBUG: Task 2271 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e38ee920-d98f-4a37-a43e-9afa535be748_isrForCrosstalkSources_996963_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e38ee920-d98f-4a37-a43e-9afa535be748_isrForCrosstalkSources_996963_57.stderr parsl.dataflow.dflow INFO: Task 2231 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5b50305ab3941acc8e0f1d254f12ace with result from task 2231 parsl.dataflow.dflow DEBUG: Task 2231 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9660d3e1-70dd-4683-8460-68e8d8f783fa_isrForCrosstalkSources_997022_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9660d3e1-70dd-4683-8460-68e8d8f783fa_isrForCrosstalkSources_997022_57.stderr parsl.dataflow.dflow INFO: Task 2080 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ccc4cc506cb641398cd7c85f2433b66 with result from task 2080 parsl.dataflow.dflow DEBUG: Task 2080 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e6623b08-77ec-40db-a367-aafef1b12d8d_isrForCrosstalkSources_997031_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e6623b08-77ec-40db-a367-aafef1b12d8d_isrForCrosstalkSources_997031_24.stderr parsl.dataflow.dflow INFO: Task 2284 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a07a1b8f046885d3008e5a8a2a1d77b with result from task 2284 parsl.dataflow.dflow DEBUG: Task 2284 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7b8b24b4-7655-4878-9b39-e315906475a9_isrForCrosstalkSources_997005_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7b8b24b4-7655-4878-9b39-e315906475a9_isrForCrosstalkSources_997005_10.stderr parsl.dataflow.dflow INFO: Task 2042 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dab47e2c79a1364d915e86f8ea70c755 with result from task 2042 parsl.dataflow.dflow DEBUG: Task 2042 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/21960d2c-e651-47ba-8f85-52e30ed8f5e6_isrForCrosstalkSources_996981_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/21960d2c-e651-47ba-8f85-52e30ed8f5e6_isrForCrosstalkSources_996981_21.stderr parsl.dataflow.dflow INFO: Task 1961 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 548b552d634343a857609dbb8211ab6b with result from task 1961 parsl.dataflow.dflow DEBUG: Task 1961 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/64cbd154-4a52-43fa-87a5-bcebba7f11ee_isrForCrosstalkSources_997017_49.stdout parsl.dataflow.dflow INFO: Standard error for task 1961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/64cbd154-4a52-43fa-87a5-bcebba7f11ee_isrForCrosstalkSources_997017_49.stderr parsl.dataflow.dflow INFO: Task 2234 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce8d45a47bb0a33c543931c4f3c2d649 with result from task 2234 parsl.dataflow.dflow DEBUG: Task 2234 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/fc8456ee-e393-4a65-ab4b-dc64bba0e4d2_isrForCrosstalkSources_996957_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/fc8456ee-e393-4a65-ab4b-dc64bba0e4d2_isrForCrosstalkSources_996957_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2061 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry e27fe48612f6326e801e65e62b8a418a with result from task 2061 parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2061 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:53 proc_lsst.multi:146 [INFO] found job 17075232 in provider checkpoint proc_lsst.multi INFO: found job 17075232 in provider checkpoint parsl.dataflow.dflow INFO: Standard output for task 2061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2b4706d1-1eee-4ce5-bc14-9f49d5cacd14_isrForCrosstalkSources_996969_22.stdout parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Standard error for task 2061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/2b4706d1-1eee-4ce5-bc14-9f49d5cacd14_isrForCrosstalkSources_996969_22.stderr parsl.dataflow.dflow INFO: Task 2131 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 612a21bb2080b52fc17292966d093460 with result from task 2131 parsl.dataflow.dflow DEBUG: Task 2131 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/afe28813-1fb9-4275-b269-13024403a21f_isrForCrosstalkSources_997001_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/afe28813-1fb9-4275-b269-13024403a21f_isrForCrosstalkSources_997001_9.stderr parsl.dataflow.dflow INFO: Task 1966 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59c7e7ff4e76dca7f7c373e852877362 with result from task 1966 parsl.dataflow.dflow DEBUG: Task 1966 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/2127d0b1-e721-4338-9d63-9647b790182f_isrForCrosstalkSources_997020_30.stdout parsl.dataflow.dflow INFO: Standard error for task 1966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/2127d0b1-e721-4338-9d63-9647b790182f_isrForCrosstalkSources_997020_30.stderr parsl.dataflow.dflow INFO: Task 2037 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e25880e77ebb880ef35663ced6877f03 with result from task 2037 parsl.dataflow.dflow DEBUG: Task 2037 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/4f998954-4b75-423d-ab13-40038a482173_isrForCrosstalkSources_996955_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/4f998954-4b75-423d-ab13-40038a482173_isrForCrosstalkSources_996955_9.stderr parsl.dataflow.dflow INFO: Task 2227 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d8020cf0df4e68f6806cfdeecd80024 with result from task 2227 parsl.dataflow.dflow DEBUG: Task 2227 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c78c546c-99ba-4ba7-b616-e80b3b535ab7_isrForCrosstalkSources_996961_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/c78c546c-99ba-4ba7-b616-e80b3b535ab7_isrForCrosstalkSources_996961_42.stderr parsl.dataflow.dflow INFO: Task 2260 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d90d4bb5fa282f62fe03c5601e715ed6 with result from task 2260 parsl.dataflow.dflow DEBUG: Task 2260 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/86846414-849c-4714-8a90-f94442450042_isrForCrosstalkSources_996969_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/86846414-849c-4714-8a90-f94442450042_isrForCrosstalkSources_996969_1.stderr parsl.dataflow.dflow INFO: Task 2237 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b97ef19cfa9837aa9c7101f2ed7ab0a with result from task 2237 parsl.dataflow.dflow DEBUG: Task 2237 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/2b3c44b4-76f4-4410-9dd7-e6ae019217dd_isrForCrosstalkSources_997013_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/2b3c44b4-76f4-4410-9dd7-e6ae019217dd_isrForCrosstalkSources_997013_25.stderr parsl.dataflow.dflow INFO: Task 2257 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26577bece696caf3063fdfec93292d73 with result from task 2257 parsl.dataflow.dflow DEBUG: Task 2257 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/42f1b564-9f52-4e12-bf48-5e7b68b7483f_isrForCrosstalkSources_996996_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/42f1b564-9f52-4e12-bf48-5e7b68b7483f_isrForCrosstalkSources_996996_16.stderr parsl.dataflow.dflow INFO: Task 2100 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86ab75496b71eb0e8d1e8ee8cb6cfd82 with result from task 2100 parsl.dataflow.dflow DEBUG: Task 2100 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/44ae0ece-daaa-4d9a-a771-12c0cfaf4840_isrForCrosstalkSources_997021_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/44ae0ece-daaa-4d9a-a771-12c0cfaf4840_isrForCrosstalkSources_997021_57.stderr parsl.dataflow.dflow INFO: Task 2219 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6326ac01b40fdae15f6e7779ca87da4 with result from task 2219 parsl.dataflow.dflow DEBUG: Task 2219 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5ae23cf7-e2e8-4e48-a5cd-33e49a4f457d_isrForCrosstalkSources_996985_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5ae23cf7-e2e8-4e48-a5cd-33e49a4f457d_isrForCrosstalkSources_996985_17.stderr parsl.dataflow.dflow INFO: Task 2026 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 318327419503a6cd9b3e096921c27a45 with result from task 2026 parsl.dataflow.dflow DEBUG: Task 2026 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6d9ed66e-eb3c-4cf3-a424-615294c34b89_isrForCrosstalkSources_996974_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6d9ed66e-eb3c-4cf3-a424-615294c34b89_isrForCrosstalkSources_996974_37.stderr parsl.dataflow.dflow INFO: Task 2298 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39ea1864b497bf9f4dcc4151b58cd13f with result from task 2298 parsl.dataflow.dflow DEBUG: Task 2298 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc761b97-bc75-4cb6-b95c-e29c5e1553fa_isrForCrosstalkSources_997025_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc761b97-bc75-4cb6-b95c-e29c5e1553fa_isrForCrosstalkSources_997025_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:54 proc_lsst.multi:146 [INFO] found job 17075233 in provider checkpoint proc_lsst.multi INFO: found job 17075233 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2119 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9206d75b5bb6c53eab0d06d819b4eb4 with result from task 2119 parsl.dataflow.dflow DEBUG: Task 2119 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b79e850b-b9ea-4ac4-96f6-5ddb8588df6b_isrForCrosstalkSources_996963_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/b79e850b-b9ea-4ac4-96f6-5ddb8588df6b_isrForCrosstalkSources_996963_7.stderr parsl.dataflow.dflow INFO: Task 2085 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85b472d194630daf4862ff1400ccf8b2 with result from task 2085 parsl.dataflow.dflow DEBUG: Task 2085 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c7975230-a0f3-46e6-aa67-e4f42296fd52_isrForCrosstalkSources_996997_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c7975230-a0f3-46e6-aa67-e4f42296fd52_isrForCrosstalkSources_996997_18.stderr parsl.dataflow.dflow INFO: Task 2261 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f5f9e5f0e0c9b1380a729bb5290b71b with result from task 2261 parsl.dataflow.dflow DEBUG: Task 2261 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/a00b213d-64b3-4eff-b311-3ac94c3040e4_isrForCrosstalkSources_996989_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/a00b213d-64b3-4eff-b311-3ac94c3040e4_isrForCrosstalkSources_996989_21.stderr parsl.dataflow.dflow INFO: Task 2024 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e6a484e9664238cafe6c7677e37925f with result from task 2024 parsl.dataflow.dflow DEBUG: Task 2024 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/3125989d-c617-4228-bbb6-e3871121a1e6_isrForCrosstalkSources_997032_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/3125989d-c617-4228-bbb6-e3871121a1e6_isrForCrosstalkSources_997032_59.stderr parsl.dataflow.dflow INFO: Task 2299 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f489279c1f6ebca5bdd031e7a9394962 with result from task 2299 parsl.dataflow.dflow DEBUG: Task 2299 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/61b50259-ce11-49a8-9f58-e5a2dc76ec81_isrForCrosstalkSources_996958_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/61b50259-ce11-49a8-9f58-e5a2dc76ec81_isrForCrosstalkSources_996958_34.stderr parsl.dataflow.dflow INFO: Task 2313 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 35ee51dc92947312d4284bebdabf3a37 with result from task 2313 parsl.dataflow.dflow DEBUG: Task 2313 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5c418e62-1f77-49ff-b579-9e02223b6a26_isrForCrosstalkSources_996972_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5c418e62-1f77-49ff-b579-9e02223b6a26_isrForCrosstalkSources_996972_29.stderr parsl.dataflow.dflow INFO: Task 2300 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec5fb436118ff2faa876b682cf1b8b76 with result from task 2300 parsl.dataflow.dflow DEBUG: Task 2300 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d3df1fa7-a626-4320-a93f-d15b1aa620ac_isrForCrosstalkSources_996972_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d3df1fa7-a626-4320-a93f-d15b1aa620ac_isrForCrosstalkSources_996972_2.stderr parsl.dataflow.dflow INFO: Task 2259 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19fc54bdf126089ecce01b5c1b3ded7d with result from task 2259 parsl.dataflow.dflow DEBUG: Task 2259 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/db38e2fe-6f5f-41af-a93b-78c7554cab49_isrForCrosstalkSources_996971_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/db38e2fe-6f5f-41af-a93b-78c7554cab49_isrForCrosstalkSources_996971_1.stderr parsl.dataflow.dflow INFO: Task 1833 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5896ffd8501247cf68a6da4565e8946 with result from task 1833 parsl.dataflow.dflow DEBUG: Task 1833 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/772900ec-28f6-4210-ab44-b5faa2229dc8_isrForCrosstalkSources_996966_60.stdout parsl.dataflow.dflow INFO: Standard error for task 1833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/772900ec-28f6-4210-ab44-b5faa2229dc8_isrForCrosstalkSources_996966_60.stderr parsl.dataflow.dflow INFO: Task 2051 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e948ea80e105eecc47374cb932201187 with result from task 2051 parsl.dataflow.dflow DEBUG: Task 2051 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d8ac8d6a-2092-47f6-872d-4e3a77ee742d_isrForCrosstalkSources_996998_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d8ac8d6a-2092-47f6-872d-4e3a77ee742d_isrForCrosstalkSources_996998_2.stderr parsl.dataflow.dflow INFO: Task 2107 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7446cde50c010adca252d5db5c668578 with result from task 2107 parsl.dataflow.dflow DEBUG: Task 2107 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5ed3c61c-02d9-42a7-a586-fa1c7e509ff8_isrForCrosstalkSources_996955_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5ed3c61c-02d9-42a7-a586-fa1c7e509ff8_isrForCrosstalkSources_996955_47.stderr parsl.dataflow.dflow INFO: Task 2137 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb8668b4e6825ca9f4c1965384798a70 with result from task 2137 parsl.dataflow.dflow DEBUG: Task 2137 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/6d4f6ec3-f404-425b-98e7-dee6eac8a322_isrForCrosstalkSources_996965_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/6d4f6ec3-f404-425b-98e7-dee6eac8a322_isrForCrosstalkSources_996965_2.stderr parsl.dataflow.dflow INFO: Task 2161 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 684f9008be593ed0ebd59ce08a722ab8 with result from task 2161 parsl.dataflow.dflow DEBUG: Task 2161 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/70fc038e-5e2d-4eb1-ab69-b35f0165c0fe_isrForCrosstalkSources_996988_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/70fc038e-5e2d-4eb1-ab69-b35f0165c0fe_isrForCrosstalkSources_996988_59.stderr parsl.dataflow.dflow INFO: Task 2123 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ec421073f0fbeb86363f5209c56f146 with result from task 2123 parsl.dataflow.dflow DEBUG: Task 2123 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/7cc35d3a-e44a-4296-8e69-19e1e22aa0fd_isrForCrosstalkSources_997020_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/7cc35d3a-e44a-4296-8e69-19e1e22aa0fd_isrForCrosstalkSources_997020_37.stderr parsl.dataflow.dflow INFO: Task 2099 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6458a28fc429d7fb3a7270e8a22cc406 with result from task 2099 parsl.dataflow.dflow DEBUG: Task 2099 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/aba25683-c5fe-4ad5-872d-ccd4cf17de4e_isrForCrosstalkSources_997010_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/aba25683-c5fe-4ad5-872d-ccd4cf17de4e_isrForCrosstalkSources_997010_24.stderr parsl.dataflow.dflow INFO: Task 2126 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d54bb87c73e904498830fbbe185848f8 with result from task 2126 parsl.dataflow.dflow DEBUG: Task 2126 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e89bd236-0292-46d5-b5b1-857eaefe7f08_isrForCrosstalkSources_996952_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e89bd236-0292-46d5-b5b1-857eaefe7f08_isrForCrosstalkSources_996952_12.stderr parsl.dataflow.dflow INFO: Task 1965 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 43abec267c2db6c86fdda7feaf46f1ec with result from task 1965 parsl.dataflow.dflow DEBUG: Task 1965 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/64740a62-edce-4137-971d-b277165fa822_isrForCrosstalkSources_997002_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/64740a62-edce-4137-971d-b277165fa822_isrForCrosstalkSources_997002_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:55 proc_lsst.multi:146 [INFO] found job 17075234 in provider checkpoint proc_lsst.multi INFO: found job 17075234 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2169 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c042369bbcbe459ba526e7924468a72 with result from task 2169 parsl.dataflow.dflow DEBUG: Task 2169 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/a38780e4-f193-4677-a8c7-d4e05aee82f8_isrForCrosstalkSources_996997_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/a38780e4-f193-4677-a8c7-d4e05aee82f8_isrForCrosstalkSources_996997_36.stderr parsl.dataflow.dflow INFO: Task 2182 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry edc14a4f78c8eb24091182abee2e116a with result from task 2182 parsl.dataflow.dflow DEBUG: Task 2182 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/9b46810e-7781-47ae-a1da-5b9005a4f994_isrForCrosstalkSources_997027_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/9b46810e-7781-47ae-a1da-5b9005a4f994_isrForCrosstalkSources_997027_2.stderr parsl.dataflow.dflow INFO: Task 2076 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8454839e3c756c44492d979677c79aa1 with result from task 2076 parsl.dataflow.dflow DEBUG: Task 2076 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f4d3b35d-5cdd-42d0-af58-77583e7d1dcc_isrForCrosstalkSources_996957_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f4d3b35d-5cdd-42d0-af58-77583e7d1dcc_isrForCrosstalkSources_996957_2.stderr parsl.dataflow.dflow INFO: Task 1885 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f0770ac7f0ac85402760ca00928f12ff with result from task 1885 parsl.dataflow.dflow DEBUG: Task 1885 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/95959b3d-002a-450e-90a3-77cef5228713_isrForCrosstalkSources_997015_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/95959b3d-002a-450e-90a3-77cef5228713_isrForCrosstalkSources_997015_55.stderr parsl.dataflow.dflow INFO: Task 2003 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e87d82b52e4f5715ebb23d33a3a74a2 with result from task 2003 parsl.dataflow.dflow DEBUG: Task 2003 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/6dccda82-2d61-4ed3-b0b0-3e53679ebc06_isrForCrosstalkSources_996959_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/6dccda82-2d61-4ed3-b0b0-3e53679ebc06_isrForCrosstalkSources_996959_2.stderr parsl.dataflow.dflow INFO: Task 2086 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3120bfb1b49f0b78585a39ca63711ccb with result from task 2086 parsl.dataflow.dflow DEBUG: Task 2086 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/795ef32b-52e0-45dc-a9c8-b311b79e05b6_isrForCrosstalkSources_996975_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/795ef32b-52e0-45dc-a9c8-b311b79e05b6_isrForCrosstalkSources_996975_47.stderr parsl.dataflow.dflow INFO: Task 2280 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2dd2e5d64c9edf359ef4a6fad0f9bd6 with result from task 2280 parsl.dataflow.dflow DEBUG: Task 2280 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/12589320-54a6-45f6-aa34-6b707b2889bd_isrForCrosstalkSources_996952_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/12589320-54a6-45f6-aa34-6b707b2889bd_isrForCrosstalkSources_996952_35.stderr parsl.dataflow.dflow INFO: Task 2243 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da2d4306bac6938282e0d93c42a6282b with result from task 2243 parsl.dataflow.dflow DEBUG: Task 2243 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/70c9d390-557c-4f95-a0d8-415807d8bbba_isrForCrosstalkSources_996983_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/70c9d390-557c-4f95-a0d8-415807d8bbba_isrForCrosstalkSources_996983_15.stderr parsl.dataflow.dflow INFO: Task 2197 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30a40f47089d01cf2bfa312a38120c44 with result from task 2197 parsl.dataflow.dflow DEBUG: Task 2197 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5b5bd38f-8bbc-4630-901a-88c9f76d05d2_isrForCrosstalkSources_997010_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5b5bd38f-8bbc-4630-901a-88c9f76d05d2_isrForCrosstalkSources_997010_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:55 proc_lsst.multi:146 [INFO] found job 17075238 in provider checkpoint proc_lsst.multi INFO: found job 17075238 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2317 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df10ff9179afc286cf9a1d46b8130ac9 with result from task 2317 parsl.dataflow.dflow DEBUG: Task 2317 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f4e662a3-bcae-4069-ac7f-ec660633b6c3_isrForCrosstalkSources_996975_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f4e662a3-bcae-4069-ac7f-ec660633b6c3_isrForCrosstalkSources_996975_49.stderr parsl.dataflow.dflow INFO: Task 2315 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7de8c73f90ec207c8c65a95f79003ad with result from task 2315 parsl.dataflow.dflow DEBUG: Task 2315 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f19ee6af-3e72-4b94-9534-0e2ca3e9dd60_isrForCrosstalkSources_996956_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f19ee6af-3e72-4b94-9534-0e2ca3e9dd60_isrForCrosstalkSources_996956_5.stderr parsl.dataflow.dflow INFO: Task 1879 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0497ccb607d27b992ce2763b95a0b331 with result from task 1879 parsl.dataflow.dflow DEBUG: Task 1879 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/79f07025-f157-4aba-b9ea-a3004a1a598c_isrForCrosstalkSources_996957_53.stdout parsl.dataflow.dflow INFO: Standard error for task 1879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/79f07025-f157-4aba-b9ea-a3004a1a598c_isrForCrosstalkSources_996957_53.stderr parsl.dataflow.dflow INFO: Task 2321 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry adcd7a4f792bc9742ca58e5f4f910fd8 with result from task 2321 parsl.dataflow.dflow DEBUG: Task 2321 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2ea93fa7-11a4-4ec2-b89f-4c870e07444f_isrForCrosstalkSources_997003_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2ea93fa7-11a4-4ec2-b89f-4c870e07444f_isrForCrosstalkSources_997003_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:56 proc_lsst.multi:146 [INFO] found job 17075242 in provider checkpoint proc_lsst.multi INFO: found job 17075242 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2241 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33d0327b34efa59afa31b5127fc0d7bb with result from task 2241 parsl.dataflow.dflow DEBUG: Task 2241 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/77fac32a-0989-4734-a6e2-6aaaad6f7eb8_isrForCrosstalkSources_997012_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/77fac32a-0989-4734-a6e2-6aaaad6f7eb8_isrForCrosstalkSources_997012_24.stderr parsl.dataflow.dflow INFO: Task 2039 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 908a36e87c124681fc9f6d04bcb6f928 with result from task 2039 parsl.dataflow.dflow DEBUG: Task 2039 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/0d44eb83-ea90-4ac1-bcb4-1df66d95dd76_isrForCrosstalkSources_996995_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/0d44eb83-ea90-4ac1-bcb4-1df66d95dd76_isrForCrosstalkSources_996995_38.stderr parsl.dataflow.dflow INFO: Task 2235 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 68f0f7f31d49b45bb21557c25c790265 with result from task 2235 parsl.dataflow.dflow DEBUG: Task 2235 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/f9417734-5ffa-4ab8-a437-51acfa45623e_isrForCrosstalkSources_997007_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/f9417734-5ffa-4ab8-a437-51acfa45623e_isrForCrosstalkSources_997007_59.stderr parsl.dataflow.dflow INFO: Task 2213 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8fb45dd6633fea77c3824cd17e86d573 with result from task 2213 parsl.dataflow.dflow DEBUG: Task 2213 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/65f23818-6214-4824-bd6a-e835f2b9f671_isrForCrosstalkSources_997024_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/65f23818-6214-4824-bd6a-e835f2b9f671_isrForCrosstalkSources_997024_36.stderr parsl.dataflow.dflow INFO: Task 2089 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd0dd222ca7ccfc572f599989f67715d with result from task 2089 parsl.dataflow.dflow DEBUG: Task 2089 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/383c304d-7748-4632-9607-f96851408338_isrForCrosstalkSources_996965_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/383c304d-7748-4632-9607-f96851408338_isrForCrosstalkSources_996965_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:57 proc_lsst.multi:146 [INFO] found job 17075245 in provider checkpoint proc_lsst.multi INFO: found job 17075245 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2327 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84b605b9d3bbefa383228e9f0e0111cc with result from task 2327 parsl.dataflow.dflow DEBUG: Task 2327 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/b393910a-37a5-446f-86b6-514de41396f3_isrForCrosstalkSources_997023_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/b393910a-37a5-446f-86b6-514de41396f3_isrForCrosstalkSources_997023_41.stderr parsl.dataflow.dflow INFO: Task 2319 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry caf7f59801e30f2d3b647a1ce2ffb774 with result from task 2319 parsl.dataflow.dflow DEBUG: Task 2319 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/7e16fa75-1547-41dc-a36f-ec710d302b4f_isrForCrosstalkSources_996955_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/7e16fa75-1547-41dc-a36f-ec710d302b4f_isrForCrosstalkSources_996955_39.stderr parsl.dataflow.dflow INFO: Task 2296 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fca6ff367c5506c79489c9b1e97a686 with result from task 2296 parsl.dataflow.dflow DEBUG: Task 2296 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/8626658e-4568-4120-bf54-d591a8db30b3_isrForCrosstalkSources_997031_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/8626658e-4568-4120-bf54-d591a8db30b3_isrForCrosstalkSources_997031_29.stderr parsl.dataflow.dflow INFO: Task 2294 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93702a911fd512d9974d61ab31b2cd2a with result from task 2294 parsl.dataflow.dflow DEBUG: Task 2294 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/02b3a8a3-f33a-4aae-b351-0d37b039e45b_isrForCrosstalkSources_996953_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/02b3a8a3-f33a-4aae-b351-0d37b039e45b_isrForCrosstalkSources_996953_51.stderr parsl.dataflow.dflow INFO: Task 1969 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7682a978409222ef52c888b6722f3323 with result from task 1969 parsl.dataflow.dflow DEBUG: Task 1969 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/3b603dc8-122b-47e6-bf74-5afcd9d5f2b9_isrForCrosstalkSources_996993_38.stdout parsl.dataflow.dflow INFO: Standard error for task 1969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/3b603dc8-122b-47e6-bf74-5afcd9d5f2b9_isrForCrosstalkSources_996993_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:57 proc_lsst.multi:146 [INFO] found job 17075246 in provider checkpoint proc_lsst.multi INFO: found job 17075246 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2078 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e9bc8440625b0c069645ac39709a889 with result from task 2078 parsl.dataflow.dflow DEBUG: Task 2078 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/00234f11-cbed-4d31-b607-498b321fcb04_isrForCrosstalkSources_996965_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/00234f11-cbed-4d31-b607-498b321fcb04_isrForCrosstalkSources_996965_42.stderr parsl.dataflow.dflow INFO: Task 2326 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da413637f565cce703cfe68c76c01ee7 with result from task 2326 parsl.dataflow.dflow DEBUG: Task 2326 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/b12f7786-096a-4f76-9bdf-6f56ccd63e9f_isrForCrosstalkSources_996974_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/b12f7786-096a-4f76-9bdf-6f56ccd63e9f_isrForCrosstalkSources_996974_61.stderr parsl.dataflow.dflow INFO: Task 2324 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a993a47afc38f4cd67794fa6dd52103 with result from task 2324 parsl.dataflow.dflow DEBUG: Task 2324 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7c517b2a-1633-4e0d-aa36-06a022fac9f5_isrForCrosstalkSources_997033_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7c517b2a-1633-4e0d-aa36-06a022fac9f5_isrForCrosstalkSources_997033_44.stderr parsl.dataflow.dflow INFO: Task 2277 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c787befff00965a824d918b4d7cf96d0 with result from task 2277 parsl.dataflow.dflow DEBUG: Task 2277 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/c94c42ce-b86c-48e0-a197-ccded7a7722d_isrForCrosstalkSources_997018_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/c94c42ce-b86c-48e0-a197-ccded7a7722d_isrForCrosstalkSources_997018_11.stderr parsl.dataflow.dflow INFO: Task 2273 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e911f646b764318b62f5d1fa2ada9370 with result from task 2273 parsl.dataflow.dflow DEBUG: Task 2273 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/b32be9ec-57c6-42df-915e-8331344c221c_isrForCrosstalkSources_996989_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/b32be9ec-57c6-42df-915e-8331344c221c_isrForCrosstalkSources_996989_19.stderr parsl.dataflow.dflow INFO: Task 2236 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 530995d7f17a82633cb082e799868f5d with result from task 2236 parsl.dataflow.dflow DEBUG: Task 2236 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d3bc9205-9f96-4a28-ac31-4a082cb6e6d8_isrForCrosstalkSources_997023_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d3bc9205-9f96-4a28-ac31-4a082cb6e6d8_isrForCrosstalkSources_997023_1.stderr parsl.dataflow.dflow INFO: Task 2242 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a324f228acea98b6e40cf6018973335 with result from task 2242 parsl.dataflow.dflow DEBUG: Task 2242 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/fc44ebdf-13d2-43a6-9c30-be0c357a9b0b_isrForCrosstalkSources_996965_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/fc44ebdf-13d2-43a6-9c30-be0c357a9b0b_isrForCrosstalkSources_996965_28.stderr parsl.dataflow.dflow INFO: Task 1989 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e33ea605d2852af728ba9477481f1f7e with result from task 1989 parsl.dataflow.dflow DEBUG: Task 1989 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/647ae71b-6e8d-44ae-96d4-938b1ff80cdd_isrForCrosstalkSources_996961_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/647ae71b-6e8d-44ae-96d4-938b1ff80cdd_isrForCrosstalkSources_996961_36.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:58 proc_lsst.multi:146 [INFO] found job 17075247 in provider checkpoint proc_lsst.multi INFO: found job 17075247 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2054 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d281b652cd396d3208a5a2aae7bfd8cf with result from task 2054 parsl.dataflow.dflow DEBUG: Task 2054 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/cfca7db3-3479-40ff-906a-317d9173abfb_isrForCrosstalkSources_997032_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/cfca7db3-3479-40ff-906a-317d9173abfb_isrForCrosstalkSources_997032_1.stderr parsl.dataflow.dflow INFO: Task 2283 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c02a52cc49ffd6e4989d176d13b66c4 with result from task 2283 parsl.dataflow.dflow DEBUG: Task 2283 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/c81522dd-cb26-4619-93d8-42d8400b088b_isrForCrosstalkSources_997011_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/c81522dd-cb26-4619-93d8-42d8400b088b_isrForCrosstalkSources_997011_60.stderr parsl.dataflow.dflow INFO: Task 2331 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e44d39183eb9ba22b0ec7bc609f1ce7e with result from task 2331 parsl.dataflow.dflow DEBUG: Task 2331 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c5966daf-5fee-4b1d-b7ab-f67a08e2e1ae_isrForCrosstalkSources_997006_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c5966daf-5fee-4b1d-b7ab-f67a08e2e1ae_isrForCrosstalkSources_997006_17.stderr parsl.dataflow.dflow INFO: Task 1976 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c071acf07f9831c7b132c1586df470c with result from task 1976 parsl.dataflow.dflow DEBUG: Task 1976 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/94b1bd81-8c82-4792-a0f3-59f384fe755f_isrForCrosstalkSources_997022_42.stdout parsl.dataflow.dflow INFO: Standard error for task 1976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/94b1bd81-8c82-4792-a0f3-59f384fe755f_isrForCrosstalkSources_997022_42.stderr parsl.dataflow.dflow INFO: Task 2240 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef704453814d879da8f9f39525ba3827 with result from task 2240 parsl.dataflow.dflow DEBUG: Task 2240 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/59cd39ce-842f-4523-87f9-2649af3d8f48_isrForCrosstalkSources_996975_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/59cd39ce-842f-4523-87f9-2649af3d8f48_isrForCrosstalkSources_996975_43.stderr parsl.dataflow.dflow INFO: Task 2238 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 34d5eda9b16422cc6291f3bb95e63eea with result from task 2238 parsl.dataflow.dflow DEBUG: Task 2238 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6f48b8b1-4d14-4d68-9b6e-d9586c104d0e_isrForCrosstalkSources_997030_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6f48b8b1-4d14-4d68-9b6e-d9586c104d0e_isrForCrosstalkSources_997030_31.stderr parsl.dataflow.dflow INFO: Task 2330 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ba203a200a9e9d362d8abc6bc89085e with result from task 2330 parsl.dataflow.dflow DEBUG: Task 2330 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/585c3807-0007-43ac-8b82-dab2edad9d46_isrForCrosstalkSources_997031_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/585c3807-0007-43ac-8b82-dab2edad9d46_isrForCrosstalkSources_997031_54.stderr parsl.dataflow.dflow INFO: Task 2301 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 308830a56d5454f088cf479839f256d8 with result from task 2301 parsl.dataflow.dflow DEBUG: Task 2301 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/134fe129-e407-4822-9f00-2115478ff8e2_isrForCrosstalkSources_997000_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/134fe129-e407-4822-9f00-2115478ff8e2_isrForCrosstalkSources_997000_61.stderr parsl.dataflow.dflow INFO: Task 2342 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f73937dbf80e40dc27b3dd9ae820502 with result from task 2342 parsl.dataflow.dflow DEBUG: Task 2342 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4fd2b40e-2f7a-4d3d-98e7-00f0e9b7d140_isrForCrosstalkSources_996962_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4fd2b40e-2f7a-4d3d-98e7-00f0e9b7d140_isrForCrosstalkSources_996962_36.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:59 proc_lsst.multi:146 [INFO] found job 17075250 in provider checkpoint proc_lsst.multi INFO: found job 17075250 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2239 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e00a1a44d33dff9e688ccaaa5e17d854 with result from task 2239 parsl.dataflow.dflow DEBUG: Task 2239 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c841a656-38fc-4322-a0ba-d0e8dd939f2f_isrForCrosstalkSources_997013_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c841a656-38fc-4322-a0ba-d0e8dd939f2f_isrForCrosstalkSources_997013_62.stderr parsl.dataflow.dflow INFO: Task 2136 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8168c36f7777956594ea40e283c4c375 with result from task 2136 parsl.dataflow.dflow DEBUG: Task 2136 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1fb17c38-d822-4ba2-aac1-544eab1858fa_isrForCrosstalkSources_997019_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1fb17c38-d822-4ba2-aac1-544eab1858fa_isrForCrosstalkSources_997019_43.stderr parsl.dataflow.dflow INFO: Task 2352 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02eb689a0071c4e699973e07d939c655 with result from task 2352 parsl.dataflow.dflow DEBUG: Task 2352 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a98f46c2-5418-4598-8758-668049960e86_isrForCrosstalkSources_996977_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a98f46c2-5418-4598-8758-668049960e86_isrForCrosstalkSources_996977_27.stderr parsl.dataflow.dflow INFO: Task 1963 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e33bb6143952f0655975315fe187f44 with result from task 1963 parsl.dataflow.dflow DEBUG: Task 1963 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/6084eeee-fd04-404c-a442-20a144eeaf63_isrForCrosstalkSources_996990_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/6084eeee-fd04-404c-a442-20a144eeaf63_isrForCrosstalkSources_996990_58.stderr parsl.dataflow.dflow INFO: Task 2332 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7545e7d21602261d5c750f3f72ee39b4 with result from task 2332 parsl.dataflow.dflow DEBUG: Task 2332 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/13021847-a9c9-4e24-9979-93e1384949ae_isrForCrosstalkSources_996985_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/13021847-a9c9-4e24-9979-93e1384949ae_isrForCrosstalkSources_996985_57.stderr parsl.dataflow.dflow INFO: Task 2101 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 779873a430146843fda34c90a47875e7 with result from task 2101 parsl.dataflow.dflow DEBUG: Task 2101 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6622f1a9-cbb3-4e8a-b5a3-31e956f0c0c6_isrForCrosstalkSources_997003_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/6622f1a9-cbb3-4e8a-b5a3-31e956f0c0c6_isrForCrosstalkSources_997003_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:55:59 proc_lsst.multi:146 [INFO] found job 17075254 in provider checkpoint proc_lsst.multi INFO: found job 17075254 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2208 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c541b13d6b01be1980ae3d72e705863 with result from task 2208 parsl.dataflow.dflow DEBUG: Task 2208 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6131745f-dbbe-4d77-b53c-80e7b1c04bc8_isrForCrosstalkSources_996983_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6131745f-dbbe-4d77-b53c-80e7b1c04bc8_isrForCrosstalkSources_996983_53.stderr parsl.dataflow.dflow INFO: Task 2311 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e614061aa3104a0886411a2e30788e7 with result from task 2311 parsl.dataflow.dflow DEBUG: Task 2311 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f1292f42-c500-44ec-a352-c90173ac1293_isrForCrosstalkSources_996958_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/f1292f42-c500-44ec-a352-c90173ac1293_isrForCrosstalkSources_996958_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:00 proc_lsst.multi:146 [INFO] found job 17075255 in provider checkpoint proc_lsst.multi INFO: found job 17075255 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:00 proc_lsst.multi:146 [INFO] found job 17075256 in provider checkpoint proc_lsst.multi INFO: found job 17075256 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2359 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46d865cc14698a9d4e102363163be648 with result from task 2359 parsl.dataflow.dflow DEBUG: Task 2359 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/88496502-1e01-4dcb-8749-b170b1854d10_isrForCrosstalkSources_996993_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/88496502-1e01-4dcb-8749-b170b1854d10_isrForCrosstalkSources_996993_47.stderr parsl.dataflow.dflow INFO: Task 2000 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fef5fc26f6950e6caa77dab8b3e3309 with result from task 2000 parsl.dataflow.dflow DEBUG: Task 2000 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/9264fe03-d8ef-4991-ba65-5d2c654e6600_isrForCrosstalkSources_997005_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/9264fe03-d8ef-4991-ba65-5d2c654e6600_isrForCrosstalkSources_997005_47.stderr parsl.dataflow.dflow INFO: Task 2266 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7feaafe87aebb5970f4bf37d13678a19 with result from task 2266 parsl.dataflow.dflow DEBUG: Task 2266 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/466a0032-e4d7-4227-9a52-89fa1d35a7d6_isrForCrosstalkSources_996955_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/466a0032-e4d7-4227-9a52-89fa1d35a7d6_isrForCrosstalkSources_996955_25.stderr parsl.dataflow.dflow INFO: Task 2303 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd679444d09739899e8d4e81991f7bb5 with result from task 2303 parsl.dataflow.dflow DEBUG: Task 2303 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/370cd279-35fb-42cd-bd6e-c373a3a3a387_isrForCrosstalkSources_996958_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/370cd279-35fb-42cd-bd6e-c373a3a3a387_isrForCrosstalkSources_996958_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:01 proc_lsst.multi:146 [INFO] found job 17075257 in provider checkpoint proc_lsst.multi INFO: found job 17075257 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2156 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2c20a31c73143b45810b069dc698985 with result from task 2156 parsl.dataflow.dflow DEBUG: Task 2156 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/752dd4d5-4cf7-46a9-ace3-83e130580128_isrForCrosstalkSources_996960_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/752dd4d5-4cf7-46a9-ace3-83e130580128_isrForCrosstalkSources_996960_34.stderr parsl.dataflow.dflow INFO: Task 2120 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a753d5261fad4a62a74059bbcd245b0 with result from task 2120 parsl.dataflow.dflow DEBUG: Task 2120 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d18f913c-86a9-4908-9d14-56819c4fed45_isrForCrosstalkSources_996988_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d18f913c-86a9-4908-9d14-56819c4fed45_isrForCrosstalkSources_996988_55.stderr parsl.dataflow.dflow INFO: Task 2102 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04472939d075bf34ada51ccc498ce0c7 with result from task 2102 parsl.dataflow.dflow DEBUG: Task 2102 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/134964ff-76e8-4501-be0c-29d2000d0d29_isrForCrosstalkSources_997004_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/134964ff-76e8-4501-be0c-29d2000d0d29_isrForCrosstalkSources_997004_58.stderr parsl.dataflow.dflow INFO: Task 2369 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc2e323e1d000cf339bb89b29d017c5e with result from task 2369 parsl.dataflow.dflow DEBUG: Task 2369 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/50a6cf56-b69b-492f-8e6b-bbc57e5b724e_isrForCrosstalkSources_996978_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/50a6cf56-b69b-492f-8e6b-bbc57e5b724e_isrForCrosstalkSources_996978_22.stderr parsl.dataflow.dflow INFO: Task 2361 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c7941d098dd36137cd7574b333ffb4c with result from task 2361 parsl.dataflow.dflow DEBUG: Task 2361 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f3b4401c-0211-444a-88e4-6f547e451910_isrForCrosstalkSources_996966_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f3b4401c-0211-444a-88e4-6f547e451910_isrForCrosstalkSources_996966_4.stderr parsl.dataflow.dflow INFO: Task 2308 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ed81afed29cf8b09b3b5bde9d0670ba with result from task 2308 parsl.dataflow.dflow DEBUG: Task 2308 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b0d5cd1a-76ea-49ed-871b-87dc45786a83_isrForCrosstalkSources_997001_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b0d5cd1a-76ea-49ed-871b-87dc45786a83_isrForCrosstalkSources_997001_22.stderr parsl.dataflow.dflow INFO: Task 2121 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a59279dd131766c10c5d88a1c612958 with result from task 2121 parsl.dataflow.dflow DEBUG: Task 2121 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/18529793-fec3-454a-9b1e-61a87e198b68_isrForCrosstalkSources_997006_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/18529793-fec3-454a-9b1e-61a87e198b68_isrForCrosstalkSources_997006_59.stderr parsl.dataflow.dflow INFO: Task 2152 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e36187ef85854a8946b81bf3f2375b30 with result from task 2152 parsl.dataflow.dflow DEBUG: Task 2152 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3129e9e4-0eb1-4e15-bad5-b0e03bf845c7_isrForCrosstalkSources_996984_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3129e9e4-0eb1-4e15-bad5-b0e03bf845c7_isrForCrosstalkSources_996984_24.stderr parsl.dataflow.dflow INFO: Task 1984 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b781f2756daefe6db88355f81f01888 with result from task 1984 parsl.dataflow.dflow DEBUG: Task 1984 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/269494aa-afa7-4487-b1f0-2a7e11028e1d_isrForCrosstalkSources_996960_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/269494aa-afa7-4487-b1f0-2a7e11028e1d_isrForCrosstalkSources_996960_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:02 proc_lsst.multi:146 [INFO] found job 17075258 in provider checkpoint proc_lsst.multi INFO: found job 17075258 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1884 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32f88bd7334b024d91a6e3e0bf1963a8 with result from task 1884 parsl.dataflow.dflow DEBUG: Task 1884 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/d8475ca3-2302-40e4-bf0b-865b5938a842_isrForCrosstalkSources_996978_50.stdout parsl.dataflow.dflow INFO: Standard error for task 1884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/d8475ca3-2302-40e4-bf0b-865b5938a842_isrForCrosstalkSources_996978_50.stderr parsl.dataflow.dflow INFO: Task 2256 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a95861e8fb29f2b1fbe5097037397ff2 with result from task 2256 parsl.dataflow.dflow DEBUG: Task 2256 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/99bd8550-0ace-4838-8af9-96fbf1b076fb_isrForCrosstalkSources_996953_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/99bd8550-0ace-4838-8af9-96fbf1b076fb_isrForCrosstalkSources_996953_27.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:02 proc_lsst.multi:146 [INFO] found job 17075264 in provider checkpoint proc_lsst.multi INFO: found job 17075264 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 1818 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 853726e27453b020153416d89918d69d with result from task 1818 parsl.dataflow.dflow DEBUG: Task 1818 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7ad802ec-214f-4f62-a838-f55c0007b2ab_isrForCrosstalkSources_996986_55.stdout parsl.dataflow.dflow INFO: Standard error for task 1818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7ad802ec-214f-4f62-a838-f55c0007b2ab_isrForCrosstalkSources_996986_55.stderr parsl.dataflow.dflow INFO: Task 1822 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 28a718eba1f82c97f22de8ef67e0acc2 with result from task 1822 parsl.dataflow.dflow DEBUG: Task 1822 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/10a85776-b4a2-4724-a7ba-c1723aee74e0_isrForCrosstalkSources_996995_36.stdout parsl.dataflow.dflow INFO: Standard error for task 1822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/10a85776-b4a2-4724-a7ba-c1723aee74e0_isrForCrosstalkSources_996995_36.stderr parsl.dataflow.dflow INFO: Task 2333 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb139be7dfc4898820446094a3d24930 with result from task 2333 parsl.dataflow.dflow DEBUG: Task 2333 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/4fd1d483-7b1b-477e-ab4c-c23b06167ed2_isrForCrosstalkSources_997027_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/4fd1d483-7b1b-477e-ab4c-c23b06167ed2_isrForCrosstalkSources_997027_45.stderr parsl.dataflow.dflow INFO: Task 1823 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf2a597906554184ce75e8f426dbffe5 with result from task 1823 parsl.dataflow.dflow DEBUG: Task 1823 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/29a08579-8a0b-45f6-a512-1471c8fafe4c_isrForCrosstalkSources_996987_24.stdout parsl.dataflow.dflow INFO: Standard error for task 1823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/29a08579-8a0b-45f6-a512-1471c8fafe4c_isrForCrosstalkSources_996987_24.stderr parsl.dataflow.dflow INFO: Task 1819 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a7ba272b448aab4112fbf6e95ea5c007 with result from task 1819 parsl.dataflow.dflow DEBUG: Task 1819 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/84560068-762c-4b7a-a7ed-638ccc1251ec_isrForCrosstalkSources_996986_57.stdout parsl.dataflow.dflow INFO: Standard error for task 1819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/84560068-762c-4b7a-a7ed-638ccc1251ec_isrForCrosstalkSources_996986_57.stderr parsl.dataflow.dflow INFO: Task 1820 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 983abeb9282b8ef3f4d270561b12ce42 with result from task 1820 parsl.dataflow.dflow DEBUG: Task 1820 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/ba7bebd4-c6e4-4865-b705-e9cd7fe2ea98_isrForCrosstalkSources_996997_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/ba7bebd4-c6e4-4865-b705-e9cd7fe2ea98_isrForCrosstalkSources_996997_29.stderr parsl.dataflow.dflow INFO: Task 1817 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e0bb106f77ff3e2814669c8b233f203 with result from task 1817 parsl.dataflow.dflow DEBUG: Task 1817 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/0b1bdef3-29b5-435d-80e6-c6f4631d12dd_isrForCrosstalkSources_997024_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/0b1bdef3-29b5-435d-80e6-c6f4631d12dd_isrForCrosstalkSources_997024_21.stderr parsl.dataflow.dflow INFO: Task 2040 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46cb17e222df259600a74b07e68fe1b9 with result from task 2040 parsl.dataflow.dflow DEBUG: Task 2040 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/83988cea-47fc-47a4-9509-fadc9ae77649_isrForCrosstalkSources_997015_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/83988cea-47fc-47a4-9509-fadc9ae77649_isrForCrosstalkSources_997015_62.stderr parsl.dataflow.dflow INFO: Task 1816 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5eae9db39d8bf618caba6686b2bb400 with result from task 1816 parsl.dataflow.dflow DEBUG: Task 1816 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/97ac01ba-bdcc-4267-82f6-5f1e1f2f1a67_isrForCrosstalkSources_996973_22.stdout parsl.dataflow.dflow INFO: Standard error for task 1816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/97ac01ba-bdcc-4267-82f6-5f1e1f2f1a67_isrForCrosstalkSources_996973_22.stderr parsl.dataflow.dflow INFO: Task 1821 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f5d5dc8d5a795a0cd2148fcc2b7a5af with result from task 1821 parsl.dataflow.dflow DEBUG: Task 1821 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0d723bfd-ac80-4087-a278-7a614dd729bb_isrForCrosstalkSources_996956_52.stdout parsl.dataflow.dflow INFO: Standard error for task 1821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0d723bfd-ac80-4087-a278-7a614dd729bb_isrForCrosstalkSources_996956_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:03 proc_lsst.multi:146 [INFO] found job 17075265 in provider checkpoint proc_lsst.multi INFO: found job 17075265 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2204 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 902d227ca71337cb4f36134133467ff9 with result from task 2204 parsl.dataflow.dflow DEBUG: Task 2204 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/6af65e2c-80f1-4d6c-ab72-70b1f11e4c06_isrForCrosstalkSources_996986_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/6af65e2c-80f1-4d6c-ab72-70b1f11e4c06_isrForCrosstalkSources_996986_19.stderr parsl.dataflow.dflow INFO: Task 2383 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30d51808976a16436f80966e47f23f66 with result from task 2383 parsl.dataflow.dflow DEBUG: Task 2383 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/11d053dc-d160-4344-928b-65da9daaa21a_isrForCrosstalkSources_996975_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/11d053dc-d160-4344-928b-65da9daaa21a_isrForCrosstalkSources_996975_51.stderr parsl.dataflow.dflow INFO: Task 2253 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b538382c43657b0e5a057373c9a9e08 with result from task 2253 parsl.dataflow.dflow DEBUG: Task 2253 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/233dcb1a-553e-41ae-9414-edba00dbab55_isrForCrosstalkSources_996998_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/233dcb1a-553e-41ae-9414-edba00dbab55_isrForCrosstalkSources_996998_4.stderr parsl.dataflow.dflow INFO: Task 2310 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f8998b99d313fd013dc5738b50bc9f2c with result from task 2310 parsl.dataflow.dflow DEBUG: Task 2310 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/9af04eba-280f-4b0c-8a65-8f189ef461a0_isrForCrosstalkSources_996973_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/9af04eba-280f-4b0c-8a65-8f189ef461a0_isrForCrosstalkSources_996973_31.stderr parsl.dataflow.dflow INFO: Task 2181 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d2802afdedbd0dea4a84a91b7aa4339a with result from task 2181 parsl.dataflow.dflow DEBUG: Task 2181 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/236371bb-cf82-4335-895c-13a2305c9d74_isrForCrosstalkSources_997020_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/236371bb-cf82-4335-895c-13a2305c9d74_isrForCrosstalkSources_997020_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:03 proc_lsst.multi:146 [INFO] found job 17075273 in provider checkpoint proc_lsst.multi INFO: found job 17075273 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2195 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d80fd866518608e5b6cdaf65815f42bb with result from task 2195 parsl.dataflow.dflow DEBUG: Task 2195 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/8866391a-cf3d-4632-9056-53376d8385b6_isrForCrosstalkSources_996959_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/8866391a-cf3d-4632-9056-53376d8385b6_isrForCrosstalkSources_996959_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:04 proc_lsst.multi:146 [INFO] found job 17075274 in provider checkpoint proc_lsst.multi INFO: found job 17075274 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2249 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fcf91f70ca5e3c267ce96038aced4fcc with result from task 2249 parsl.dataflow.dflow DEBUG: Task 2249 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4f079502-8718-4562-a799-1dce8edc8d8c_isrForCrosstalkSources_996987_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4f079502-8718-4562-a799-1dce8edc8d8c_isrForCrosstalkSources_996987_56.stderr parsl.dataflow.dflow INFO: Task 2388 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec4dadad0f5472ae29948335a23a661b with result from task 2388 parsl.dataflow.dflow DEBUG: Task 2388 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/38ba85de-2615-4860-81e9-c84d8c7df94a_isrForCrosstalkSources_996994_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/38ba85de-2615-4860-81e9-c84d8c7df94a_isrForCrosstalkSources_996994_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:04 proc_lsst.multi:146 [INFO] found job 17075275 in provider checkpoint proc_lsst.multi INFO: found job 17075275 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2410 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c25c7c5553129c3c6a1e41fd29cd230 with result from task 2410 parsl.dataflow.dflow DEBUG: Task 2410 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/65ceb63c-17bd-4789-8aeb-5089d31f0083_isrForCrosstalkSources_996966_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/65ceb63c-17bd-4789-8aeb-5089d31f0083_isrForCrosstalkSources_996966_38.stderr parsl.dataflow.dflow INFO: Task 2305 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae76bfa2b59a6b03c07f1b1425a8367f with result from task 2305 parsl.dataflow.dflow DEBUG: Task 2305 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/dc9f8eb9-1d0e-43a2-b8c7-69b8daa31cbe_isrForCrosstalkSources_996955_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/dc9f8eb9-1d0e-43a2-b8c7-69b8daa31cbe_isrForCrosstalkSources_996955_11.stderr parsl.dataflow.dflow INFO: Task 1881 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1efb4572a697cc405fe1d3d773b4af91 with result from task 1881 parsl.dataflow.dflow DEBUG: Task 1881 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a5715e6f-1dc3-4dfa-8702-1fe9ca793f84_isrForCrosstalkSources_997018_62.stdout parsl.dataflow.dflow INFO: Standard error for task 1881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a5715e6f-1dc3-4dfa-8702-1fe9ca793f84_isrForCrosstalkSources_997018_62.stderr parsl.dataflow.dflow INFO: Task 2409 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a55a6d4ab2529e1d229d18d474da98fa with result from task 2409 parsl.dataflow.dflow DEBUG: Task 2409 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/76e54c44-1c11-4e1e-b3aa-303162554bd4_isrForCrosstalkSources_997029_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/76e54c44-1c11-4e1e-b3aa-303162554bd4_isrForCrosstalkSources_997029_20.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:05 proc_lsst.multi:146 [INFO] found job 17075276 in provider checkpoint proc_lsst.multi INFO: found job 17075276 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:05 proc_lsst.multi:146 [INFO] found job 17075278 in provider checkpoint proc_lsst.multi INFO: found job 17075278 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2391 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2509c009e44339e633b08a423b5ff07d with result from task 2391 parsl.dataflow.dflow DEBUG: Task 2391 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bc40cf74-283f-4ce5-b383-000ec9fa5508_isrForCrosstalkSources_997026_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bc40cf74-283f-4ce5-b383-000ec9fa5508_isrForCrosstalkSources_997026_30.stderr parsl.dataflow.dflow INFO: Task 2134 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25534cecb0a6da34e44c0ba481681264 with result from task 2134 parsl.dataflow.dflow DEBUG: Task 2134 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a3bff4f6-d131-46dd-8e41-74a0d2b7fefa_isrForCrosstalkSources_996991_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a3bff4f6-d131-46dd-8e41-74a0d2b7fefa_isrForCrosstalkSources_996991_41.stderr parsl.dataflow.dflow INFO: Task 2347 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82e6927c85796de801fbab8253e1c5b8 with result from task 2347 parsl.dataflow.dflow DEBUG: Task 2347 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/08ecb673-2cd6-49a1-8bcd-dbd666546e46_isrForCrosstalkSources_997014_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/08ecb673-2cd6-49a1-8bcd-dbd666546e46_isrForCrosstalkSources_997014_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:06 proc_lsst.multi:146 [INFO] found job 17075279 in provider checkpoint proc_lsst.multi INFO: found job 17075279 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2378 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a58f73e89226db50ba87bd34256bb77b with result from task 2378 parsl.dataflow.dflow DEBUG: Task 2378 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8e70616b-b673-44b0-8a69-e37685f7167f_isrForCrosstalkSources_996967_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8e70616b-b673-44b0-8a69-e37685f7167f_isrForCrosstalkSources_996967_2.stderr parsl.dataflow.dflow INFO: Task 2349 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 64f6fd8e5399be4ce358f8aa21abc4ab with result from task 2349 parsl.dataflow.dflow DEBUG: Task 2349 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a3bfede6-ebdb-461d-bb4e-db00b3998a45_isrForCrosstalkSources_996957_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a3bfede6-ebdb-461d-bb4e-db00b3998a45_isrForCrosstalkSources_996957_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2228 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 9f4cc17b636fa1dd1007c5425d5c81c1 with result from task 2228 parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2228 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/3453907b-acef-413d-bb02-0e0f501aae31_isrForCrosstalkSources_996971_22.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/3453907b-acef-413d-bb02-0e0f501aae31_isrForCrosstalkSources_996971_22.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:07 proc_lsst.multi:146 [INFO] found job 17075280 in provider checkpoint proc_lsst.multi INFO: found job 17075280 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2433 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3e4249754c0f655167b07e5513028d99 with result from task 2433 parsl.dataflow.dflow DEBUG: Task 2433 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c9096242-5b0a-477e-8a9d-d92701c4019e_isrForCrosstalkSources_997019_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c9096242-5b0a-477e-8a9d-d92701c4019e_isrForCrosstalkSources_997019_31.stderr parsl.dataflow.dflow INFO: Task 2194 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa1312379d3d9b64d6b359023b61856b with result from task 2194 parsl.dataflow.dflow DEBUG: Task 2194 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bdcd8e20-3bee-411c-aff2-a3248adeae4e_isrForCrosstalkSources_996960_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bdcd8e20-3bee-411c-aff2-a3248adeae4e_isrForCrosstalkSources_996960_24.stderr parsl.dataflow.dflow INFO: Task 2411 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d36051d8fd8273010403131bdfa9bce4 with result from task 2411 parsl.dataflow.dflow DEBUG: Task 2411 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/364605aa-ecf8-4ce8-80d7-c3d0ee05d59e_isrForCrosstalkSources_996976_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/364605aa-ecf8-4ce8-80d7-c3d0ee05d59e_isrForCrosstalkSources_996976_26.stderr parsl.dataflow.dflow INFO: Task 2431 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f8a25c60699202b7cfeb2f6e8c9997f with result from task 2431 parsl.dataflow.dflow DEBUG: Task 2431 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1c751a80-1e97-4e29-ac36-1cbd6fbf4c3a_isrForCrosstalkSources_996977_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1c751a80-1e97-4e29-ac36-1cbd6fbf4c3a_isrForCrosstalkSources_996977_59.stderr parsl.dataflow.dflow INFO: Task 2439 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9319b3019a67ed43445dc3890f7b38f with result from task 2439 parsl.dataflow.dflow DEBUG: Task 2439 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c6f82afe-3c56-425c-9839-33bcc9162ac8_isrForCrosstalkSources_996973_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c6f82afe-3c56-425c-9839-33bcc9162ac8_isrForCrosstalkSources_996973_39.stderr parsl.dataflow.dflow INFO: Task 2268 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a254129a91457c7d6ab01a47878adb94 with result from task 2268 parsl.dataflow.dflow DEBUG: Task 2268 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/038acdd3-55d5-4140-894b-34ce95fc2f30_isrForCrosstalkSources_997030_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/038acdd3-55d5-4140-894b-34ce95fc2f30_isrForCrosstalkSources_997030_10.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 2366 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 43cefc72b2162085f21e65d42ee068ee with result from task 2366 parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2366 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/7f4635cb-c5cc-437a-9cc0-e5dc5e951fd7_isrForCrosstalkSources_997009_57.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/7f4635cb-c5cc-437a-9cc0-e5dc5e951fd7_isrForCrosstalkSources_997009_57.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2458 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 8708ac4031919e23c3a477caab87d0e7 with result from task 2458 parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:07 proc_lsst.multi:146 [INFO] found job 17075281 in provider checkpoint proc_lsst.multi INFO: found job 17075281 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2458 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/724c8f7e-96cb-4872-af97-9c022b86fa82_isrForCrosstalkSources_996993_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/724c8f7e-96cb-4872-af97-9c022b86fa82_isrForCrosstalkSources_996993_11.stderr parsl.dataflow.dflow INFO: Task 2397 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c96c426b955bbfbb86844d52af09b07e with result from task 2397 parsl.dataflow.dflow DEBUG: Task 2397 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/1a69faea-feca-496a-8552-571b2e32712d_isrForCrosstalkSources_996997_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/1a69faea-feca-496a-8552-571b2e32712d_isrForCrosstalkSources_996997_25.stderr parsl.dataflow.dflow INFO: Task 2445 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78c60508df45bee4780c06b6a9554296 with result from task 2445 parsl.dataflow.dflow DEBUG: Task 2445 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/8c57645e-6903-4d2d-8290-c0b00925546c_isrForCrosstalkSources_996998_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/8c57645e-6903-4d2d-8290-c0b00925546c_isrForCrosstalkSources_996998_39.stderr parsl.dataflow.dflow INFO: Task 2306 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23ea64fbbf4c4389092e93e7ec4c1437 with result from task 2306 parsl.dataflow.dflow DEBUG: Task 2306 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a8abc9e2-c90c-48cc-b245-59407b65737b_isrForCrosstalkSources_997020_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a8abc9e2-c90c-48cc-b245-59407b65737b_isrForCrosstalkSources_997020_47.stderr parsl.dataflow.dflow INFO: Task 2091 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a9b407d2cc4bdde331cd573212496e4 with result from task 2091 parsl.dataflow.dflow DEBUG: Task 2091 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/29e602aa-398f-453f-9a38-4da696078aec_isrForCrosstalkSources_996991_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/29e602aa-398f-453f-9a38-4da696078aec_isrForCrosstalkSources_996991_28.stderr parsl.dataflow.dflow INFO: Task 2463 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 498c76a5ee74c1063cddd4605f5504f9 with result from task 2463 parsl.dataflow.dflow DEBUG: Task 2463 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4fdc2511-159d-46ae-88d2-0bccf62888c4_isrForCrosstalkSources_996966_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4fdc2511-159d-46ae-88d2-0bccf62888c4_isrForCrosstalkSources_996966_27.stderr parsl.dataflow.dflow INFO: Task 2132 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 502e1dc6af29d441820aa34dbc870251 with result from task 2132 parsl.dataflow.dflow DEBUG: Task 2132 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a3a2d3b2-2682-4abd-ab69-892619efdfe6_isrForCrosstalkSources_997001_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a3a2d3b2-2682-4abd-ab69-892619efdfe6_isrForCrosstalkSources_997001_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:08 proc_lsst.multi:146 [INFO] found job 17075282 in provider checkpoint proc_lsst.multi INFO: found job 17075282 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2385 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 746ac9c8a8448419e6077d89f759b7ff with result from task 2385 parsl.dataflow.dflow DEBUG: Task 2385 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8c5eb432-7ae8-4d9b-9596-eeec28b7cf4b_isrForCrosstalkSources_997017_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/8c5eb432-7ae8-4d9b-9596-eeec28b7cf4b_isrForCrosstalkSources_997017_60.stderr parsl.dataflow.dflow INFO: Task 2377 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1f506e359c8757a5fa55f5e325e35aa with result from task 2377 parsl.dataflow.dflow DEBUG: Task 2377 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/7899b428-f394-4161-9114-6ce37c1fb0f5_isrForCrosstalkSources_996978_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/7899b428-f394-4161-9114-6ce37c1fb0f5_isrForCrosstalkSources_996978_35.stderr parsl.dataflow.dflow INFO: Task 2440 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4a833179b2e50ae096b74c3c0a960d0 with result from task 2440 parsl.dataflow.dflow DEBUG: Task 2440 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/fe2b8a88-d89e-47a4-b099-43701377807a_isrForCrosstalkSources_996973_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/fe2b8a88-d89e-47a4-b099-43701377807a_isrForCrosstalkSources_996973_8.stderr parsl.dataflow.dflow INFO: Task 2212 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 464036a1b427dded3e06f52acfcb8191 with result from task 2212 parsl.dataflow.dflow DEBUG: Task 2212 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b2790512-f656-4bce-9192-01f2ab27590d_isrForCrosstalkSources_997022_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b2790512-f656-4bce-9192-01f2ab27590d_isrForCrosstalkSources_997022_26.stderr parsl.dataflow.dflow INFO: Task 2168 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 892b5d5a2f90f7699225b194b7a07d0a with result from task 2168 parsl.dataflow.dflow DEBUG: Task 2168 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fb3930ba-c0e4-4592-93cb-8a3615170a09_isrForCrosstalkSources_996955_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fb3930ba-c0e4-4592-93cb-8a3615170a09_isrForCrosstalkSources_996955_26.stderr parsl.dataflow.dflow INFO: Task 2408 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af1d59bdc526b9e4840a24c2a07e772e with result from task 2408 parsl.dataflow.dflow DEBUG: Task 2408 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e0e4d389-d8d1-45d5-bdb2-82ddbded0df6_isrForCrosstalkSources_997001_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e0e4d389-d8d1-45d5-bdb2-82ddbded0df6_isrForCrosstalkSources_997001_18.stderr parsl.dataflow.dflow INFO: Task 2153 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03ed02fc4ae591ee0cf97fbb9428caf8 with result from task 2153 parsl.dataflow.dflow DEBUG: Task 2153 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/57fdc729-8b0e-4f85-9282-5b8eb434ba3e_isrForCrosstalkSources_997007_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/57fdc729-8b0e-4f85-9282-5b8eb434ba3e_isrForCrosstalkSources_997007_11.stderr parsl.dataflow.dflow INFO: Task 2276 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc9850fd80cd14fc6914bc7a38a019b3 with result from task 2276 parsl.dataflow.dflow DEBUG: Task 2276 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/01dbba16-c818-4614-b8b2-a307031088b7_isrForCrosstalkSources_996965_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/01dbba16-c818-4614-b8b2-a307031088b7_isrForCrosstalkSources_996965_11.stderr parsl.dataflow.dflow INFO: Task 2149 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7afc01c014f5a1e124547b0ff4a6117a with result from task 2149 parsl.dataflow.dflow DEBUG: Task 2149 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b58ddf98-08bd-4a37-b334-724921c8e486_isrForCrosstalkSources_996969_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b58ddf98-08bd-4a37-b334-724921c8e486_isrForCrosstalkSources_996969_36.stderr parsl.dataflow.dflow INFO: Task 2460 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 68cabebaf32759d49082995f415a039e with result from task 2460 parsl.dataflow.dflow DEBUG: Task 2460 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/15f14bc9-f730-4345-b9d6-ea3fe490240b_isrForCrosstalkSources_996972_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/15f14bc9-f730-4345-b9d6-ea3fe490240b_isrForCrosstalkSources_996972_11.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:08 proc_lsst.multi:146 [INFO] found job 17075283 in provider checkpoint proc_lsst.multi INFO: found job 17075283 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2424 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ba3318075ee78147c5e36e9f48a5e2d with result from task 2424 parsl.dataflow.dflow DEBUG: Task 2424 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4cbbf5b2-2f05-4abb-8930-a1bc5425e772_isrForCrosstalkSources_996986_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4cbbf5b2-2f05-4abb-8930-a1bc5425e772_isrForCrosstalkSources_996986_22.stderr parsl.dataflow.dflow INFO: Task 2386 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1a953b576d72588a7cfc3699de36434 with result from task 2386 parsl.dataflow.dflow DEBUG: Task 2386 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/41205893-5e13-4f96-ad41-25117b1cbf8e_isrForCrosstalkSources_997006_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/41205893-5e13-4f96-ad41-25117b1cbf8e_isrForCrosstalkSources_997006_49.stderr parsl.dataflow.dflow INFO: Task 2390 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c896e1c527288a8d8f33c393c745a100 with result from task 2390 parsl.dataflow.dflow DEBUG: Task 2390 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6f00fc4a-000d-4970-869d-6fa9f7b4f0c4_isrForCrosstalkSources_996982_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6f00fc4a-000d-4970-869d-6fa9f7b4f0c4_isrForCrosstalkSources_996982_13.stderr parsl.dataflow.dflow INFO: Task 2371 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5152cb63b668ec549c53a64d2ec739e7 with result from task 2371 parsl.dataflow.dflow DEBUG: Task 2371 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/ef58bdd8-1353-4649-ad5c-1feab72a0960_isrForCrosstalkSources_997031_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/ef58bdd8-1353-4649-ad5c-1feab72a0960_isrForCrosstalkSources_997031_48.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:09 proc_lsst.multi:146 [INFO] found job 17075284 in provider checkpoint proc_lsst.multi INFO: found job 17075284 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2135 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6773c85e02beae9ac4a5445b9ac74d7f with result from task 2135 parsl.dataflow.dflow DEBUG: Task 2135 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a0a27864-897a-4887-8d68-1209e51771ab_isrForCrosstalkSources_997021_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a0a27864-897a-4887-8d68-1209e51771ab_isrForCrosstalkSources_997021_37.stderr parsl.dataflow.dflow INFO: Task 2282 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4f28553e0bcc75a0bc05184598683be with result from task 2282 parsl.dataflow.dflow DEBUG: Task 2282 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c63af9f5-a364-46b5-a442-c2c8ff6f03a1_isrForCrosstalkSources_996972_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c63af9f5-a364-46b5-a442-c2c8ff6f03a1_isrForCrosstalkSources_996972_12.stderr parsl.dataflow.dflow INFO: Task 2403 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97f362183d3038e70817a0a14d9407c9 with result from task 2403 parsl.dataflow.dflow DEBUG: Task 2403 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c969a6d8-421c-42ff-983b-888023b6a480_isrForCrosstalkSources_996992_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c969a6d8-421c-42ff-983b-888023b6a480_isrForCrosstalkSources_996992_52.stderr parsl.dataflow.dflow INFO: Task 2441 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f0a3dc4f54b169fa3cb67a19b8d182e9 with result from task 2441 parsl.dataflow.dflow DEBUG: Task 2441 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/e44fe8d3-654f-4413-8507-99d0e8741d13_isrForCrosstalkSources_996966_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/e44fe8d3-654f-4413-8507-99d0e8741d13_isrForCrosstalkSources_996966_25.stderr parsl.dataflow.dflow INFO: Task 2267 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca2ca0fcb4734202fa1d783630fb5079 with result from task 2267 parsl.dataflow.dflow DEBUG: Task 2267 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/05e0e374-0ff8-4cfd-ae4d-ef8d8e0f5145_isrForCrosstalkSources_997032_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/05e0e374-0ff8-4cfd-ae4d-ef8d8e0f5145_isrForCrosstalkSources_997032_10.stderr parsl.dataflow.dflow INFO: Task 2454 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a876414f11a0961e971e5a27f07a3e26 with result from task 2454 parsl.dataflow.dflow DEBUG: Task 2454 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/fc9ca2e1-f850-4551-8228-53d808e69358_isrForCrosstalkSources_997017_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/fc9ca2e1-f850-4551-8228-53d808e69358_isrForCrosstalkSources_997017_38.stderr parsl.dataflow.dflow INFO: Task 2339 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry beb660800f587a192c173623e97b92b5 with result from task 2339 parsl.dataflow.dflow DEBUG: Task 2339 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/7d900203-d89c-4f77-933c-4b5f12403f4b_isrForCrosstalkSources_996966_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/7d900203-d89c-4f77-933c-4b5f12403f4b_isrForCrosstalkSources_996966_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2157 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 783c8cf300f7979e4770d2a37c1d339e with result from task 2157 parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2157 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/3e0a74ee-d6e8-46ec-87bf-6b8f862cff27_isrForCrosstalkSources_996977_44.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/3e0a74ee-d6e8-46ec-87bf-6b8f862cff27_isrForCrosstalkSources_996977_44.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:09 proc_lsst.multi:146 [INFO] found job 17075285 in provider checkpoint proc_lsst.multi INFO: found job 17075285 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2432 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f186277d561e5064d831c81d77cd0d87 with result from task 2432 parsl.dataflow.dflow DEBUG: Task 2432 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c747872f-aae6-4744-8eb1-8a0b3ef70368_isrForCrosstalkSources_996965_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c747872f-aae6-4744-8eb1-8a0b3ef70368_isrForCrosstalkSources_996965_41.stderr parsl.dataflow.dflow INFO: Task 2381 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 98f6148ce782f8dcc5bc978da6a1d446 with result from task 2381 parsl.dataflow.dflow DEBUG: Task 2381 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/33283a1a-4caf-401c-84df-188c17c0bb4c_isrForCrosstalkSources_996973_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/33283a1a-4caf-401c-84df-188c17c0bb4c_isrForCrosstalkSources_996973_4.stderr parsl.dataflow.dflow INFO: Task 2177 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2cc1e88d2125b3e9cea07ed74c68bc6d with result from task 2177 parsl.dataflow.dflow DEBUG: Task 2177 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2354f6b4-2336-4bf1-8b65-e9cc05bcd780_isrForCrosstalkSources_996963_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2354f6b4-2336-4bf1-8b65-e9cc05bcd780_isrForCrosstalkSources_996963_30.stderr parsl.dataflow.dflow INFO: Task 2384 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 213439039c6db695c33988a93ba576b5 with result from task 2384 parsl.dataflow.dflow DEBUG: Task 2384 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/d40fba4e-bb8f-49db-95b0-36ed290ced33_isrForCrosstalkSources_996968_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/d40fba4e-bb8f-49db-95b0-36ed290ced33_isrForCrosstalkSources_996968_16.stderr parsl.dataflow.dflow INFO: Task 2218 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84c0e3fefd2343bca72bb4564159e10f with result from task 2218 parsl.dataflow.dflow DEBUG: Task 2218 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/cb706d19-cb8b-44d2-b923-539fc10a1108_isrForCrosstalkSources_996970_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/cb706d19-cb8b-44d2-b923-539fc10a1108_isrForCrosstalkSources_996970_26.stderr parsl.dataflow.dflow INFO: Task 2343 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry facebf3539f3bd435744a26564ee1f63 with result from task 2343 parsl.dataflow.dflow DEBUG: Task 2343 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/195b8f09-1561-4bb9-8984-c1e710b05d70_isrForCrosstalkSources_997020_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/195b8f09-1561-4bb9-8984-c1e710b05d70_isrForCrosstalkSources_997020_1.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:10 proc_lsst.multi:146 [INFO] found job 17075286 in provider checkpoint proc_lsst.multi INFO: found job 17075286 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2143 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8ac2248649de22c4f6687b6647c412b with result from task 2143 parsl.dataflow.dflow DEBUG: Task 2143 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/683395e9-6c88-414b-be83-244a57cdabbb_isrForCrosstalkSources_997019_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/683395e9-6c88-414b-be83-244a57cdabbb_isrForCrosstalkSources_997019_36.stderr parsl.dataflow.dflow INFO: Task 2223 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26858646df75ea88286d1e543702d60d with result from task 2223 parsl.dataflow.dflow DEBUG: Task 2223 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/56849b40-4da7-47d9-bc79-1443bbb6790c_isrForCrosstalkSources_996979_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/56849b40-4da7-47d9-bc79-1443bbb6790c_isrForCrosstalkSources_996979_28.stderr parsl.dataflow.dflow INFO: Task 2465 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c5bddacd231c835b998d9e42496370e9 with result from task 2465 parsl.dataflow.dflow DEBUG: Task 2465 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/484dc16c-5b0d-44f2-94d7-f5e9244f16b1_isrForCrosstalkSources_996999_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/484dc16c-5b0d-44f2-94d7-f5e9244f16b1_isrForCrosstalkSources_996999_1.stderr parsl.dataflow.dflow INFO: Task 2379 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3bfa0595c1c7dc229b014d1f3377395a with result from task 2379 parsl.dataflow.dflow DEBUG: Task 2379 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/fc58b97a-ba11-4584-b549-e33455b30fcc_isrForCrosstalkSources_996999_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/fc58b97a-ba11-4584-b549-e33455b30fcc_isrForCrosstalkSources_996999_23.stderr parsl.dataflow.dflow INFO: Task 2291 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8595ec834d181da415f4ff8c55154564 with result from task 2291 parsl.dataflow.dflow DEBUG: Task 2291 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8472e40c-4f4a-4bb5-bcb2-6f45cf93dbf3_isrForCrosstalkSources_996956_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8472e40c-4f4a-4bb5-bcb2-6f45cf93dbf3_isrForCrosstalkSources_996956_25.stderr parsl.dataflow.dflow INFO: Task 2476 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 059601dbdf9c011f4fcaa2381d4c77f2 with result from task 2476 parsl.dataflow.dflow DEBUG: Task 2476 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/9ba5ef64-c34b-4780-88a7-6f45bd9e8285_isrForCrosstalkSources_997016_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/9ba5ef64-c34b-4780-88a7-6f45bd9e8285_isrForCrosstalkSources_997016_26.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:10 proc_lsst.multi:146 [INFO] found job 17075287 in provider checkpoint proc_lsst.multi INFO: found job 17075287 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2374 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1b0801f4523f08f251b413edd6c2c850 with result from task 2374 parsl.dataflow.dflow DEBUG: Task 2374 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f66a3714-58ae-4a58-89e8-58680541e8f5_isrForCrosstalkSources_996980_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f66a3714-58ae-4a58-89e8-58680541e8f5_isrForCrosstalkSources_996980_10.stderr parsl.dataflow.dflow INFO: Task 2396 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 288d00c5cf820890304500d731b494c2 with result from task 2396 parsl.dataflow.dflow DEBUG: Task 2396 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2689f274-13da-4b87-a203-307f7c3b603d_isrForCrosstalkSources_996981_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2689f274-13da-4b87-a203-307f7c3b603d_isrForCrosstalkSources_996981_54.stderr parsl.dataflow.dflow INFO: Task 2436 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ad1b96b3e3278399345ac5f6b349af0 with result from task 2436 parsl.dataflow.dflow DEBUG: Task 2436 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/49d348af-1fde-44b0-9e15-d555cbb09835_isrForCrosstalkSources_996963_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/49d348af-1fde-44b0-9e15-d555cbb09835_isrForCrosstalkSources_996963_2.stderr parsl.dataflow.dflow INFO: Task 2340 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75d53baa1c53abb333fc894f70280e0e with result from task 2340 parsl.dataflow.dflow DEBUG: Task 2340 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/15766452-75c0-4266-a55e-51ae9d3a27f7_isrForCrosstalkSources_996996_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/15766452-75c0-4266-a55e-51ae9d3a27f7_isrForCrosstalkSources_996996_26.stderr parsl.dataflow.dflow INFO: Task 2426 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 200d00e90bce5f78086555df3ab12116 with result from task 2426 parsl.dataflow.dflow DEBUG: Task 2426 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9bd7f08d-987a-4d0c-9f87-16b9198954e8_isrForCrosstalkSources_996964_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9bd7f08d-987a-4d0c-9f87-16b9198954e8_isrForCrosstalkSources_996964_26.stderr parsl.dataflow.dflow INFO: Task 2446 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b1563024c39ec72d8e43ef35c9bf1c0 with result from task 2446 parsl.dataflow.dflow DEBUG: Task 2446 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a0381627-cc6d-42a3-904b-7de3fbbba8b8_isrForCrosstalkSources_997008_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a0381627-cc6d-42a3-904b-7de3fbbba8b8_isrForCrosstalkSources_997008_15.stderr parsl.dataflow.dflow INFO: Task 2430 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b7d07d9d4350bfb97a891885b516c3f with result from task 2430 parsl.dataflow.dflow DEBUG: Task 2430 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/65bae40d-8ba1-4d11-a404-fe61f30e54e4_isrForCrosstalkSources_997029_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/65bae40d-8ba1-4d11-a404-fe61f30e54e4_isrForCrosstalkSources_997029_6.stderr parsl.dataflow.dflow INFO: Task 2442 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7030d124275ee84f8802ed4b99bab688 with result from task 2442 parsl.dataflow.dflow DEBUG: Task 2442 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/d37c86b8-960c-4d87-9611-6c10a86d517c_isrForCrosstalkSources_997015_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/d37c86b8-960c-4d87-9611-6c10a86d517c_isrForCrosstalkSources_997015_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:11 proc_lsst.multi:146 [INFO] found job 17075288 in provider checkpoint proc_lsst.multi INFO: found job 17075288 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2380 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd111a20541b9b424d48b40cfe9fb5a8 with result from task 2380 parsl.dataflow.dflow DEBUG: Task 2380 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/477ef399-4ab9-4d13-b50f-5a175a205387_isrForCrosstalkSources_997010_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/477ef399-4ab9-4d13-b50f-5a175a205387_isrForCrosstalkSources_997010_16.stderr parsl.dataflow.dflow INFO: Task 2376 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5f030dbfc7b78b9891979da4151cc5c with result from task 2376 parsl.dataflow.dflow DEBUG: Task 2376 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/78259061-b009-493e-be19-7e7a08107207_isrForCrosstalkSources_996996_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/78259061-b009-493e-be19-7e7a08107207_isrForCrosstalkSources_996996_24.stderr parsl.dataflow.dflow INFO: Task 2155 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c057b48e34a28f17950cb530b7f6bc3 with result from task 2155 parsl.dataflow.dflow DEBUG: Task 2155 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/6bf944b6-2ed9-4d43-9038-62e0fec84c59_isrForCrosstalkSources_997024_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/6bf944b6-2ed9-4d43-9038-62e0fec84c59_isrForCrosstalkSources_997024_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:12 proc_lsst.multi:146 [INFO] found job 17075289 in provider checkpoint proc_lsst.multi INFO: found job 17075289 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2290 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f1a0740422c88d731cb597b91ca9c19 with result from task 2290 parsl.dataflow.dflow DEBUG: Task 2290 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7daa3566-174b-4bcc-8810-25445ea85b3c_isrForCrosstalkSources_996953_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7daa3566-174b-4bcc-8810-25445ea85b3c_isrForCrosstalkSources_996953_32.stderr parsl.dataflow.dflow INFO: Task 2224 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15a4db629fc599ac85efffb23f1cd8ed with result from task 2224 parsl.dataflow.dflow DEBUG: Task 2224 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/aa1623fb-0f6c-465c-be5d-c548730c04ea_isrForCrosstalkSources_997023_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/aa1623fb-0f6c-465c-be5d-c548730c04ea_isrForCrosstalkSources_997023_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:12 proc_lsst.multi:146 [INFO] found job 17075290 in provider checkpoint proc_lsst.multi INFO: found job 17075290 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2360 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f19bfcf2bf65581bc9b3d93808610ed with result from task 2360 parsl.dataflow.dflow DEBUG: Task 2360 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/760e0a35-cf73-438b-8a15-b6120b151f19_isrForCrosstalkSources_997019_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/760e0a35-cf73-438b-8a15-b6120b151f19_isrForCrosstalkSources_997019_44.stderr parsl.dataflow.dflow INFO: Task 2252 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 68e5d074c3a4095c3026a074e95ae97f with result from task 2252 parsl.dataflow.dflow DEBUG: Task 2252 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/ca0513d6-53c0-41e3-a7a7-75b173e4a97f_isrForCrosstalkSources_996958_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/ca0513d6-53c0-41e3-a7a7-75b173e4a97f_isrForCrosstalkSources_996958_28.stderr parsl.dataflow.dflow INFO: Task 1781 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8491abe83c8d7bacf598d1d0efcef2f4 with result from task 1781 parsl.dataflow.dflow DEBUG: Task 1781 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/c1af2275-244d-4ccb-b1bc-94a06891465e_isrForCrosstalkSources_997027_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/c1af2275-244d-4ccb-b1bc-94a06891465e_isrForCrosstalkSources_997027_5.stderr parsl.dataflow.dflow INFO: Task 1784 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 05c3996fe9745745bc6fbab24ae4885c with result from task 1784 parsl.dataflow.dflow DEBUG: Task 1784 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7fce77a5-07bb-4f2a-9320-8f17daaa338c_isrForCrosstalkSources_996953_14.stdout parsl.dataflow.dflow INFO: Standard error for task 1784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7fce77a5-07bb-4f2a-9320-8f17daaa338c_isrForCrosstalkSources_996953_14.stderr parsl.dataflow.dflow INFO: Task 1786 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3ed83deea2410b9d5955cad38203008e with result from task 1786 parsl.dataflow.dflow DEBUG: Task 1786 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/62f36b7b-aa65-4e72-a336-8def265b395e_isrForCrosstalkSources_996982_20.stdout parsl.dataflow.dflow INFO: Standard error for task 1786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/62f36b7b-aa65-4e72-a336-8def265b395e_isrForCrosstalkSources_996982_20.stderr parsl.dataflow.dflow INFO: Task 1785 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd0b14ca52ee61fcce7ef6d1302433d2 with result from task 1785 parsl.dataflow.dflow DEBUG: Task 1785 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/575c26a5-d7f2-4ce4-9d33-50900947c5cd_isrForCrosstalkSources_996987_1.stdout parsl.dataflow.dflow INFO: Standard error for task 1785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/575c26a5-d7f2-4ce4-9d33-50900947c5cd_isrForCrosstalkSources_996987_1.stderr parsl.dataflow.dflow INFO: Task 1782 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 575b8ec282fe5b2c2ba9293edc34ac7e with result from task 1782 parsl.dataflow.dflow DEBUG: Task 1782 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/9a44e585-0b62-4db7-bf4c-618874073f07_isrForCrosstalkSources_996953_21.stdout parsl.dataflow.dflow INFO: Standard error for task 1782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/9a44e585-0b62-4db7-bf4c-618874073f07_isrForCrosstalkSources_996953_21.stderr parsl.dataflow.dflow INFO: Task 1783 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e860a959e4ac20ab69b3f8d0588252e6 with result from task 1783 parsl.dataflow.dflow DEBUG: Task 1783 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/fd2fbf35-04a2-4ef0-a4da-ff19ecdab52b_isrForCrosstalkSources_996988_3.stdout parsl.dataflow.dflow INFO: Standard error for task 1783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/fd2fbf35-04a2-4ef0-a4da-ff19ecdab52b_isrForCrosstalkSources_996988_3.stderr parsl.dataflow.dflow INFO: Task 2362 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d13e6594e3df6c9648663d0f74ec601 with result from task 2362 parsl.dataflow.dflow DEBUG: Task 2362 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/95dfffa6-e5e4-40ba-815d-9ea5107735b3_isrForCrosstalkSources_997006_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/95dfffa6-e5e4-40ba-815d-9ea5107735b3_isrForCrosstalkSources_997006_40.stderr parsl.dataflow.dflow INFO: Task 1787 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1beb3f3479fa4dc96181e25f8921794 with result from task 1787 parsl.dataflow.dflow DEBUG: Task 1787 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/2b264544-2551-4a7c-a3b5-48eefaf65923_isrForCrosstalkSources_997020_5.stdout parsl.dataflow.dflow INFO: Standard error for task 1787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/2b264544-2551-4a7c-a3b5-48eefaf65923_isrForCrosstalkSources_997020_5.stderr parsl.dataflow.dflow INFO: Task 2309 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd70242675cf1c7bfbf8960e20b36c4c with result from task 2309 parsl.dataflow.dflow DEBUG: Task 2309 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a3dcb1e9-2ce1-4ecb-858e-e0d7d91fc3a0_isrForCrosstalkSources_996984_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a3dcb1e9-2ce1-4ecb-858e-e0d7d91fc3a0_isrForCrosstalkSources_996984_61.stderr parsl.dataflow.dflow INFO: Task 2469 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 562d3f7035bd07f3b8cef91d964facc9 with result from task 2469 parsl.dataflow.dflow DEBUG: Task 2469 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/4a8e0011-e1cd-42e7-a81c-02dd1621a718_isrForCrosstalkSources_997026_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/4a8e0011-e1cd-42e7-a81c-02dd1621a718_isrForCrosstalkSources_997026_59.stderr parsl.dataflow.dflow INFO: Task 2279 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d160e61ea809342c35d67b5db551151c with result from task 2279 parsl.dataflow.dflow DEBUG: Task 2279 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0aedb66b-80b5-41d0-ab54-f6ad70992aeb_isrForCrosstalkSources_996956_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0aedb66b-80b5-41d0-ab54-f6ad70992aeb_isrForCrosstalkSources_996956_59.stderr parsl.dataflow.dflow INFO: Task 2477 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07e34658a450737cc46fcc3b23336afa with result from task 2477 parsl.dataflow.dflow DEBUG: Task 2477 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8ea4466b-04e3-47c5-aa75-4e88c1148067_isrForCrosstalkSources_997027_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8ea4466b-04e3-47c5-aa75-4e88c1148067_isrForCrosstalkSources_997027_3.stderr parsl.dataflow.dflow INFO: Task 1780 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1df48f301bb3f64e01f5ea19dff984db with result from task 1780 parsl.dataflow.dflow DEBUG: Task 1780 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6008842e-04d9-4fb7-b408-913b641f2f0b_isrForCrosstalkSources_997030_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/6008842e-04d9-4fb7-b408-913b641f2f0b_isrForCrosstalkSources_997030_58.stderr parsl.dataflow.dflow INFO: Task 2203 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab07a0183a37675cefc8130e7820077a with result from task 2203 parsl.dataflow.dflow DEBUG: Task 2203 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/570be287-6a04-43a0-8949-234732709a6b_isrForCrosstalkSources_996968_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/570be287-6a04-43a0-8949-234732709a6b_isrForCrosstalkSources_996968_28.stderr parsl.dataflow.dflow INFO: Task 2450 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 036250e2c6bbf35b1c2fea34cdca7df1 with result from task 2450 parsl.dataflow.dflow DEBUG: Task 2450 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1a985e81-3f63-4ba3-8f8a-28f807b4e0e6_isrForCrosstalkSources_996957_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1a985e81-3f63-4ba3-8f8a-28f807b4e0e6_isrForCrosstalkSources_996957_27.stderr parsl.dataflow.dflow INFO: Task 2468 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2bd2615bc8c04257998925b0be80f5f6 with result from task 2468 parsl.dataflow.dflow DEBUG: Task 2468 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1b7314ed-c185-49dd-b4e8-9cf61969ef6e_isrForCrosstalkSources_997032_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1b7314ed-c185-49dd-b4e8-9cf61969ef6e_isrForCrosstalkSources_997032_19.stderr parsl.dataflow.dflow INFO: Task 2449 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d3169f589b290cc5b66554cb2e0d8740 with result from task 2449 parsl.dataflow.dflow DEBUG: Task 2449 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a08f790f-0ba9-4bc9-9028-3773def0df93_isrForCrosstalkSources_996961_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a08f790f-0ba9-4bc9-9028-3773def0df93_isrForCrosstalkSources_996961_33.stderr parsl.dataflow.dflow INFO: Task 2428 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry def46531bf389f8c73d090b7d4bb05f1 with result from task 2428 parsl.dataflow.dflow DEBUG: Task 2428 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bf149585-6159-493b-845e-d34f2d53f379_isrForCrosstalkSources_996992_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/bf149585-6159-493b-845e-d34f2d53f379_isrForCrosstalkSources_996992_26.stderr parsl.dataflow.dflow INFO: Task 2382 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5693d352b3ee1e3eb6fe30690ce811ee with result from task 2382 parsl.dataflow.dflow DEBUG: Task 2382 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/218b6921-021f-4185-a321-44762812b00a_isrForCrosstalkSources_996991_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/218b6921-021f-4185-a321-44762812b00a_isrForCrosstalkSources_996991_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:14 proc_lsst.multi:146 [INFO] found job 17075293 in provider checkpoint proc_lsst.multi INFO: found job 17075293 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2427 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 851c0114bdbe8cf85768ea669ec957af with result from task 2427 parsl.dataflow.dflow DEBUG: Task 2427 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/1f0fdf87-4dfc-4b02-8dd1-7ef91aa4e55d_isrForCrosstalkSources_996994_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/1f0fdf87-4dfc-4b02-8dd1-7ef91aa4e55d_isrForCrosstalkSources_996994_41.stderr parsl.dataflow.dflow INFO: Task 2434 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e87bad6c59b8f052a1567aec70e5bbc7 with result from task 2434 parsl.dataflow.dflow DEBUG: Task 2434 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/cb234b67-67d5-49f2-9a46-6574dda2edfe_isrForCrosstalkSources_997031_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/cb234b67-67d5-49f2-9a46-6574dda2edfe_isrForCrosstalkSources_997031_6.stderr parsl.dataflow.dflow INFO: Task 2406 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dabf49e2c8c2e2f4028183aec1ea2ed6 with result from task 2406 parsl.dataflow.dflow DEBUG: Task 2406 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8b1dd569-8556-4de6-8c56-1bde38c24d7e_isrForCrosstalkSources_996973_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8b1dd569-8556-4de6-8c56-1bde38c24d7e_isrForCrosstalkSources_996973_1.stderr parsl.dataflow.dflow INFO: Task 2495 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2eb3a7561784f9c6b1a06475fd895e65 with result from task 2495 parsl.dataflow.dflow DEBUG: Task 2495 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/87fca8ad-ed7f-4ff8-b086-e42d3ac53f38_isrForCrosstalkSources_997013_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/87fca8ad-ed7f-4ff8-b086-e42d3ac53f38_isrForCrosstalkSources_997013_10.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:15 proc_lsst.multi:146 [INFO] found job 17075298 in provider checkpoint proc_lsst.multi INFO: found job 17075298 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2307 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc4b7967fd2db112c633949bdc544f92 with result from task 2307 parsl.dataflow.dflow DEBUG: Task 2307 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/98e3f3e5-bd41-4c3e-b032-90e775f7d7f3_isrForCrosstalkSources_997021_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/98e3f3e5-bd41-4c3e-b032-90e775f7d7f3_isrForCrosstalkSources_997021_50.stderr parsl.dataflow.dflow INFO: Task 2304 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0bb70e4674486597c6b9e4907ed008f8 with result from task 2304 parsl.dataflow.dflow DEBUG: Task 2304 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8407d9fc-c0ad-4bbb-83a7-6a048bd9dc08_isrForCrosstalkSources_996971_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8407d9fc-c0ad-4bbb-83a7-6a048bd9dc08_isrForCrosstalkSources_996971_50.stderr parsl.dataflow.dflow INFO: Task 2205 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7dc116ec08e5dbb8c3ccec79b6e2c62 with result from task 2205 parsl.dataflow.dflow DEBUG: Task 2205 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c435e237-f8ac-47f3-89ac-739e3f74f7be_isrForCrosstalkSources_996969_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c435e237-f8ac-47f3-89ac-739e3f74f7be_isrForCrosstalkSources_996969_44.stderr parsl.dataflow.dflow INFO: Task 2416 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89c767f56dbace3c1196800882428de7 with result from task 2416 parsl.dataflow.dflow DEBUG: Task 2416 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d5ee5dbb-2b87-4deb-88bc-6c6f458e4d8e_isrForCrosstalkSources_996993_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d5ee5dbb-2b87-4deb-88bc-6c6f458e4d8e_isrForCrosstalkSources_996993_25.stderr parsl.dataflow.dflow INFO: Task 2459 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fc9824ac246af854bb61742027a47209 with result from task 2459 parsl.dataflow.dflow DEBUG: Task 2459 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f20081f4-fbcd-4799-b97f-62252ce28131_isrForCrosstalkSources_996974_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/f20081f4-fbcd-4799-b97f-62252ce28131_isrForCrosstalkSources_996974_36.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:15 proc_lsst.multi:146 [INFO] found job 17075299 in provider checkpoint proc_lsst.multi INFO: found job 17075299 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2461 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f892b67a5ca21d346eb591221a149da1 with result from task 2461 parsl.dataflow.dflow DEBUG: Task 2461 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/72ecae83-c79a-4b9c-8613-bb69978ee144_isrForCrosstalkSources_996990_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/72ecae83-c79a-4b9c-8613-bb69978ee144_isrForCrosstalkSources_996990_9.stderr parsl.dataflow.dflow INFO: Task 2429 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fabaa299d8727fa3fdc96b4c16fca021 with result from task 2429 parsl.dataflow.dflow DEBUG: Task 2429 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4436cd2a-05a5-4391-92be-1758638ea0e9_isrForCrosstalkSources_996963_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4436cd2a-05a5-4391-92be-1758638ea0e9_isrForCrosstalkSources_996963_24.stderr parsl.dataflow.dflow INFO: Task 2398 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1aec00d0d8d56beaa96edf5d1250cd38 with result from task 2398 parsl.dataflow.dflow DEBUG: Task 2398 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/c4a20854-ad56-4e70-9c1b-fb9183a1c8d4_isrForCrosstalkSources_997017_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/c4a20854-ad56-4e70-9c1b-fb9183a1c8d4_isrForCrosstalkSources_997017_41.stderr parsl.dataflow.dflow INFO: Task 2166 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3733234f69321e24bb4d92511a26948a with result from task 2166 parsl.dataflow.dflow DEBUG: Task 2166 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c4d82630-e51b-4a80-9346-4ace8dc26297_isrForCrosstalkSources_997012_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c4d82630-e51b-4a80-9346-4ace8dc26297_isrForCrosstalkSources_997012_4.stderr parsl.dataflow.dflow INFO: Task 2412 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8b18914ece0cbd7c56d05ad30887b11 with result from task 2412 parsl.dataflow.dflow DEBUG: Task 2412 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/14a6f43a-650b-4793-be66-0c1ab2bb8064_isrForCrosstalkSources_996956_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/14a6f43a-650b-4793-be66-0c1ab2bb8064_isrForCrosstalkSources_996956_45.stderr parsl.dataflow.dflow INFO: Task 2365 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb003ed40a29be64df73b8a0fdd6c2c6 with result from task 2365 parsl.dataflow.dflow DEBUG: Task 2365 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a0fba2b3-d9c6-4148-8028-b991df2bb15e_isrForCrosstalkSources_997028_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a0fba2b3-d9c6-4148-8028-b991df2bb15e_isrForCrosstalkSources_997028_60.stderr parsl.dataflow.dflow INFO: Task 2467 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 36cb604b22f6f043fbfaddbb72623181 with result from task 2467 parsl.dataflow.dflow DEBUG: Task 2467 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/03e5d046-77c3-42f2-a8a2-0aedd569a6c8_isrForCrosstalkSources_996960_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/03e5d046-77c3-42f2-a8a2-0aedd569a6c8_isrForCrosstalkSources_996960_5.stderr parsl.dataflow.dflow INFO: Task 2341 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f3c60a7dffde3621c498701fcfbd83c with result from task 2341 parsl.dataflow.dflow DEBUG: Task 2341 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ac9668e7-52c2-4d98-a6de-353aca3dce4d_isrForCrosstalkSources_996984_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ac9668e7-52c2-4d98-a6de-353aca3dce4d_isrForCrosstalkSources_996984_5.stderr parsl.dataflow.dflow INFO: Task 2501 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d8f8594cff654b75d55b49845b4f300e with result from task 2501 parsl.dataflow.dflow DEBUG: Task 2501 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/b2904d3b-2683-451f-a2d1-fd1275089632_isrForCrosstalkSources_997002_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/b2904d3b-2683-451f-a2d1-fd1275089632_isrForCrosstalkSources_997002_57.stderr parsl.dataflow.dflow INFO: Task 2250 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e0d0e5ceecf0aa426397a72d09d14556 with result from task 2250 parsl.dataflow.dflow DEBUG: Task 2250 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c957badd-2c85-4981-ba8e-d2fe52826c75_isrForCrosstalkSources_997020_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/c957badd-2c85-4981-ba8e-d2fe52826c75_isrForCrosstalkSources_997020_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2422 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 9b76b22f42b6fe4bff3f711c01bf7ef1 with result from task 2422 parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2422 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/4ef8eded-c7aa-48c7-9864-c05c5fddf6f0_isrForCrosstalkSources_997019_11.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/4ef8eded-c7aa-48c7-9864-c05c5fddf6f0_isrForCrosstalkSources_997019_11.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:16 proc_lsst.multi:146 [INFO] found job 17075301 in provider checkpoint proc_lsst.multi INFO: found job 17075301 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2186 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eba831ab14afe0a88d24a5ed52f4d3d1 with result from task 2186 parsl.dataflow.dflow DEBUG: Task 2186 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/9c733c52-040c-4186-8536-99569fac2ade_isrForCrosstalkSources_996981_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/9c733c52-040c-4186-8536-99569fac2ade_isrForCrosstalkSources_996981_37.stderr parsl.dataflow.dflow INFO: Task 2421 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ee3c796025eaa90d3cc3617651509ac with result from task 2421 parsl.dataflow.dflow DEBUG: Task 2421 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7a4331d4-bc2f-4a77-98b5-47abaf00e95f_isrForCrosstalkSources_996974_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7a4331d4-bc2f-4a77-98b5-47abaf00e95f_isrForCrosstalkSources_996974_10.stderr parsl.dataflow.dflow INFO: Task 2146 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 915f60ed797b7fe9fc83fa41d8feaf22 with result from task 2146 parsl.dataflow.dflow DEBUG: Task 2146 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/822daef6-7645-43a3-8e47-e570e7c59916_isrForCrosstalkSources_997020_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/822daef6-7645-43a3-8e47-e570e7c59916_isrForCrosstalkSources_997020_59.stderr parsl.dataflow.dflow INFO: Task 2457 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e5f8e01a06aebc6df39ab3adbe227cd3 with result from task 2457 parsl.dataflow.dflow DEBUG: Task 2457 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/492263b8-11cc-44eb-8c51-0903220296be_isrForCrosstalkSources_997017_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/492263b8-11cc-44eb-8c51-0903220296be_isrForCrosstalkSources_997017_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:16 proc_lsst.multi:146 [INFO] found job 17075306 in provider checkpoint proc_lsst.multi INFO: found job 17075306 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2394 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99d6e3a2f1a6f86d8d630b8838ee42cd with result from task 2394 parsl.dataflow.dflow DEBUG: Task 2394 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8459a4f1-ec2c-4512-98a9-1dc0dd6213bf_isrForCrosstalkSources_996992_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8459a4f1-ec2c-4512-98a9-1dc0dd6213bf_isrForCrosstalkSources_996992_38.stderr parsl.dataflow.dflow INFO: Task 2329 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9356dca9fb2532e98ce424ca54b8772 with result from task 2329 parsl.dataflow.dflow DEBUG: Task 2329 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a46628d5-9765-4267-a426-d2a6a8e3a74e_isrForCrosstalkSources_996983_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a46628d5-9765-4267-a426-d2a6a8e3a74e_isrForCrosstalkSources_996983_39.stderr parsl.dataflow.dflow INFO: Task 2502 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e36598c00f2eac1ed89eb0d34a280eaf with result from task 2502 parsl.dataflow.dflow DEBUG: Task 2502 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/45d860bc-81d6-4ff7-85be-7fded96dbf90_isrForCrosstalkSources_997027_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/45d860bc-81d6-4ff7-85be-7fded96dbf90_isrForCrosstalkSources_997027_18.stderr parsl.dataflow.dflow INFO: Task 2200 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f96b207778e71a0d3cdd186580fa76a7 with result from task 2200 parsl.dataflow.dflow DEBUG: Task 2200 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/b0424264-8dcc-4b00-ab82-174676435f42_isrForCrosstalkSources_996971_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/b0424264-8dcc-4b00-ab82-174676435f42_isrForCrosstalkSources_996971_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:17 proc_lsst.multi:146 [INFO] found job 17075307 in provider checkpoint proc_lsst.multi INFO: found job 17075307 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2505 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 92d37ca69b9cb157c18612970a9679c1 with result from task 2505 parsl.dataflow.dflow DEBUG: Task 2505 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/394682c5-8e1d-4b39-b647-15739f1a658a_isrForCrosstalkSources_996977_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/394682c5-8e1d-4b39-b647-15739f1a658a_isrForCrosstalkSources_996977_55.stderr parsl.dataflow.dflow INFO: Task 2348 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d682ff1e123ec71be23497986cd8df1c with result from task 2348 parsl.dataflow.dflow DEBUG: Task 2348 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/b969317e-641f-4702-841f-a79d273d0317_isrForCrosstalkSources_997019_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/b969317e-641f-4702-841f-a79d273d0317_isrForCrosstalkSources_997019_41.stderr parsl.dataflow.dflow INFO: Task 2184 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c3a1c32c1beb67185fa180dc0a1fd37 with result from task 2184 parsl.dataflow.dflow DEBUG: Task 2184 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/961cb588-97a2-48ce-b9eb-73868949f096_isrForCrosstalkSources_996980_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/961cb588-97a2-48ce-b9eb-73868949f096_isrForCrosstalkSources_996980_29.stderr parsl.dataflow.dflow INFO: Task 2199 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ced0c664e12ae9cd37368306e1f41504 with result from task 2199 parsl.dataflow.dflow DEBUG: Task 2199 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/dbfdcc6a-e68a-458a-bdd5-cf6189168ae3_isrForCrosstalkSources_996972_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/dbfdcc6a-e68a-458a-bdd5-cf6189168ae3_isrForCrosstalkSources_996972_48.stderr parsl.dataflow.dflow INFO: Task 2524 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 849bf1054e542eb98d29921074a9f290 with result from task 2524 parsl.dataflow.dflow DEBUG: Task 2524 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/868d37d6-b274-4f24-b22d-8d9e4d8a46a5_isrForCrosstalkSources_997014_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/868d37d6-b274-4f24-b22d-8d9e4d8a46a5_isrForCrosstalkSources_997014_13.stderr parsl.dataflow.dflow INFO: Task 2318 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8a5a8143f97236dd8074f342d889f74 with result from task 2318 parsl.dataflow.dflow DEBUG: Task 2318 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1b35cfa5-5166-47ef-a34d-a265584fface_isrForCrosstalkSources_996958_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1b35cfa5-5166-47ef-a34d-a265584fface_isrForCrosstalkSources_996958_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:18 proc_lsst.multi:146 [INFO] found job 17075308 in provider checkpoint proc_lsst.multi INFO: found job 17075308 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2419 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ecf48d8baf880cd9a10cf1dc4360357 with result from task 2419 parsl.dataflow.dflow DEBUG: Task 2419 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/91492f0d-b716-4f0a-9b72-4da7fcdec4bb_isrForCrosstalkSources_996965_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/91492f0d-b716-4f0a-9b72-4da7fcdec4bb_isrForCrosstalkSources_996965_39.stderr parsl.dataflow.dflow INFO: Task 2325 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42048a06ddfb7296048137624027e242 with result from task 2325 parsl.dataflow.dflow DEBUG: Task 2325 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ded12831-5a56-4c49-a64a-64db7727d7fd_isrForCrosstalkSources_997020_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/ded12831-5a56-4c49-a64a-64db7727d7fd_isrForCrosstalkSources_997020_11.stderr parsl.dataflow.dflow INFO: Task 2265 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d81a3c37c8d6dcefdc1d3a11d3bea98 with result from task 2265 parsl.dataflow.dflow DEBUG: Task 2265 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/3132096b-367a-4954-aff2-fa2cce736e71_isrForCrosstalkSources_996987_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/3132096b-367a-4954-aff2-fa2cce736e71_isrForCrosstalkSources_996987_23.stderr parsl.dataflow.dflow INFO: Task 2437 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f42bc254909fccc437417752d58b5731 with result from task 2437 parsl.dataflow.dflow DEBUG: Task 2437 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/7e99dbda-8d2d-4f2d-ad88-73c1cd9ec446_isrForCrosstalkSources_996982_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/7e99dbda-8d2d-4f2d-ad88-73c1cd9ec446_isrForCrosstalkSources_996982_10.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 2506 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 4cf3e116837c379716037ec4ffb44458 with result from task 2506 parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2506 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5460541a-e91e-4a97-a592-96d9e131ea99_isrForCrosstalkSources_996981_33.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5460541a-e91e-4a97-a592-96d9e131ea99_isrForCrosstalkSources_996981_33.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:18 proc_lsst.multi:146 [INFO] found job 17075309 in provider checkpoint proc_lsst.multi INFO: found job 17075309 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2456 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b7cdd18ccef5f7e97e24bfc0cab4f9d with result from task 2456 parsl.dataflow.dflow DEBUG: Task 2456 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f1b9d339-8abc-40bf-9a5e-0d6e1d6f37f1_isrForCrosstalkSources_996971_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f1b9d339-8abc-40bf-9a5e-0d6e1d6f37f1_isrForCrosstalkSources_996971_54.stderr parsl.dataflow.dflow INFO: Task 2180 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e84b790c84f9cec3d5c85c0db43fbe16 with result from task 2180 parsl.dataflow.dflow DEBUG: Task 2180 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/132c1dde-40e8-4f4d-9dce-4b2cadbfbcb8_isrForCrosstalkSources_996974_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/132c1dde-40e8-4f4d-9dce-4b2cadbfbcb8_isrForCrosstalkSources_996974_55.stderr parsl.dataflow.dflow INFO: Task 2201 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abfcb54f47858fc9185fc4066ebcb726 with result from task 2201 parsl.dataflow.dflow DEBUG: Task 2201 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/97d284a0-ffc1-47b8-84af-d46a0cb0f98b_isrForCrosstalkSources_997014_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/97d284a0-ffc1-47b8-84af-d46a0cb0f98b_isrForCrosstalkSources_997014_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:19 proc_lsst.multi:146 [INFO] found job 17075310 in provider checkpoint proc_lsst.multi INFO: found job 17075310 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2515 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6817eaac74498ff3348ae30c9273707f with result from task 2515 parsl.dataflow.dflow DEBUG: Task 2515 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/59204e0e-5518-4ed2-8029-e0be336d48fe_isrForCrosstalkSources_996991_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/59204e0e-5518-4ed2-8029-e0be336d48fe_isrForCrosstalkSources_996991_56.stderr parsl.dataflow.dflow INFO: Task 2453 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1126cedbf377441295bd1a1504bbfd97 with result from task 2453 parsl.dataflow.dflow DEBUG: Task 2453 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e867ae5d-2253-4819-a14d-1085ec94fd1b_isrForCrosstalkSources_996984_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/e867ae5d-2253-4819-a14d-1085ec94fd1b_isrForCrosstalkSources_996984_59.stderr parsl.dataflow.dflow INFO: Task 2401 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 128a0a9defd27d32da446a3cc8dcac9d with result from task 2401 parsl.dataflow.dflow DEBUG: Task 2401 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/05056ad7-6842-4dce-9d2c-401bfd824e34_isrForCrosstalkSources_997028_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/05056ad7-6842-4dce-9d2c-401bfd824e34_isrForCrosstalkSources_997028_9.stderr parsl.dataflow.dflow INFO: Task 2351 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3e3287f0aee99991114aa080992b856d with result from task 2351 parsl.dataflow.dflow DEBUG: Task 2351 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/15c62a46-4a2d-45f1-95ff-2b8fd50b0bd8_isrForCrosstalkSources_997005_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/15c62a46-4a2d-45f1-95ff-2b8fd50b0bd8_isrForCrosstalkSources_997005_24.stderr parsl.dataflow.dflow INFO: Task 2206 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78d9a0fde7c1bfa0104d783967f89ccb with result from task 2206 parsl.dataflow.dflow DEBUG: Task 2206 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/3ee155c4-e026-4b67-bd69-f46c173edc47_isrForCrosstalkSources_996987_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/3ee155c4-e026-4b67-bd69-f46c173edc47_isrForCrosstalkSources_996987_36.stderr parsl.dataflow.dflow INFO: Task 2323 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d89bce5aa42abbabc284735f90f8d8e7 with result from task 2323 parsl.dataflow.dflow DEBUG: Task 2323 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5eb0b2ed-cf44-43d8-9fa7-3f2d59941ba4_isrForCrosstalkSources_997018_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5eb0b2ed-cf44-43d8-9fa7-3f2d59941ba4_isrForCrosstalkSources_997018_46.stderr parsl.dataflow.dflow INFO: Task 2545 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c0e9192849296f7cd50ebc871da7bd4 with result from task 2545 parsl.dataflow.dflow DEBUG: Task 2545 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/da9b95f2-85f4-4049-9123-66b4308abe0b_isrForCrosstalkSources_996955_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/da9b95f2-85f4-4049-9123-66b4308abe0b_isrForCrosstalkSources_996955_5.stderr parsl.dataflow.dflow INFO: Task 2191 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b9bdae34b6270b5e3e9df688c56135a with result from task 2191 parsl.dataflow.dflow DEBUG: Task 2191 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e715981b-51fd-4960-9bba-706c599e6c14_isrForCrosstalkSources_996992_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e715981b-51fd-4960-9bba-706c599e6c14_isrForCrosstalkSources_996992_62.stderr parsl.dataflow.dflow INFO: Task 2521 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08e24e9e6c82d6a39feab5be0543d5f2 with result from task 2521 parsl.dataflow.dflow DEBUG: Task 2521 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b2b673d3-e075-45ac-92f9-60bea2581246_isrForCrosstalkSources_996996_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b2b673d3-e075-45ac-92f9-60bea2581246_isrForCrosstalkSources_996996_54.stderr parsl.dataflow.dflow INFO: Task 2516 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 729b51b58fca56a820da52eac97c8965 with result from task 2516 parsl.dataflow.dflow DEBUG: Task 2516 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6a67ea77-85da-4374-98fb-cc78d10823eb_isrForCrosstalkSources_996996_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6a67ea77-85da-4374-98fb-cc78d10823eb_isrForCrosstalkSources_996996_18.stderr parsl.dataflow.dflow INFO: Task 2418 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2be992483963b9305e075ec5d3f5f1fc with result from task 2418 parsl.dataflow.dflow DEBUG: Task 2418 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/645d30c1-d123-48b0-9c4c-4615498a7796_isrForCrosstalkSources_996996_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/645d30c1-d123-48b0-9c4c-4615498a7796_isrForCrosstalkSources_996996_56.stderr parsl.dataflow.dflow INFO: Task 2346 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b056e941ac6aa0c9f5b0b6d53497f457 with result from task 2346 parsl.dataflow.dflow DEBUG: Task 2346 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0baafbfe-a18f-44c6-957e-7181a455151a_isrForCrosstalkSources_997010_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/0baafbfe-a18f-44c6-957e-7181a455151a_isrForCrosstalkSources_997010_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:19 proc_lsst.multi:146 [INFO] found job 17075311 in provider checkpoint proc_lsst.multi INFO: found job 17075311 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2183 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 512bf1a8847131b83f281b7ae2b0d7c9 with result from task 2183 parsl.dataflow.dflow DEBUG: Task 2183 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b16a7e0d-978c-42ad-bc30-b3561edb2197_isrForCrosstalkSources_997014_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b16a7e0d-978c-42ad-bc30-b3561edb2197_isrForCrosstalkSources_997014_50.stderr parsl.dataflow.dflow INFO: Task 2531 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a67e122a1127943337806e234baf4f03 with result from task 2531 parsl.dataflow.dflow DEBUG: Task 2531 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/de156fff-c812-4c97-a8e0-d3a683b0db3f_isrForCrosstalkSources_996974_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/de156fff-c812-4c97-a8e0-d3a683b0db3f_isrForCrosstalkSources_996974_23.stderr parsl.dataflow.dflow INFO: Task 2350 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 160c44e1700dedfaeb22d52d098fcbc6 with result from task 2350 parsl.dataflow.dflow DEBUG: Task 2350 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0ec4b600-b272-4517-ade4-befa804a129a_isrForCrosstalkSources_996970_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0ec4b600-b272-4517-ade4-befa804a129a_isrForCrosstalkSources_996970_3.stderr parsl.dataflow.dflow INFO: Task 2399 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb132719983704b9ae65f12b68ac5482 with result from task 2399 parsl.dataflow.dflow DEBUG: Task 2399 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/4ab1dfb4-08fa-4ad6-9b92-0edc725765cb_isrForCrosstalkSources_997000_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/4ab1dfb4-08fa-4ad6-9b92-0edc725765cb_isrForCrosstalkSources_997000_62.stderr parsl.dataflow.dflow INFO: Task 2519 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 030a5ab8d8c99932fe07e4e71a6c607f with result from task 2519 parsl.dataflow.dflow DEBUG: Task 2519 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/51773e66-985a-44e6-91a2-3792fb879878_isrForCrosstalkSources_996990_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/51773e66-985a-44e6-91a2-3792fb879878_isrForCrosstalkSources_996990_60.stderr parsl.dataflow.dflow INFO: Task 2151 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c2819a246b6775e20dcfe4087652942 with result from task 2151 parsl.dataflow.dflow DEBUG: Task 2151 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/82fcd16d-3479-47e7-aa9d-8a099c12f722_isrForCrosstalkSources_996969_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/82fcd16d-3479-47e7-aa9d-8a099c12f722_isrForCrosstalkSources_996969_30.stderr parsl.dataflow.dflow INFO: Task 2198 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ddc4c49f3828f9abc980ac4352107777 with result from task 2198 parsl.dataflow.dflow DEBUG: Task 2198 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/72498c58-80a4-435a-86e4-ad3d97a11c7e_isrForCrosstalkSources_997031_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/72498c58-80a4-435a-86e4-ad3d97a11c7e_isrForCrosstalkSources_997031_11.stderr parsl.dataflow.dflow INFO: Task 2512 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2604eb843130f9a305135e7e857d049 with result from task 2512 parsl.dataflow.dflow DEBUG: Task 2512 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5a96248f-e412-43e3-9b24-77bfcdeda066_isrForCrosstalkSources_996981_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/5a96248f-e412-43e3-9b24-77bfcdeda066_isrForCrosstalkSources_996981_60.stderr parsl.dataflow.dflow INFO: Task 2535 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13ba6241938afd4fea36b9598bbfe47f with result from task 2535 parsl.dataflow.dflow DEBUG: Task 2535 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5cf45e66-bdef-466f-a6c5-85d17ff20cb8_isrForCrosstalkSources_996979_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5cf45e66-bdef-466f-a6c5-85d17ff20cb8_isrForCrosstalkSources_996979_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:20 proc_lsst.multi:146 [INFO] found job 17075312 in provider checkpoint proc_lsst.multi INFO: found job 17075312 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2447 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ddd8c04e548c1f6794b4a38f097b8c47 with result from task 2447 parsl.dataflow.dflow DEBUG: Task 2447 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/dbe5a498-f1ab-4c05-b938-cc974d35dd9c_isrForCrosstalkSources_996969_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/dbe5a498-f1ab-4c05-b938-cc974d35dd9c_isrForCrosstalkSources_996969_49.stderr parsl.dataflow.dflow INFO: Task 2217 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11542e1b67002cede5caf5483fdd1879 with result from task 2217 parsl.dataflow.dflow DEBUG: Task 2217 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/af3600c4-a1a0-40a5-940c-d774d8db326d_isrForCrosstalkSources_996995_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/af3600c4-a1a0-40a5-940c-d774d8db326d_isrForCrosstalkSources_996995_23.stderr parsl.dataflow.dflow INFO: Task 2556 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f2b5bea9ec9a11079ab89a8af8633303 with result from task 2556 parsl.dataflow.dflow DEBUG: Task 2556 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a7cc4fc9-8af3-48c9-940a-5bef9ef6fa51_isrForCrosstalkSources_996960_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a7cc4fc9-8af3-48c9-940a-5bef9ef6fa51_isrForCrosstalkSources_996960_18.stderr parsl.dataflow.dflow INFO: Task 2559 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 042e102cb8a25faca5ff3cad90671602 with result from task 2559 parsl.dataflow.dflow DEBUG: Task 2559 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c2abc604-e524-4229-9cf1-a57ccd5aaf61_isrForCrosstalkSources_996991_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c2abc604-e524-4229-9cf1-a57ccd5aaf61_isrForCrosstalkSources_996991_44.stderr parsl.dataflow.dflow INFO: Task 2509 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75b3c7237a2402994b659c8a4fc50634 with result from task 2509 parsl.dataflow.dflow DEBUG: Task 2509 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/5385e89d-ebf1-44cd-991d-7a40cd57190a_isrForCrosstalkSources_996962_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/5385e89d-ebf1-44cd-991d-7a40cd57190a_isrForCrosstalkSources_996962_59.stderr parsl.dataflow.dflow INFO: Task 2417 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db67748a48b7a9d4d96c723c6bf1c6eb with result from task 2417 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:21 proc_lsst.multi:146 [INFO] found job 17075315 in provider checkpoint proc_lsst.multi INFO: found job 17075315 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2417 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2a07c6cb-dee7-401b-ab92-b65ed7ca3c4a_isrForCrosstalkSources_997005_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2a07c6cb-dee7-401b-ab92-b65ed7ca3c4a_isrForCrosstalkSources_997005_61.stderr parsl.dataflow.dflow INFO: Task 2226 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d8966f398fe305dfc3fb42be46ca4022 with result from task 2226 parsl.dataflow.dflow DEBUG: Task 2226 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/be524e8c-c3b2-47bb-9dd4-dfb54482fa11_isrForCrosstalkSources_996959_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/be524e8c-c3b2-47bb-9dd4-dfb54482fa11_isrForCrosstalkSources_996959_5.stderr parsl.dataflow.dflow INFO: Task 2414 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19af73b078245c4db4df84fae2bd7c6f with result from task 2414 parsl.dataflow.dflow DEBUG: Task 2414 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/0c01c62d-d5fc-4baa-a79f-200d0b57dfc1_isrForCrosstalkSources_996999_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/0c01c62d-d5fc-4baa-a79f-200d0b57dfc1_isrForCrosstalkSources_996999_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:22 proc_lsst.multi:146 [INFO] found job 17075318 in provider checkpoint proc_lsst.multi INFO: found job 17075318 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:22 proc_lsst.multi:146 [INFO] found job 17075319 in provider checkpoint proc_lsst.multi INFO: found job 17075319 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2164 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b528d019cdf349525a306a0fe6069a9 with result from task 2164 parsl.dataflow.dflow DEBUG: Task 2164 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/348d8b58-7246-485d-98ce-b90353acf3b9_isrForCrosstalkSources_997001_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/348d8b58-7246-485d-98ce-b90353acf3b9_isrForCrosstalkSources_997001_15.stderr parsl.dataflow.dflow INFO: Task 2368 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c96a65bf35758cad24eec6db02cf5f8d with result from task 2368 parsl.dataflow.dflow DEBUG: Task 2368 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b38a189a-715c-49c2-bdda-1ce6a221ed9b_isrForCrosstalkSources_996979_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b38a189a-715c-49c2-bdda-1ce6a221ed9b_isrForCrosstalkSources_996979_53.stderr parsl.dataflow.dflow INFO: Task 2246 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1444ce5b5e6853576765fdbcd66de94f with result from task 2246 parsl.dataflow.dflow DEBUG: Task 2246 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6391be9d-8fd4-4454-ba2a-654f112095ce_isrForCrosstalkSources_997015_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6391be9d-8fd4-4454-ba2a-654f112095ce_isrForCrosstalkSources_997015_42.stderr parsl.dataflow.dflow INFO: Task 2542 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f426943cf9786dee02e9153a49b3a8dc with result from task 2542 parsl.dataflow.dflow DEBUG: Task 2542 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a9a002b0-6053-434b-a53b-85d70c1825f3_isrForCrosstalkSources_997028_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a9a002b0-6053-434b-a53b-85d70c1825f3_isrForCrosstalkSources_997028_18.stderr parsl.dataflow.dflow INFO: Task 2251 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 130cfb5dcd5c750f552a774c6266e6b3 with result from task 2251 parsl.dataflow.dflow DEBUG: Task 2251 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/13cb8a34-07d3-43ba-ad9b-eef47bb41aeb_isrForCrosstalkSources_996992_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/13cb8a34-07d3-43ba-ad9b-eef47bb41aeb_isrForCrosstalkSources_996992_44.stderr parsl.dataflow.dflow INFO: Task 2202 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9dd7ebc4c4678a7a209c3c0133966e0 with result from task 2202 parsl.dataflow.dflow DEBUG: Task 2202 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1af38e32-764b-49b6-af0a-eee4fd3469c8_isrForCrosstalkSources_996983_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1af38e32-764b-49b6-af0a-eee4fd3469c8_isrForCrosstalkSources_996983_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:23 proc_lsst.multi:146 [INFO] found job 17075320 in provider checkpoint proc_lsst.multi INFO: found job 17075320 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2536 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38a4b7b3d44845eb5bd470071a4d6a83 with result from task 2536 parsl.dataflow.dflow DEBUG: Task 2536 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/1f6a0812-ddbd-426a-a30a-aa6143e52cf7_isrForCrosstalkSources_997033_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/1f6a0812-ddbd-426a-a30a-aa6143e52cf7_isrForCrosstalkSources_997033_8.stderr parsl.dataflow.dflow INFO: Task 2244 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b85b3c7ad1f36e953c8a211eedc59fc with result from task 2244 parsl.dataflow.dflow DEBUG: Task 2244 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/4eb28046-c471-4e2f-9028-881fc946c6f5_isrForCrosstalkSources_997009_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/4eb28046-c471-4e2f-9028-881fc946c6f5_isrForCrosstalkSources_997009_26.stderr parsl.dataflow.dflow INFO: Task 2584 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe18abbb5ec7ba5bba96fef581680dd9 with result from task 2584 parsl.dataflow.dflow DEBUG: Task 2584 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/36808f1e-818d-47bc-9013-4002573683b2_isrForCrosstalkSources_996976_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/36808f1e-818d-47bc-9013-4002573683b2_isrForCrosstalkSources_996976_10.stderr parsl.dataflow.dflow INFO: Task 2245 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50548ba1add922d8139a19b982615071 with result from task 2245 parsl.dataflow.dflow DEBUG: Task 2245 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/4f498bec-6512-44af-b1fd-61ab059a92fa_isrForCrosstalkSources_996992_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/4f498bec-6512-44af-b1fd-61ab059a92fa_isrForCrosstalkSources_996992_60.stderr parsl.dataflow.dflow INFO: Task 2565 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27491767cc1b98ae6c4a789a92fee361 with result from task 2565 parsl.dataflow.dflow DEBUG: Task 2565 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c595c899-3311-4727-93b1-197d8103878c_isrForCrosstalkSources_997007_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c595c899-3311-4727-93b1-197d8103878c_isrForCrosstalkSources_997007_52.stderr parsl.dataflow.dflow INFO: Task 2548 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef5d9115bb2dd4465f83c9b9c1c9c94d with result from task 2548 parsl.dataflow.dflow DEBUG: Task 2548 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/66efa14a-c4e3-4ec7-9dcc-bd9f1e30ca43_isrForCrosstalkSources_997013_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/66efa14a-c4e3-4ec7-9dcc-bd9f1e30ca43_isrForCrosstalkSources_997013_8.stderr parsl.dataflow.dflow INFO: Task 2586 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16a5ac785396a7b56b19a78a06b91a1f with result from task 2586 parsl.dataflow.dflow DEBUG: Task 2586 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0278204a-2dd8-4068-b9d1-5c20fda73e61_isrForCrosstalkSources_996970_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0278204a-2dd8-4068-b9d1-5c20fda73e61_isrForCrosstalkSources_996970_10.stderr parsl.dataflow.dflow INFO: Task 2160 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19f0b95f459d858daef026dc7a918f77 with result from task 2160 parsl.dataflow.dflow DEBUG: Task 2160 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a8569c92-2ddd-421b-a332-ffefb337a59e_isrForCrosstalkSources_997021_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a8569c92-2ddd-421b-a332-ffefb337a59e_isrForCrosstalkSources_997021_44.stderr parsl.dataflow.dflow INFO: Task 2557 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 045a9bed7561f5dc433d807bfbbfa415 with result from task 2557 parsl.dataflow.dflow DEBUG: Task 2557 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/39c452e7-9090-4e8a-b15d-47bea445cc6c_isrForCrosstalkSources_997032_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/39c452e7-9090-4e8a-b15d-47bea445cc6c_isrForCrosstalkSources_997032_21.stderr parsl.dataflow.dflow INFO: Task 2162 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5297ef727c935d5406aebd67ca9dd3d0 with result from task 2162 parsl.dataflow.dflow DEBUG: Task 2162 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3cf3efc0-0990-4a3f-aed8-0a0453c264a5_isrForCrosstalkSources_996969_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3cf3efc0-0990-4a3f-aed8-0a0453c264a5_isrForCrosstalkSources_996969_28.stderr parsl.dataflow.dflow INFO: Task 2389 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d2134e4c84abe530721c987109b92a11 with result from task 2389 parsl.dataflow.dflow DEBUG: Task 2389 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/737149d2-6be7-4c1b-828c-3f9903ee5b6a_isrForCrosstalkSources_997016_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/737149d2-6be7-4c1b-828c-3f9903ee5b6a_isrForCrosstalkSources_997016_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:24 proc_lsst.multi:146 [INFO] found job 17075321 in provider checkpoint proc_lsst.multi INFO: found job 17075321 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2592 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb4b489d658818e8df6f9fe2ceb567d7 with result from task 2592 parsl.dataflow.dflow DEBUG: Task 2592 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/83cb04a9-350a-4805-9495-54bc5bfe0187_isrForCrosstalkSources_997003_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/83cb04a9-350a-4805-9495-54bc5bfe0187_isrForCrosstalkSources_997003_19.stderr parsl.dataflow.dflow INFO: Task 2392 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01d7f043e8f6382d87e63dc94d7f5282 with result from task 2392 parsl.dataflow.dflow DEBUG: Task 2392 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cc8d876e-1dbf-4a7b-99f5-a4eb752f6858_isrForCrosstalkSources_996956_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cc8d876e-1dbf-4a7b-99f5-a4eb752f6858_isrForCrosstalkSources_996956_27.stderr parsl.dataflow.dflow INFO: Task 2504 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9bc6ef324859262a4d5771c19db253a6 with result from task 2504 parsl.dataflow.dflow DEBUG: Task 2504 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/bd71fbae-748d-4b5c-ba5b-b88b238a644c_isrForCrosstalkSources_997017_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/bd71fbae-748d-4b5c-ba5b-b88b238a644c_isrForCrosstalkSources_997017_56.stderr parsl.dataflow.dflow INFO: Task 2393 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e0c8ad6efdc67f734c6a79f44f7a6c8 with result from task 2393 parsl.dataflow.dflow DEBUG: Task 2393 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/49c9839d-b3b0-49ea-a34c-12a152c76d05_isrForCrosstalkSources_997005_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/49c9839d-b3b0-49ea-a34c-12a152c76d05_isrForCrosstalkSources_997005_33.stderr parsl.dataflow.dflow INFO: Task 2407 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4855e6d51d3ecbff72588898e878fd6c with result from task 2407 parsl.dataflow.dflow DEBUG: Task 2407 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/b637eada-9ee7-4fcf-9ed3-6786d812be2d_isrForCrosstalkSources_996995_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/b637eada-9ee7-4fcf-9ed3-6786d812be2d_isrForCrosstalkSources_996995_19.stderr parsl.dataflow.dflow INFO: Task 2292 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23c0ecc8d60b7674ac8e9f3e9c2bd9c5 with result from task 2292 parsl.dataflow.dflow DEBUG: Task 2292 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f978894b-c412-4e29-a463-847e0203483a_isrForCrosstalkSources_997028_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f978894b-c412-4e29-a463-847e0203483a_isrForCrosstalkSources_997028_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:25 proc_lsst.multi:146 [INFO] found job 17075326 in provider checkpoint proc_lsst.multi INFO: found job 17075326 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2568 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ffb9bee71e00014e27d11c66854ef1e2 with result from task 2568 parsl.dataflow.dflow DEBUG: Task 2568 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a5ab9a3c-b4b2-40d3-b424-76cd44e4ba4b_isrForCrosstalkSources_997008_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a5ab9a3c-b4b2-40d3-b424-76cd44e4ba4b_isrForCrosstalkSources_997008_16.stderr parsl.dataflow.dflow INFO: Task 2314 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b64372c4131e701cf2115edfaa6aceb with result from task 2314 parsl.dataflow.dflow DEBUG: Task 2314 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/62f3367a-f01c-49a4-96e4-c02353766d00_isrForCrosstalkSources_996953_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/62f3367a-f01c-49a4-96e4-c02353766d00_isrForCrosstalkSources_996953_7.stderr parsl.dataflow.dflow INFO: Task 2178 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9ab3537568b822f72243ef3196b9650 with result from task 2178 parsl.dataflow.dflow DEBUG: Task 2178 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c2ceda99-d840-4a06-8b22-a7c07ddbe6b6_isrForCrosstalkSources_997007_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c2ceda99-d840-4a06-8b22-a7c07ddbe6b6_isrForCrosstalkSources_997007_37.stderr parsl.dataflow.dflow INFO: Task 2353 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a00318272b6b8d54a78c9ca92d27f359 with result from task 2353 parsl.dataflow.dflow DEBUG: Task 2353 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/400dd857-950d-4c04-8efb-d0b8fe5e5b42_isrForCrosstalkSources_997032_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/400dd857-950d-4c04-8efb-d0b8fe5e5b42_isrForCrosstalkSources_997032_8.stderr parsl.dataflow.dflow INFO: Task 2611 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aed49d9f9853ac6b1839a6d40575fd14 with result from task 2611 parsl.dataflow.dflow DEBUG: Task 2611 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e97d17bb-34ec-4905-8c6e-1cec0f7af9dc_isrForCrosstalkSources_997020_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e97d17bb-34ec-4905-8c6e-1cec0f7af9dc_isrForCrosstalkSources_997020_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2438 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 0207b81697b1c8d9333c724e2b6767d0 with result from task 2438 parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2438 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/610affb9-841b-465c-94e7-daee2c1c8add_isrForCrosstalkSources_997003_51.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/610affb9-841b-465c-94e7-daee2c1c8add_isrForCrosstalkSources_997003_51.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:26 proc_lsst.multi:146 [INFO] found job 17075329 in provider checkpoint proc_lsst.multi INFO: found job 17075329 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2334 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63ba768684b609e070e508713ccc4fab with result from task 2334 parsl.dataflow.dflow DEBUG: Task 2334 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e9c698c8-8cca-4b19-9460-5d3fea9761d4_isrForCrosstalkSources_996952_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e9c698c8-8cca-4b19-9460-5d3fea9761d4_isrForCrosstalkSources_996952_51.stderr parsl.dataflow.dflow INFO: Task 2482 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9dfc1eb554e4e311cfb982ae742d5d2c with result from task 2482 parsl.dataflow.dflow DEBUG: Task 2482 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4a6a226d-cc30-4365-a423-8824a217a3ab_isrForCrosstalkSources_997005_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4a6a226d-cc30-4365-a423-8824a217a3ab_isrForCrosstalkSources_997005_18.stderr parsl.dataflow.dflow INFO: Task 2387 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fbd80f4730899a4b44bcdce1b41e133 with result from task 2387 parsl.dataflow.dflow DEBUG: Task 2387 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/f61e2d11-fe4c-4231-89c5-2e58d1802a8c_isrForCrosstalkSources_996981_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/f61e2d11-fe4c-4231-89c5-2e58d1802a8c_isrForCrosstalkSources_996981_8.stderr parsl.dataflow.dflow INFO: Task 2597 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa4e30a0959d95f055287729b66f03e6 with result from task 2597 parsl.dataflow.dflow DEBUG: Task 2597 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7a58cd63-0897-42d2-9cc0-dfabedc02b2b_isrForCrosstalkSources_996984_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7a58cd63-0897-42d2-9cc0-dfabedc02b2b_isrForCrosstalkSources_996984_2.stderr parsl.dataflow.dflow INFO: Task 2275 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2026eb552709d95ad5c5ba9059b393f9 with result from task 2275 parsl.dataflow.dflow DEBUG: Task 2275 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a9ac976d-2fca-414d-bb60-fe192e792eb4_isrForCrosstalkSources_996977_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a9ac976d-2fca-414d-bb60-fe192e792eb4_isrForCrosstalkSources_996977_37.stderr parsl.dataflow.dflow INFO: Task 2543 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 210e359ee95d7c590380cee79d0cf48a with result from task 2543 parsl.dataflow.dflow DEBUG: Task 2543 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9442ad18-939e-4ee5-a229-91d64b56c929_isrForCrosstalkSources_996979_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9442ad18-939e-4ee5-a229-91d64b56c929_isrForCrosstalkSources_996979_39.stderr parsl.dataflow.dflow INFO: Task 2190 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d32308ce280075501fd483fee33faa0 with result from task 2190 parsl.dataflow.dflow DEBUG: Task 2190 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/2c62b367-d5c6-4dc3-a409-6e7301224925_isrForCrosstalkSources_996976_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/2c62b367-d5c6-4dc3-a409-6e7301224925_isrForCrosstalkSources_996976_44.stderr parsl.dataflow.dflow INFO: Task 2569 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f4ee247348b743605a5ac4b57224826 with result from task 2569 parsl.dataflow.dflow DEBUG: Task 2569 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4ca84b95-0cd6-4d90-acff-065f0507d488_isrForCrosstalkSources_996966_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/4ca84b95-0cd6-4d90-acff-065f0507d488_isrForCrosstalkSources_996966_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2413 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 103c719bd2d876e8bb7318d0cce2d6aa with result from task 2413 parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2413 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/af5fa671-4b50-43ab-b98a-853def799f8c_isrForCrosstalkSources_996994_8.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/af5fa671-4b50-43ab-b98a-853def799f8c_isrForCrosstalkSources_996994_8.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:26 proc_lsst.multi:146 [INFO] found job 17075330 in provider checkpoint proc_lsst.multi INFO: found job 17075330 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2561 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e2bb78cefd7a3a9f4461169155e3ff9 with result from task 2561 parsl.dataflow.dflow DEBUG: Task 2561 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6180b11a-9b47-46d9-a18d-beca6b400a07_isrForCrosstalkSources_996974_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/6180b11a-9b47-46d9-a18d-beca6b400a07_isrForCrosstalkSources_996974_39.stderr parsl.dataflow.dflow INFO: Task 2549 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2232c44b540c52dcb1c5df28090bea53 with result from task 2549 parsl.dataflow.dflow DEBUG: Task 2549 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9e59db63-d377-416c-addc-8b51b903b519_isrForCrosstalkSources_996959_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9e59db63-d377-416c-addc-8b51b903b519_isrForCrosstalkSources_996959_31.stderr parsl.dataflow.dflow INFO: Task 2220 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a0c20ee1b6d3ee6a741f9ce360c0ba5 with result from task 2220 parsl.dataflow.dflow DEBUG: Task 2220 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/c32dc25c-babb-42c1-8473-8dbd897aa1a0_isrForCrosstalkSources_997031_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/c32dc25c-babb-42c1-8473-8dbd897aa1a0_isrForCrosstalkSources_997031_30.stderr parsl.dataflow.dflow INFO: Task 1807 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63c19320ddb133930ac4dc2794c927b0 with result from task 1807 parsl.dataflow.dflow DEBUG: Task 1807 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/aa4b87bd-f4e4-478a-8ec6-9f27e53cf12b_isrForCrosstalkSources_996975_25.stdout parsl.dataflow.dflow INFO: Standard error for task 1807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/aa4b87bd-f4e4-478a-8ec6-9f27e53cf12b_isrForCrosstalkSources_996975_25.stderr parsl.dataflow.dflow INFO: Task 1809 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry acb5f5a62c81fb8cbd98e6c8352edf57 with result from task 1809 parsl.dataflow.dflow DEBUG: Task 1809 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/0e8a9c4b-bf23-450a-afe0-7256b978055a_isrForCrosstalkSources_996957_19.stdout parsl.dataflow.dflow INFO: Standard error for task 1809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/0e8a9c4b-bf23-450a-afe0-7256b978055a_isrForCrosstalkSources_996957_19.stderr parsl.dataflow.dflow INFO: Task 2566 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry caa238fa946f2a919515e46237d37c9a with result from task 2566 parsl.dataflow.dflow DEBUG: Task 2566 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/fd5e8791-ed28-4a56-834f-c0a50373ee0a_isrForCrosstalkSources_996969_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/fd5e8791-ed28-4a56-834f-c0a50373ee0a_isrForCrosstalkSources_996969_4.stderr parsl.dataflow.dflow INFO: Task 1805 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 80d389f3a91b802b2906f88868cc3958 with result from task 1805 parsl.dataflow.dflow DEBUG: Task 1805 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d1032567-6fd2-4ee8-ab2a-97b63c42b622_isrForCrosstalkSources_996972_58.stdout parsl.dataflow.dflow INFO: Standard error for task 1805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/d1032567-6fd2-4ee8-ab2a-97b63c42b622_isrForCrosstalkSources_996972_58.stderr parsl.dataflow.dflow INFO: Task 1806 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a584a77191bd74bb8dfdd2f230245199 with result from task 1806 parsl.dataflow.dflow DEBUG: Task 1806 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0556cb73-0553-4b10-93cd-07eb3d6cd51f_isrForCrosstalkSources_996975_29.stdout parsl.dataflow.dflow INFO: Standard error for task 1806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0556cb73-0553-4b10-93cd-07eb3d6cd51f_isrForCrosstalkSources_996975_29.stderr parsl.dataflow.dflow INFO: Task 1802 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0200ee92f120c501c27c5b78a83ac1eb with result from task 1802 parsl.dataflow.dflow DEBUG: Task 1802 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/9a4141ca-c897-4027-b751-5a7136d4ac24_isrForCrosstalkSources_997033_11.stdout parsl.dataflow.dflow INFO: Standard error for task 1802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/9a4141ca-c897-4027-b751-5a7136d4ac24_isrForCrosstalkSources_997033_11.stderr parsl.dataflow.dflow INFO: Task 1803 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9e29dbb65f0164e402a2deef7588e35 with result from task 1803 parsl.dataflow.dflow DEBUG: Task 1803 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/4f38a6b8-f6eb-40fc-8e47-8711c658a5a1_isrForCrosstalkSources_996996_47.stdout parsl.dataflow.dflow INFO: Standard error for task 1803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/4f38a6b8-f6eb-40fc-8e47-8711c658a5a1_isrForCrosstalkSources_996996_47.stderr parsl.dataflow.dflow INFO: Task 1804 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a4095720be8b8109034c3bbd8dbf1e8 with result from task 1804 parsl.dataflow.dflow DEBUG: Task 1804 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/51f3547d-8eb5-47a5-b8c7-aeb3799ef058_isrForCrosstalkSources_997019_54.stdout parsl.dataflow.dflow INFO: Standard error for task 1804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/51f3547d-8eb5-47a5-b8c7-aeb3799ef058_isrForCrosstalkSources_997019_54.stderr parsl.dataflow.dflow INFO: Task 1808 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2622283cb272a0bea1904a7288225c84 with result from task 1808 parsl.dataflow.dflow DEBUG: Task 1808 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 1808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2e889981-edba-4ffc-99c0-32a3e8e4adc0_isrForCrosstalkSources_996995_10.stdout parsl.dataflow.dflow INFO: Standard error for task 1808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/2e889981-edba-4ffc-99c0-32a3e8e4adc0_isrForCrosstalkSources_996995_10.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:27 proc_lsst.multi:146 [INFO] found job 17075332 in provider checkpoint proc_lsst.multi INFO: found job 17075332 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2375 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 66c078e3c054202310c8b2e5fb59a228 with result from task 2375 parsl.dataflow.dflow DEBUG: Task 2375 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1d7243f9-c156-4ef2-9423-29d852e2c45d_isrForCrosstalkSources_997005_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1d7243f9-c156-4ef2-9423-29d852e2c45d_isrForCrosstalkSources_997005_1.stderr parsl.dataflow.dflow INFO: Task 2354 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da85923ded72fe7c087ce5ecf8411900 with result from task 2354 parsl.dataflow.dflow DEBUG: Task 2354 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/ae8c5241-939d-4c59-8b82-0f5993ddd889_isrForCrosstalkSources_996955_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/ae8c5241-939d-4c59-8b82-0f5993ddd889_isrForCrosstalkSources_996955_15.stderr parsl.dataflow.dflow INFO: Task 2444 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0833283e51a80221e832db4cb3164812 with result from task 2444 parsl.dataflow.dflow DEBUG: Task 2444 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/2729b324-7e87-459a-98c0-8f4edf821115_isrForCrosstalkSources_996952_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/2729b324-7e87-459a-98c0-8f4edf821115_isrForCrosstalkSources_996952_25.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:28 proc_lsst.multi:146 [INFO] found job 17075338 in provider checkpoint proc_lsst.multi INFO: found job 17075338 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2603 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56aea219653b0999928b73525b5a8773 with result from task 2603 parsl.dataflow.dflow DEBUG: Task 2603 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/21bbfa9b-3fef-4be1-9a2a-bef3445f62e9_isrForCrosstalkSources_996978_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/21bbfa9b-3fef-4be1-9a2a-bef3445f62e9_isrForCrosstalkSources_996978_16.stderr parsl.dataflow.dflow INFO: Task 2423 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d31f6c4ddafa8244893149d60b2ca23 with result from task 2423 parsl.dataflow.dflow DEBUG: Task 2423 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/be141674-e723-4a29-b14a-9f49e9536bc7_isrForCrosstalkSources_996976_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/be141674-e723-4a29-b14a-9f49e9536bc7_isrForCrosstalkSources_996976_25.stderr parsl.dataflow.dflow INFO: Task 2620 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6784977b84ce6649e50599e3e2b2e0b with result from task 2620 parsl.dataflow.dflow DEBUG: Task 2620 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f53a0e22-2f3a-4bc8-bd18-827255e463e0_isrForCrosstalkSources_996980_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f53a0e22-2f3a-4bc8-bd18-827255e463e0_isrForCrosstalkSources_996980_20.stderr parsl.dataflow.dflow INFO: Task 2464 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 52bc1bc0d1424a2c2505dc754e55b353 with result from task 2464 parsl.dataflow.dflow DEBUG: Task 2464 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9cdebbfb-7ea2-40d1-a1a2-211471c5379b_isrForCrosstalkSources_997022_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/9cdebbfb-7ea2-40d1-a1a2-211471c5379b_isrForCrosstalkSources_997022_9.stderr parsl.dataflow.dflow INFO: Task 2622 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d3da08888e6b2042b0f918948e02886 with result from task 2622 parsl.dataflow.dflow DEBUG: Task 2622 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/2d671726-7e83-45e0-bc24-511e573a135f_isrForCrosstalkSources_996994_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/2d671726-7e83-45e0-bc24-511e573a135f_isrForCrosstalkSources_996994_48.stderr parsl.dataflow.dflow INFO: Task 2626 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f508ade7686e353327a83b99fa77048 with result from task 2626 parsl.dataflow.dflow DEBUG: Task 2626 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/8ab45938-6b41-4a6f-a61e-23080ef96ef4_isrForCrosstalkSources_996965_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/8ab45938-6b41-4a6f-a61e-23080ef96ef4_isrForCrosstalkSources_996965_9.stderr parsl.dataflow.dflow INFO: Task 2295 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 76c9ff24184d26ab2d5a147d3dbb10f6 with result from task 2295 parsl.dataflow.dflow DEBUG: Task 2295 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f6fba253-d409-4da3-95dd-09946f57cc7d_isrForCrosstalkSources_996988_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f6fba253-d409-4da3-95dd-09946f57cc7d_isrForCrosstalkSources_996988_19.stderr parsl.dataflow.dflow INFO: Task 2595 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ee5ffb1e89b7edceff25368b3743720d with result from task 2595 parsl.dataflow.dflow DEBUG: Task 2595 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/265b42d9-367b-4fd3-9144-711257904332_isrForCrosstalkSources_997005_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/265b42d9-367b-4fd3-9144-711257904332_isrForCrosstalkSources_997005_55.stderr parsl.dataflow.dflow INFO: Task 2605 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d6fe9d10617fd76cabd40b52314186b4 with result from task 2605 parsl.dataflow.dflow DEBUG: Task 2605 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/557f4f17-703b-4c33-a404-a3681524e05d_isrForCrosstalkSources_996958_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/557f4f17-703b-4c33-a404-a3681524e05d_isrForCrosstalkSources_996958_41.stderr parsl.dataflow.dflow INFO: Task 2617 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7160393b534e0d00c057ee98c6ac544e with result from task 2617 parsl.dataflow.dflow DEBUG: Task 2617 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/7b3c40c0-e8c0-4307-8633-fb4edfbc29f9_isrForCrosstalkSources_996952_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/7b3c40c0-e8c0-4307-8633-fb4edfbc29f9_isrForCrosstalkSources_996952_28.stderr parsl.dataflow.dflow INFO: Task 2608 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 72356c7d8d61ffd7470978a7ca7d63cf with result from task 2608 parsl.dataflow.dflow DEBUG: Task 2608 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7b19e421-56f9-40f6-843e-28e6347a3a56_isrForCrosstalkSources_996991_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7b19e421-56f9-40f6-843e-28e6347a3a56_isrForCrosstalkSources_996991_45.stderr parsl.dataflow.dflow INFO: Task 2590 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8d840a9da40449c5f68cf7d2263002b with result from task 2590 parsl.dataflow.dflow DEBUG: Task 2590 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/36d2aa9c-6d7d-44e3-85d2-11b2295c9aa6_isrForCrosstalkSources_996966_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/36d2aa9c-6d7d-44e3-85d2-11b2295c9aa6_isrForCrosstalkSources_996966_52.stderr parsl.dataflow.dflow INFO: Task 2593 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3077f9948569c2772e6dcf831a0160e with result from task 2593 parsl.dataflow.dflow DEBUG: Task 2593 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b64c3941-b1f7-4c3f-9035-6dd648b1dfaa_isrForCrosstalkSources_997021_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b64c3941-b1f7-4c3f-9035-6dd648b1dfaa_isrForCrosstalkSources_997021_59.stderr parsl.dataflow.dflow INFO: Task 2322 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef1443d77e3d9ee7856c3328f876d082 with result from task 2322 parsl.dataflow.dflow DEBUG: Task 2322 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/49717cbe-fda2-48be-ba7d-4faf2b3af477_isrForCrosstalkSources_997029_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/49717cbe-fda2-48be-ba7d-4faf2b3af477_isrForCrosstalkSources_997029_44.stderr parsl.dataflow.dflow INFO: Task 2567 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3c43f74f19265673d661a65327fd25e with result from task 2567 parsl.dataflow.dflow DEBUG: Task 2567 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/27f805d0-4f12-4356-a945-798ef6e1a64c_isrForCrosstalkSources_996959_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/27f805d0-4f12-4356-a945-798ef6e1a64c_isrForCrosstalkSources_996959_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:30 proc_lsst.multi:146 [INFO] found job 17075344 in provider checkpoint proc_lsst.multi INFO: found job 17075344 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2602 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93ab47e5bfe0ccf5de174bbcfc74d92f with result from task 2602 parsl.dataflow.dflow DEBUG: Task 2602 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/dd714b4c-38fb-4c6b-98c3-94b0644a5e6a_isrForCrosstalkSources_997024_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/dd714b4c-38fb-4c6b-98c3-94b0644a5e6a_isrForCrosstalkSources_997024_61.stderr parsl.dataflow.dflow INFO: Task 2591 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e31c550a617881eb906c13fb18e9343 with result from task 2591 parsl.dataflow.dflow DEBUG: Task 2591 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/770ec2ca-0647-48a9-b9a5-b8881c727c68_isrForCrosstalkSources_996981_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/770ec2ca-0647-48a9-b9a5-b8881c727c68_isrForCrosstalkSources_996981_44.stderr parsl.dataflow.dflow INFO: Task 2609 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85b2b11749af0d87df5e08543322ea2a with result from task 2609 parsl.dataflow.dflow DEBUG: Task 2609 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/68c3614e-9d2a-4129-8f3e-bd86e81638d2_isrForCrosstalkSources_997014_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/68c3614e-9d2a-4129-8f3e-bd86e81638d2_isrForCrosstalkSources_997014_12.stderr parsl.dataflow.dflow INFO: Task 2633 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a5d0464672e0eb2afbbf50c6b73145b with result from task 2633 parsl.dataflow.dflow DEBUG: Task 2633 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/538b2ae7-af13-41ce-949f-a07052f9342d_isrForCrosstalkSources_997027_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/538b2ae7-af13-41ce-949f-a07052f9342d_isrForCrosstalkSources_997027_1.stderr parsl.dataflow.dflow INFO: Task 2589 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 440fa3bcf262879d5aff5fb04a82fd57 with result from task 2589 parsl.dataflow.dflow DEBUG: Task 2589 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/acd5be88-6143-47ab-907b-a65d5f788446_isrForCrosstalkSources_996988_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/acd5be88-6143-47ab-907b-a65d5f788446_isrForCrosstalkSources_996988_42.stderr parsl.dataflow.dflow INFO: Task 2636 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d2ee2aabe32bd57a04532375dec2354 with result from task 2636 parsl.dataflow.dflow DEBUG: Task 2636 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/f6a2f39d-5ee2-461c-bf99-8c20527891b5_isrForCrosstalkSources_997006_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/f6a2f39d-5ee2-461c-bf99-8c20527891b5_isrForCrosstalkSources_997006_21.stderr parsl.dataflow.dflow INFO: Task 2165 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e34c0d4cfc4cfa93c66135e8d795917f with result from task 2165 parsl.dataflow.dflow DEBUG: Task 2165 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/855c876b-b3b6-4659-a39b-d078358683ca_isrForCrosstalkSources_996997_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/855c876b-b3b6-4659-a39b-d078358683ca_isrForCrosstalkSources_996997_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:30 proc_lsst.multi:146 [INFO] found job 17075345 in provider checkpoint proc_lsst.multi INFO: found job 17075345 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2345 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad40f45959d4df9f9b888020e12f07bc with result from task 2345 parsl.dataflow.dflow DEBUG: Task 2345 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/50aabd07-16ab-4121-ad26-867684a5b22d_isrForCrosstalkSources_996989_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/50aabd07-16ab-4121-ad26-867684a5b22d_isrForCrosstalkSources_996989_29.stderr parsl.dataflow.dflow INFO: Task 2659 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23844cc22d7ee518babe15e9175f9bcd with result from task 2659 parsl.dataflow.dflow DEBUG: Task 2659 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e207fd2a-2b03-43b8-8b33-df22a72a253c_isrForCrosstalkSources_996989_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e207fd2a-2b03-43b8-8b33-df22a72a253c_isrForCrosstalkSources_996989_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:31 proc_lsst.multi:146 [INFO] found job 17075346 in provider checkpoint proc_lsst.multi INFO: found job 17075346 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2302 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eac6ad66baa78512706bc4db84f935e0 with result from task 2302 parsl.dataflow.dflow DEBUG: Task 2302 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b6d5aebd-03d4-4d8e-8fbf-df34cb614d5c_isrForCrosstalkSources_997004_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b6d5aebd-03d4-4d8e-8fbf-df34cb614d5c_isrForCrosstalkSources_997004_39.stderr parsl.dataflow.dflow INFO: Task 2402 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10a156c51647f297975c59a85211c4d0 with result from task 2402 parsl.dataflow.dflow DEBUG: Task 2402 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c6423196-6aa3-4ec3-894b-d5ec2774c65e_isrForCrosstalkSources_997001_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c6423196-6aa3-4ec3-894b-d5ec2774c65e_isrForCrosstalkSources_997001_13.stderr parsl.dataflow.dflow INFO: Task 2644 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 208b025887b508dfa38e8df4ba574ddf with result from task 2644 parsl.dataflow.dflow DEBUG: Task 2644 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/853073c9-70de-4979-8775-ac96af0ff720_isrForCrosstalkSources_997024_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/853073c9-70de-4979-8775-ac96af0ff720_isrForCrosstalkSources_997024_15.stderr parsl.dataflow.dflow INFO: Task 2642 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d2481618f47b996c864c9d3a5eca92a with result from task 2642 parsl.dataflow.dflow DEBUG: Task 2642 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/df67196a-0f1c-4d6f-81d6-6f96a0f3d754_isrForCrosstalkSources_996986_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/df67196a-0f1c-4d6f-81d6-6f96a0f3d754_isrForCrosstalkSources_996986_37.stderr parsl.dataflow.dflow INFO: Task 2367 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed4ff1d596626080f158872c59aa201d with result from task 2367 parsl.dataflow.dflow DEBUG: Task 2367 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a08ed3f2-1b95-4623-a453-c06408898a8b_isrForCrosstalkSources_996957_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a08ed3f2-1b95-4623-a453-c06408898a8b_isrForCrosstalkSources_996957_50.stderr parsl.dataflow.dflow INFO: Task 2225 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1b7497b8b7fe114d3d355f9cb448fd57 with result from task 2225 parsl.dataflow.dflow DEBUG: Task 2225 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a94b204e-8b5b-404f-9bfd-dc0ffda93573_isrForCrosstalkSources_997008_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a94b204e-8b5b-404f-9bfd-dc0ffda93573_isrForCrosstalkSources_997008_38.stderr parsl.dataflow.dflow INFO: Task 2373 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2343dfaa31186a5a3803cb02a377b906 with result from task 2373 parsl.dataflow.dflow DEBUG: Task 2373 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d3dfaea4-0754-412b-be0e-adee9d783261_isrForCrosstalkSources_997001_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d3dfaea4-0754-412b-be0e-adee9d783261_isrForCrosstalkSources_997001_60.stderr parsl.dataflow.dflow INFO: Task 2652 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d56fd0b17a09f58ad3d18b9612dc7020 with result from task 2652 parsl.dataflow.dflow DEBUG: Task 2652 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/52c7c4c7-03f5-4a8b-ad82-838f9741eeaa_isrForCrosstalkSources_996961_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/52c7c4c7-03f5-4a8b-ad82-838f9741eeaa_isrForCrosstalkSources_996961_21.stderr parsl.dataflow.dflow INFO: Task 2344 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6a96f3636feda4feff149c361ee6189 with result from task 2344 parsl.dataflow.dflow DEBUG: Task 2344 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/db941cf0-74db-47bb-a16b-a1c374367d35_isrForCrosstalkSources_997000_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/db941cf0-74db-47bb-a16b-a1c374367d35_isrForCrosstalkSources_997000_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:32 proc_lsst.multi:146 [INFO] found job 17075347 in provider checkpoint proc_lsst.multi INFO: found job 17075347 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2528 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 297adee4b391c9e3f7eed228b9b5eb57 with result from task 2528 parsl.dataflow.dflow DEBUG: Task 2528 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e0232251-e2f4-4e0b-a157-cb45ccc84f66_isrForCrosstalkSources_997031_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e0232251-e2f4-4e0b-a157-cb45ccc84f66_isrForCrosstalkSources_997031_60.stderr parsl.dataflow.dflow INFO: Task 2632 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7f92495a8f05fbfc45c17a9a09234484 with result from task 2632 parsl.dataflow.dflow DEBUG: Task 2632 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5d3046f5-85a0-4a27-9d01-e237ff057162_isrForCrosstalkSources_996988_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/5d3046f5-85a0-4a27-9d01-e237ff057162_isrForCrosstalkSources_996988_40.stderr parsl.dataflow.dflow INFO: Task 2645 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e68d10da1ac735a7d211031ab78205d with result from task 2645 parsl.dataflow.dflow DEBUG: Task 2645 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e99e4df5-bdf1-43cd-ad6c-1e41b26a2466_isrForCrosstalkSources_997031_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/e99e4df5-bdf1-43cd-ad6c-1e41b26a2466_isrForCrosstalkSources_997031_18.stderr parsl.dataflow.dflow INFO: Task 2316 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8af353d2868ebc1b76768cb134c717b4 with result from task 2316 parsl.dataflow.dflow DEBUG: Task 2316 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/38f3938a-a124-4eab-986f-52dfd63e44d8_isrForCrosstalkSources_996952_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/38f3938a-a124-4eab-986f-52dfd63e44d8_isrForCrosstalkSources_996952_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:32 proc_lsst.multi:146 [INFO] found job 17075348 in provider checkpoint proc_lsst.multi INFO: found job 17075348 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2320 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6cf1a30fd2bd1a1fe1a6283e3123bb36 with result from task 2320 parsl.dataflow.dflow DEBUG: Task 2320 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6148703e-3260-4dfa-88cc-b056dcd24a7a_isrForCrosstalkSources_996957_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6148703e-3260-4dfa-88cc-b056dcd24a7a_isrForCrosstalkSources_996957_59.stderr parsl.dataflow.dflow INFO: Task 2656 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5cb3f1e41343298515f50856d37e8aed with result from task 2656 parsl.dataflow.dflow DEBUG: Task 2656 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3d99429d-249b-40ce-aae5-094faa1cbca8_isrForCrosstalkSources_997014_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3d99429d-249b-40ce-aae5-094faa1cbca8_isrForCrosstalkSources_997014_6.stderr parsl.dataflow.dflow INFO: Task 2176 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9133befea5be8299691b8f75736b988 with result from task 2176 parsl.dataflow.dflow DEBUG: Task 2176 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/fd74c6bd-67c3-4b10-910a-d472e6215e4f_isrForCrosstalkSources_996958_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/fd74c6bd-67c3-4b10-910a-d472e6215e4f_isrForCrosstalkSources_996958_62.stderr parsl.dataflow.dflow INFO: Task 2491 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8477bfcc29f56ae1c58e456b82203a3a with result from task 2491 parsl.dataflow.dflow DEBUG: Task 2491 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/62cabc1c-21c4-417a-8203-df71e1ba332a_isrForCrosstalkSources_997028_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/62cabc1c-21c4-417a-8203-df71e1ba332a_isrForCrosstalkSources_997028_23.stderr parsl.dataflow.dflow INFO: Task 2537 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44ac4c799bebe3cfad5bbcea73735197 with result from task 2537 parsl.dataflow.dflow DEBUG: Task 2537 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b39cff8b-aa1f-4a37-9a5b-f805dbe46e6b_isrForCrosstalkSources_996964_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b39cff8b-aa1f-4a37-9a5b-f805dbe46e6b_isrForCrosstalkSources_996964_59.stderr parsl.dataflow.dflow INFO: Task 2660 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ceec27d6e2cfc75f128594a420fdd0cf with result from task 2660 parsl.dataflow.dflow DEBUG: Task 2660 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/eaad31b2-94a5-4b1c-bc30-164763efea1d_isrForCrosstalkSources_996953_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/eaad31b2-94a5-4b1c-bc30-164763efea1d_isrForCrosstalkSources_996953_15.stderr parsl.dataflow.dflow INFO: Task 2297 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 517ea7ca57370e339ce47a9cc7f98ea3 with result from task 2297 parsl.dataflow.dflow DEBUG: Task 2297 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/35167d03-9076-428b-bf4f-77b6c2b3a570_isrForCrosstalkSources_997002_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/35167d03-9076-428b-bf4f-77b6c2b3a570_isrForCrosstalkSources_997002_8.stderr parsl.dataflow.dflow INFO: Task 2562 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a35b9493466ef4368103693755b34c6f with result from task 2562 parsl.dataflow.dflow DEBUG: Task 2562 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/087ff17d-a266-40bf-a9ac-728311263af0_isrForCrosstalkSources_997022_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/087ff17d-a266-40bf-a9ac-728311263af0_isrForCrosstalkSources_997022_52.stderr parsl.dataflow.dflow INFO: Task 2336 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5def5c50252b430300132e5c121ab6b5 with result from task 2336 parsl.dataflow.dflow DEBUG: Task 2336 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ce6ad8ca-22f5-4298-b4ca-8d5df929a612_isrForCrosstalkSources_997029_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/ce6ad8ca-22f5-4298-b4ca-8d5df929a612_isrForCrosstalkSources_997029_29.stderr parsl.dataflow.dflow INFO: Task 2337 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d92105bc9c53d88c031778ba0411c98b with result from task 2337 parsl.dataflow.dflow DEBUG: Task 2337 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/686f3651-adf3-4ef3-a66f-896f09b34367_isrForCrosstalkSources_996986_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/686f3651-adf3-4ef3-a66f-896f09b34367_isrForCrosstalkSources_996986_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:33 proc_lsst.multi:146 [INFO] found job 17075351 in provider checkpoint proc_lsst.multi INFO: found job 17075351 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2462 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02ddc54f1072666e22b2ffe185e5d2e3 with result from task 2462 parsl.dataflow.dflow DEBUG: Task 2462 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/e607574e-e2a4-465f-8fd5-d3a706f60b10_isrForCrosstalkSources_997029_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/e607574e-e2a4-465f-8fd5-d3a706f60b10_isrForCrosstalkSources_997029_23.stderr parsl.dataflow.dflow INFO: Task 2338 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b1b0622996f54cce7f7e995794abb0b with result from task 2338 parsl.dataflow.dflow DEBUG: Task 2338 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7f8deade-5b9e-4461-8ee4-a678369317f7_isrForCrosstalkSources_996963_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/7f8deade-5b9e-4461-8ee4-a678369317f7_isrForCrosstalkSources_996963_49.stderr parsl.dataflow.dflow INFO: Task 2493 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b03c2f887f68b7b295aafe7bf68841ee with result from task 2493 parsl.dataflow.dflow DEBUG: Task 2493 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c5b07f0c-aa3c-462d-ae73-f44b94528d80_isrForCrosstalkSources_996977_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c5b07f0c-aa3c-462d-ae73-f44b94528d80_isrForCrosstalkSources_996977_34.stderr parsl.dataflow.dflow INFO: Task 2171 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3e69fc5f633e0edfcb554615d547c911 with result from task 2171 parsl.dataflow.dflow DEBUG: Task 2171 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/85816360-4aa9-4ff4-abee-d41cd6eab71d_isrForCrosstalkSources_996972_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/85816360-4aa9-4ff4-abee-d41cd6eab71d_isrForCrosstalkSources_996972_30.stderr parsl.dataflow.dflow INFO: Task 2655 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 074c40c51c92e6848d0931816ce7af97 with result from task 2655 parsl.dataflow.dflow DEBUG: Task 2655 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/60811b70-0a58-46f6-a277-d3c37f1f8b56_isrForCrosstalkSources_996992_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/60811b70-0a58-46f6-a277-d3c37f1f8b56_isrForCrosstalkSources_996992_54.stderr parsl.dataflow.dflow INFO: Task 2425 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ff560ce536161e6b8592119ac7c8843 with result from task 2425 parsl.dataflow.dflow DEBUG: Task 2425 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a1c4edd9-8d28-4d50-826d-c9bbc3bcd2b6_isrForCrosstalkSources_996956_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a1c4edd9-8d28-4d50-826d-c9bbc3bcd2b6_isrForCrosstalkSources_996956_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:34 proc_lsst.multi:146 [INFO] found job 17075353 in provider checkpoint proc_lsst.multi INFO: found job 17075353 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2174 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6693d8689c8f5a768dc375ffc5eb8974 with result from task 2174 parsl.dataflow.dflow DEBUG: Task 2174 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/57215ccf-ec7b-496b-b665-439be1576b99_isrForCrosstalkSources_996970_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/57215ccf-ec7b-496b-b665-439be1576b99_isrForCrosstalkSources_996970_59.stderr parsl.dataflow.dflow INFO: Task 2489 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3b2305ca3dadbc497e18ed4303018ee with result from task 2489 parsl.dataflow.dflow DEBUG: Task 2489 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5f54fc04-60a6-49ee-b4b8-42d420a5ef84_isrForCrosstalkSources_996954_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5f54fc04-60a6-49ee-b4b8-42d420a5ef84_isrForCrosstalkSources_996954_14.stderr parsl.dataflow.dflow INFO: Task 2372 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e43fd98b82c7765282cf86d0a13a58a with result from task 2372 parsl.dataflow.dflow DEBUG: Task 2372 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/0369cc5d-7ed1-4b20-bcd4-2fa42cb9e2e4_isrForCrosstalkSources_996962_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/0369cc5d-7ed1-4b20-bcd4-2fa42cb9e2e4_isrForCrosstalkSources_996962_49.stderr parsl.dataflow.dflow INFO: Task 2673 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 958507f3782a07d7b6a55a2c3fad2695 with result from task 2673 parsl.dataflow.dflow DEBUG: Task 2673 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c272a04c-1427-4d59-9e39-033aea46d87e_isrForCrosstalkSources_997013_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c272a04c-1427-4d59-9e39-033aea46d87e_isrForCrosstalkSources_997013_58.stderr parsl.dataflow.dflow INFO: Task 2287 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 621e1d68b5bd4021283ab79eeb479f70 with result from task 2287 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:34 proc_lsst.multi:146 [INFO] found job 17075354 in provider checkpoint proc_lsst.multi INFO: found job 17075354 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2287 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f3895cb1-0870-4185-8200-714eae2dad49_isrForCrosstalkSources_996955_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f3895cb1-0870-4185-8200-714eae2dad49_isrForCrosstalkSources_996955_52.stderr parsl.dataflow.dflow INFO: Task 2526 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5afe9a79c32a8378c0b023f4cf052b08 with result from task 2526 parsl.dataflow.dflow DEBUG: Task 2526 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3fa69f95-c7de-4048-bea3-d270e19a3d9f_isrForCrosstalkSources_997019_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3fa69f95-c7de-4048-bea3-d270e19a3d9f_isrForCrosstalkSources_997019_29.stderr parsl.dataflow.dflow INFO: Task 2148 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9236ae806033164002b9fbad6ef9dc28 with result from task 2148 parsl.dataflow.dflow DEBUG: Task 2148 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/49565dec-5553-4542-b1c8-321c83b20c46_isrForCrosstalkSources_996998_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/49565dec-5553-4542-b1c8-321c83b20c46_isrForCrosstalkSources_996998_60.stderr parsl.dataflow.dflow INFO: Task 2527 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86a3ae909dea49d8f355f7ad47b32145 with result from task 2527 parsl.dataflow.dflow DEBUG: Task 2527 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/3948b96a-b3d8-4690-a8ca-8c79213e0b66_isrForCrosstalkSources_996980_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/3948b96a-b3d8-4690-a8ca-8c79213e0b66_isrForCrosstalkSources_996980_28.stderr parsl.dataflow.dflow INFO: Task 2400 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ecb948f8ed7a7b06dded197fcaa28ac with result from task 2400 parsl.dataflow.dflow DEBUG: Task 2400 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/72fafcad-3cb8-45d5-8398-ae4cbbe50dd9_isrForCrosstalkSources_996962_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/72fafcad-3cb8-45d5-8398-ae4cbbe50dd9_isrForCrosstalkSources_996962_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:35 proc_lsst.multi:146 [INFO] found job 17075355 in provider checkpoint proc_lsst.multi INFO: found job 17075355 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2179 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e89952a3f5119412fb2921d5fa7757d with result from task 2179 parsl.dataflow.dflow DEBUG: Task 2179 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4c9388f9-2867-439a-8d75-fef7dfbe1a34_isrForCrosstalkSources_997033_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/4c9388f9-2867-439a-8d75-fef7dfbe1a34_isrForCrosstalkSources_997033_54.stderr parsl.dataflow.dflow INFO: Task 2649 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42de1ab06c5cf001d81ca3a024013a82 with result from task 2649 parsl.dataflow.dflow DEBUG: Task 2649 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/cac9fac6-1829-45e9-b799-381975e69395_isrForCrosstalkSources_997010_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/cac9fac6-1829-45e9-b799-381975e69395_isrForCrosstalkSources_997010_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:35 proc_lsst.multi:146 [INFO] found job 17075357 in provider checkpoint proc_lsst.multi INFO: found job 17075357 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2551 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 460fcfa92a17ea4fa16ae96f71c35a59 with result from task 2551 parsl.dataflow.dflow DEBUG: Task 2551 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ebdf3984-c96e-4a49-b716-635e0d216e26_isrForCrosstalkSources_996978_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ebdf3984-c96e-4a49-b716-635e0d216e26_isrForCrosstalkSources_996978_55.stderr parsl.dataflow.dflow INFO: Task 2286 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry afefd1f9ea57072f94ed384be36e10fa with result from task 2286 parsl.dataflow.dflow DEBUG: Task 2286 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/68de810e-90e2-44fc-b93b-73cc436991af_isrForCrosstalkSources_996976_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/68de810e-90e2-44fc-b93b-73cc436991af_isrForCrosstalkSources_996976_23.stderr parsl.dataflow.dflow INFO: Task 2677 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 498615d2535cc9a29829d25679efa8c8 with result from task 2677 parsl.dataflow.dflow DEBUG: Task 2677 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/c8f748ed-82c5-4fba-b1a9-defbb568459d_isrForCrosstalkSources_997032_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/c8f748ed-82c5-4fba-b1a9-defbb568459d_isrForCrosstalkSources_997032_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:36 proc_lsst.multi:146 [INFO] found job 17075358 in provider checkpoint proc_lsst.multi INFO: found job 17075358 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2364 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 68b03ff06709dff6edfa583f9b2ae328 with result from task 2364 parsl.dataflow.dflow DEBUG: Task 2364 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/cd8a2dcc-0c45-44ed-a552-d8345ea09422_isrForCrosstalkSources_996954_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/cd8a2dcc-0c45-44ed-a552-d8345ea09422_isrForCrosstalkSources_996954_16.stderr parsl.dataflow.dflow INFO: Task 2692 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39f8c72cc2dc9ce56438b2cb9bb4b0f4 with result from task 2692 parsl.dataflow.dflow DEBUG: Task 2692 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ede4c99b-ea13-4c63-9ae5-835be2cf60a8_isrForCrosstalkSources_997000_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ede4c99b-ea13-4c63-9ae5-835be2cf60a8_isrForCrosstalkSources_997000_10.stderr parsl.dataflow.dflow INFO: Task 2466 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f98ea358fee44726827be2de0efc19f8 with result from task 2466 parsl.dataflow.dflow DEBUG: Task 2466 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/685920a1-6843-4699-a8a8-a01d5cfe8976_isrForCrosstalkSources_997026_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/685920a1-6843-4699-a8a8-a01d5cfe8976_isrForCrosstalkSources_997026_16.stderr parsl.dataflow.dflow INFO: Task 2415 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a629ebe6e047d800a69fcf1b7d7afbd with result from task 2415 parsl.dataflow.dflow DEBUG: Task 2415 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8bf87953-7879-4593-bba8-0e894197105e_isrForCrosstalkSources_996992_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8bf87953-7879-4593-bba8-0e894197105e_isrForCrosstalkSources_996992_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:36 proc_lsst.multi:146 [INFO] found job 17075359 in provider checkpoint proc_lsst.multi INFO: found job 17075359 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2697 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 218cba28f0dd7efec9de39560f088b31 with result from task 2697 parsl.dataflow.dflow DEBUG: Task 2697 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/99448460-e315-45c5-8c2e-1aba5dea15c3_isrForCrosstalkSources_997025_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/99448460-e315-45c5-8c2e-1aba5dea15c3_isrForCrosstalkSources_997025_10.stderr parsl.dataflow.dflow INFO: Task 2661 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 95a1bdfc4cf0a29aec368409c77902c9 with result from task 2661 parsl.dataflow.dflow DEBUG: Task 2661 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4b429a0d-d051-4873-a8e6-d530c490ff94_isrForCrosstalkSources_997030_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4b429a0d-d051-4873-a8e6-d530c490ff94_isrForCrosstalkSources_997030_7.stderr parsl.dataflow.dflow INFO: Task 2420 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48824af55f7a3b06218917c32927fa54 with result from task 2420 parsl.dataflow.dflow DEBUG: Task 2420 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4c1109c8-7b9a-4045-8bf2-a921a2d5baf8_isrForCrosstalkSources_997017_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4c1109c8-7b9a-4045-8bf2-a921a2d5baf8_isrForCrosstalkSources_997017_2.stderr parsl.dataflow.dflow INFO: Task 2475 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb686b2aaa37567c09383e1d6d8b93f6 with result from task 2475 parsl.dataflow.dflow DEBUG: Task 2475 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/2080cdc4-6ecb-4ee9-8e83-85e04556475b_isrForCrosstalkSources_997029_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/2080cdc4-6ecb-4ee9-8e83-85e04556475b_isrForCrosstalkSources_997029_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:37 proc_lsst.multi:146 [INFO] found job 17075360 in provider checkpoint proc_lsst.multi INFO: found job 17075360 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2233 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c211390e0cdcc1941fdd99f21a8b8494 with result from task 2233 parsl.dataflow.dflow DEBUG: Task 2233 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ec3ce184-1335-4803-82ed-a6ad37b4f7e7_isrForCrosstalkSources_996992_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ec3ce184-1335-4803-82ed-a6ad37b4f7e7_isrForCrosstalkSources_996992_48.stderr parsl.dataflow.dflow INFO: Task 2675 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 47e0b3fa35855540d854af9638d22960 with result from task 2675 parsl.dataflow.dflow DEBUG: Task 2675 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a366daa5-ca30-4b73-9df5-17e639fee7d0_isrForCrosstalkSources_996994_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a366daa5-ca30-4b73-9df5-17e639fee7d0_isrForCrosstalkSources_996994_50.stderr parsl.dataflow.dflow INFO: Task 2596 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4022f48289057d48a38b9811bbb7393a with result from task 2596 parsl.dataflow.dflow DEBUG: Task 2596 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/7a980967-2f99-41df-84db-d0c43fb332c3_isrForCrosstalkSources_996962_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/7a980967-2f99-41df-84db-d0c43fb332c3_isrForCrosstalkSources_996962_20.stderr parsl.dataflow.dflow INFO: Task 2594 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d54892718bae715d1c4edd878447e10 with result from task 2594 parsl.dataflow.dflow DEBUG: Task 2594 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/3492f5ae-37eb-44bd-9b7f-32e994af9bdb_isrForCrosstalkSources_996973_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/3492f5ae-37eb-44bd-9b7f-32e994af9bdb_isrForCrosstalkSources_996973_34.stderr parsl.dataflow.dflow INFO: Task 2264 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf456a29cf4b31dfb4c742d54241b220 with result from task 2264 parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:37 proc_lsst.multi:146 [INFO] found job 17075364 in provider checkpoint proc_lsst.multi INFO: found job 17075364 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2264 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/4239f5b0-82e6-4322-969a-0f635e519ffa_isrForCrosstalkSources_997009_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/4239f5b0-82e6-4322-969a-0f635e519ffa_isrForCrosstalkSources_997009_52.stderr parsl.dataflow.dflow INFO: Task 2193 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9cb75a158c72cac297822419b8ed8702 with result from task 2193 parsl.dataflow.dflow DEBUG: Task 2193 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/94897950-2944-4302-bb5f-a620443f9f23_isrForCrosstalkSources_996988_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/94897950-2944-4302-bb5f-a620443f9f23_isrForCrosstalkSources_996988_52.stderr parsl.dataflow.dflow INFO: Task 2668 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 438ade7c3c56ce3b7e6dc544fefbced3 with result from task 2668 parsl.dataflow.dflow DEBUG: Task 2668 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/acace142-a7d0-4279-af54-ee6ff6f02f6f_isrForCrosstalkSources_996964_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/acace142-a7d0-4279-af54-ee6ff6f02f6f_isrForCrosstalkSources_996964_21.stderr parsl.dataflow.dflow INFO: Task 2405 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b2ee1dc5f7f7d2a0fb78159f1723b0cb with result from task 2405 parsl.dataflow.dflow DEBUG: Task 2405 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/860b07e2-e5a2-414e-a678-af4d9be0b237_isrForCrosstalkSources_996999_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/860b07e2-e5a2-414e-a678-af4d9be0b237_isrForCrosstalkSources_996999_9.stderr parsl.dataflow.dflow INFO: Task 2470 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c9f9447a4f06cdec92882ad84d4dce1 with result from task 2470 parsl.dataflow.dflow DEBUG: Task 2470 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/08921616-d482-4177-8707-1f59c89e32b3_isrForCrosstalkSources_997032_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/08921616-d482-4177-8707-1f59c89e32b3_isrForCrosstalkSources_997032_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:38 proc_lsst.multi:146 [INFO] found job 17075368 in provider checkpoint proc_lsst.multi INFO: found job 17075368 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2651 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 047af9b4a11a8bcd377937ec610c64e1 with result from task 2651 parsl.dataflow.dflow DEBUG: Task 2651 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f76d4f31-0a2b-4f29-99c8-6d5128dd4990_isrForCrosstalkSources_997032_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f76d4f31-0a2b-4f29-99c8-6d5128dd4990_isrForCrosstalkSources_997032_56.stderr parsl.dataflow.dflow INFO: Task 2395 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f29fb9ba819a61df081b7af5c7807fa3 with result from task 2395 parsl.dataflow.dflow DEBUG: Task 2395 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/a096cc60-3ce6-4c01-947e-308a1393dc4d_isrForCrosstalkSources_996968_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/a096cc60-3ce6-4c01-947e-308a1393dc4d_isrForCrosstalkSources_996968_11.stderr parsl.dataflow.dflow INFO: Task 2293 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8cfc2dbd969ddee15524d3981ba9dbc5 with result from task 2293 parsl.dataflow.dflow DEBUG: Task 2293 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/b1559aab-078f-4e4f-84a1-de6bbddd4720_isrForCrosstalkSources_996954_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/b1559aab-078f-4e4f-84a1-de6bbddd4720_isrForCrosstalkSources_996954_1.stderr parsl.dataflow.dflow INFO: Task 2288 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c703812c55cb35194710b5d359b712f2 with result from task 2288 parsl.dataflow.dflow DEBUG: Task 2288 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b292662f-ac07-4c9d-bbd7-da431635549b_isrForCrosstalkSources_996979_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b292662f-ac07-4c9d-bbd7-da431635549b_isrForCrosstalkSources_996979_57.stderr parsl.dataflow.dflow INFO: Task 2255 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f0449b009f6d3b7cdf970609033467ef with result from task 2255 parsl.dataflow.dflow DEBUG: Task 2255 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8c7afdb1-f57e-48ff-bd55-76c31b6c70c2_isrForCrosstalkSources_996976_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8c7afdb1-f57e-48ff-bd55-76c31b6c70c2_isrForCrosstalkSources_996976_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2363 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 16d94d98edd3274a44354c08b02c246e with result from task 2363 parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2363 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/86eab017-047f-4bae-b26b-5cd283ca5e26_isrForCrosstalkSources_996971_52.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/86eab017-047f-4bae-b26b-5cd283ca5e26_isrForCrosstalkSources_996971_52.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:39 proc_lsst.multi:146 [INFO] found job 17075372 in provider checkpoint proc_lsst.multi INFO: found job 17075372 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2723 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00bc7120c3ff7e95259a76935198eb4c with result from task 2723 parsl.dataflow.dflow DEBUG: Task 2723 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/1eb69ac1-fe69-4321-9372-ce07dfe9abaa_isrForCrosstalkSources_997015_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/1eb69ac1-fe69-4321-9372-ce07dfe9abaa_isrForCrosstalkSources_997015_13.stderr parsl.dataflow.dflow INFO: Task 2474 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry efced680dd6914d72f6d2519c467b40a with result from task 2474 parsl.dataflow.dflow DEBUG: Task 2474 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e3383d57-ed9b-4120-a091-85e161433571_isrForCrosstalkSources_996964_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e3383d57-ed9b-4120-a091-85e161433571_isrForCrosstalkSources_996964_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2722 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry ab2a9f77e60ee09e3a4811044c17aaae with result from task 2722 parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2722 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4205fb2a-1e1d-496d-90cf-6c51c579f917_isrForCrosstalkSources_996976_21.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4205fb2a-1e1d-496d-90cf-6c51c579f917_isrForCrosstalkSources_996976_21.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2637 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 79b3e3069253be8f4be1fcf68b59d703 with result from task 2637 parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2637 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/770052b8-b730-46c5-850b-95466bc50c9b_isrForCrosstalkSources_997031_41.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:40 proc_lsst.multi:146 [INFO] found job 17075376 in provider checkpoint proc_lsst.multi INFO: found job 17075376 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Standard error for task 2637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/770052b8-b730-46c5-850b-95466bc50c9b_isrForCrosstalkSources_997031_41.stderr parsl.dataflow.dflow INFO: Task 2720 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 20f49d6bff9fb7a5593991cb694a0668 with result from task 2720 parsl.dataflow.dflow DEBUG: Task 2720 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/e40e8074-143c-419d-9ca8-3918cf3eb76f_isrForCrosstalkSources_997025_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/e40e8074-143c-419d-9ca8-3918cf3eb76f_isrForCrosstalkSources_997025_47.stderr parsl.dataflow.dflow INFO: Task 2560 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58d88f852f9aaecddfbee7109bfe93f4 with result from task 2560 parsl.dataflow.dflow DEBUG: Task 2560 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/1dc1edf6-fdcd-4521-a361-cade7b1e97ce_isrForCrosstalkSources_996981_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/1dc1edf6-fdcd-4521-a361-cade7b1e97ce_isrForCrosstalkSources_996981_11.stderr parsl.dataflow.dflow INFO: Task 2745 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4130e5392ff65e6bcd73ae355f9e8e8f with result from task 2745 parsl.dataflow.dflow DEBUG: Task 2745 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/7e43c469-0b13-4b47-a039-c2217ef5eac0_isrForCrosstalkSources_997000_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/7e43c469-0b13-4b47-a039-c2217ef5eac0_isrForCrosstalkSources_997000_2.stderr parsl.dataflow.dflow INFO: Task 2479 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e75bb72910aa7bc5bba8ca9f3683e5ab with result from task 2479 parsl.dataflow.dflow DEBUG: Task 2479 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4c2c0b3b-ecaa-4871-8102-6bc9c9e6fb33_isrForCrosstalkSources_997030_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4c2c0b3b-ecaa-4871-8102-6bc9c9e6fb33_isrForCrosstalkSources_997030_54.stderr parsl.dataflow.dflow INFO: Task 2738 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c68c5a002fcc144985a861975ce1dc0 with result from task 2738 parsl.dataflow.dflow DEBUG: Task 2738 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/355d8f3b-3de2-4fb1-8785-4cb238b5c776_isrForCrosstalkSources_997004_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/355d8f3b-3de2-4fb1-8785-4cb238b5c776_isrForCrosstalkSources_997004_10.stderr parsl.dataflow.dflow INFO: Task 2514 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b7c645ba5c37205373c29d369c1b65e with result from task 2514 parsl.dataflow.dflow DEBUG: Task 2514 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/3172333c-75f5-4104-9619-8e8c04c8b768_isrForCrosstalkSources_997018_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/3172333c-75f5-4104-9619-8e8c04c8b768_isrForCrosstalkSources_997018_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2699 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:41 proc_lsst.multi:146 [INFO] found job 17075379 in provider checkpoint proc_lsst.multi INFO: found job 17075379 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.memoization DEBUG: Storing app cache entry 47ea78b67b8086e8269ffcb7f47d423b with result from task 2699 parsl.dataflow.dflow DEBUG: Task 2699 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc94f3e6-b2fd-4dbf-828e-453413ded5f0_isrForCrosstalkSources_997025_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc94f3e6-b2fd-4dbf-828e-453413ded5f0_isrForCrosstalkSources_997025_32.stderr parsl.dataflow.dflow INFO: Task 2541 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5146f5c498f4e2d6baf3e2feaa74a1dd with result from task 2541 parsl.dataflow.dflow DEBUG: Task 2541 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/38fa187a-5416-4344-bd36-98ab40652641_isrForCrosstalkSources_996987_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/38fa187a-5416-4344-bd36-98ab40652641_isrForCrosstalkSources_996987_34.stderr parsl.dataflow.dflow INFO: Task 2448 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 872672f35d81a99de6f7348ab2f12429 with result from task 2448 parsl.dataflow.dflow DEBUG: Task 2448 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/3708f8ad-aad2-45db-9273-e0c385798f16_isrForCrosstalkSources_997032_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/3708f8ad-aad2-45db-9273-e0c385798f16_isrForCrosstalkSources_997032_20.stderr parsl.dataflow.dflow INFO: Task 2553 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2cd567e28d03dcfd35dbc035322f1355 with result from task 2553 parsl.dataflow.dflow DEBUG: Task 2553 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c343e056-dbd9-4114-a859-3b99a9e2b7e3_isrForCrosstalkSources_997003_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c343e056-dbd9-4114-a859-3b99a9e2b7e3_isrForCrosstalkSources_997003_37.stderr parsl.dataflow.dflow INFO: Task 2563 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c55cf614819cf5edee80085c635d17ea with result from task 2563 parsl.dataflow.dflow DEBUG: Task 2563 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d1521340-7aa7-4dfd-93ea-27da545e50d3_isrForCrosstalkSources_996954_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/d1521340-7aa7-4dfd-93ea-27da545e50d3_isrForCrosstalkSources_996954_20.stderr parsl.dataflow.dflow INFO: Task 2682 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e468e4aeda841c7739501bb922952ee8 with result from task 2682 parsl.dataflow.dflow DEBUG: Task 2682 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d8c2c21f-94db-4e31-bb36-8d261c50d1f8_isrForCrosstalkSources_996988_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d8c2c21f-94db-4e31-bb36-8d261c50d1f8_isrForCrosstalkSources_996988_1.stderr parsl.dataflow.dflow INFO: Task 2741 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de832a6f3087f128736c77183a3d3e7e with result from task 2741 parsl.dataflow.dflow DEBUG: Task 2741 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/35da15c8-08e3-498e-bc15-b07837a17895_isrForCrosstalkSources_997018_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/35da15c8-08e3-498e-bc15-b07837a17895_isrForCrosstalkSources_997018_18.stderr parsl.dataflow.dflow INFO: Task 2737 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3817793ace9dd9f817a2b1f193c5ae00 with result from task 2737 parsl.dataflow.dflow DEBUG: Task 2737 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/99c061e2-a1fb-40c6-a6cb-8f0dc2a1beac_isrForCrosstalkSources_996968_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/99c061e2-a1fb-40c6-a6cb-8f0dc2a1beac_isrForCrosstalkSources_996968_25.stderr parsl.dataflow.dflow INFO: Task 2357 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc544781642608ee16f3694ded082fd7 with result from task 2357 parsl.dataflow.dflow DEBUG: Task 2357 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2be186b5-cd16-4cff-8c0a-4097e1d9d384_isrForCrosstalkSources_996974_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2be186b5-cd16-4cff-8c0a-4097e1d9d384_isrForCrosstalkSources_996974_31.stderr parsl.dataflow.dflow INFO: Task 2520 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19d0a0fbebcf8f49fc0d701a8f7e8e51 with result from task 2520 parsl.dataflow.dflow DEBUG: Task 2520 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/21e0a183-290c-443a-a185-fbcac2759de6_isrForCrosstalkSources_997010_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/21e0a183-290c-443a-a185-fbcac2759de6_isrForCrosstalkSources_997010_49.stderr parsl.dataflow.dflow INFO: Task 2538 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f347afc6cb5624a868db70c0cb8b0d19 with result from task 2538 parsl.dataflow.dflow DEBUG: Task 2538 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/d2680210-80a9-4585-82f7-88a65cdd93f7_isrForCrosstalkSources_996959_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/d2680210-80a9-4585-82f7-88a65cdd93f7_isrForCrosstalkSources_996959_34.stderr parsl.dataflow.dflow INFO: Task 2452 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b95587336b3417106b3f1eef9207ca93 with result from task 2452 parsl.dataflow.dflow DEBUG: Task 2452 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/579c9296-e156-4ea6-ba6e-4e5a9e8848c4_isrForCrosstalkSources_997001_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/579c9296-e156-4ea6-ba6e-4e5a9e8848c4_isrForCrosstalkSources_997001_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2481 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry d64f5a5a1beafa3dd84ad6a83489a9f6 with result from task 2481 parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2481 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/143bd40d-6928-4262-92a0-7002f1a0abbd_isrForCrosstalkSources_996997_49.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/143bd40d-6928-4262-92a0-7002f1a0abbd_isrForCrosstalkSources_996997_49.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2356 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 6f3807bea90b2bcc80a8a126e74847c5 with result from task 2356 parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2356 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/e3467e9f-0323-49fa-99d3-b6c502dceb7b_isrForCrosstalkSources_997012_57.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/e3467e9f-0323-49fa-99d3-b6c502dceb7b_isrForCrosstalkSources_997012_57.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2478 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 2392741d05ff22cdbe754565430daf43 with result from task 2478 parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2478 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/81247663-5844-4260-a9c4-66b5f246ab9c_isrForCrosstalkSources_997007_25.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/81247663-5844-4260-a9c4-66b5f246ab9c_isrForCrosstalkSources_997007_25.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2640 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 08ea03e09fcfb932a9ebf0f54ead0332 with result from task 2640 parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2640 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/528a6a40-f5db-4743-a091-701bd41a88c6_isrForCrosstalkSources_997028_14.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/528a6a40-f5db-4743-a091-701bd41a88c6_isrForCrosstalkSources_997028_14.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:43 proc_lsst.multi:146 [INFO] found job 17075380 in provider checkpoint proc_lsst.multi INFO: found job 17075380 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2666 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e1d690038956b55bafafc05cb589f50 with result from task 2666 parsl.dataflow.dflow DEBUG: Task 2666 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c22a55c8-2f8b-4632-a6b6-51d84d0a119c_isrForCrosstalkSources_996997_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c22a55c8-2f8b-4632-a6b6-51d84d0a119c_isrForCrosstalkSources_996997_34.stderr parsl.dataflow.dflow INFO: Task 2756 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 001c1b41f7a297a20679e8d94fcde4a0 with result from task 2756 parsl.dataflow.dflow DEBUG: Task 2756 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8dc0fba1-16c5-4773-937a-36af601e09a1_isrForCrosstalkSources_997032_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8dc0fba1-16c5-4773-937a-36af601e09a1_isrForCrosstalkSources_997032_32.stderr parsl.dataflow.dflow INFO: Task 2753 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry efdfcdd58c5f834fec5a49fc720eac42 with result from task 2753 parsl.dataflow.dflow DEBUG: Task 2753 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a9d773bf-5477-4483-a07b-7468093ba10a_isrForCrosstalkSources_996970_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a9d773bf-5477-4483-a07b-7468093ba10a_isrForCrosstalkSources_996970_20.stderr parsl.dataflow.dflow INFO: Task 2744 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b377526c75a13cd4977bb1d14d690fe with result from task 2744 parsl.dataflow.dflow DEBUG: Task 2744 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d8517c4d-a561-47a4-b3f2-ebd6dafe87bd_isrForCrosstalkSources_996971_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d8517c4d-a561-47a4-b3f2-ebd6dafe87bd_isrForCrosstalkSources_996971_24.stderr parsl.dataflow.dflow INFO: Task 2281 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9647359631b66bf056fc6c9117dbe4c with result from task 2281 parsl.dataflow.dflow DEBUG: Task 2281 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b458cb13-9de5-4adf-b2ca-ab1845486254_isrForCrosstalkSources_996996_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/b458cb13-9de5-4adf-b2ca-ab1845486254_isrForCrosstalkSources_996996_29.stderr parsl.dataflow.dflow INFO: Task 2749 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f1cd897e365e5da0ef013a3e6245dc56 with result from task 2749 parsl.dataflow.dflow DEBUG: Task 2749 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6d4fd76e-1368-4851-a147-1b29a3e6028a_isrForCrosstalkSources_996957_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/6d4fd76e-1368-4851-a147-1b29a3e6028a_isrForCrosstalkSources_996957_51.stderr parsl.dataflow.dflow INFO: Task 2404 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ee8c597a297c06df9a70cdff8c8a1d7 with result from task 2404 parsl.dataflow.dflow DEBUG: Task 2404 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e78b924c-9b09-400a-aa6f-4411a9c43c28_isrForCrosstalkSources_996996_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e78b924c-9b09-400a-aa6f-4411a9c43c28_isrForCrosstalkSources_996996_55.stderr parsl.dataflow.dflow INFO: Task 2492 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4ed7ef5ea64020c2f280412e874b652 with result from task 2492 parsl.dataflow.dflow DEBUG: Task 2492 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/579844c0-081b-4b15-b94c-2e968bbce65b_isrForCrosstalkSources_996955_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/579844c0-081b-4b15-b94c-2e968bbce65b_isrForCrosstalkSources_996955_6.stderr parsl.dataflow.dflow INFO: Task 2485 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89dc1d8819db0bd1f612956ee6a0c190 with result from task 2485 parsl.dataflow.dflow DEBUG: Task 2485 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/07b53967-722f-47da-8b96-c776a6a98019_isrForCrosstalkSources_996991_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/07b53967-722f-47da-8b96-c776a6a98019_isrForCrosstalkSources_996991_8.stderr parsl.dataflow.dflow INFO: Task 2484 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17e05cd2f6819ee1f656f035b05e7aeb with result from task 2484 parsl.dataflow.dflow DEBUG: Task 2484 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/3c368c17-3eec-4892-b776-78198c59fa48_isrForCrosstalkSources_996956_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/3c368c17-3eec-4892-b776-78198c59fa48_isrForCrosstalkSources_996956_31.stderr parsl.dataflow.dflow INFO: Task 2370 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dc29d51a04c8c9a4641afac7d0df4293 with result from task 2370 parsl.dataflow.dflow DEBUG: Task 2370 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/27c25d35-9692-406f-b1bc-603452afb85c_isrForCrosstalkSources_997032_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/27c25d35-9692-406f-b1bc-603452afb85c_isrForCrosstalkSources_997032_36.stderr parsl.dataflow.dflow INFO: Task 2488 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 557385d7ec7d701bb43162b56b566195 with result from task 2488 parsl.dataflow.dflow DEBUG: Task 2488 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7ad86f58-710a-4c0f-b3f6-23b129cbde74_isrForCrosstalkSources_997008_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7ad86f58-710a-4c0f-b3f6-23b129cbde74_isrForCrosstalkSources_997008_17.stderr parsl.dataflow.dflow INFO: Task 2487 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ea46e1dc323ee0ba459732f68441a57 with result from task 2487 parsl.dataflow.dflow DEBUG: Task 2487 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/754cf6b7-4e54-4b7f-aeb2-23469d72e2ea_isrForCrosstalkSources_996967_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/754cf6b7-4e54-4b7f-aeb2-23469d72e2ea_isrForCrosstalkSources_996967_24.stderr parsl.dataflow.dflow INFO: Task 2508 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2a64ba4d851d76dafae9027295b7dc7 with result from task 2508 parsl.dataflow.dflow DEBUG: Task 2508 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/feac3256-5eba-4877-9b2a-c9d123437b7f_isrForCrosstalkSources_996953_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/feac3256-5eba-4877-9b2a-c9d123437b7f_isrForCrosstalkSources_996953_17.stderr parsl.dataflow.dflow INFO: Task 2490 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ce7ee62d3641dd2192cb02cd3496ab4 with result from task 2490 parsl.dataflow.dflow DEBUG: Task 2490 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/650b7206-afc9-4367-b9df-144f24bf23bd_isrForCrosstalkSources_996979_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/650b7206-afc9-4367-b9df-144f24bf23bd_isrForCrosstalkSources_996979_59.stderr parsl.dataflow.dflow INFO: Task 2641 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e5067b321eab9cd2019b3cb612aefcb5 with result from task 2641 parsl.dataflow.dflow DEBUG: Task 2641 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/fe37300a-59a9-4758-bd41-a83469830801_isrForCrosstalkSources_997000_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/fe37300a-59a9-4758-bd41-a83469830801_isrForCrosstalkSources_997000_20.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075064 PD 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 2510 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eec0cdff45e777b69e475d201dc504ba with result from task 2510 parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2510 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/764fe377-13be-424e-a011-5c09bb1e2cfd_isrForCrosstalkSources_996995_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/764fe377-13be-424e-a011-5c09bb1e2cfd_isrForCrosstalkSources_996995_39.stderr parsl.dataflow.dflow INFO: Task 2443 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c6fd650011383e93d7c66ae3df779bce with result from task 2443 parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2443 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e42547e2-847c-4e83-9451-c6fd89f5499e_isrForCrosstalkSources_996996_60.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/e42547e2-847c-4e83-9451-c6fd89f5499e_isrForCrosstalkSources_996996_60.stderr parsl.dataflow.dflow INFO: Task 2581 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe3f11765bae80c49793df5e6ee1f70a with result from task 2581 parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2581 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/eb4f0903-e5dc-4753-a2cd-fa64f31506b3_isrForCrosstalkSources_996978_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/eb4f0903-e5dc-4753-a2cd-fa64f31506b3_isrForCrosstalkSources_996978_2.stderr parsl.dataflow.dflow INFO: Task 2278 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56ae3a22b4efb3fb326ea57a98a3dcde with result from task 2278 parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:44 proc_lsst.multi:146 [INFO] found job 17075381 in provider checkpoint proc_lsst.multi INFO: found job 17075381 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2278 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/4bd41279-1372-4d0c-919c-17bfc99a50ef_isrForCrosstalkSources_996983_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/4bd41279-1372-4d0c-919c-17bfc99a50ef_isrForCrosstalkSources_996983_52.stderr parsl.dataflow.dflow INFO: Task 2610 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac7de8bb475d34519a65ce5c8cdeb9b3 with result from task 2610 parsl.dataflow.dflow DEBUG: Task 2610 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/777fa400-579e-4f6e-aa18-94b2a4e046ad_isrForCrosstalkSources_996957_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/777fa400-579e-4f6e-aa18-94b2a4e046ad_isrForCrosstalkSources_996957_23.stderr parsl.dataflow.dflow INFO: Task 2726 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8340b287bb8480f3bd5df217faed6a96 with result from task 2726 parsl.dataflow.dflow DEBUG: Task 2726 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/4ff17dc3-7166-4196-8b8f-961caad43808_isrForCrosstalkSources_997024_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/4ff17dc3-7166-4196-8b8f-961caad43808_isrForCrosstalkSources_997024_11.stderr parsl.dataflow.dflow INFO: Task 2628 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4cbe33823c24579e24aac254f9b18d5 with result from task 2628 parsl.dataflow.dflow DEBUG: Task 2628 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/977acd62-33c4-4826-a154-0a1cdf7e013d_isrForCrosstalkSources_996974_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/977acd62-33c4-4826-a154-0a1cdf7e013d_isrForCrosstalkSources_996974_4.stderr parsl.dataflow.dflow INFO: Task 2546 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96e6df47d14c05d02fb2f4354600556c with result from task 2546 parsl.dataflow.dflow DEBUG: Task 2546 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/1a76e343-5c9a-42d9-b71f-b66036f2a9c1_isrForCrosstalkSources_997031_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/1a76e343-5c9a-42d9-b71f-b66036f2a9c1_isrForCrosstalkSources_997031_53.stderr parsl.dataflow.dflow INFO: Task 2496 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ced802752026dcade0d825eec58bdc00 with result from task 2496 parsl.dataflow.dflow DEBUG: Task 2496 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e7e16adb-6898-4987-84ce-1ba6bf8772b6_isrForCrosstalkSources_997008_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e7e16adb-6898-4987-84ce-1ba6bf8772b6_isrForCrosstalkSources_997008_9.stderr parsl.dataflow.dflow INFO: Task 2725 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13b67740236c542e72b8d75e9ebf010e with result from task 2725 parsl.dataflow.dflow DEBUG: Task 2725 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1a4eff7c-6f00-43cb-b2b6-2ace3dd82665_isrForCrosstalkSources_996953_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/1a4eff7c-6f00-43cb-b2b6-2ace3dd82665_isrForCrosstalkSources_996953_3.stderr parsl.dataflow.dflow INFO: Task 2729 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e887e715bf460caa11f0cfcd90b4c08 with result from task 2729 parsl.dataflow.dflow DEBUG: Task 2729 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/263af9d7-a423-4438-af50-fee0740ba310_isrForCrosstalkSources_997012_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/263af9d7-a423-4438-af50-fee0740ba310_isrForCrosstalkSources_997012_15.stderr parsl.dataflow.dflow INFO: Task 2732 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1577a26ba61ecfa6eb40577e1ac19803 with result from task 2732 parsl.dataflow.dflow DEBUG: Task 2732 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8955b14b-70f4-43ed-9875-623e90292554_isrForCrosstalkSources_996973_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8955b14b-70f4-43ed-9875-623e90292554_isrForCrosstalkSources_996973_7.stderr parsl.dataflow.dflow INFO: Task 2599 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a68339f2f6eaad8cab8e4f19e0d73c43 with result from task 2599 parsl.dataflow.dflow DEBUG: Task 2599 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/4eb0951c-daf1-4ac1-9dff-f65fe4986bc5_isrForCrosstalkSources_996977_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/4eb0951c-daf1-4ac1-9dff-f65fe4986bc5_isrForCrosstalkSources_996977_40.stderr parsl.dataflow.dflow INFO: Task 2662 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f845264e025b13a9e01f42966f8ca8a3 with result from task 2662 parsl.dataflow.dflow DEBUG: Task 2662 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4f376097-7f40-44a7-b544-122def519df7_isrForCrosstalkSources_996997_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4f376097-7f40-44a7-b544-122def519df7_isrForCrosstalkSources_996997_15.stderr parsl.dataflow.dflow INFO: Task 2728 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9948f75b8cec83cbca49d53bc05034af with result from task 2728 parsl.dataflow.dflow DEBUG: Task 2728 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/aee5958f-6301-4f8f-a929-76b6c30ac830_isrForCrosstalkSources_996996_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/aee5958f-6301-4f8f-a929-76b6c30ac830_isrForCrosstalkSources_996996_43.stderr parsl.dataflow.dflow INFO: Task 2511 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cae6e1ab3c6d9127e4b4d7bef75e84a4 with result from task 2511 parsl.dataflow.dflow DEBUG: Task 2511 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/03731a2e-3cf8-434b-9fd9-5af968cb2539_isrForCrosstalkSources_996985_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/03731a2e-3cf8-434b-9fd9-5af968cb2539_isrForCrosstalkSources_996985_26.stderr parsl.dataflow.dflow INFO: Task 2503 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6e643e9be5aba110caeb714f269100c8 with result from task 2503 parsl.dataflow.dflow DEBUG: Task 2503 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/76183e15-1ecc-4def-8c97-4c7cecb44fc7_isrForCrosstalkSources_997008_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/76183e15-1ecc-4def-8c97-4c7cecb44fc7_isrForCrosstalkSources_997008_23.stderr parsl.dataflow.dflow INFO: Task 2733 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88851a6ee14ea3849b09bb490a3d47df with result from task 2733 parsl.dataflow.dflow DEBUG: Task 2733 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e478823b-a506-4fa8-ada8-01d1a074c769_isrForCrosstalkSources_997033_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e478823b-a506-4fa8-ada8-01d1a074c769_isrForCrosstalkSources_997033_38.stderr parsl.dataflow.dflow INFO: Task 2486 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 98e8749d43ce7b2817ebd51b02c4f563 with result from task 2486 parsl.dataflow.dflow DEBUG: Task 2486 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/a2a2e00a-1564-490a-8b5e-730478417679_isrForCrosstalkSources_997014_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/a2a2e00a-1564-490a-8b5e-730478417679_isrForCrosstalkSources_997014_23.stderr parsl.dataflow.dflow INFO: Task 2718 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d02bc6b59a16db575215ad268554a7d2 with result from task 2718 parsl.dataflow.dflow DEBUG: Task 2718 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e51fb5d3-2398-4460-9d5c-2f7ccee6a138_isrForCrosstalkSources_997022_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e51fb5d3-2398-4460-9d5c-2f7ccee6a138_isrForCrosstalkSources_997022_54.stderr parsl.dataflow.dflow INFO: Task 2734 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0b34a692d82a00577bb0e3a4a4fc5cf5 with result from task 2734 parsl.dataflow.dflow DEBUG: Task 2734 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ef252fea-f8e9-44b6-9563-155141a89ef9_isrForCrosstalkSources_997006_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ef252fea-f8e9-44b6-9563-155141a89ef9_isrForCrosstalkSources_997006_24.stderr parsl.dataflow.dflow INFO: Task 2736 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2e46325d27b577b2fdde2a1111bce84 with result from task 2736 parsl.dataflow.dflow DEBUG: Task 2736 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/169cda47-e1a0-4dcd-bf0a-f4323d085694_isrForCrosstalkSources_997031_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/169cda47-e1a0-4dcd-bf0a-f4323d085694_isrForCrosstalkSources_997031_15.stderr parsl.dataflow.dflow INFO: Task 2472 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d1f5fde84c553d3cd8393fec4e84b331 with result from task 2472 parsl.dataflow.dflow DEBUG: Task 2472 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/8ac482ed-aee0-4b71-b9d5-45d3b046e2d7_isrForCrosstalkSources_997015_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/8ac482ed-aee0-4b71-b9d5-45d3b046e2d7_isrForCrosstalkSources_997015_4.stderr parsl.dataflow.dflow INFO: Task 2727 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da55ca339f77f2b0b7bc19d80aa0ea56 with result from task 2727 parsl.dataflow.dflow DEBUG: Task 2727 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/2716d60b-7e3c-497a-b4d8-dff4d9b9f8f9_isrForCrosstalkSources_997021_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/2716d60b-7e3c-497a-b4d8-dff4d9b9f8f9_isrForCrosstalkSources_997021_3.stderr parsl.dataflow.dflow INFO: Task 2648 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 69eb14ef7fd749578c1f0397990c9cc5 with result from task 2648 parsl.dataflow.dflow DEBUG: Task 2648 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/166d257d-3a87-4515-8e09-5b03c5ce1a64_isrForCrosstalkSources_996974_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/166d257d-3a87-4515-8e09-5b03c5ce1a64_isrForCrosstalkSources_996974_19.stderr parsl.dataflow.dflow INFO: Task 2643 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82a280d4c4c8721ff4ff67abc73b9b04 with result from task 2643 parsl.dataflow.dflow DEBUG: Task 2643 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/5bd36215-6e00-4c3a-add0-b8ca8e61611f_isrForCrosstalkSources_997027_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/5bd36215-6e00-4c3a-add0-b8ca8e61611f_isrForCrosstalkSources_997027_9.stderr parsl.dataflow.dflow INFO: Task 2760 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54e4c1db8592c713464b0a97bd471ac6 with result from task 2760 parsl.dataflow.dflow DEBUG: Task 2760 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e1f69863-7fbd-4e54-84c6-7048c8c1d2b9_isrForCrosstalkSources_997001_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e1f69863-7fbd-4e54-84c6-7048c8c1d2b9_isrForCrosstalkSources_997001_53.stderr parsl.dataflow.dflow INFO: Task 2607 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ce6c129667384003e1c5d5e1f241bdc with result from task 2607 parsl.dataflow.dflow DEBUG: Task 2607 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cbe6dc4f-05da-4110-be4c-9278ae59b966_isrForCrosstalkSources_996956_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/cbe6dc4f-05da-4110-be4c-9278ae59b966_isrForCrosstalkSources_996956_57.stderr parsl.dataflow.dflow INFO: Task 2335 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c27747e1d1f16ae150c76a10eb3c05ad with result from task 2335 parsl.dataflow.dflow DEBUG: Task 2335 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/95c4b543-aec5-45a8-9c64-3f2d76468d9d_isrForCrosstalkSources_996986_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/95c4b543-aec5-45a8-9c64-3f2d76468d9d_isrForCrosstalkSources_996986_50.stderr parsl.dataflow.dflow INFO: Task 2471 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 781b5231ea0a7a00f7e4a6cc34a124ea with result from task 2471 parsl.dataflow.dflow DEBUG: Task 2471 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/38f02035-7cc3-4c76-8368-231bce1c5ac3_isrForCrosstalkSources_997029_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/38f02035-7cc3-4c76-8368-231bce1c5ac3_isrForCrosstalkSources_997029_32.stderr parsl.dataflow.dflow INFO: Task 2752 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3af71f789396e53bbe2804451214ef92 with result from task 2752 parsl.dataflow.dflow DEBUG: Task 2752 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0c228d1d-0875-4cfa-9616-fd1a21fcdd6f_isrForCrosstalkSources_996980_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0c228d1d-0875-4cfa-9616-fd1a21fcdd6f_isrForCrosstalkSources_996980_4.stderr parsl.dataflow.dflow INFO: Task 2175 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5967c2fc6936dee4ddeffa229361e6f1 with result from task 2175 parsl.dataflow.dflow DEBUG: Task 2175 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ec0da860-9176-4aeb-afb7-e375ca3c5f74_isrForCrosstalkSources_997028_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ec0da860-9176-4aeb-afb7-e375ca3c5f74_isrForCrosstalkSources_997028_62.stderr parsl.dataflow.dflow INFO: Task 2285 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46315a5a0ac1973ba56dcb9dd8d98d1a with result from task 2285 parsl.dataflow.dflow DEBUG: Task 2285 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/05270c06-1a63-4414-94c5-c0d6bb7718e8_isrForCrosstalkSources_996997_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/05270c06-1a63-4414-94c5-c0d6bb7718e8_isrForCrosstalkSources_996997_57.stderr parsl.dataflow.dflow INFO: Task 2769 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f0fc418e1900e0e685712650d94d1a24 with result from task 2769 parsl.dataflow.dflow DEBUG: Task 2769 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/7290379d-f020-4986-b3ec-d6efe8157972_isrForCrosstalkSources_996957_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/7290379d-f020-4986-b3ec-d6efe8157972_isrForCrosstalkSources_996957_12.stderr parsl.dataflow.dflow INFO: Task 2784 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b530ed491f3bcac52c55953646510ba with result from task 2784 parsl.dataflow.dflow DEBUG: Task 2784 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/51fb3834-996f-445b-a482-417ea9222335_isrForCrosstalkSources_997030_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/51fb3834-996f-445b-a482-417ea9222335_isrForCrosstalkSources_997030_19.stderr parsl.dataflow.dflow INFO: Task 2730 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fbb196a9d02714a79436e71011d0349d with result from task 2730 parsl.dataflow.dflow DEBUG: Task 2730 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/3c2b15f3-ea07-4a8f-9f66-3f7d80b8f056_isrForCrosstalkSources_997017_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/3c2b15f3-ea07-4a8f-9f66-3f7d80b8f056_isrForCrosstalkSources_997017_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2650 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry c57ceb62eea32c6450f21ede94220a0f with result from task 2650 parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2650 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/67a6ea98-d648-4be9-b9fd-5277fb6463af_isrForCrosstalkSources_997031_38.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/67a6ea98-d648-4be9-b9fd-5277fb6463af_isrForCrosstalkSources_997031_38.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2774 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry d69c9255f70d729d8d5d0330468b6d1b with result from task 2774 parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2774 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/2dd7a00a-1fdb-452b-8e11-32e888e49c6d_isrForCrosstalkSources_996975_48.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/2dd7a00a-1fdb-452b-8e11-32e888e49c6d_isrForCrosstalkSources_996975_48.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:46 proc_lsst.multi:146 [INFO] found job 17075382 in provider checkpoint proc_lsst.multi INFO: found job 17075382 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2550 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f30b9bf5a4add14b0fa132c39e88842 with result from task 2550 parsl.dataflow.dflow DEBUG: Task 2550 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a4fcdb89-f046-42f7-878a-9cdf558f28c5_isrForCrosstalkSources_997002_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/a4fcdb89-f046-42f7-878a-9cdf558f28c5_isrForCrosstalkSources_997002_51.stderr parsl.dataflow.dflow INFO: Task 2701 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9abb7fd30f3bcb64cffc624472fc07b3 with result from task 2701 parsl.dataflow.dflow DEBUG: Task 2701 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/29d77549-47aa-41b5-93e5-f2ffb86bb66e_isrForCrosstalkSources_996987_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/29d77549-47aa-41b5-93e5-f2ffb86bb66e_isrForCrosstalkSources_996987_57.stderr parsl.dataflow.dflow INFO: Task 2539 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ea89b6f8cafbcf803abd19eff21ab42 with result from task 2539 parsl.dataflow.dflow DEBUG: Task 2539 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ded3e5c8-5caa-4fdd-9e3f-e245a7424138_isrForCrosstalkSources_996986_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ded3e5c8-5caa-4fdd-9e3f-e245a7424138_isrForCrosstalkSources_996986_30.stderr parsl.dataflow.dflow INFO: Task 2494 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18d9406b38d8d348243849f7bb4fee0a with result from task 2494 parsl.dataflow.dflow DEBUG: Task 2494 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8ca2649b-6d3e-4f40-9270-a4f90741c4f9_isrForCrosstalkSources_996976_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8ca2649b-6d3e-4f40-9270-a4f90741c4f9_isrForCrosstalkSources_996976_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2328 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry d9865362fe8c01578c7b8ea934237230 with result from task 2328 parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2328 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/322afac2-ec5c-4d41-a621-a8fbf199d08d_isrForCrosstalkSources_996984_50.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/322afac2-ec5c-4d41-a621-a8fbf199d08d_isrForCrosstalkSources_996984_50.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2789 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry d0647137c55b097fd830c3dd15e43ec3 with result from task 2789 parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2789 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/ea7008d9-e15c-46c9-b9bd-cf0f94410fb7_isrForCrosstalkSources_997017_46.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/ea7008d9-e15c-46c9-b9bd-cf0f94410fb7_isrForCrosstalkSources_997017_46.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2585 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 8f5f7c95241802ea657f286e42eb9b28 with result from task 2585 parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2585 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c32f6c55-d22f-4c91-abb1-c81bee90dce7_isrForCrosstalkSources_996991_42.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/c32f6c55-d22f-4c91-abb1-c81bee90dce7_isrForCrosstalkSources_996991_42.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2601 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 921d1152b417dba4d8865aba29aaa0a8 with result from task 2601 parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2601 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/08a0bd20-d045-454e-a692-a2e5d67e1785_isrForCrosstalkSources_996994_16.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/08a0bd20-d045-454e-a692-a2e5d67e1785_isrForCrosstalkSources_996994_16.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2455 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry e54fd4d166d594b302114e00c0b67a43 with result from task 2455 parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2455 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0471d374-d912-4f99-8e28-4aa0d131415a_isrForCrosstalkSources_996966_61.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0471d374-d912-4f99-8e28-4aa0d131415a_isrForCrosstalkSources_996966_61.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:46 proc_lsst.multi:146 [INFO] found job 17075384 in provider checkpoint proc_lsst.multi INFO: found job 17075384 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2483 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08646ce45971d1711841e7af7d9d5be4 with result from task 2483 parsl.dataflow.dflow DEBUG: Task 2483 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b911106d-ee8a-4703-b84d-81a0937a869f_isrForCrosstalkSources_997006_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b911106d-ee8a-4703-b84d-81a0937a869f_isrForCrosstalkSources_997006_15.stderr parsl.dataflow.dflow INFO: Task 2625 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1866f787dc64aa45cf11e63a7985fe1e with result from task 2625 parsl.dataflow.dflow DEBUG: Task 2625 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e19ecad4-2960-4595-93c0-7365be2795a5_isrForCrosstalkSources_996995_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e19ecad4-2960-4595-93c0-7365be2795a5_isrForCrosstalkSources_996995_4.stderr parsl.dataflow.dflow INFO: Task 2451 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57f1fd5080077486d600de957266f54e with result from task 2451 parsl.dataflow.dflow DEBUG: Task 2451 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/17fd1367-aba9-4b53-a984-b540f70c6369_isrForCrosstalkSources_997009_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/17fd1367-aba9-4b53-a984-b540f70c6369_isrForCrosstalkSources_997009_59.stderr parsl.dataflow.dflow INFO: Task 2639 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25ca26ad255e75ef283eb4a230b2371e with result from task 2639 parsl.dataflow.dflow DEBUG: Task 2639 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d62cabdd-6cb6-4bf5-9fc5-c70d7167e314_isrForCrosstalkSources_997025_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d62cabdd-6cb6-4bf5-9fc5-c70d7167e314_isrForCrosstalkSources_997025_6.stderr parsl.dataflow.dflow INFO: Task 2751 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9f7f6befd9b881b86c9e11a374f4d7fc with result from task 2751 parsl.dataflow.dflow DEBUG: Task 2751 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/180fe417-d6ab-4562-bc09-0a4083589a24_isrForCrosstalkSources_997026_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/180fe417-d6ab-4562-bc09-0a4083589a24_isrForCrosstalkSources_997026_46.stderr parsl.dataflow.dflow INFO: Task 2646 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d5be44c45e9d753e2d9791eeb88ed625 with result from task 2646 parsl.dataflow.dflow DEBUG: Task 2646 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2ec96e00-1d73-41b7-be7d-2de9955a3547_isrForCrosstalkSources_996963_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2ec96e00-1d73-41b7-be7d-2de9955a3547_isrForCrosstalkSources_996963_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 2573 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 8acada2d4058c4a2142aa8dc4ecba13a with result from task 2573 parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2573 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ac4d8e26-96f9-4c1c-82f9-2586505d6024_isrForCrosstalkSources_997002_43.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ac4d8e26-96f9-4c1c-82f9-2586505d6024_isrForCrosstalkSources_997002_43.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2518 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 15c94904ba770187f7956282605c5691 with result from task 2518 parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2518 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/413ab978-1b98-4b0f-9b9b-fabd1a0f672c_isrForCrosstalkSources_997026_49.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/413ab978-1b98-4b0f-9b9b-fabd1a0f672c_isrForCrosstalkSources_997026_49.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:47 proc_lsst.multi:146 [INFO] found job 17075385 in provider checkpoint proc_lsst.multi INFO: found job 17075385 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2435 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f158f19292d8a1d18cff880725cc0775 with result from task 2435 parsl.dataflow.dflow DEBUG: Task 2435 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c4ff4b8c-64e6-4f7a-9ad0-a30607d641c4_isrForCrosstalkSources_997015_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c4ff4b8c-64e6-4f7a-9ad0-a30607d641c4_isrForCrosstalkSources_997015_58.stderr parsl.dataflow.dflow INFO: Task 2619 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3acdfbcea847a749888ccfc8f1899227 with result from task 2619 parsl.dataflow.dflow DEBUG: Task 2619 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/efeb379a-d01a-4841-86fd-46ad34c38012_isrForCrosstalkSources_996965_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/efeb379a-d01a-4841-86fd-46ad34c38012_isrForCrosstalkSources_996965_58.stderr parsl.dataflow.dflow INFO: Task 2653 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1932a311ebe0e57f1b60c5f2cb81ceff with result from task 2653 parsl.dataflow.dflow DEBUG: Task 2653 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/40265502-ed2e-407d-9e24-d8867da1bf25_isrForCrosstalkSources_996963_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/40265502-ed2e-407d-9e24-d8867da1bf25_isrForCrosstalkSources_996963_36.stderr parsl.dataflow.dflow INFO: Task 2781 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d16992d30d8e5254cda8b92889c64062 with result from task 2781 parsl.dataflow.dflow DEBUG: Task 2781 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Standard output for task 2781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/a385a6e7-cc7d-4cb6-8598-d69deefe602a_isrForCrosstalkSources_997016_14.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/a385a6e7-cc7d-4cb6-8598-d69deefe602a_isrForCrosstalkSources_997016_14.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2679 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 9b811e8a83b9abb04a4027b777593a11 with result from task 2679 parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2679 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/831b72d9-39dc-4672-850e-42e0df5a0ff6_isrForCrosstalkSources_997023_24.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/831b72d9-39dc-4672-850e-42e0df5a0ff6_isrForCrosstalkSources_997023_24.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2808 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 7cb399f9229fc12aa0ee2e435d9898b6 with result from task 2808 parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2808 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f1e2caa8-7242-4fa0-bf49-63c64503c99a_isrForCrosstalkSources_996956_26.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/f1e2caa8-7242-4fa0-bf49-63c64503c99a_isrForCrosstalkSources_996956_26.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2708 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 6f8f6117bbffb1c52b2340a1cbabaed5 with result from task 2708 parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2708 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/11753d3f-9ddd-46fc-ab83-7e6bd98f0c4f_isrForCrosstalkSources_996983_51.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/11753d3f-9ddd-46fc-ab83-7e6bd98f0c4f_isrForCrosstalkSources_996983_51.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2786 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 9b9747c878e3dac62f5be78a8b3bccfc with result from task 2786 parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:49 proc_lsst.multi:146 [INFO] found job 17075386 in provider checkpoint proc_lsst.multi INFO: found job 17075386 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2786 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/795d9a79-d9b4-4ca0-aca6-2b1ae706e10b_isrForCrosstalkSources_996992_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/795d9a79-d9b4-4ca0-aca6-2b1ae706e10b_isrForCrosstalkSources_996992_28.stderr parsl.dataflow.dflow INFO: Task 2588 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 632350d8bbd3bcdcfa045b48b0a00c05 with result from task 2588 parsl.dataflow.dflow DEBUG: Task 2588 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7445fe4f-e616-4127-90a1-14cbea6e3871_isrForCrosstalkSources_996973_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7445fe4f-e616-4127-90a1-14cbea6e3871_isrForCrosstalkSources_996973_46.stderr parsl.dataflow.dflow INFO: Task 2627 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf2f7463a019afa3581bddfb9f42aa2e with result from task 2627 parsl.dataflow.dflow DEBUG: Task 2627 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/992ef550-3514-42f0-8a8d-0e991a0c2ae9_isrForCrosstalkSources_996963_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/992ef550-3514-42f0-8a8d-0e991a0c2ae9_isrForCrosstalkSources_996963_37.stderr parsl.dataflow.dflow INFO: Task 2814 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c9b127c6e18e363569b4950483ed3ef with result from task 2814 parsl.dataflow.dflow DEBUG: Task 2814 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/be469688-d600-4e38-92dc-84d3f1ce5508_isrForCrosstalkSources_997017_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/be469688-d600-4e38-92dc-84d3f1ce5508_isrForCrosstalkSources_997017_24.stderr parsl.dataflow.dflow INFO: Task 2762 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a9525fc261cf499b1260417285406e5c with result from task 2762 parsl.dataflow.dflow DEBUG: Task 2762 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/bada876c-d775-4b4b-a2f2-f2953a786b18_isrForCrosstalkSources_996997_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/bada876c-d775-4b4b-a2f2-f2953a786b18_isrForCrosstalkSources_996997_12.stderr parsl.dataflow.dflow INFO: Task 2684 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 611133bc1f74d4d8d6584f9b812f6fd3 with result from task 2684 parsl.dataflow.dflow DEBUG: Task 2684 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/feec94d9-faa8-4a19-b3a6-522d92eef2cb_isrForCrosstalkSources_996996_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/feec94d9-faa8-4a19-b3a6-522d92eef2cb_isrForCrosstalkSources_996996_23.stderr parsl.dataflow.dflow INFO: Task 2598 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fcb3c85a46fba0d67b1def8a35a7155 with result from task 2598 parsl.dataflow.dflow DEBUG: Task 2598 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/3916e32b-66fa-47df-81da-7893a58702d4_isrForCrosstalkSources_997027_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/3916e32b-66fa-47df-81da-7893a58702d4_isrForCrosstalkSources_997027_41.stderr parsl.dataflow.dflow INFO: Task 2530 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d177dd68143a2ec83baf68c91e94e1d with result from task 2530 parsl.dataflow.dflow DEBUG: Task 2530 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6ca57d18-4a67-4396-90d2-64dcd0f5a348_isrForCrosstalkSources_997023_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6ca57d18-4a67-4396-90d2-64dcd0f5a348_isrForCrosstalkSources_997023_16.stderr parsl.dataflow.dflow INFO: Task 2525 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7edad8c4ec3ba5ecc70d1de6762531d with result from task 2525 parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2525 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f9fbd15a-0dc8-4636-94d4-f6a4dfd58dfc_isrForCrosstalkSources_997015_10.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f9fbd15a-0dc8-4636-94d4-f6a4dfd58dfc_isrForCrosstalkSources_997015_10.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2688 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry c7b400f22ffc471fc3c6c8f97a2c41ef with result from task 2688 parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2688 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/42e12d3a-8818-4096-85f1-1b4a0691bf2f_isrForCrosstalkSources_996970_9.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/42e12d3a-8818-4096-85f1-1b4a0691bf2f_isrForCrosstalkSources_996970_9.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2612 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 956b93814802fdb7b3ce0eaec758d77c with result from task 2612 parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2612 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7a53f2f4-e351-4b3d-908e-df81eb086838_isrForCrosstalkSources_997013_38.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/7a53f2f4-e351-4b3d-908e-df81eb086838_isrForCrosstalkSources_997013_38.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2358 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:50 proc_lsst.multi:146 [INFO] found job 17075387 in provider checkpoint proc_lsst.multi INFO: found job 17075387 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.memoization DEBUG: Storing app cache entry 7a5b0214686010431162432c99d27af5 with result from task 2358 parsl.dataflow.dflow DEBUG: Task 2358 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/55165652-a43e-4b59-b606-2e8b750ee0aa_isrForCrosstalkSources_996988_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/55165652-a43e-4b59-b606-2e8b750ee0aa_isrForCrosstalkSources_996988_61.stderr parsl.dataflow.dflow INFO: Task 2821 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b269bfd1776323dcc1422a155bf6e4de with result from task 2821 parsl.dataflow.dflow DEBUG: Task 2821 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7791c427-db51-4a84-bbd5-596ab4e172d3_isrForCrosstalkSources_997015_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7791c427-db51-4a84-bbd5-596ab4e172d3_isrForCrosstalkSources_997015_33.stderr parsl.dataflow.dflow INFO: Task 2703 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd6da7382bcd0963d9befdc0b86759e3 with result from task 2703 parsl.dataflow.dflow DEBUG: Task 2703 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/e3c8bf69-2e31-4b03-a1fb-f80c2348ebd4_isrForCrosstalkSources_996990_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/e3c8bf69-2e31-4b03-a1fb-f80c2348ebd4_isrForCrosstalkSources_996990_2.stderr parsl.dataflow.dflow INFO: Task 2618 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b557fc8434f2adb1a0cc1556c3c0314 with result from task 2618 parsl.dataflow.dflow DEBUG: Task 2618 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/71ec74cf-45d9-436b-a681-cbf6b314a6e0_isrForCrosstalkSources_997030_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/71ec74cf-45d9-436b-a681-cbf6b314a6e0_isrForCrosstalkSources_997030_11.stderr parsl.dataflow.dflow INFO: Task 2735 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b159a75f9c4dc61cd267aea649c82a5 with result from task 2735 parsl.dataflow.dflow DEBUG: Task 2735 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/6671c8aa-e5ae-4b16-8cee-21658ece6c1c_isrForCrosstalkSources_997019_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/6671c8aa-e5ae-4b16-8cee-21658ece6c1c_isrForCrosstalkSources_997019_50.stderr parsl.dataflow.dflow INFO: Task 2810 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a552668bf1daf17aa17887f86f0bb60b with result from task 2810 parsl.dataflow.dflow DEBUG: Task 2810 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b4166ea4-f80d-4cde-9267-16e6470b5fbe_isrForCrosstalkSources_997004_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b4166ea4-f80d-4cde-9267-16e6470b5fbe_isrForCrosstalkSources_997004_56.stderr parsl.dataflow.dflow INFO: Task 2497 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec9400d761c239592411ab4ded60f28c with result from task 2497 parsl.dataflow.dflow DEBUG: Task 2497 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/fe80ea70-4405-417a-8655-6b8650e43324_isrForCrosstalkSources_996988_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/fe80ea70-4405-417a-8655-6b8650e43324_isrForCrosstalkSources_996988_35.stderr parsl.dataflow.dflow INFO: Task 2600 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cad85ef95f4b0fb40440a01405b8bd44 with result from task 2600 parsl.dataflow.dflow DEBUG: Task 2600 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/036606b3-a2a2-40d6-b183-15750e3e3b44_isrForCrosstalkSources_996953_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/036606b3-a2a2-40d6-b183-15750e3e3b44_isrForCrosstalkSources_996953_11.stderr parsl.dataflow.dflow INFO: Task 2544 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a0e4e9895b549704c0075b11bd86894f with result from task 2544 parsl.dataflow.dflow DEBUG: Task 2544 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b9f8fc73-64cf-448b-86a4-ea68ad349bfe_isrForCrosstalkSources_997013_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/b9f8fc73-64cf-448b-86a4-ea68ad349bfe_isrForCrosstalkSources_997013_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:50 proc_lsst.multi:146 [INFO] found job 17075388 in provider checkpoint proc_lsst.multi INFO: found job 17075388 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2670 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry adbed8e0701bd8d001ed939f0fb44bb0 with result from task 2670 parsl.dataflow.dflow DEBUG: Task 2670 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a09c2209-d2ea-452d-886f-5dcfbeff99d7_isrForCrosstalkSources_997011_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a09c2209-d2ea-452d-886f-5dcfbeff99d7_isrForCrosstalkSources_997011_25.stderr parsl.dataflow.dflow INFO: Task 2604 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe616df3f411c7a749d758042bc18ac0 with result from task 2604 parsl.dataflow.dflow DEBUG: Task 2604 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/6969d4a8-0d67-40ae-a123-46b20f323c78_isrForCrosstalkSources_996972_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/6969d4a8-0d67-40ae-a123-46b20f323c78_isrForCrosstalkSources_996972_13.stderr parsl.dataflow.dflow INFO: Task 2657 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4fa7cb47bf6bffda8a517eaa41be0af with result from task 2657 parsl.dataflow.dflow DEBUG: Task 2657 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f4e241e2-6e12-4283-bf75-56d2c133aed3_isrForCrosstalkSources_996987_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f4e241e2-6e12-4283-bf75-56d2c133aed3_isrForCrosstalkSources_996987_17.stderr parsl.dataflow.dflow INFO: Task 2498 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 653f69e8d691e799a634422c555e0bb8 with result from task 2498 parsl.dataflow.dflow DEBUG: Task 2498 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/db8f6832-f73c-40ab-872e-d3a12bde39a6_isrForCrosstalkSources_997025_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/db8f6832-f73c-40ab-872e-d3a12bde39a6_isrForCrosstalkSources_997025_55.stderr parsl.dataflow.dflow INFO: Task 2638 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5df2fd4f95c17a9cee2062b27b49bd14 with result from task 2638 parsl.dataflow.dflow DEBUG: Task 2638 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/db90e1b1-a7fb-4bbb-8cf6-1c4f45d4e4d2_isrForCrosstalkSources_997014_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/db90e1b1-a7fb-4bbb-8cf6-1c4f45d4e4d2_isrForCrosstalkSources_997014_19.stderr parsl.dataflow.dflow INFO: Task 2554 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f2e75a52dbded46f592bae290f15e0e0 with result from task 2554 parsl.dataflow.dflow DEBUG: Task 2554 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/feec21b0-a4fd-4273-8390-95e3c42face1_isrForCrosstalkSources_996965_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/feec21b0-a4fd-4273-8390-95e3c42face1_isrForCrosstalkSources_996965_45.stderr parsl.dataflow.dflow INFO: Task 2606 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e47f6fb9f62d6c646496af21f6a1bbd0 with result from task 2606 parsl.dataflow.dflow DEBUG: Task 2606 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/35e2ac81-78d7-4373-9a0d-71ad0505a66f_isrForCrosstalkSources_996995_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/35e2ac81-78d7-4373-9a0d-71ad0505a66f_isrForCrosstalkSources_996995_26.stderr parsl.dataflow.dflow INFO: Task 2534 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.memoization DEBUG: Storing app cache entry 8732bc1e0015fadb18f56dfac6db5155 with result from task 2534 parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2534 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/62f2341b-bcd8-4701-876f-b8ea82df081e_isrForCrosstalkSources_997017_13.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/62f2341b-bcd8-4701-876f-b8ea82df081e_isrForCrosstalkSources_997017_13.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2794 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry fac549a60a6990685f7d681efedaa825 with result from task 2794 parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2794 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/344e9d0f-b558-481b-8093-4476711a0584_isrForCrosstalkSources_996991_46.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/344e9d0f-b558-481b-8093-4476711a0584_isrForCrosstalkSources_996991_46.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2522 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 853e196ca5f09a982e6d37833bb6c816 with result from task 2522 parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2522 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/30b477fa-457c-478d-9ceb-a91664ba836a_isrForCrosstalkSources_997023_35.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/30b477fa-457c-478d-9ceb-a91664ba836a_isrForCrosstalkSources_997023_35.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:51 proc_lsst.multi:146 [INFO] found job 17075392 in provider checkpoint proc_lsst.multi INFO: found job 17075392 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2797 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d0230ce4cc1c4db536faded9b1d5d0c with result from task 2797 parsl.dataflow.dflow DEBUG: Task 2797 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/73bac0a0-24d6-4897-a51e-b5eda58281b4_isrForCrosstalkSources_996965_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/73bac0a0-24d6-4897-a51e-b5eda58281b4_isrForCrosstalkSources_996965_19.stderr parsl.dataflow.dflow INFO: Task 2816 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6e1d9a8a155fef1ac50b92bc90e31a4f with result from task 2816 parsl.dataflow.dflow DEBUG: Task 2816 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/cf650ffd-45d6-4704-a963-60a7b5ce0eb4_isrForCrosstalkSources_996969_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/cf650ffd-45d6-4704-a963-60a7b5ce0eb4_isrForCrosstalkSources_996969_53.stderr parsl.dataflow.dflow INFO: Task 2517 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5aa081cfc8a0cd923c3786e53ffed8af with result from task 2517 parsl.dataflow.dflow DEBUG: Task 2517 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c020edf7-b683-4a2a-8f84-a174fa2dee2c_isrForCrosstalkSources_996962_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c020edf7-b683-4a2a-8f84-a174fa2dee2c_isrForCrosstalkSources_996962_48.stderr parsl.dataflow.dflow INFO: Task 2272 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 87c85341614f239082890b9717442ae3 with result from task 2272 parsl.dataflow.dflow DEBUG: Task 2272 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/edb9ed54-5eb0-4f46-87cf-1a252a24bed2_isrForCrosstalkSources_996980_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/edb9ed54-5eb0-4f46-87cf-1a252a24bed2_isrForCrosstalkSources_996980_52.stderr parsl.dataflow.dflow INFO: Task 2799 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 436901567ae10a704f262a02e13baf16 with result from task 2799 parsl.dataflow.dflow DEBUG: Task 2799 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/49bc34a5-e473-465c-9cac-8ec26f8413ab_isrForCrosstalkSources_997013_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/49bc34a5-e473-465c-9cac-8ec26f8413ab_isrForCrosstalkSources_997013_3.stderr parsl.dataflow.dflow INFO: Task 2747 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe4577e91a48a3a5bb6fbef87ff6b7c6 with result from task 2747 parsl.dataflow.dflow DEBUG: Task 2747 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Standard output for task 2747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e7102f36-e8b8-4f1f-b687-8577ab974bea_isrForCrosstalkSources_997033_20.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e7102f36-e8b8-4f1f-b687-8577ab974bea_isrForCrosstalkSources_997033_20.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2624 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 51e15a2269e5778231795f03cbb0b936 with result from task 2624 parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2624 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f79804a0-b1e6-46b1-a14c-1fffcd630cbe_isrForCrosstalkSources_996973_11.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/f79804a0-b1e6-46b1-a14c-1fffcd630cbe_isrForCrosstalkSources_996973_11.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2689 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 1ca4c857a71580e67159995843be2180 with result from task 2689 parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2689 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/387b9a35-7c24-4f83-bdb4-edaff4ad47ce_isrForCrosstalkSources_996965_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/387b9a35-7c24-4f83-bdb4-edaff4ad47ce_isrForCrosstalkSources_996965_8.stderr parsl.dataflow.dflow INFO: Task 2615 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 036c4fc21f3470ffc54da106a2cdfba7 with result from task 2615 parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2615 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dde14437-5b34-4979-af4e-a1fdfa93f649_isrForCrosstalkSources_997001_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dde14437-5b34-4979-af4e-a1fdfa93f649_isrForCrosstalkSources_997001_19.stderr parsl.dataflow.dflow INFO: Task 2523 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e357cd7e4622c7a6aebe1561ca828ae6 with result from task 2523 parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2523 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/bc53c342-b5a9-47dc-9c69-3c24a26100d5_isrForCrosstalkSources_997011_22.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/bc53c342-b5a9-47dc-9c69-3c24a26100d5_isrForCrosstalkSources_997011_22.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2575 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 0dbb0b34537f629ab2b165cdeaaaddb2 with result from task 2575 parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2575 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/39d1fc9f-93aa-4602-b8e4-741fa167910e_isrForCrosstalkSources_997008_33.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/39d1fc9f-93aa-4602-b8e4-741fa167910e_isrForCrosstalkSources_997008_33.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2790 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 70d943684594990f5c2ea4c0bf60a5b2 with result from task 2790 parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2790 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/00f23fae-8122-4a87-b78b-48f88067f203_isrForCrosstalkSources_996996_59.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/00f23fae-8122-4a87-b78b-48f88067f203_isrForCrosstalkSources_996996_59.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2827 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 6a129f392900a6aadb1989701d60fc74 with result from task 2827 parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2827 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/000d4ea1-3e3c-4ef8-b2da-eac9ae1ccf63_isrForCrosstalkSources_996992_45.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/000d4ea1-3e3c-4ef8-b2da-eac9ae1ccf63_isrForCrosstalkSources_996992_45.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2499 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 6b2a17db9100e636863fac6625bbb1ed with result from task 2499 parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2499 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/73648851-7089-4d14-bb8c-324b585fd67b_isrForCrosstalkSources_996984_4.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/73648851-7089-4d14-bb8c-324b585fd67b_isrForCrosstalkSources_996984_4.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2812 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry ef1071854e0e33bfb9188c6532ecca9b with result from task 2812 parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2812 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/3104352d-e608-42dd-8345-33e886562fec_isrForCrosstalkSources_996954_41.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/3104352d-e608-42dd-8345-33e886562fec_isrForCrosstalkSources_996954_41.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:52 proc_lsst.multi:146 [INFO] found job 17075394 in provider checkpoint proc_lsst.multi INFO: found job 17075394 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2824 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6758d5dd18ea7842cc4c7681440b66fe with result from task 2824 parsl.dataflow.dflow DEBUG: Task 2824 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/34ce65a2-210f-4646-a7b2-78357a9114dc_isrForCrosstalkSources_996975_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/34ce65a2-210f-4646-a7b2-78357a9114dc_isrForCrosstalkSources_996975_40.stderr parsl.dataflow.dflow INFO: Task 2580 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29a4fd3fbab5faebd6534670fb48cb5a with result from task 2580 parsl.dataflow.dflow DEBUG: Task 2580 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/8afba572-5a69-417e-96ba-866893c0b569_isrForCrosstalkSources_996964_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/8afba572-5a69-417e-96ba-866893c0b569_isrForCrosstalkSources_996964_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2547 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 15896911c0a8c32d74fafffb55b70315 with result from task 2547 parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2547 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/71fd6127-99fa-4335-a4f5-f8404aa626cf_isrForCrosstalkSources_997000_56.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/71fd6127-99fa-4335-a4f5-f8404aa626cf_isrForCrosstalkSources_997000_56.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2312 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 557b8a28098c9ea209529d0aea414a97 with result from task 2312 parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2312 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7610221a-6fb0-4df4-bc92-22b22fb34a73_isrForCrosstalkSources_997033_30.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/7610221a-6fb0-4df4-bc92-22b22fb34a73_isrForCrosstalkSources_997033_30.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2837 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry a72d3cb13f2d40b4a005cad0dfdcc6a1 with result from task 2837 parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2837 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/95f1616e-e8c3-4f8d-9f9e-e6af59b3cab4_isrForCrosstalkSources_996998_9.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/95f1616e-e8c3-4f8d-9f9e-e6af59b3cab4_isrForCrosstalkSources_996998_9.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2664 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 49dded4e1c007a156fa24bc51d06f154 with result from task 2664 parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2664 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e97bb3e8-30b3-4b27-ae33-272ce2b069dd_isrForCrosstalkSources_996968_2.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e97bb3e8-30b3-4b27-ae33-272ce2b069dd_isrForCrosstalkSources_996968_2.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2704 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 304727c01eff49cfe9be59f80a4fdaff with result from task 2704 parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2704 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/28771f35-aff3-4d0a-83e1-9eb8fdb8dbcc_isrForCrosstalkSources_997018_47.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/28771f35-aff3-4d0a-83e1-9eb8fdb8dbcc_isrForCrosstalkSources_997018_47.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2839 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:53 proc_lsst.multi:146 [INFO] found job 17075399 in provider checkpoint parsl.dataflow.memoization DEBUG: Storing app cache entry adacd6c06f53f905c9d04ae6716970e2 with result from task 2839 proc_lsst.multi INFO: found job 17075399 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2839 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/294da685-f945-4704-90a4-7274aeec8f21_isrForCrosstalkSources_996953_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/294da685-f945-4704-90a4-7274aeec8f21_isrForCrosstalkSources_996953_35.stderr parsl.dataflow.dflow INFO: Task 2746 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b596a80cb782ed1003e5a7f74807d90 with result from task 2746 parsl.dataflow.dflow DEBUG: Task 2746 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/c3b23665-8dc2-40f5-9f27-a3f2fbd63d43_isrForCrosstalkSources_996964_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/c3b23665-8dc2-40f5-9f27-a3f2fbd63d43_isrForCrosstalkSources_996964_44.stderr parsl.dataflow.dflow INFO: Task 2818 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4d364dc9780a4fba85f1b5243337135a with result from task 2818 parsl.dataflow.dflow DEBUG: Task 2818 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f8f8091c-a2aa-4554-8fd0-fa5e43b49c2e_isrForCrosstalkSources_997016_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f8f8091c-a2aa-4554-8fd0-fa5e43b49c2e_isrForCrosstalkSources_997016_15.stderr parsl.dataflow.dflow INFO: Task 2773 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6af409b08a0b6b6a23ce5c6b0c5a5ae with result from task 2773 parsl.dataflow.dflow DEBUG: Task 2773 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/89b10738-4811-43ad-883a-066c26af2843_isrForCrosstalkSources_996997_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/89b10738-4811-43ad-883a-066c26af2843_isrForCrosstalkSources_996997_40.stderr parsl.dataflow.dflow INFO: Task 2631 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53b7512a2595fcea9d0a92990268f2ee with result from task 2631 parsl.dataflow.dflow DEBUG: Task 2631 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/53027b6f-3063-4644-b8a5-7d0ffb5d906f_isrForCrosstalkSources_996986_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/53027b6f-3063-4644-b8a5-7d0ffb5d906f_isrForCrosstalkSources_996986_7.stderr parsl.dataflow.dflow INFO: Task 2817 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c01cb8595b0e42e6bf01ff8bcf9ba84 with result from task 2817 parsl.dataflow.dflow DEBUG: Task 2817 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/144029fa-a480-4c4c-aa18-0eed805a64a4_isrForCrosstalkSources_996992_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/144029fa-a480-4c4c-aa18-0eed805a64a4_isrForCrosstalkSources_996992_5.stderr parsl.dataflow.dflow INFO: Task 2876 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 030f3b98eb340f5237e0a834ae79b47b with result from task 2876 parsl.dataflow.dflow DEBUG: Task 2876 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/2715cc49-889b-4337-938b-3111a5efae49_isrForCrosstalkSources_997025_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/2715cc49-889b-4337-938b-3111a5efae49_isrForCrosstalkSources_997025_38.stderr parsl.dataflow.dflow INFO: Task 2871 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 41e89d9a58e73b7bae171812a3bc6adf with result from task 2871 parsl.dataflow.dflow DEBUG: Task 2871 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/1c6957ca-0e25-4b13-8bf7-6e07ad27ef3d_isrForCrosstalkSources_996980_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/1c6957ca-0e25-4b13-8bf7-6e07ad27ef3d_isrForCrosstalkSources_996980_25.stderr parsl.dataflow.dflow INFO: Task 2634 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be34a0bb70836bb387408caad8d47c28 with result from task 2634 parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:55 proc_lsst.multi:146 [INFO] found job 17075400 in provider checkpoint proc_lsst.multi INFO: found job 17075400 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2634 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/e7d864c1-8140-44c3-8cfc-9c4855421c6f_isrForCrosstalkSources_996969_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/e7d864c1-8140-44c3-8cfc-9c4855421c6f_isrForCrosstalkSources_996969_61.stderr parsl.dataflow.dflow INFO: Task 2671 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50e632bc7f53f03628911f79390928b5 with result from task 2671 parsl.dataflow.dflow DEBUG: Task 2671 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/3055cf75-937d-44fa-b566-ac2931ad1b8e_isrForCrosstalkSources_997012_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/3055cf75-937d-44fa-b566-ac2931ad1b8e_isrForCrosstalkSources_997012_53.stderr parsl.dataflow.dflow INFO: Task 2695 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd11a6405b9923356d7e33b554dbf375 with result from task 2695 parsl.dataflow.dflow DEBUG: Task 2695 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/6a3b757a-9e15-4dcc-b4e2-acbc0b79d592_isrForCrosstalkSources_996967_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/6a3b757a-9e15-4dcc-b4e2-acbc0b79d592_isrForCrosstalkSources_996967_47.stderr parsl.dataflow.dflow INFO: Task 2570 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a485cf90f0c5dc89820f7f463d8721af with result from task 2570 parsl.dataflow.dflow DEBUG: Task 2570 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/39872200-09b6-4e2f-87c4-233009294f96_isrForCrosstalkSources_997016_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/39872200-09b6-4e2f-87c4-233009294f96_isrForCrosstalkSources_997016_2.stderr parsl.dataflow.dflow INFO: Task 2811 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18ea5f3f2341c7b7a98f090434dd8e0b with result from task 2811 parsl.dataflow.dflow DEBUG: Task 2811 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ee30ae68-7c5e-4f53-b4d6-a9a6c80bcc07_isrForCrosstalkSources_996982_15.stdout parsl.dataflow.dflow INFO: Standard error for task 2811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ee30ae68-7c5e-4f53-b4d6-a9a6c80bcc07_isrForCrosstalkSources_996982_15.stderr parsl.dataflow.dflow INFO: Task 2614 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58570e4f8de30f521fafc50fc18887db with result from task 2614 parsl.dataflow.dflow DEBUG: Task 2614 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/645014c2-9766-4167-933c-f7308a25b6f4_isrForCrosstalkSources_996967_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/645014c2-9766-4167-933c-f7308a25b6f4_isrForCrosstalkSources_996967_44.stderr parsl.dataflow.dflow INFO: Task 2532 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9de0f8d6ff42ca83d4d93f96d252fb61 with result from task 2532 parsl.dataflow.dflow DEBUG: Task 2532 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/20bd8833-d712-42f6-9a06-50b38fbd287f_isrForCrosstalkSources_997012_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/20bd8833-d712-42f6-9a06-50b38fbd287f_isrForCrosstalkSources_997012_43.stderr parsl.dataflow.dflow INFO: Task 2613 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef0ec4073c9cbe374f4ec4c7e0d6c748 with result from task 2613 parsl.dataflow.dflow DEBUG: Task 2613 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/fe8a36b5-08ab-4e34-9d22-cb734a875064_isrForCrosstalkSources_996993_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/fe8a36b5-08ab-4e34-9d22-cb734a875064_isrForCrosstalkSources_996993_42.stderr parsl.dataflow.dflow INFO: Task 2835 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae80533c71cb9b72f4b0876866eaedbd with result from task 2835 parsl.dataflow.dflow DEBUG: Task 2835 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/546d477c-5271-480f-a584-934ad65006b2_isrForCrosstalkSources_996953_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/546d477c-5271-480f-a584-934ad65006b2_isrForCrosstalkSources_996953_59.stderr parsl.dataflow.dflow INFO: Task 2748 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b02c891cab416ecb2702ddaa47f2967 with result from task 2748 parsl.dataflow.dflow DEBUG: Task 2748 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/62198719-c0d6-4cb6-8a79-ae06b1a2df66_isrForCrosstalkSources_997003_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/62198719-c0d6-4cb6-8a79-ae06b1a2df66_isrForCrosstalkSources_997003_43.stderr parsl.dataflow.dflow INFO: Task 2533 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88bb6bd844feef6e01b18c08511851a4 with result from task 2533 parsl.dataflow.dflow DEBUG: Task 2533 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/42e8012f-63ac-4590-9217-9e342f4a13a7_isrForCrosstalkSources_997012_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/42e8012f-63ac-4590-9217-9e342f4a13a7_isrForCrosstalkSources_997012_49.stderr parsl.dataflow.dflow INFO: Task 2674 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a55484edb1efae9e731857862b514d8 with result from task 2674 parsl.dataflow.dflow DEBUG: Task 2674 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4fe22c91-50d2-47e8-8867-a0eab64c042a_isrForCrosstalkSources_997017_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4fe22c91-50d2-47e8-8867-a0eab64c042a_isrForCrosstalkSources_997017_29.stderr parsl.dataflow.dflow INFO: Task 2860 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf9a36531b8a7261e336c8a53833a4a3 with result from task 2860 parsl.dataflow.dflow DEBUG: Task 2860 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3cd9f0e1-6d68-4e1e-adff-919e9fe5569c_isrForCrosstalkSources_997031_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3cd9f0e1-6d68-4e1e-adff-919e9fe5569c_isrForCrosstalkSources_997031_51.stderr parsl.dataflow.dflow INFO: Task 2647 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f3319a5afd276eefc8226d0ebc35d032 with result from task 2647 parsl.dataflow.dflow DEBUG: Task 2647 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/62a43de6-ff9a-4d40-8562-7e1d554158f9_isrForCrosstalkSources_996970_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/62a43de6-ff9a-4d40-8562-7e1d554158f9_isrForCrosstalkSources_996970_50.stderr parsl.dataflow.dflow INFO: Task 2583 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17eed2a8dd1b90f6a72523e2c273f93b with result from task 2583 parsl.dataflow.dflow DEBUG: Task 2583 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7a39fd16-c28a-4088-b688-48844a38bd36_isrForCrosstalkSources_997006_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7a39fd16-c28a-4088-b688-48844a38bd36_isrForCrosstalkSources_997006_44.stderr parsl.dataflow.dflow INFO: Task 2836 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7f3a4f9a45b376ebe710d8ee9115428d with result from task 2836 parsl.dataflow.dflow DEBUG: Task 2836 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8676e33b-6910-4987-86eb-88948aac7ae7_isrForCrosstalkSources_996952_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8676e33b-6910-4987-86eb-88948aac7ae7_isrForCrosstalkSources_996952_57.stderr parsl.dataflow.dflow INFO: Task 2884 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51f8510f8225e3b31782a1360ea25f97 with result from task 2884 parsl.dataflow.dflow DEBUG: Task 2884 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/398c8453-16d2-4550-9416-fa89a8f69dba_isrForCrosstalkSources_996981_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/398c8453-16d2-4550-9416-fa89a8f69dba_isrForCrosstalkSources_996981_24.stderr parsl.dataflow.dflow INFO: Task 2676 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3de2cdd7174ce40a42f6e3170c3b6949 with result from task 2676 parsl.dataflow.dflow DEBUG: Task 2676 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e4f67bde-9165-4430-9f4f-ff0589daa1e3_isrForCrosstalkSources_996986_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e4f67bde-9165-4430-9f4f-ff0589daa1e3_isrForCrosstalkSources_996986_62.stderr parsl.dataflow.dflow INFO: Task 2529 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6484dee228eda5c5d3d2b5fa75af6e3e with result from task 2529 parsl.dataflow.dflow DEBUG: Task 2529 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/98746efa-895b-483c-b3cd-a6c71e697dfd_isrForCrosstalkSources_997029_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/98746efa-895b-483c-b3cd-a6c71e697dfd_isrForCrosstalkSources_997029_53.stderr parsl.dataflow.dflow INFO: Task 2782 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f40b0631f74746c3168c3cadce29b33 with result from task 2782 parsl.dataflow.dflow DEBUG: Task 2782 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/76fd61ba-5bc4-4b76-91fc-629fe212ea43_isrForCrosstalkSources_997018_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/76fd61ba-5bc4-4b76-91fc-629fe212ea43_isrForCrosstalkSources_997018_50.stderr parsl.dataflow.dflow INFO: Task 2853 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03987f5b94ae22e24600f42e6022ea06 with result from task 2853 parsl.dataflow.dflow DEBUG: Task 2853 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/5a6be999-5741-4a0c-b25f-162301207710_isrForCrosstalkSources_996983_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/5a6be999-5741-4a0c-b25f-162301207710_isrForCrosstalkSources_996983_12.stderr parsl.dataflow.dflow INFO: Task 2807 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f402b4a49583a03911f1ab300c4e51c9 with result from task 2807 parsl.dataflow.dflow DEBUG: Task 2807 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9052b222-f575-43e2-aab1-828f3d992794_isrForCrosstalkSources_997000_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/9052b222-f575-43e2-aab1-828f3d992794_isrForCrosstalkSources_997000_51.stderr parsl.dataflow.dflow INFO: Task 2507 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e58cf9b9f463fe6f56ed989562b7df0b with result from task 2507 parsl.dataflow.dflow DEBUG: Task 2507 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a6192808-b073-4ce9-8dff-8edd986d57fe_isrForCrosstalkSources_997027_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/a6192808-b073-4ce9-8dff-8edd986d57fe_isrForCrosstalkSources_997027_17.stderr parsl.dataflow.dflow INFO: Task 2480 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a8b5e819527d534d24c729576827543 with result from task 2480 parsl.dataflow.dflow DEBUG: Task 2480 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/9ce3b733-8748-4685-bbbc-4a98009a8a81_isrForCrosstalkSources_996975_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/9ce3b733-8748-4685-bbbc-4a98009a8a81_isrForCrosstalkSources_996975_7.stderr parsl.dataflow.dflow INFO: Task 2623 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f2dbf207cfac3c3ee40586ddadd72236 with result from task 2623 parsl.dataflow.dflow DEBUG: Task 2623 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/9c413603-45f9-4a17-82b0-fd42bdf7f18f_isrForCrosstalkSources_996955_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/9c413603-45f9-4a17-82b0-fd42bdf7f18f_isrForCrosstalkSources_996955_62.stderr parsl.dataflow.dflow INFO: Task 2571 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a33795a99a40ac2c064fcea1df4db70 with result from task 2571 parsl.dataflow.dflow DEBUG: Task 2571 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6e642f81-acb5-4cb0-b552-3b7e4fc65b82_isrForCrosstalkSources_996983_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/6e642f81-acb5-4cb0-b552-3b7e4fc65b82_isrForCrosstalkSources_996983_35.stderr parsl.dataflow.dflow INFO: Task 2572 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25f2b10b416f6c829ea46ce27acd70b7 with result from task 2572 parsl.dataflow.dflow DEBUG: Task 2572 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/281259b6-c15d-46b7-bb19-e88a26758b6b_isrForCrosstalkSources_996983_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/281259b6-c15d-46b7-bb19-e88a26758b6b_isrForCrosstalkSources_996983_36.stderr parsl.dataflow.dflow INFO: Task 2680 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 756c68b8d882a7988455ad5c4875ab18 with result from task 2680 parsl.dataflow.dflow DEBUG: Task 2680 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/aa049fb6-a45b-45df-9143-9b5ba7eb948a_isrForCrosstalkSources_996961_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/aa049fb6-a45b-45df-9143-9b5ba7eb948a_isrForCrosstalkSources_996961_32.stderr parsl.dataflow.dflow INFO: Task 2555 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e2f0de0cf72e5d5eab479be32e5014c with result from task 2555 parsl.dataflow.dflow DEBUG: Task 2555 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3e25aceb-39c0-4872-882a-f26cb0ce7d19_isrForCrosstalkSources_997001_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3e25aceb-39c0-4872-882a-f26cb0ce7d19_isrForCrosstalkSources_997001_29.stderr parsl.dataflow.dflow INFO: Task 2669 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd8b1dfcb42598cbbbb3b7e2ec92d400 with result from task 2669 parsl.dataflow.dflow DEBUG: Task 2669 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e2430c38-b27e-4d11-b7f8-b0fc6443daa1_isrForCrosstalkSources_997005_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e2430c38-b27e-4d11-b7f8-b0fc6443daa1_isrForCrosstalkSources_997005_54.stderr parsl.dataflow.dflow INFO: Task 2552 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f55bb01db5201bb84ea777d8df7bb250 with result from task 2552 parsl.dataflow.dflow DEBUG: Task 2552 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/dc286c2e-bc72-4c30-8743-9c7fdc105ef8_isrForCrosstalkSources_996972_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/dc286c2e-bc72-4c30-8743-9c7fdc105ef8_isrForCrosstalkSources_996972_25.stderr parsl.dataflow.dflow INFO: Task 2771 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ad6938befdca9d3a753bb8d8bf62af6 with result from task 2771 parsl.dataflow.dflow DEBUG: Task 2771 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/93dd7b97-ecd4-4d66-afc1-a284443c5a70_isrForCrosstalkSources_997008_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/93dd7b97-ecd4-4d66-afc1-a284443c5a70_isrForCrosstalkSources_997008_13.stderr parsl.dataflow.dflow INFO: Task 2813 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0729006b52ef876228258564a005945f with result from task 2813 parsl.dataflow.dflow DEBUG: Task 2813 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2c9728c1-7f0a-44dc-be89-e08615f02005_isrForCrosstalkSources_996977_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2c9728c1-7f0a-44dc-be89-e08615f02005_isrForCrosstalkSources_996977_18.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:56 proc_lsst.multi:146 [INFO] found job 17075401 in provider checkpoint proc_lsst.multi INFO: found job 17075401 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2763 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c687bcb6e75937ff6dd37d820792fc3e with result from task 2763 parsl.dataflow.dflow DEBUG: Task 2763 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fcf07496-8789-4b05-a97e-d54f7687ed38_isrForCrosstalkSources_997012_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fcf07496-8789-4b05-a97e-d54f7687ed38_isrForCrosstalkSources_997012_5.stderr parsl.dataflow.dflow INFO: Task 2690 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5e39a44360fbece22a9c1edfc86006c with result from task 2690 parsl.dataflow.dflow DEBUG: Task 2690 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/966d198f-37d5-437e-a9ff-2d081e3727e3_isrForCrosstalkSources_997008_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/966d198f-37d5-437e-a9ff-2d081e3727e3_isrForCrosstalkSources_997008_20.stderr parsl.dataflow.dflow INFO: Task 2587 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4fe49efd5514a0dda96993361dad6cb with result from task 2587 parsl.dataflow.dflow DEBUG: Task 2587 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a80af2af-788f-4b4f-94c7-243a4372e717_isrForCrosstalkSources_996977_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a80af2af-788f-4b4f-94c7-243a4372e717_isrForCrosstalkSources_996977_11.stderr parsl.dataflow.dflow INFO: Task 2867 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 792aec35622ffe07bf764dc17ebc5f4b with result from task 2867 parsl.dataflow.dflow DEBUG: Task 2867 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/98547152-fdad-4d6a-85f6-490fe247dc54_isrForCrosstalkSources_996967_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/98547152-fdad-4d6a-85f6-490fe247dc54_isrForCrosstalkSources_996967_32.stderr parsl.dataflow.dflow INFO: Task 2500 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be036de564dd67efdf8428f2b242d8ab with result from task 2500 parsl.dataflow.dflow DEBUG: Task 2500 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6ef61b85-fc2e-45cf-b8d0-ba12b77b90b1_isrForCrosstalkSources_997023_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/6ef61b85-fc2e-45cf-b8d0-ba12b77b90b1_isrForCrosstalkSources_997023_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2859 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry d2d734b69f7d74724875112dcadf7633 with result from task 2859 parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2859 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/64dc6861-944c-4986-a801-a1b21df3fb50_isrForCrosstalkSources_997023_17.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/64dc6861-944c-4986-a801-a1b21df3fb50_isrForCrosstalkSources_997023_17.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2887 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry aa6c72d24c1edc3ab26c900703130a59 with result from task 2887 parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2887 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ff015dcb-4cce-4aaa-80e2-8524000672c7_isrForCrosstalkSources_996982_57.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ff015dcb-4cce-4aaa-80e2-8524000672c7_isrForCrosstalkSources_996982_57.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:57 proc_lsst.multi:146 [INFO] found job 17075405 in provider checkpoint proc_lsst.multi INFO: found job 17075405 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2893 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 37e5df953fd6d0497fbe639b8b8a9d22 with result from task 2893 parsl.dataflow.dflow DEBUG: Task 2893 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e296968c-2108-44c5-8972-ac351c63a21a_isrForCrosstalkSources_996955_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e296968c-2108-44c5-8972-ac351c63a21a_isrForCrosstalkSources_996955_22.stderr parsl.dataflow.dflow INFO: Task 2883 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 659f3b6632323c25982b7e7cf90265b2 with result from task 2883 parsl.dataflow.dflow DEBUG: Task 2883 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/54f7eb22-59e6-4be4-9afc-7775ce6fe912_isrForCrosstalkSources_996959_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/54f7eb22-59e6-4be4-9afc-7775ce6fe912_isrForCrosstalkSources_996959_47.stderr parsl.dataflow.dflow INFO: Task 2582 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f11df8e6a9613c973d4f7628e12ccf6 with result from task 2582 parsl.dataflow.dflow DEBUG: Task 2582 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2fdc53db-d588-4460-af40-81d8dfe71e80_isrForCrosstalkSources_996989_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2fdc53db-d588-4460-af40-81d8dfe71e80_isrForCrosstalkSources_996989_51.stderr parsl.dataflow.dflow INFO: Task 2355 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51149444bd6ef0b49d8f5605da9566b7 with result from task 2355 parsl.dataflow.dflow DEBUG: Task 2355 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/103d5168-2eac-4860-97b4-17f165647879_isrForCrosstalkSources_996974_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/103d5168-2eac-4860-97b4-17f165647879_isrForCrosstalkSources_996974_48.stderr parsl.dataflow.dflow INFO: Task 2578 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17df761c0dfe8bf2ca0d99061f51642a with result from task 2578 parsl.dataflow.dflow DEBUG: Task 2578 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/9a09f7f5-e970-4a9c-b285-d11e90486c53_isrForCrosstalkSources_996970_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/9a09f7f5-e970-4a9c-b285-d11e90486c53_isrForCrosstalkSources_996970_11.stderr parsl.dataflow.dflow INFO: Task 2864 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d5c1ee1394056c1ce711790fc6b6fba with result from task 2864 parsl.dataflow.dflow DEBUG: Task 2864 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a8e1e54a-706e-457d-989a-8bd3b5511bb2_isrForCrosstalkSources_997020_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/a8e1e54a-706e-457d-989a-8bd3b5511bb2_isrForCrosstalkSources_997020_19.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:58 proc_lsst.multi:146 [INFO] found job 17075408 in provider checkpoint proc_lsst.multi INFO: found job 17075408 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2654 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 425dedc3e42dd69d21ee88c6b2ef171e with result from task 2654 parsl.dataflow.dflow DEBUG: Task 2654 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b176d987-c76e-4bad-bcc7-20cc6a226588_isrForCrosstalkSources_997004_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b176d987-c76e-4bad-bcc7-20cc6a226588_isrForCrosstalkSources_997004_16.stderr parsl.dataflow.dflow INFO: Task 2766 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 69283de9c9f979259a819601893636a3 with result from task 2766 parsl.dataflow.dflow DEBUG: Task 2766 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3dfc5091-2d50-43f7-acda-5b3d79e7a5e5_isrForCrosstalkSources_997001_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3dfc5091-2d50-43f7-acda-5b3d79e7a5e5_isrForCrosstalkSources_997001_2.stderr parsl.dataflow.dflow INFO: Task 2576 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9edb174fc673c7301896495d3476e03 with result from task 2576 parsl.dataflow.dflow DEBUG: Task 2576 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/e4ab543e-d052-4430-9c47-48ce79e89f90_isrForCrosstalkSources_996971_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/e4ab543e-d052-4430-9c47-48ce79e89f90_isrForCrosstalkSources_996971_23.stderr parsl.dataflow.dflow INFO: Task 2764 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3061bb43590e51afc39f111566bfec91 with result from task 2764 parsl.dataflow.dflow DEBUG: Task 2764 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/74aaabf7-4766-4600-b4dc-a85b4d021585_isrForCrosstalkSources_997026_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/74aaabf7-4766-4600-b4dc-a85b4d021585_isrForCrosstalkSources_997026_48.stderr parsl.dataflow.dflow INFO: Task 2793 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd04ef7e30955a8974eb14f070aa0a28 with result from task 2793 parsl.dataflow.dflow DEBUG: Task 2793 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9dc3bd22-ccd1-4a74-a1ce-bcf5a292f29e_isrForCrosstalkSources_996984_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9dc3bd22-ccd1-4a74-a1ce-bcf5a292f29e_isrForCrosstalkSources_996984_21.stderr parsl.dataflow.dflow INFO: Task 2635 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 550b5d5be8acd87588b32970fad29155 with result from task 2635 parsl.dataflow.dflow DEBUG: Task 2635 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/a38df11e-d576-4f8a-85df-ffebcb59c64d_isrForCrosstalkSources_997017_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/a38df11e-d576-4f8a-85df-ffebcb59c64d_isrForCrosstalkSources_997017_28.stderr parsl.dataflow.dflow INFO: Task 2910 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5761af31c7e4f127c2663e80aa2e3ca5 with result from task 2910 parsl.dataflow.dflow DEBUG: Task 2910 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ec33553f-3450-4aa8-b976-d243a6577371_isrForCrosstalkSources_997015_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ec33553f-3450-4aa8-b976-d243a6577371_isrForCrosstalkSources_997015_7.stderr parsl.dataflow.dflow INFO: Task 2616 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0972565de7a6ccf1419b9ed80c688b81 with result from task 2616 parsl.dataflow.dflow DEBUG: Task 2616 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d90414c4-8499-4221-90c6-09ad9e85b10e_isrForCrosstalkSources_996964_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/d90414c4-8499-4221-90c6-09ad9e85b10e_isrForCrosstalkSources_996964_1.stderr parsl.dataflow.dflow INFO: Task 2473 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2c3a175d9a6f5a1a8cbb76d3d8f0fe7 with result from task 2473 parsl.dataflow.dflow DEBUG: Task 2473 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1c9cd0c9-4744-4539-8ad7-955b52bf14b6_isrForCrosstalkSources_997005_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/1c9cd0c9-4744-4539-8ad7-955b52bf14b6_isrForCrosstalkSources_997005_42.stderr parsl.dataflow.dflow INFO: Task 2912 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f9488b8add8aa78f66f597d9d831195d with result from task 2912 parsl.dataflow.dflow DEBUG: Task 2912 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/00644a80-af0e-4674-983f-78c18e1f1552_isrForCrosstalkSources_996962_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/00644a80-af0e-4674-983f-78c18e1f1552_isrForCrosstalkSources_996962_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:56:59 proc_lsst.multi:146 [INFO] found job 17075417 in provider checkpoint proc_lsst.multi INFO: found job 17075417 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2795 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 77a0f827e5205c5fc4982277590df78c with result from task 2795 parsl.dataflow.dflow DEBUG: Task 2795 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/b964848f-0a85-430c-905e-7d6a3acfaddf_isrForCrosstalkSources_996970_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/b964848f-0a85-430c-905e-7d6a3acfaddf_isrForCrosstalkSources_996970_58.stderr parsl.dataflow.dflow INFO: Task 2667 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82f8692f1a0a3dc26266e708f5b476e7 with result from task 2667 parsl.dataflow.dflow DEBUG: Task 2667 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d976cf3d-7e89-4a11-aa0f-4e325ad2f29e_isrForCrosstalkSources_997009_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d976cf3d-7e89-4a11-aa0f-4e325ad2f29e_isrForCrosstalkSources_997009_22.stderr parsl.dataflow.dflow INFO: Task 2802 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b37e99199f07566516f23051d6047ef4 with result from task 2802 parsl.dataflow.dflow DEBUG: Task 2802 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ea36fbee-7a36-4a8f-8782-e1430997f1f9_isrForCrosstalkSources_996972_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/ea36fbee-7a36-4a8f-8782-e1430997f1f9_isrForCrosstalkSources_996972_57.stderr parsl.dataflow.dflow INFO: Task 2825 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f5c158f2b26c8db7d00892aefa63722 with result from task 2825 parsl.dataflow.dflow DEBUG: Task 2825 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9a4710c0-c6ef-4b0c-a4fd-75e38645ee93_isrForCrosstalkSources_996968_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9a4710c0-c6ef-4b0c-a4fd-75e38645ee93_isrForCrosstalkSources_996968_35.stderr parsl.dataflow.dflow INFO: Task 2823 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9989e35d7ff6c01176c10417f6631a26 with result from task 2823 parsl.dataflow.dflow DEBUG: Task 2823 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/c2179676-2c98-412f-92f8-6337b28e3d6f_isrForCrosstalkSources_996983_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/c2179676-2c98-412f-92f8-6337b28e3d6f_isrForCrosstalkSources_996983_1.stderr parsl.dataflow.dflow INFO: Task 2855 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d66a3c0d51d576b970a2af1d688be1c with result from task 2855 parsl.dataflow.dflow DEBUG: Task 2855 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/38caacbd-6102-45cd-8c8d-e2233bcf191b_isrForCrosstalkSources_996989_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/38caacbd-6102-45cd-8c8d-e2233bcf191b_isrForCrosstalkSources_996989_23.stderr parsl.dataflow.dflow INFO: Task 2540 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eee50c68eacb25a4e78605dc5e215ec6 with result from task 2540 parsl.dataflow.dflow DEBUG: Task 2540 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b1ca13d8-cc56-4693-9dc3-79315d5b8b94_isrForCrosstalkSources_997033_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b1ca13d8-cc56-4693-9dc3-79315d5b8b94_isrForCrosstalkSources_997033_29.stderr parsl.dataflow.dflow INFO: Task 2777 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d54930bf54e9f9de8fc0e627f7e29c5b with result from task 2777 parsl.dataflow.dflow DEBUG: Task 2777 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/eeb9fdb9-ee04-445a-aa81-2173dc06a2d4_isrForCrosstalkSources_997022_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/eeb9fdb9-ee04-445a-aa81-2173dc06a2d4_isrForCrosstalkSources_997022_53.stderr parsl.dataflow.dflow INFO: Task 2796 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2bb97e7eef3b94e477bd0f35dc59b181 with result from task 2796 parsl.dataflow.dflow DEBUG: Task 2796 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/cde69ff8-7560-4eb6-9dbb-1d470c8bd091_isrForCrosstalkSources_996964_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/cde69ff8-7560-4eb6-9dbb-1d470c8bd091_isrForCrosstalkSources_996964_47.stderr parsl.dataflow.dflow INFO: Task 2908 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b1e86007b26668dd55d5a60e71f6b106 with result from task 2908 parsl.dataflow.dflow DEBUG: Task 2908 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/83e0a3cb-8b30-4078-9d15-e769011f44ff_isrForCrosstalkSources_996954_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/83e0a3cb-8b30-4078-9d15-e769011f44ff_isrForCrosstalkSources_996954_26.stderr parsl.dataflow.dflow INFO: Task 2700 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82a59cf9c558e62f142e9905aecdb15e with result from task 2700 parsl.dataflow.dflow DEBUG: Task 2700 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/aecbf6da-b367-42e5-8397-a8a06ddb2cc4_isrForCrosstalkSources_997006_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/aecbf6da-b367-42e5-8397-a8a06ddb2cc4_isrForCrosstalkSources_997006_10.stderr parsl.dataflow.dflow INFO: Task 2915 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a2c081e8ca5dd2603ad96e42e0757fc with result from task 2915 parsl.dataflow.dflow DEBUG: Task 2915 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/199053dc-5b7a-478c-9875-8a6fda4a8c9d_isrForCrosstalkSources_996968_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/199053dc-5b7a-478c-9875-8a6fda4a8c9d_isrForCrosstalkSources_996968_24.stderr parsl.dataflow.dflow INFO: Task 2880 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7bebe503391a43539352b7c6e92850d0 with result from task 2880 parsl.dataflow.dflow DEBUG: Task 2880 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/404ee456-4526-49c4-a05b-e5684a267a06_isrForCrosstalkSources_996976_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/404ee456-4526-49c4-a05b-e5684a267a06_isrForCrosstalkSources_996976_49.stderr parsl.dataflow.dflow INFO: Task 2873 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry baa102b5f7c579a6957a005d6c16d753 with result from task 2873 parsl.dataflow.dflow DEBUG: Task 2873 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3528f0cf-6506-4fc1-81dd-d952514849ce_isrForCrosstalkSources_996955_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3528f0cf-6506-4fc1-81dd-d952514849ce_isrForCrosstalkSources_996955_60.stderr parsl.dataflow.dflow INFO: Task 2577 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56b8fc92df7c0c3673bad71c23f2b91d with result from task 2577 parsl.dataflow.dflow DEBUG: Task 2577 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/071f24d3-e3e3-435e-b623-d19ceddb2658_isrForCrosstalkSources_997025_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/071f24d3-e3e3-435e-b623-d19ceddb2658_isrForCrosstalkSources_997025_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 2574 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 248e5d3a74969eff2212b90317e405d4 with result from task 2574 parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2574 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2483d421-9186-441f-b13b-cf696ae4202e_isrForCrosstalkSources_997007_44.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2483d421-9186-441f-b13b-cf696ae4202e_isrForCrosstalkSources_997007_44.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:00 proc_lsst.multi:146 [INFO] found job 17075428 in provider checkpoint proc_lsst.multi INFO: found job 17075428 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2869 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d73444ad55fee393a0eb32cb9b27774e with result from task 2869 parsl.dataflow.dflow DEBUG: Task 2869 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/9e69b7f6-d131-49a6-8a92-9e0575977a83_isrForCrosstalkSources_996996_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/9e69b7f6-d131-49a6-8a92-9e0575977a83_isrForCrosstalkSources_996996_11.stderr parsl.dataflow.dflow INFO: Task 2870 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e354b12a4c6c439ec7af0dbd460cbfb6 with result from task 2870 parsl.dataflow.dflow DEBUG: Task 2870 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/45e48bc8-85fd-4239-b740-4bfafaa83600_isrForCrosstalkSources_996969_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/45e48bc8-85fd-4239-b740-4bfafaa83600_isrForCrosstalkSources_996969_48.stderr parsl.dataflow.dflow INFO: Task 2928 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a080b2721f332bf3f6617a47a851dc7b with result from task 2928 parsl.dataflow.dflow DEBUG: Task 2928 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/8bbbe872-e6a2-4d64-9e55-ffb5c184de39_isrForCrosstalkSources_996962_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/8bbbe872-e6a2-4d64-9e55-ffb5c184de39_isrForCrosstalkSources_996962_28.stderr parsl.dataflow.dflow INFO: Task 2861 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7c1a744ceb3ae93d37e5ba494e77100 with result from task 2861 parsl.dataflow.dflow DEBUG: Task 2861 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/84975943-1ff6-49d2-b5f6-0abcb6cc78fb_isrForCrosstalkSources_996955_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/84975943-1ff6-49d2-b5f6-0abcb6cc78fb_isrForCrosstalkSources_996955_53.stderr parsl.dataflow.dflow INFO: Task 2621 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30f3a62c87c7b9f3064b72e03e2ddc66 with result from task 2621 parsl.dataflow.dflow DEBUG: Task 2621 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/22504e2b-9ac3-44ee-af4b-1f6b80649aea_isrForCrosstalkSources_996993_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/22504e2b-9ac3-44ee-af4b-1f6b80649aea_isrForCrosstalkSources_996993_41.stderr parsl.dataflow.dflow INFO: Task 2678 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25a7b572384841dcc82f9963e13623e2 with result from task 2678 parsl.dataflow.dflow DEBUG: Task 2678 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e1fa7cfe-df69-4854-b19f-5f96b08080ca_isrForCrosstalkSources_996985_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/e1fa7cfe-df69-4854-b19f-5f96b08080ca_isrForCrosstalkSources_996985_44.stderr parsl.dataflow.dflow INFO: Task 2759 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c892a66ab1bb17526267da55ae5a702 with result from task 2759 parsl.dataflow.dflow DEBUG: Task 2759 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ddd9ca66-29fe-47ae-b4eb-c29e4cc207de_isrForCrosstalkSources_996988_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ddd9ca66-29fe-47ae-b4eb-c29e4cc207de_isrForCrosstalkSources_996988_24.stderr parsl.dataflow.dflow INFO: Task 2956 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 736f77632354c5ca17447455c85ea475 with result from task 2956 parsl.dataflow.dflow DEBUG: Task 2956 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c66fdfde-2efb-4bdd-9f71-c91c5b516b11_isrForCrosstalkSources_996959_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c66fdfde-2efb-4bdd-9f71-c91c5b516b11_isrForCrosstalkSources_996959_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:01 proc_lsst.multi:146 [INFO] found job 17075430 in provider checkpoint proc_lsst.multi INFO: found job 17075430 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2776 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd357c5e803b41e30d8389d602773f66 with result from task 2776 parsl.dataflow.dflow DEBUG: Task 2776 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ae8a9225-4ddc-4cd2-89aa-d4d8ca2edf20_isrForCrosstalkSources_996976_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ae8a9225-4ddc-4cd2-89aa-d4d8ca2edf20_isrForCrosstalkSources_996976_22.stderr parsl.dataflow.dflow INFO: Task 2943 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4684439a4745998711d8dc3e0aa3a9c with result from task 2943 parsl.dataflow.dflow DEBUG: Task 2943 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/165a4f75-14af-4ac0-98ff-da55136073d5_isrForCrosstalkSources_997008_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/165a4f75-14af-4ac0-98ff-da55136073d5_isrForCrosstalkSources_997008_56.stderr parsl.dataflow.dflow INFO: Task 2513 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d437cae2cd533975ca868d365d3e3b5c with result from task 2513 parsl.dataflow.dflow DEBUG: Task 2513 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/06b8cd52-0a81-4d95-85ca-9cfe0540d2ed_isrForCrosstalkSources_996980_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/06b8cd52-0a81-4d95-85ca-9cfe0540d2ed_isrForCrosstalkSources_996980_53.stderr parsl.dataflow.dflow INFO: Task 2778 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ecc68fd8b68ffe8cb54132cd2e40d886 with result from task 2778 parsl.dataflow.dflow DEBUG: Task 2778 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/3747aa24-0450-4ba1-9e20-1f0881a868cf_isrForCrosstalkSources_996976_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/3747aa24-0450-4ba1-9e20-1f0881a868cf_isrForCrosstalkSources_996976_2.stderr parsl.dataflow.dflow INFO: Task 2806 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 834da497916f624c02566e86c523ece5 with result from task 2806 parsl.dataflow.dflow DEBUG: Task 2806 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7f7fde2b-b86d-46f5-b45e-c8446a7d0efe_isrForCrosstalkSources_997008_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7f7fde2b-b86d-46f5-b45e-c8446a7d0efe_isrForCrosstalkSources_997008_14.stderr parsl.dataflow.dflow INFO: Task 2629 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 288ca0b0e384902f59ad6e177c3852e7 with result from task 2629 parsl.dataflow.dflow DEBUG: Task 2629 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/1e65e63e-92ea-4f2b-b9b8-d6c388f24934_isrForCrosstalkSources_997007_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/1e65e63e-92ea-4f2b-b9b8-d6c388f24934_isrForCrosstalkSources_997007_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2665 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:03 proc_lsst.multi:146 [INFO] found job 17075431 in provider checkpoint proc_lsst.multi INFO: found job 17075431 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.memoization DEBUG: Storing app cache entry 0d70425291b43ed89fa72250fe4b6114 with result from task 2665 parsl.dataflow.dflow DEBUG: Task 2665 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/27c67197-9865-497c-898f-98d4fe0e7769_isrForCrosstalkSources_996995_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/27c67197-9865-497c-898f-98d4fe0e7769_isrForCrosstalkSources_996995_24.stderr parsl.dataflow.dflow INFO: Task 2961 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aeb0f535be2e3fc975cd181843b6657b with result from task 2961 parsl.dataflow.dflow DEBUG: Task 2961 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b65c622e-559e-4aa7-827f-e738b7467506_isrForCrosstalkSources_996983_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b65c622e-559e-4aa7-827f-e738b7467506_isrForCrosstalkSources_996983_55.stderr parsl.dataflow.dflow INFO: Task 2731 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 592217cd7ec871c4ea4c88676cc324f7 with result from task 2731 parsl.dataflow.dflow DEBUG: Task 2731 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d7aee4b9-8ef8-444f-9b9e-d591d4c02231_isrForCrosstalkSources_997026_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d7aee4b9-8ef8-444f-9b9e-d591d4c02231_isrForCrosstalkSources_997026_22.stderr parsl.dataflow.dflow INFO: Task 2957 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dceba03f4845ce9a23f44ae7fa0db8ec with result from task 2957 parsl.dataflow.dflow DEBUG: Task 2957 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/2c27e4eb-8e44-478c-8e3d-214e8c410dc6_isrForCrosstalkSources_997032_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/2c27e4eb-8e44-478c-8e3d-214e8c410dc6_isrForCrosstalkSources_997032_6.stderr parsl.dataflow.dflow INFO: Task 2970 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48692d6964073c993e370c49dc8a28f9 with result from task 2970 parsl.dataflow.dflow DEBUG: Task 2970 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/44659357-96ab-435f-9e97-94638dba2fa0_isrForCrosstalkSources_996953_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/44659357-96ab-435f-9e97-94638dba2fa0_isrForCrosstalkSources_996953_6.stderr parsl.dataflow.dflow INFO: Task 2889 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a00849f477a790891b5fedb86d1572a4 with result from task 2889 parsl.dataflow.dflow DEBUG: Task 2889 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/f7ed49d4-1980-4834-8012-2619aa020684_isrForCrosstalkSources_997004_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/f7ed49d4-1980-4834-8012-2619aa020684_isrForCrosstalkSources_997004_57.stderr parsl.dataflow.dflow INFO: Task 2868 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 773c9f937e782ae4a77dd13c68adbbe4 with result from task 2868 parsl.dataflow.dflow DEBUG: Task 2868 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/b9ed2a27-fed9-4dca-aaff-5fc5b7355954_isrForCrosstalkSources_996965_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/b9ed2a27-fed9-4dca-aaff-5fc5b7355954_isrForCrosstalkSources_996965_27.stderr parsl.dataflow.dflow INFO: Task 2940 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82d2fa97d4eb0c391ba9395a30be071f with result from task 2940 parsl.dataflow.dflow DEBUG: Task 2940 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/e6a8eefa-feee-4def-a899-542b79e05a0f_isrForCrosstalkSources_996962_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/e6a8eefa-feee-4def-a899-542b79e05a0f_isrForCrosstalkSources_996962_7.stderr parsl.dataflow.dflow INFO: Task 2798 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a9e65c77dee021b02395898043745e8 with result from task 2798 parsl.dataflow.dflow DEBUG: Task 2798 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a246a335-1cdd-46cb-aa35-ad7cb32d9011_isrForCrosstalkSources_997008_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a246a335-1cdd-46cb-aa35-ad7cb32d9011_isrForCrosstalkSources_997008_59.stderr parsl.dataflow.dflow INFO: Task 2822 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3cf4fe82890250692de92793b3846b15 with result from task 2822 parsl.dataflow.dflow DEBUG: Task 2822 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d5b2bb89-aeb0-4486-a182-6f3ef66ea2b2_isrForCrosstalkSources_996993_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d5b2bb89-aeb0-4486-a182-6f3ef66ea2b2_isrForCrosstalkSources_996993_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:04 proc_lsst.multi:146 [INFO] found job 17075432 in provider checkpoint proc_lsst.multi INFO: found job 17075432 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2716 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d838c603f22412e9c74c5fb5b81b1686 with result from task 2716 parsl.dataflow.dflow DEBUG: Task 2716 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/785c7b59-701c-4ff0-8134-393e3160c4c5_isrForCrosstalkSources_997011_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/785c7b59-701c-4ff0-8134-393e3160c4c5_isrForCrosstalkSources_997011_26.stderr parsl.dataflow.dflow INFO: Task 2702 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 976b1b7001c8e8918a89204f7405f601 with result from task 2702 parsl.dataflow.dflow DEBUG: Task 2702 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/12b25834-cefd-4d3e-b4ea-38528dcf5a2c_isrForCrosstalkSources_996962_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/12b25834-cefd-4d3e-b4ea-38528dcf5a2c_isrForCrosstalkSources_996962_27.stderr parsl.dataflow.dflow INFO: Task 2850 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ff4e469ef57e6b4feee0ecda7225ee3 with result from task 2850 parsl.dataflow.dflow DEBUG: Task 2850 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/472ef291-56c8-4fa4-9332-31aa2d9c4a46_isrForCrosstalkSources_997026_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/472ef291-56c8-4fa4-9332-31aa2d9c4a46_isrForCrosstalkSources_997026_55.stderr parsl.dataflow.dflow INFO: Task 2964 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e7000a403e903d53f0c4cc95a76eb35 with result from task 2964 parsl.dataflow.dflow DEBUG: Task 2964 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d57ff839-0a93-4591-bf46-f6f519989dc1_isrForCrosstalkSources_996980_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d57ff839-0a93-4591-bf46-f6f519989dc1_isrForCrosstalkSources_996980_48.stderr parsl.dataflow.dflow INFO: Task 2988 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f42b49810b95117895148eae901d8de6 with result from task 2988 parsl.dataflow.dflow DEBUG: Task 2988 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b90339c2-f14f-4f66-b751-01a753fbcfdf_isrForCrosstalkSources_996980_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b90339c2-f14f-4f66-b751-01a753fbcfdf_isrForCrosstalkSources_996980_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:06 proc_lsst.multi:146 [INFO] found job 17075433 in provider checkpoint proc_lsst.multi INFO: found job 17075433 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2768 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5209171d0d98fc7b548e9ff7a07e83fe with result from task 2768 parsl.dataflow.dflow DEBUG: Task 2768 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9a1dd9f6-14b0-4ba7-8519-74878b8d685c_isrForCrosstalkSources_996979_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9a1dd9f6-14b0-4ba7-8519-74878b8d685c_isrForCrosstalkSources_996979_3.stderr parsl.dataflow.dflow INFO: Task 2903 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2944e3b294ecfb0da9970edde83a9720 with result from task 2903 parsl.dataflow.dflow DEBUG: Task 2903 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ada51573-b39a-43a9-b66e-66c8dc8e2b4f_isrForCrosstalkSources_996978_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ada51573-b39a-43a9-b66e-66c8dc8e2b4f_isrForCrosstalkSources_996978_13.stderr parsl.dataflow.dflow INFO: Task 2847 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c325e06a517290bb493e62e30220b447 with result from task 2847 parsl.dataflow.dflow DEBUG: Task 2847 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/196cba82-9e1a-47ba-abd9-eab40d470994_isrForCrosstalkSources_996980_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/196cba82-9e1a-47ba-abd9-eab40d470994_isrForCrosstalkSources_996980_34.stderr parsl.dataflow.dflow INFO: Task 2980 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55474ddf23c05c44e3adef81962f1bbb with result from task 2980 parsl.dataflow.dflow DEBUG: Task 2980 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/93a5aae3-5cab-409c-a9ed-a3afffc1c5fe_isrForCrosstalkSources_996973_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/93a5aae3-5cab-409c-a9ed-a3afffc1c5fe_isrForCrosstalkSources_996973_17.stderr parsl.dataflow.dflow INFO: Task 2946 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 128ca4e13c5952cfae74bacf0f2611b8 with result from task 2946 parsl.dataflow.dflow DEBUG: Task 2946 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/9a179c26-5938-47ad-9be7-4af07ecf8c65_isrForCrosstalkSources_997032_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/9a179c26-5938-47ad-9be7-4af07ecf8c65_isrForCrosstalkSources_997032_55.stderr parsl.dataflow.dflow INFO: Task 2953 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 47f619a6c3e612692f4e0c8ed5480539 with result from task 2953 parsl.dataflow.dflow DEBUG: Task 2953 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/678a5929-8c7e-4044-8801-ea74bc5847e9_isrForCrosstalkSources_996964_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/678a5929-8c7e-4044-8801-ea74bc5847e9_isrForCrosstalkSources_996964_7.stderr parsl.dataflow.dflow INFO: Task 2693 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ec964d51f7e6f94bd2cef6f617375c0 with result from task 2693 parsl.dataflow.dflow DEBUG: Task 2693 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/b2ae47c9-0b8c-4ce2-b9a8-49e7accc5cbf_isrForCrosstalkSources_997020_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/b2ae47c9-0b8c-4ce2-b9a8-49e7accc5cbf_isrForCrosstalkSources_997020_20.stderr parsl.dataflow.dflow INFO: Task 2715 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9354cec10e85e567334dc3256e0c9b0f with result from task 2715 parsl.dataflow.dflow DEBUG: Task 2715 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/097764a2-5d18-4982-a1ed-a5eef672f84f_isrForCrosstalkSources_996982_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/097764a2-5d18-4982-a1ed-a5eef672f84f_isrForCrosstalkSources_996982_14.stderr parsl.dataflow.dflow INFO: Task 2840 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cbb309b446a21e28d2d2155e0a7b4ed7 with result from task 2840 parsl.dataflow.dflow DEBUG: Task 2840 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/b108b748-696e-4f2f-935d-8d29a0184da7_isrForCrosstalkSources_996965_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/b108b748-696e-4f2f-935d-8d29a0184da7_isrForCrosstalkSources_996965_14.stderr parsl.dataflow.dflow INFO: Task 2905 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8b1bc111c073d78c375098445a7d315 with result from task 2905 parsl.dataflow.dflow DEBUG: Task 2905 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/baa2ee2d-40b1-4731-b328-ab721fcf8f21_isrForCrosstalkSources_997004_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/baa2ee2d-40b1-4731-b328-ab721fcf8f21_isrForCrosstalkSources_997004_27.stderr parsl.dataflow.dflow INFO: Task 2927 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e2db0eca09259024757577ba2978923 with result from task 2927 parsl.dataflow.dflow DEBUG: Task 2927 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/2c26d667-a37b-452d-bec0-5600ef579fe4_isrForCrosstalkSources_996994_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/2c26d667-a37b-452d-bec0-5600ef579fe4_isrForCrosstalkSources_996994_45.stderr parsl.dataflow.dflow INFO: Task 2765 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 45d2e36f1297d5a3e85a091e604ba168 with result from task 2765 parsl.dataflow.dflow DEBUG: Task 2765 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fb6cd9e1-2ec5-474a-b8e4-792a00120b1b_isrForCrosstalkSources_997008_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fb6cd9e1-2ec5-474a-b8e4-792a00120b1b_isrForCrosstalkSources_997008_27.stderr parsl.dataflow.dflow INFO: Task 2987 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dc2f397698ae3080173d563d24dc8105 with result from task 2987 parsl.dataflow.dflow DEBUG: Task 2987 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/dd173c1f-ec9c-4524-a3e2-3b5b716521e6_isrForCrosstalkSources_996990_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/dd173c1f-ec9c-4524-a3e2-3b5b716521e6_isrForCrosstalkSources_996990_46.stderr parsl.dataflow.dflow INFO: Task 2705 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bdbcdbbbd4a90eb8402fc394ab9f6223 with result from task 2705 parsl.dataflow.dflow DEBUG: Task 2705 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/48d62580-f7b0-4b3f-8742-578e4da76a1f_isrForCrosstalkSources_996977_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/48d62580-f7b0-4b3f-8742-578e4da76a1f_isrForCrosstalkSources_996977_2.stderr parsl.dataflow.dflow INFO: Task 2877 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff5467530ed8534f9f3814a60f7247a5 with result from task 2877 parsl.dataflow.dflow DEBUG: Task 2877 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/016824c8-e6cc-49bf-a866-ac3c2b21fa20_isrForCrosstalkSources_996969_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/016824c8-e6cc-49bf-a866-ac3c2b21fa20_isrForCrosstalkSources_996969_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2755 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry dbe173eb9b6b4300df5e433a417867ea with result from task 2755 parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2755 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0e6380ec-c176-4cc2-8bc0-444541cf5f49_isrForCrosstalkSources_997013_56.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0e6380ec-c176-4cc2-8bc0-444541cf5f49_isrForCrosstalkSources_997013_56.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2851 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry a3d215860ff1afc24b1a1fbbcbb9f449 with result from task 2851 parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2851 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/70c11f32-7018-42ff-afe1-0dc18882af6f_isrForCrosstalkSources_997019_22.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/70c11f32-7018-42ff-afe1-0dc18882af6f_isrForCrosstalkSources_997019_22.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2779 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry faf69495ac9800d675f31c8689319e37 with result from task 2779 parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 2779 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 2779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/84673b74-329e-4474-b660-e5311fc0a238_isrForCrosstalkSources_997029_50.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 2779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/84673b74-329e-4474-b660-e5311fc0a238_isrForCrosstalkSources_997029_50.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 2800 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 6944ae5a20bbaeaabfec5c4d3a17ca28 with result from task 2800 parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:07 proc_lsst.multi:146 [INFO] found job 17075434 in provider checkpoint proc_lsst.multi INFO: found job 17075434 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2800 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7afabad8-4875-4e13-8488-e1e4d23e55ed_isrForCrosstalkSources_997028_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7afabad8-4875-4e13-8488-e1e4d23e55ed_isrForCrosstalkSources_997028_13.stderr parsl.dataflow.dflow INFO: Task 2852 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f18c9faac81ad8561074024d0825b3f with result from task 2852 parsl.dataflow.dflow DEBUG: Task 2852 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/51b35427-f3e8-43a8-8b89-c3103bc22a30_isrForCrosstalkSources_997007_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/51b35427-f3e8-43a8-8b89-c3103bc22a30_isrForCrosstalkSources_997007_36.stderr parsl.dataflow.dflow INFO: Task 2681 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 79d58a1bdc87b1bd08ee29a816b8f38f with result from task 2681 parsl.dataflow.dflow DEBUG: Task 2681 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/17b41f17-df5b-4e7d-93ef-916832d2cfc0_isrForCrosstalkSources_996953_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/17b41f17-df5b-4e7d-93ef-916832d2cfc0_isrForCrosstalkSources_996953_52.stderr parsl.dataflow.dflow INFO: Task 2857 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6fda3ea0743dd71bed7990be1e41b0a5 with result from task 2857 parsl.dataflow.dflow DEBUG: Task 2857 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/58d22a18-9e2f-4a29-ae36-4a9e356e7635_isrForCrosstalkSources_997029_60.stdout parsl.dataflow.dflow INFO: Standard error for task 2857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/58d22a18-9e2f-4a29-ae36-4a9e356e7635_isrForCrosstalkSources_997029_60.stderr parsl.dataflow.dflow INFO: Task 2854 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11c2d68890bd4934c69c4278aded8d34 with result from task 2854 parsl.dataflow.dflow DEBUG: Task 2854 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/a26a73ae-e1d9-4fc0-b5a4-7895bdc7c964_isrForCrosstalkSources_996997_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/a26a73ae-e1d9-4fc0-b5a4-7895bdc7c964_isrForCrosstalkSources_996997_41.stderr parsl.dataflow.dflow INFO: Task 2672 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 34e2f3943abcc7b5f0c40ea3abfd1179 with result from task 2672 parsl.dataflow.dflow DEBUG: Task 2672 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a9840771-17fb-45ae-a7cd-315d105515da_isrForCrosstalkSources_996954_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a9840771-17fb-45ae-a7cd-315d105515da_isrForCrosstalkSources_996954_36.stderr parsl.dataflow.dflow INFO: Task 2757 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e0413f801a28b6a3cdeb4fe6c942d3a with result from task 2757 parsl.dataflow.dflow DEBUG: Task 2757 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fc99ac1e-85a6-4e0f-841b-2353732dd4ac_isrForCrosstalkSources_997008_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/fc99ac1e-85a6-4e0f-841b-2353732dd4ac_isrForCrosstalkSources_997008_44.stderr parsl.dataflow.dflow INFO: Task 2761 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 344319d79aa25bb4a946f4672421780a with result from task 2761 parsl.dataflow.dflow DEBUG: Task 2761 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6298f33-5797-4f3a-add9-c8747b0acba1_isrForCrosstalkSources_996976_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a6298f33-5797-4f3a-add9-c8747b0acba1_isrForCrosstalkSources_996976_14.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 2754 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1323a18e9e7047c54add2c00743985cf with result from task 2754 parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2754 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:08 proc_lsst.multi:146 [INFO] found job 17075435 in provider checkpoint proc_lsst.multi INFO: found job 17075435 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ca0065b3-20fd-4db9-a76d-ae78612dd75f_isrForCrosstalkSources_996969_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ca0065b3-20fd-4db9-a76d-ae78612dd75f_isrForCrosstalkSources_996969_26.stderr parsl.dataflow.dflow INFO: Task 2882 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53d6dd31779878c04bff5fa174590bf2 with result from task 2882 parsl.dataflow.dflow DEBUG: Task 2882 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/67df462c-89b2-4a04-8f19-db8cbe9b780d_isrForCrosstalkSources_997023_26.stdout parsl.dataflow.dflow INFO: Standard error for task 2882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/67df462c-89b2-4a04-8f19-db8cbe9b780d_isrForCrosstalkSources_997023_26.stderr parsl.dataflow.dflow INFO: Task 3011 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ba5407d313de982727a6d343004dacc with result from task 3011 parsl.dataflow.dflow DEBUG: Task 3011 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/096802de-d9fb-4ce7-b096-93deecc06472_isrForCrosstalkSources_996979_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/096802de-d9fb-4ce7-b096-93deecc06472_isrForCrosstalkSources_996979_52.stderr parsl.dataflow.dflow INFO: Task 3012 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4517a23d92db807add388ea6ff141d9e with result from task 3012 parsl.dataflow.dflow DEBUG: Task 3012 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b2e85978-c066-4be5-aff5-dd6e33156c59_isrForCrosstalkSources_997007_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/b2e85978-c066-4be5-aff5-dd6e33156c59_isrForCrosstalkSources_997007_8.stderr parsl.dataflow.dflow INFO: Task 2973 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 28c7214b31c333dbb104e92071d6bbe5 with result from task 2973 parsl.dataflow.dflow DEBUG: Task 2973 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f693525c-1bd5-4d6c-aa81-f9ead8265f15_isrForCrosstalkSources_996999_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f693525c-1bd5-4d6c-aa81-f9ead8265f15_isrForCrosstalkSources_996999_25.stderr parsl.dataflow.dflow INFO: Task 2899 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df51685c14efbe62f6322c87bf1b4773 with result from task 2899 parsl.dataflow.dflow DEBUG: Task 2899 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/74cb9618-e6c2-4682-8556-42b8d406947b_isrForCrosstalkSources_996979_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/74cb9618-e6c2-4682-8556-42b8d406947b_isrForCrosstalkSources_996979_33.stderr parsl.dataflow.dflow INFO: Task 2804 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32dd83cd69e301ad98e70ed9502ea51f with result from task 2804 parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:10 proc_lsst.multi:146 [INFO] found job 17075436 in provider checkpoint proc_lsst.multi INFO: found job 17075436 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 2804 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0146f99c-03df-40ca-849b-bbe9ef43c06d_isrForCrosstalkSources_996966_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0146f99c-03df-40ca-849b-bbe9ef43c06d_isrForCrosstalkSources_996966_8.stderr parsl.dataflow.dflow INFO: Task 3013 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fd7e17a7ea53101b18b1e0071f2195b with result from task 3013 parsl.dataflow.dflow DEBUG: Task 3013 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e4ad14f8-0715-4151-8270-6a8b903740ee_isrForCrosstalkSources_996961_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e4ad14f8-0715-4151-8270-6a8b903740ee_isrForCrosstalkSources_996961_10.stderr parsl.dataflow.dflow INFO: Task 3004 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba3f491bea3f5e158e0fc54437ff48ad with result from task 3004 parsl.dataflow.dflow DEBUG: Task 3004 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0ff322b0-33f5-40ea-b0e2-275bd9d6c434_isrForCrosstalkSources_996991_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0ff322b0-33f5-40ea-b0e2-275bd9d6c434_isrForCrosstalkSources_996991_33.stderr parsl.dataflow.dflow INFO: Task 2630 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a490f01c6503feecaf9738f2945d044c with result from task 2630 parsl.dataflow.dflow DEBUG: Task 2630 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/64d44d0e-f5b1-4d7a-b10f-d30c639456ac_isrForCrosstalkSources_996984_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/64d44d0e-f5b1-4d7a-b10f-d30c639456ac_isrForCrosstalkSources_996984_28.stderr parsl.dataflow.dflow INFO: Task 3036 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9f7f9f3962a4d2e82f560f2c1f597a3e with result from task 3036 parsl.dataflow.dflow DEBUG: Task 3036 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/edc50e1d-c603-458c-befc-f63731bc9f9a_isrForCrosstalkSources_997002_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/edc50e1d-c603-458c-befc-f63731bc9f9a_isrForCrosstalkSources_997002_25.stderr parsl.dataflow.dflow INFO: Task 2896 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de47e409225523efad5cadc758bb0874 with result from task 2896 parsl.dataflow.dflow DEBUG: Task 2896 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8159cd99-79a8-437c-8416-de7ec5a6ac98_isrForCrosstalkSources_996987_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/8159cd99-79a8-437c-8416-de7ec5a6ac98_isrForCrosstalkSources_996987_4.stderr parsl.dataflow.dflow INFO: Task 2717 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b3cc96dda2810d6d7d73ef7c4c7e6d2 with result from task 2717 parsl.dataflow.dflow DEBUG: Task 2717 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7b76f480-9a32-4023-88ff-2b1ac8474336_isrForCrosstalkSources_996991_19.stdout parsl.dataflow.dflow INFO: Standard error for task 2717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/7b76f480-9a32-4023-88ff-2b1ac8474336_isrForCrosstalkSources_996991_19.stderr parsl.dataflow.dflow INFO: Task 2923 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f66ab91aaded5a002dc8e9601e10ef0 with result from task 2923 parsl.dataflow.dflow DEBUG: Task 2923 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/25621a10-68e8-4f6e-8ff3-686ad8cf4dbc_isrForCrosstalkSources_997006_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/25621a10-68e8-4f6e-8ff3-686ad8cf4dbc_isrForCrosstalkSources_997006_34.stderr parsl.dataflow.dflow INFO: Task 2564 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a6650dfc180e232edeba8c0b6a4364c with result from task 2564 parsl.dataflow.dflow DEBUG: Task 2564 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/5fdf38de-fb87-40ca-924f-e2cd44eeba8a_isrForCrosstalkSources_996958_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/5fdf38de-fb87-40ca-924f-e2cd44eeba8a_isrForCrosstalkSources_996958_44.stderr parsl.dataflow.dflow INFO: Task 2805 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78c42576504f79173bf77707adfda7b1 with result from task 2805 parsl.dataflow.dflow DEBUG: Task 2805 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ff958221-a5c2-4896-8d23-4dc68fec81d2_isrForCrosstalkSources_996969_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ff958221-a5c2-4896-8d23-4dc68fec81d2_isrForCrosstalkSources_996969_8.stderr parsl.dataflow.dflow INFO: Task 2911 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21a3fdf5a250178738cd3a5b5da6722d with result from task 2911 parsl.dataflow.dflow DEBUG: Task 2911 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a4987a81-1dcd-4b92-9804-91ba6df6ec36_isrForCrosstalkSources_996956_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a4987a81-1dcd-4b92-9804-91ba6df6ec36_isrForCrosstalkSources_996956_36.stderr parsl.dataflow.dflow INFO: Task 3044 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a828a4fb09995d7e2fd427d82ec1e3da with result from task 3044 parsl.dataflow.dflow DEBUG: Task 3044 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/de9eab24-f14e-4e6e-9997-78e47e03d8b0_isrForCrosstalkSources_996968_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/de9eab24-f14e-4e6e-9997-78e47e03d8b0_isrForCrosstalkSources_996968_32.stderr parsl.dataflow.dflow INFO: Task 2925 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08949a5cf1b329a62d55556f7180ca49 with result from task 2925 parsl.dataflow.dflow DEBUG: Task 2925 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/098becc5-a9cf-4d93-bef9-e8c2839b5cb9_isrForCrosstalkSources_997020_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/098becc5-a9cf-4d93-bef9-e8c2839b5cb9_isrForCrosstalkSources_997020_52.stderr parsl.dataflow.dflow INFO: Task 2872 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f40a01e52ec90794bc22e8a21bdd19d with result from task 2872 parsl.dataflow.dflow DEBUG: Task 2872 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/c0e870b7-0e31-4bac-8fb1-c07a565d6a4d_isrForCrosstalkSources_996988_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/c0e870b7-0e31-4bac-8fb1-c07a565d6a4d_isrForCrosstalkSources_996988_16.stderr parsl.dataflow.dflow INFO: Task 2901 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d388d3568050b0114c5f77d46bb2a0a7 with result from task 2901 parsl.dataflow.dflow DEBUG: Task 2901 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/77b3c704-cf5b-46ae-aa08-d481a89a931d_isrForCrosstalkSources_996999_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/77b3c704-cf5b-46ae-aa08-d481a89a931d_isrForCrosstalkSources_996999_38.stderr parsl.dataflow.dflow INFO: Task 2894 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6588ca6b0fb7b40eb66c897f8ec5f243 with result from task 2894 parsl.dataflow.dflow DEBUG: Task 2894 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/430af234-623f-43f6-800f-d18f71cda8c3_isrForCrosstalkSources_997014_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/430af234-623f-43f6-800f-d18f71cda8c3_isrForCrosstalkSources_997014_18.stderr parsl.dataflow.dflow INFO: Task 3024 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56e34e04fbd21279873be22974384d06 with result from task 3024 parsl.dataflow.dflow DEBUG: Task 3024 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/7962e892-a9f6-429e-90ff-b0c71b42b6e7_isrForCrosstalkSources_996969_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/7962e892-a9f6-429e-90ff-b0c71b42b6e7_isrForCrosstalkSources_996969_56.stderr parsl.dataflow.dflow INFO: Task 2966 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93d6e7faf56712308e476563de2e7409 with result from task 2966 parsl.dataflow.dflow DEBUG: Task 2966 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/66f338f4-14da-4010-bc29-b1d9b1c8c9ef_isrForCrosstalkSources_996953_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/66f338f4-14da-4010-bc29-b1d9b1c8c9ef_isrForCrosstalkSources_996953_1.stderr parsl.dataflow.dflow INFO: Task 2714 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63c5e855c2ee494ba56e7bfb5d1e3266 with result from task 2714 parsl.dataflow.dflow DEBUG: Task 2714 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/35f76edf-2ea6-4a09-8e5f-2bc3142cf40b_isrForCrosstalkSources_996978_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/35f76edf-2ea6-4a09-8e5f-2bc3142cf40b_isrForCrosstalkSources_996978_40.stderr parsl.dataflow.dflow INFO: Task 2954 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f05c463baebbd03560b10915784c623 with result from task 2954 parsl.dataflow.dflow DEBUG: Task 2954 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d2de8f0d-6bcb-462a-a24b-7924907818a0_isrForCrosstalkSources_997023_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d2de8f0d-6bcb-462a-a24b-7924907818a0_isrForCrosstalkSources_997023_39.stderr parsl.dataflow.dflow INFO: Task 2916 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac8387780ee1c71ff1331359f7e4e484 with result from task 2916 parsl.dataflow.dflow DEBUG: Task 2916 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/505e72f3-8f0a-46fe-b614-c08c1f469103_isrForCrosstalkSources_996987_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/505e72f3-8f0a-46fe-b614-c08c1f469103_isrForCrosstalkSources_996987_38.stderr parsl.dataflow.dflow INFO: Task 3041 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 560958136f08fa185114e5e5295a91e7 with result from task 3041 parsl.dataflow.dflow DEBUG: Task 3041 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ce4f8ba5-ee26-4628-9073-ea03ce3b9ad9_isrForCrosstalkSources_997019_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ce4f8ba5-ee26-4628-9073-ea03ce3b9ad9_isrForCrosstalkSources_997019_34.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:10 proc_lsst.multi:146 [INFO] found job 17075438 in provider checkpoint proc_lsst.multi INFO: found job 17075438 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2904 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1338e1484eb33b4203d83ff87c718813 with result from task 2904 parsl.dataflow.dflow DEBUG: Task 2904 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/18599bbd-7a14-488b-bceb-e2749a2d046e_isrForCrosstalkSources_997006_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/18599bbd-7a14-488b-bceb-e2749a2d046e_isrForCrosstalkSources_997006_42.stderr parsl.dataflow.dflow INFO: Task 3027 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9158eb78cd6caea7271d5f25506fbe65 with result from task 3027 parsl.dataflow.dflow DEBUG: Task 3027 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/4254ade1-db05-418f-8ec8-8cceab8dbf76_isrForCrosstalkSources_996967_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/4254ade1-db05-418f-8ec8-8cceab8dbf76_isrForCrosstalkSources_996967_39.stderr parsl.dataflow.dflow INFO: Task 2683 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea3b3e608e504efaeede09498599ef6e with result from task 2683 parsl.dataflow.dflow DEBUG: Task 2683 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/16eeb570-bcbf-4ddb-8c51-7268f017d4dc_isrForCrosstalkSources_997003_38.stdout parsl.dataflow.dflow INFO: Standard error for task 2683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/16eeb570-bcbf-4ddb-8c51-7268f017d4dc_isrForCrosstalkSources_997003_38.stderr parsl.dataflow.dflow INFO: Task 3053 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba1e28c2a3b67efa6d2d8e1f4fa2b997 with result from task 3053 parsl.dataflow.dflow DEBUG: Task 3053 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ea9bcd71-cbee-4a99-8614-7e63a244f79e_isrForCrosstalkSources_996962_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ea9bcd71-cbee-4a99-8614-7e63a244f79e_isrForCrosstalkSources_996962_58.stderr parsl.dataflow.dflow INFO: Task 3047 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a37e742db8518be3b31c3267b4ee3ba0 with result from task 3047 parsl.dataflow.dflow DEBUG: Task 3047 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/e4606221-2e5b-4561-86c4-9c47e3842139_isrForCrosstalkSources_996990_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/e4606221-2e5b-4561-86c4-9c47e3842139_isrForCrosstalkSources_996990_25.stderr parsl.dataflow.dflow INFO: Task 2750 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b26977ea553afb01daf97d4095f77dc3 with result from task 2750 parsl.dataflow.dflow DEBUG: Task 2750 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5f5d6688-d8fb-476c-aeef-1abfeef189bf_isrForCrosstalkSources_996955_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5f5d6688-d8fb-476c-aeef-1abfeef189bf_isrForCrosstalkSources_996955_43.stderr parsl.dataflow.dflow INFO: Task 2890 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27e73c797ba4dc2b35562344783853b8 with result from task 2890 parsl.dataflow.dflow DEBUG: Task 2890 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/adbc9df0-7930-4b00-a2d8-99a3a373f036_isrForCrosstalkSources_996994_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/adbc9df0-7930-4b00-a2d8-99a3a373f036_isrForCrosstalkSources_996994_62.stderr parsl.dataflow.dflow INFO: Task 3059 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b37c6b0491915c353181b377dc90684e with result from task 3059 parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:11 proc_lsst.multi:146 [INFO] found job 17075440 in provider checkpoint proc_lsst.multi INFO: found job 17075440 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3059 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1f0d9875-0028-485b-9f96-2e1afeb00a1c_isrForCrosstalkSources_996983_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1f0d9875-0028-485b-9f96-2e1afeb00a1c_isrForCrosstalkSources_996983_26.stderr parsl.dataflow.dflow INFO: Task 2971 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 444f005941b11c45dc57b6a7ca011ab7 with result from task 2971 parsl.dataflow.dflow DEBUG: Task 2971 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fbd49976-f9cf-4ec8-b6a5-e9cf59b4542f_isrForCrosstalkSources_996976_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fbd49976-f9cf-4ec8-b6a5-e9cf59b4542f_isrForCrosstalkSources_996976_39.stderr parsl.dataflow.dflow INFO: Task 2841 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54e371ecba867a6988618232f8541447 with result from task 2841 parsl.dataflow.dflow DEBUG: Task 2841 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b9c6432c-2b5b-4a42-bd02-cb0e759ff511_isrForCrosstalkSources_997014_4.stdout parsl.dataflow.dflow INFO: Standard error for task 2841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b9c6432c-2b5b-4a42-bd02-cb0e759ff511_isrForCrosstalkSources_997014_4.stderr parsl.dataflow.dflow INFO: Task 3008 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11d7a8667530c2498ba5de2d2e8dd5ea with result from task 3008 parsl.dataflow.dflow DEBUG: Task 3008 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/ce723bf2-010a-42bf-9843-a4e565f9b571_isrForCrosstalkSources_996967_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/ce723bf2-010a-42bf-9843-a4e565f9b571_isrForCrosstalkSources_996967_3.stderr parsl.dataflow.dflow INFO: Task 2885 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8008012574cebbf5c8fa29e9fc3a1c77 with result from task 2885 parsl.dataflow.dflow DEBUG: Task 2885 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5e50a1b4-61e9-4e3f-871d-a90389ccd63c_isrForCrosstalkSources_997031_13.stdout parsl.dataflow.dflow INFO: Standard error for task 2885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5e50a1b4-61e9-4e3f-871d-a90389ccd63c_isrForCrosstalkSources_997031_13.stderr parsl.dataflow.dflow INFO: Task 3061 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33424ab6dd3192f7f45394d487327882 with result from task 3061 parsl.dataflow.dflow DEBUG: Task 3061 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/7871d022-24c9-4a5f-8f85-1afd5d1579ed_isrForCrosstalkSources_996952_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/7871d022-24c9-4a5f-8f85-1afd5d1579ed_isrForCrosstalkSources_996952_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:12 proc_lsst.multi:146 [INFO] found job 17075441 in provider checkpoint proc_lsst.multi INFO: found job 17075441 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2739 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ee00339ff892e40a3dc9c4ba39ec5585 with result from task 2739 parsl.dataflow.dflow DEBUG: Task 2739 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/22d0bce4-7fee-4509-9afa-67be4bef9cbc_isrForCrosstalkSources_996954_48.stdout parsl.dataflow.dflow INFO: Standard error for task 2739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/22d0bce4-7fee-4509-9afa-67be4bef9cbc_isrForCrosstalkSources_996954_48.stderr parsl.dataflow.dflow INFO: Task 3039 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d6b7f3673fa94e6a8f05af7d87c9555 with result from task 3039 parsl.dataflow.dflow DEBUG: Task 3039 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/aba108d4-09e3-47de-b6bd-31eb36118d2d_isrForCrosstalkSources_997013_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/aba108d4-09e3-47de-b6bd-31eb36118d2d_isrForCrosstalkSources_997013_16.stderr parsl.dataflow.dflow INFO: Task 2698 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 127310d7a3006da4a22f2c9cf5768d7f with result from task 2698 parsl.dataflow.dflow DEBUG: Task 2698 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/77d78653-487c-4c7c-9b9c-85eff45fe016_isrForCrosstalkSources_997004_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/77d78653-487c-4c7c-9b9c-85eff45fe016_isrForCrosstalkSources_997004_55.stderr parsl.dataflow.dflow INFO: Task 2820 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0261985c9c1afd0419ff7b3024327aa6 with result from task 2820 parsl.dataflow.dflow DEBUG: Task 2820 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2c0ec059-31c6-4908-9bfd-f467b1a97ec4_isrForCrosstalkSources_996961_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2c0ec059-31c6-4908-9bfd-f467b1a97ec4_isrForCrosstalkSources_996961_55.stderr parsl.dataflow.dflow INFO: Task 2976 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 389ef78db7ca48d7ef87459585db25f0 with result from task 2976 parsl.dataflow.dflow DEBUG: Task 2976 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/37fd0a1b-f2b1-4132-bbc7-5986f3c5c14f_isrForCrosstalkSources_996978_11.stdout parsl.dataflow.dflow INFO: Standard error for task 2976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/37fd0a1b-f2b1-4132-bbc7-5986f3c5c14f_isrForCrosstalkSources_996978_11.stderr parsl.dataflow.dflow INFO: Task 2986 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f3c29899fe0a2aa08426899f1210eaa9 with result from task 2986 parsl.dataflow.dflow DEBUG: Task 2986 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/64e68dd7-36f9-4bf4-9166-1da476bf6b83_isrForCrosstalkSources_996984_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/64e68dd7-36f9-4bf4-9166-1da476bf6b83_isrForCrosstalkSources_996984_22.stderr parsl.dataflow.dflow INFO: Task 2694 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5129166a6e9229c353952230a3cc1e7a with result from task 2694 parsl.dataflow.dflow DEBUG: Task 2694 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d9629746-9a1c-4a5c-9a95-435d85e970ed_isrForCrosstalkSources_996965_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d9629746-9a1c-4a5c-9a95-435d85e970ed_isrForCrosstalkSources_996965_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2900 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry d4ea9e344c17c9cc0cfd559eac1ce120 with result from task 2900 parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2900 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/f992f867-009c-44be-a67f-5e77316f227a_isrForCrosstalkSources_996998_34.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/f992f867-009c-44be-a67f-5e77316f227a_isrForCrosstalkSources_996998_34.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:13 proc_lsst.multi:146 [INFO] found job 17075443 in provider checkpoint proc_lsst.multi INFO: found job 17075443 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3019 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55b843784df6eb37bf0c7d2b1682ddc8 with result from task 3019 parsl.dataflow.dflow DEBUG: Task 3019 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c40c26ae-2d87-4e6a-9c52-685cdcd45848_isrForCrosstalkSources_996970_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c40c26ae-2d87-4e6a-9c52-685cdcd45848_isrForCrosstalkSources_996970_49.stderr parsl.dataflow.dflow INFO: Task 3051 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83f8d39ef60f6e9d0afb96861625f527 with result from task 3051 parsl.dataflow.dflow DEBUG: Task 3051 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/4b464d88-c991-41f9-ab32-349746c1bcbe_isrForCrosstalkSources_997004_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/4b464d88-c991-41f9-ab32-349746c1bcbe_isrForCrosstalkSources_997004_43.stderr parsl.dataflow.dflow INFO: Task 2991 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0eb294141394b2d7d3a4a64216e79585 with result from task 2991 parsl.dataflow.dflow DEBUG: Task 2991 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7e67da05-bd8e-4d67-b18d-ae6bd3b35fef_isrForCrosstalkSources_997028_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7e67da05-bd8e-4d67-b18d-ae6bd3b35fef_isrForCrosstalkSources_997028_2.stderr parsl.dataflow.dflow INFO: Task 2719 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 599ca1ee3a7117346f003ecfde8b4242 with result from task 2719 parsl.dataflow.dflow DEBUG: Task 2719 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c4e738fe-f4eb-45d8-8ff9-801c20f71595_isrForCrosstalkSources_997015_46.stdout parsl.dataflow.dflow INFO: Standard error for task 2719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/c4e738fe-f4eb-45d8-8ff9-801c20f71595_isrForCrosstalkSources_997015_46.stderr parsl.dataflow.dflow INFO: Task 2770 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 706d7b58f17ff32dfe3191df0b90226f with result from task 2770 parsl.dataflow.dflow DEBUG: Task 2770 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fde6959b-c59f-4217-bb08-82cf397234c7_isrForCrosstalkSources_997002_5.stdout parsl.dataflow.dflow INFO: Standard error for task 2770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fde6959b-c59f-4217-bb08-82cf397234c7_isrForCrosstalkSources_997002_5.stderr parsl.dataflow.dflow INFO: Task 2921 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63acbd83dd52349387386f0f37c76491 with result from task 2921 parsl.dataflow.dflow DEBUG: Task 2921 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/cbd947ee-9ed1-40fd-a64c-391c72c7c10b_isrForCrosstalkSources_997029_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/cbd947ee-9ed1-40fd-a64c-391c72c7c10b_isrForCrosstalkSources_997029_37.stderr parsl.dataflow.dflow INFO: Task 2740 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 076681874598ab02a62943fd64884aab with result from task 2740 parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2740 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a1b79875-844b-48e0-aab8-099f6490fdea_isrForCrosstalkSources_997011_31.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a1b79875-844b-48e0-aab8-099f6490fdea_isrForCrosstalkSources_997011_31.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2743 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 1e118ddb0373183f0d6c294c7e49e812 with result from task 2743 parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2743 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/09948192-1f6b-4c9b-a6d2-2835e80ce03b_isrForCrosstalkSources_997005_21.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/09948192-1f6b-4c9b-a6d2-2835e80ce03b_isrForCrosstalkSources_997005_21.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 2815 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry ab1aef9caebe71bdc37927e0c4503963 with result from task 2815 parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2815 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 2815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e8c0a489-9fda-4e76-8194-5c72d39d5316_isrForCrosstalkSources_997010_46.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 2815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e8c0a489-9fda-4e76-8194-5c72d39d5316_isrForCrosstalkSources_997010_46.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:14 proc_lsst.multi:146 [INFO] found job 17075444 in provider checkpoint proc_lsst.multi INFO: found job 17075444 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 2831 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 466676ee5f95ebac7369a2330bd5aa61 with result from task 2831 parsl.dataflow.dflow DEBUG: Task 2831 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b310dacb-a51b-415a-a393-1354369d3c36_isrForCrosstalkSources_997033_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/b310dacb-a51b-415a-a393-1354369d3c36_isrForCrosstalkSources_997033_58.stderr parsl.dataflow.dflow INFO: Task 2832 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4109a575bfc014d41ae2bb6d458944a5 with result from task 2832 parsl.dataflow.dflow DEBUG: Task 2832 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e3cc12da-4671-4157-8dc3-49107cc1bc43_isrForCrosstalkSources_997028_10.stdout parsl.dataflow.dflow INFO: Standard error for task 2832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e3cc12da-4671-4157-8dc3-49107cc1bc43_isrForCrosstalkSources_997028_10.stderr parsl.dataflow.dflow INFO: Task 3005 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd78a79b5960b37063e4b926c86c474a with result from task 3005 parsl.dataflow.dflow DEBUG: Task 3005 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/e51df0eb-a9e5-4ddc-90ba-54c995125c78_isrForCrosstalkSources_996969_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/e51df0eb-a9e5-4ddc-90ba-54c995125c78_isrForCrosstalkSources_996969_62.stderr parsl.dataflow.dflow INFO: Task 2772 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3af1e7f148e2daef9ff293aeea27b93d with result from task 2772 parsl.dataflow.dflow DEBUG: Task 2772 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/06302638-8e75-4ee2-b6f7-d3160659ceb6_isrForCrosstalkSources_996984_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/06302638-8e75-4ee2-b6f7-d3160659ceb6_isrForCrosstalkSources_996984_51.stderr parsl.dataflow.dflow INFO: Task 3046 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d82e86f6206c18045809b624d6d0b677 with result from task 3046 parsl.dataflow.dflow DEBUG: Task 3046 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/13561822-c4e7-4aac-9028-1ece5f90f6f8_isrForCrosstalkSources_996992_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/13561822-c4e7-4aac-9028-1ece5f90f6f8_isrForCrosstalkSources_996992_42.stderr parsl.dataflow.dflow INFO: Task 3017 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73c1d4bb33f2f5420eeb6df79a34017c with result from task 3017 parsl.dataflow.dflow DEBUG: Task 3017 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/11090e71-c76d-4b99-bdd6-787452ca6ef2_isrForCrosstalkSources_996989_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/11090e71-c76d-4b99-bdd6-787452ca6ef2_isrForCrosstalkSources_996989_44.stderr parsl.dataflow.dflow INFO: Task 2809 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 485bb949d0da834c4c497483c7a7206c with result from task 2809 parsl.dataflow.dflow DEBUG: Task 2809 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/594fbdb2-bf1e-440d-a042-31f63ec5265a_isrForCrosstalkSources_996961_22.stdout parsl.dataflow.dflow INFO: Standard error for task 2809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/594fbdb2-bf1e-440d-a042-31f63ec5265a_isrForCrosstalkSources_996961_22.stderr parsl.dataflow.dflow INFO: Task 2792 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce3b87983da0e53e5e32db025d23d7c8 with result from task 2792 parsl.dataflow.dflow DEBUG: Task 2792 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/2eaba9d2-77a6-4945-b2c3-9b37cef27c20_isrForCrosstalkSources_997001_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/2eaba9d2-77a6-4945-b2c3-9b37cef27c20_isrForCrosstalkSources_997001_17.stderr parsl.dataflow.dflow INFO: Task 3045 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd83cadaf3bb0fe4ca8bf6355c9b171e with result from task 3045 parsl.dataflow.dflow DEBUG: Task 3045 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8b2458fe-5931-4bbd-a726-2d7ac86aee50_isrForCrosstalkSources_996973_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8b2458fe-5931-4bbd-a726-2d7ac86aee50_isrForCrosstalkSources_996973_43.stderr parsl.dataflow.dflow INFO: Task 3032 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08ae6bba737f296405fc834cad6ccf2d with result from task 3032 parsl.dataflow.dflow DEBUG: Task 3032 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7b4889b1-ee25-4134-92b1-014523f4c265_isrForCrosstalkSources_997028_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/7b4889b1-ee25-4134-92b1-014523f4c265_isrForCrosstalkSources_997028_29.stderr parsl.dataflow.dflow INFO: Task 2918 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8981d19f6eb0908f0f6077fe3e96c1d7 with result from task 2918 parsl.dataflow.dflow DEBUG: Task 2918 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/cd6e702e-5598-4311-a31f-659f52d1b054_isrForCrosstalkSources_997007_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/cd6e702e-5598-4311-a31f-659f52d1b054_isrForCrosstalkSources_997007_50.stderr parsl.dataflow.dflow INFO: Task 3010 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b2773fb5320f1cbfaea41b099cfe884 with result from task 3010 parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3010 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/49fb95d8-08b3-4bf2-a71d-418ed13d6c07_isrForCrosstalkSources_996984_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/49fb95d8-08b3-4bf2-a71d-418ed13d6c07_isrForCrosstalkSources_996984_23.stderr parsl.dataflow.dflow INFO: Task 2843 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f941a711b5f4a3eab69624c9de222ca7 with result from task 2843 parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2843 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/54aa51a1-3c2e-4381-8eac-16065b18ab3a_isrForCrosstalkSources_996995_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/54aa51a1-3c2e-4381-8eac-16065b18ab3a_isrForCrosstalkSources_996995_12.stderr parsl.dataflow.dflow INFO: Task 2686 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11a9def7ada4db0580f98520d494c43b with result from task 2686 parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 2686 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:15 proc_lsst.multi:146 [INFO] found job 17075446 in provider checkpoint proc_lsst.multi INFO: found job 17075446 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/80cb774e-f88d-4cb9-9116-f3806cbfddf9_isrForCrosstalkSources_996993_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/80cb774e-f88d-4cb9-9116-f3806cbfddf9_isrForCrosstalkSources_996993_61.stderr parsl.dataflow.dflow INFO: Task 2984 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78ded03b5e3ca54a1cbb8d3ce1c8b285 with result from task 2984 parsl.dataflow.dflow DEBUG: Task 2984 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/0e3d9767-4b54-414e-add0-004083a540a3_isrForCrosstalkSources_996989_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/0e3d9767-4b54-414e-add0-004083a540a3_isrForCrosstalkSources_996989_43.stderr parsl.dataflow.dflow INFO: Task 3063 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8deb75307181dab2025cbf53cef9c4db with result from task 3063 parsl.dataflow.dflow DEBUG: Task 3063 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e7eb1d2d-6c90-4cfa-b969-af0313cfe4fe_isrForCrosstalkSources_997018_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e7eb1d2d-6c90-4cfa-b969-af0313cfe4fe_isrForCrosstalkSources_997018_19.stderr parsl.dataflow.dflow INFO: Task 3040 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e0d7abd151a88a4967d90ec33a4c4791 with result from task 3040 parsl.dataflow.dflow DEBUG: Task 3040 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4f192782-6a2f-4ba9-b257-ba906efb6eee_isrForCrosstalkSources_996957_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4f192782-6a2f-4ba9-b257-ba906efb6eee_isrForCrosstalkSources_996957_7.stderr parsl.dataflow.dflow INFO: Task 3033 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 627e464f9e5bd9d5e175c4989ba3076a with result from task 3033 parsl.dataflow.dflow DEBUG: Task 3033 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/78696bd3-faa9-441a-8330-ae78d8cc2b36_isrForCrosstalkSources_997007_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/78696bd3-faa9-441a-8330-ae78d8cc2b36_isrForCrosstalkSources_997007_9.stderr parsl.dataflow.dflow INFO: Task 3050 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed684e77275beeb6c4b93eb717def826 with result from task 3050 parsl.dataflow.dflow DEBUG: Task 3050 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/619385ea-1176-42ab-b4b6-5d8632b2b716_isrForCrosstalkSources_996995_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/619385ea-1176-42ab-b4b6-5d8632b2b716_isrForCrosstalkSources_996995_27.stderr parsl.dataflow.dflow INFO: Task 3031 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 008f843f6b8087b4e4a1f48b1369cef8 with result from task 3031 parsl.dataflow.dflow DEBUG: Task 3031 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2b47ded9-08ee-40e8-b47e-a7ca75de44ca_isrForCrosstalkSources_997015_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2b47ded9-08ee-40e8-b47e-a7ca75de44ca_isrForCrosstalkSources_997015_35.stderr parsl.dataflow.dflow INFO: Task 2724 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2913fff1f2331f21715a9e3b2cd6f600 with result from task 2724 parsl.dataflow.dflow DEBUG: Task 2724 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e0592f54-7cc6-4649-9ef8-06eec036123c_isrForCrosstalkSources_997030_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e0592f54-7cc6-4649-9ef8-06eec036123c_isrForCrosstalkSources_997030_2.stderr parsl.dataflow.dflow INFO: Task 2780 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 578aae7e809141c60516591c08c89fcf with result from task 2780 parsl.dataflow.dflow DEBUG: Task 2780 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0589061b-d9de-471b-a2be-1e19f8d50d25_isrForCrosstalkSources_996958_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/0589061b-d9de-471b-a2be-1e19f8d50d25_isrForCrosstalkSources_996958_61.stderr parsl.dataflow.dflow INFO: Task 3037 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53e0156b65b5cc008ae22b3001c7bf51 with result from task 3037 parsl.dataflow.dflow DEBUG: Task 3037 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e67031e3-f9f9-4313-8889-8c6f7d73b36d_isrForCrosstalkSources_996968_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/e67031e3-f9f9-4313-8889-8c6f7d73b36d_isrForCrosstalkSources_996968_18.stderr parsl.dataflow.dflow INFO: Task 2989 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9e5b44027ec0e13c31852afc1bec811 with result from task 2989 parsl.dataflow.dflow DEBUG: Task 2989 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1ca61b3b-c605-4273-95ec-1e54c9a8b7a5_isrForCrosstalkSources_996960_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1ca61b3b-c605-4273-95ec-1e54c9a8b7a5_isrForCrosstalkSources_996960_23.stderr parsl.dataflow.dflow INFO: Task 2721 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5bf0d034f61f67b78412d3cac2d1acaf with result from task 2721 parsl.dataflow.dflow DEBUG: Task 2721 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f549354d-565c-4b4a-a947-bbb3528f6a03_isrForCrosstalkSources_997005_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f549354d-565c-4b4a-a947-bbb3528f6a03_isrForCrosstalkSources_997005_41.stderr parsl.dataflow.dflow INFO: Task 2856 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ce982d27d4ceacc9028857bece750f0 with result from task 2856 parsl.dataflow.dflow DEBUG: Task 2856 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/bd6ca355-1875-425d-958f-64a37c2b347a_isrForCrosstalkSources_996990_24.stdout parsl.dataflow.dflow INFO: Standard error for task 2856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/bd6ca355-1875-425d-958f-64a37c2b347a_isrForCrosstalkSources_996990_24.stderr parsl.dataflow.dflow INFO: Task 2663 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07c7597da331da7077535cb1e2279cd0 with result from task 2663 parsl.dataflow.dflow DEBUG: Task 2663 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f6389cce-625e-4fd7-b9a0-af2a81dfc453_isrForCrosstalkSources_996988_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f6389cce-625e-4fd7-b9a0-af2a81dfc453_isrForCrosstalkSources_996988_54.stderr parsl.dataflow.dflow INFO: Task 3065 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 87ce4923a46d31d8328e663bf24ed76f with result from task 3065 parsl.dataflow.dflow DEBUG: Task 3065 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/cb3e7e4f-dfa8-462d-8b5d-9bd9df0c9b77_isrForCrosstalkSources_997025_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/cb3e7e4f-dfa8-462d-8b5d-9bd9df0c9b77_isrForCrosstalkSources_997025_45.stderr parsl.dataflow.dflow INFO: Task 2658 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 984a43e380a6fec91669db88b3762259 with result from task 2658 parsl.dataflow.dflow DEBUG: Task 2658 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/cffa26f2-d8a2-48f2-9149-4f898a057b6f_isrForCrosstalkSources_996990_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/cffa26f2-d8a2-48f2-9149-4f898a057b6f_isrForCrosstalkSources_996990_59.stderr parsl.dataflow.dflow INFO: Task 3062 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44ff9380a95aa21ccf4523c8e05f59f4 with result from task 3062 parsl.dataflow.dflow DEBUG: Task 3062 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/29741fc6-0174-44e5-875b-10bc9cb125d5_isrForCrosstalkSources_996985_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/29741fc6-0174-44e5-875b-10bc9cb125d5_isrForCrosstalkSources_996985_39.stderr parsl.dataflow.dflow INFO: Task 2788 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aab4f92cfc8a4758cedab5b2638aa0ba with result from task 2788 parsl.dataflow.dflow DEBUG: Task 2788 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ad9cea19-ce2b-44c4-8d89-6e4cdcfa1fea_isrForCrosstalkSources_996994_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ad9cea19-ce2b-44c4-8d89-6e4cdcfa1fea_isrForCrosstalkSources_996994_51.stderr parsl.dataflow.dflow INFO: Task 3106 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9bc1287b384d9fd9f9c584b287b1e2b with result from task 3106 parsl.dataflow.dflow DEBUG: Task 3106 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/9f122589-9761-4d5b-ac08-40955bafe2ec_isrForCrosstalkSources_996973_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/9f122589-9761-4d5b-ac08-40955bafe2ec_isrForCrosstalkSources_996973_26.stderr parsl.dataflow.dflow INFO: Task 2742 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1db63237ed69d1926cafe605feeb3eb8 with result from task 2742 parsl.dataflow.dflow DEBUG: Task 2742 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/73d10808-8af1-4a5e-833b-4e510c4cf206_isrForCrosstalkSources_996999_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/73d10808-8af1-4a5e-833b-4e510c4cf206_isrForCrosstalkSources_996999_16.stderr parsl.dataflow.dflow INFO: Task 3102 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f9dafe2a5b69924eff9062ccf5816c36 with result from task 3102 parsl.dataflow.dflow DEBUG: Task 3102 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/46e468d9-9990-45f7-a69a-4b5205ac1e08_isrForCrosstalkSources_996966_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/46e468d9-9990-45f7-a69a-4b5205ac1e08_isrForCrosstalkSources_996966_32.stderr parsl.dataflow.dflow INFO: Task 2775 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f88a95c1badda884c5a3ebf54c2671c with result from task 2775 parsl.dataflow.dflow DEBUG: Task 2775 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7e84250d-b7e5-4f44-8221-56171d3ccb92_isrForCrosstalkSources_996971_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7e84250d-b7e5-4f44-8221-56171d3ccb92_isrForCrosstalkSources_996971_16.stderr parsl.dataflow.dflow INFO: Task 3072 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e2cb257591334989bfeb53f7eb7c4b0 with result from task 3072 parsl.dataflow.dflow DEBUG: Task 3072 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f523bb7f-d585-44e8-a8af-751adfa125be_isrForCrosstalkSources_996995_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f523bb7f-d585-44e8-a8af-751adfa125be_isrForCrosstalkSources_996995_2.stderr parsl.dataflow.dflow INFO: Task 2886 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5247f995f2a607c73597c74694b322a7 with result from task 2886 parsl.dataflow.dflow DEBUG: Task 2886 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/69d59daf-f7b0-4c3d-a2a1-b30bfef6664c_isrForCrosstalkSources_996956_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/69d59daf-f7b0-4c3d-a2a1-b30bfef6664c_isrForCrosstalkSources_996956_37.stderr parsl.dataflow.dflow INFO: Task 3071 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fb121b54c197e273f317eff481c3cfc with result from task 3071 parsl.dataflow.dflow DEBUG: Task 3071 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f9efee13-a9ba-43a2-8531-8814b297afe1_isrForCrosstalkSources_996977_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f9efee13-a9ba-43a2-8531-8814b297afe1_isrForCrosstalkSources_996977_41.stderr parsl.dataflow.dflow INFO: Task 2758 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 627412657227bceb5bdae67939cfdcfe with result from task 2758 parsl.dataflow.dflow DEBUG: Task 2758 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/01b47f1e-4529-4aaa-9594-58f5475b701c_isrForCrosstalkSources_997017_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/01b47f1e-4529-4aaa-9594-58f5475b701c_isrForCrosstalkSources_997017_7.stderr parsl.dataflow.dflow INFO: Task 3052 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c0cb8b186c2505105d368712b7909b8b with result from task 3052 parsl.dataflow.dflow DEBUG: Task 3052 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/81732adc-1400-47a3-9ce9-abd941898f64_isrForCrosstalkSources_997024_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/81732adc-1400-47a3-9ce9-abd941898f64_isrForCrosstalkSources_997024_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075059 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.strategy DEBUG: general strategy starting with strategy_type simple for 1 executors parsl.dataflow.strategy DEBUG: Strategizing for executor multi parsl.dataflow.dflow INFO: Task 2842 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c23ddbabb0c6f6f8dc82e084de5a605 with result from task 2842 parsl.dataflow.dflow DEBUG: Task 2842 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/4735e082-169a-4b6c-ac70-c366dccf6be2_isrForCrosstalkSources_996983_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/4735e082-169a-4b6c-ac70-c366dccf6be2_isrForCrosstalkSources_996983_7.stderr parsl.dataflow.dflow INFO: Task 3085 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a897478c1caa9cfd659ae6b5f2ec1584 with result from task 3085 parsl.dataflow.strategy DEBUG: Slot ratio calculation: active_slots = 312, active_tasks = 2157 parsl.dataflow.dflow DEBUG: Task 3085 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/0d1b2c91-5bf1-43a2-8fbb-3a4e47a33076_isrForCrosstalkSources_996965_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/0d1b2c91-5bf1-43a2-8fbb-3a4e47a33076_isrForCrosstalkSources_996965_54.stderr parsl.dataflow.dflow INFO: Task 2881 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8cf15f6f6cc6d53ebb9b149049bc9d21 with result from task 2881 parsl.dataflow.strategy DEBUG: Executor multi has 2157 active tasks, 74/238 running/pending blocks, and 281 connected workers parsl.dataflow.strategy DEBUG: Strategy case 2: slots are overloaded - (slot_ratio = active_slots/active_tasks) < parallelism parsl.dataflow.strategy DEBUG: Strategy case 2a: active_blocks 312 >= max_blocks 312 so not scaling out parsl.process_loggers DEBUG: Normal ending for _general_strategy on thread JobStatusPoller-Timer-Thread-22709913200592 2024-03-15 13:57:17 proc_lsst.multi:146 [INFO] found job 91028 in provider local proc_lsst.multi INFO: found job 91028 in provider local parsl.dataflow.dflow DEBUG: Task 2881 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/a1b09f11-b1d4-41a1-b8b5-0c4f62a5bb08_isrForCrosstalkSources_996985_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/a1b09f11-b1d4-41a1-b8b5-0c4f62a5bb08_isrForCrosstalkSources_996985_42.stderr parsl.dataflow.dflow INFO: Task 2909 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 621e2b42134ab12a06e6f7d134684e46 with result from task 2909 parsl.dataflow.dflow DEBUG: Task 2909 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e2c08f50-5d6b-4ca1-b6ee-0df6bccee3a0_isrForCrosstalkSources_997019_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e2c08f50-5d6b-4ca1-b6ee-0df6bccee3a0_isrForCrosstalkSources_997019_2.stderr 2024-03-15 13:57:18 proc_lsst.multi:146 [INFO] found job 17074833 in provider astro proc_lsst.multi INFO: found job 17074833 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2579 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93012d5ad734f5331a363ad458206cb6 with result from task 2579 parsl.dataflow.dflow DEBUG: Task 2579 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/bc4273d0-665e-4e08-9359-edc683cb50fd_isrForCrosstalkSources_996998_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/bc4273d0-665e-4e08-9359-edc683cb50fd_isrForCrosstalkSources_996998_37.stderr parsl.dataflow.dflow INFO: Task 3076 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3560b63f2a955ae3b3ae29d48e37cd4c with result from task 3076 parsl.dataflow.dflow DEBUG: Task 3076 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2ecd8fb3-30a9-4ea5-95b4-8459a8cfd81a_isrForCrosstalkSources_997002_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2ecd8fb3-30a9-4ea5-95b4-8459a8cfd81a_isrForCrosstalkSources_997002_58.stderr parsl.dataflow.dflow INFO: Task 3077 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a600fc3e8075ad4a2a93f16a7d06b06 with result from task 3077 parsl.dataflow.dflow DEBUG: Task 3077 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/b9d0a722-8984-411e-8b68-bdde85514447_isrForCrosstalkSources_996952_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/b9d0a722-8984-411e-8b68-bdde85514447_isrForCrosstalkSources_996952_26.stderr parsl.dataflow.dflow INFO: Task 3055 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ffca3351927062b82cdff3a214eebbd with result from task 3055 parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:18 proc_lsst.multi:146 [INFO] found job 17074834 in provider astro proc_lsst.multi INFO: found job 17074834 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow DEBUG: Task 3055 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/41f4562c-ebe4-460c-bf78-bed56f269c54_isrForCrosstalkSources_996993_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/41f4562c-ebe4-460c-bf78-bed56f269c54_isrForCrosstalkSources_996993_52.stderr parsl.dataflow.dflow INFO: Task 2969 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e2da2e237653e9f8bc63f417587fcd8 with result from task 2969 parsl.dataflow.dflow DEBUG: Task 2969 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/84bc61bd-f05b-47fc-80e5-efc034398aa3_isrForCrosstalkSources_996957_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/84bc61bd-f05b-47fc-80e5-efc034398aa3_isrForCrosstalkSources_996957_44.stderr parsl.dataflow.dflow INFO: Task 2846 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 090d0a896288336de03ee08fd4e550e8 with result from task 2846 parsl.dataflow.dflow DEBUG: Task 2846 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/81bd8950-a8a6-42f8-9f34-ec8b7054af85_isrForCrosstalkSources_996957_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/81bd8950-a8a6-42f8-9f34-ec8b7054af85_isrForCrosstalkSources_996957_30.stderr parsl.dataflow.dflow INFO: Task 2963 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3611dbaea1e41483e3b28d1e99788134 with result from task 2963 parsl.dataflow.dflow DEBUG: Task 2963 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/6954c032-28f4-4794-a486-10bd429c8dc2_isrForCrosstalkSources_996968_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/6954c032-28f4-4794-a486-10bd429c8dc2_isrForCrosstalkSources_996968_50.stderr parsl.dataflow.dflow INFO: Task 2558 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7db0e84450a8555b5e4f469ef9a3f2a with result from task 2558 parsl.dataflow.dflow DEBUG: Task 2558 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/10b9d48a-e43b-4e80-a571-322332c74ccb_isrForCrosstalkSources_996983_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/10b9d48a-e43b-4e80-a571-322332c74ccb_isrForCrosstalkSources_996983_58.stderr parsl.dataflow.dflow INFO: Task 3078 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27f4b76b7b83a423912066d7f94e2430 with result from task 3078 parsl.dataflow.dflow DEBUG: Task 3078 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/35716663-aec6-479c-9084-2f66155c907a_isrForCrosstalkSources_997024_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/35716663-aec6-479c-9084-2f66155c907a_isrForCrosstalkSources_997024_18.stderr parsl.dataflow.dflow INFO: Task 2974 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 470e0acb829f638b146693c282bcb5d0 with result from task 2974 parsl.dataflow.dflow DEBUG: Task 2974 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/f9706ef5-79a6-400c-be60-50023384ca1d_isrForCrosstalkSources_996984_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/f9706ef5-79a6-400c-be60-50023384ca1d_isrForCrosstalkSources_996984_58.stderr parsl.dataflow.dflow INFO: Task 2696 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a3642b07eb47c12c093de2c08436e5e with result from task 2696 parsl.dataflow.dflow DEBUG: Task 2696 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/d66b7ecd-5ba6-44e8-9f01-9621a12a9c9b_isrForCrosstalkSources_996986_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/d66b7ecd-5ba6-44e8-9f01-9621a12a9c9b_isrForCrosstalkSources_996986_3.stderr parsl.dataflow.dflow INFO: Task 3021 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4da64c92e5ac73b939465f8833b38850 with result from task 3021 parsl.dataflow.dflow DEBUG: Task 3021 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8681f580-71db-4641-a5e6-d610a12319af_isrForCrosstalkSources_996967_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8681f580-71db-4641-a5e6-d610a12319af_isrForCrosstalkSources_996967_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:19 proc_lsst.multi:146 [INFO] found job 17074835 in provider astro proc_lsst.multi INFO: found job 17074835 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3101 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f33e9de2894de45d3c4d6fca6f96cd9 with result from task 3101 parsl.dataflow.dflow DEBUG: Task 3101 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/3530f1e3-e453-4de0-a99d-0446f3855ec4_isrForCrosstalkSources_996965_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/3530f1e3-e453-4de0-a99d-0446f3855ec4_isrForCrosstalkSources_996965_51.stderr parsl.dataflow.dflow INFO: Task 3136 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 61448a1b2b8ddce78d98cf0bfef431cd with result from task 3136 parsl.dataflow.dflow DEBUG: Task 3136 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/93b9336d-dc5f-418f-a833-fd9ef095c264_isrForCrosstalkSources_996965_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/93b9336d-dc5f-418f-a833-fd9ef095c264_isrForCrosstalkSources_996965_20.stderr parsl.dataflow.dflow INFO: Task 2844 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1af92b33e50c7fc5d537476fd9d5cdc with result from task 2844 parsl.dataflow.dflow DEBUG: Task 2844 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/353923cc-d8e6-4f2b-b8fa-d799bc8bcdf5_isrForCrosstalkSources_997003_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/353923cc-d8e6-4f2b-b8fa-d799bc8bcdf5_isrForCrosstalkSources_997003_47.stderr parsl.dataflow.dflow INFO: Task 2829 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fc955bd9b8452964e5bb73c6e1243a2 with result from task 2829 parsl.dataflow.dflow DEBUG: Task 2829 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/2e6a3bb8-d391-417c-bab9-61795d2ebb3f_isrForCrosstalkSources_996959_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/2e6a3bb8-d391-417c-bab9-61795d2ebb3f_isrForCrosstalkSources_996959_16.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:19 proc_lsst.multi:146 [INFO] found job 17074836 in provider astro proc_lsst.multi INFO: found job 17074836 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3083 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 262d0d412fc91639dc2df7f10d5f2fc3 with result from task 3083 parsl.dataflow.dflow DEBUG: Task 3083 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ca264f78-0ef3-465e-816a-9325e1941a5f_isrForCrosstalkSources_997030_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ca264f78-0ef3-465e-816a-9325e1941a5f_isrForCrosstalkSources_997030_17.stderr parsl.dataflow.dflow INFO: Task 2826 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 552f6dfae671d0c138d68533a4ac382e with result from task 2826 parsl.dataflow.dflow DEBUG: Task 2826 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/6208dbf6-ba78-4896-8345-4b43a658825a_isrForCrosstalkSources_996992_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/6208dbf6-ba78-4896-8345-4b43a658825a_isrForCrosstalkSources_996992_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:20 proc_lsst.multi:146 [INFO] found job 17074837 in provider astro proc_lsst.multi INFO: found job 17074837 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3028 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65d043739a3f761b427a13790ba45a8f with result from task 3028 parsl.dataflow.dflow DEBUG: Task 3028 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/074a8b4c-4388-44e5-b25a-b1f8246f24d5_isrForCrosstalkSources_996980_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/074a8b4c-4388-44e5-b25a-b1f8246f24d5_isrForCrosstalkSources_996980_42.stderr parsl.dataflow.dflow INFO: Task 3129 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7f935eeb786560f7b795f9bc719a003 with result from task 3129 parsl.dataflow.dflow DEBUG: Task 3129 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/8172a616-97d8-4e43-9d82-d388d709dca4_isrForCrosstalkSources_997015_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/8172a616-97d8-4e43-9d82-d388d709dca4_isrForCrosstalkSources_997015_16.stderr parsl.dataflow.dflow INFO: Task 2849 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1edb84b678369c179022cfc860ed8320 with result from task 2849 parsl.dataflow.dflow DEBUG: Task 2849 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/761bfac8-fd31-486f-b742-8962ad5183e1_isrForCrosstalkSources_996958_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/761bfac8-fd31-486f-b742-8962ad5183e1_isrForCrosstalkSources_996958_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:20 proc_lsst.multi:146 [INFO] found job 17074838 in provider astro proc_lsst.multi INFO: found job 17074838 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3001 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3349f2cfc28ebb6346995609957d30a0 with result from task 3001 parsl.dataflow.dflow DEBUG: Task 3001 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/b9566bbd-120e-4169-8edd-c1f90b064009_isrForCrosstalkSources_997024_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/b9566bbd-120e-4169-8edd-c1f90b064009_isrForCrosstalkSources_997024_45.stderr parsl.dataflow.dflow INFO: Task 2711 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c825fbaf9e212c87295ed3dcb5a0d815 with result from task 2711 parsl.dataflow.dflow DEBUG: Task 2711 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/6f12dd47-3f07-4116-9cb5-5b503c0fbb89_isrForCrosstalkSources_997029_51.stdout parsl.dataflow.dflow INFO: Standard error for task 2711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/6f12dd47-3f07-4116-9cb5-5b503c0fbb89_isrForCrosstalkSources_997029_51.stderr parsl.dataflow.dflow INFO: Task 3149 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d2e2a91c6dc53df599ea6d0667f1907f with result from task 3149 parsl.dataflow.dflow DEBUG: Task 3149 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b2f368cf-7e45-42ec-bfc0-c335c44337be_isrForCrosstalkSources_996980_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/b2f368cf-7e45-42ec-bfc0-c335c44337be_isrForCrosstalkSources_996980_33.stderr parsl.dataflow.dflow INFO: Task 3147 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 438442738c0386bf3aa575dc1eacaf36 with result from task 3147 parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:21 proc_lsst.multi:146 [INFO] found job 17074839 in provider astro proc_lsst.multi INFO: found job 17074839 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow DEBUG: Task 3147 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/0002c6d8-4816-4ab0-b94a-4d22376af9bc_isrForCrosstalkSources_997004_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/0002c6d8-4816-4ab0-b94a-4d22376af9bc_isrForCrosstalkSources_997004_28.stderr parsl.dataflow.dflow INFO: Task 2895 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c06e9fe4bed6c36bbcced307a8084069 with result from task 2895 parsl.dataflow.dflow DEBUG: Task 2895 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bc4cbcd9-898d-4093-9e32-30ec1735c3ca_isrForCrosstalkSources_996960_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bc4cbcd9-898d-4093-9e32-30ec1735c3ca_isrForCrosstalkSources_996960_12.stderr parsl.dataflow.dflow INFO: Task 2713 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50c41b05f9a5dec073c84503a9c9baa6 with result from task 2713 parsl.dataflow.dflow DEBUG: Task 2713 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/977142c1-c18f-43ca-b8be-03be4019a5e8_isrForCrosstalkSources_996987_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/977142c1-c18f-43ca-b8be-03be4019a5e8_isrForCrosstalkSources_996987_12.stderr parsl.dataflow.dflow INFO: Task 2845 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef0a12038773dd84eddbbeae278873b3 with result from task 2845 parsl.dataflow.dflow DEBUG: Task 2845 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/15293833-442d-4733-a076-8764f9e48dbf_isrForCrosstalkSources_996994_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/15293833-442d-4733-a076-8764f9e48dbf_isrForCrosstalkSources_996994_36.stderr parsl.dataflow.dflow INFO: Task 2791 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f385f59a978e8e63f815ae2ae82abe55 with result from task 2791 parsl.dataflow.dflow DEBUG: Task 2791 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c6d13f08-d725-4dcd-aa39-537672c557a2_isrForCrosstalkSources_997013_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/c6d13f08-d725-4dcd-aa39-537672c557a2_isrForCrosstalkSources_997013_47.stderr parsl.dataflow.dflow INFO: Task 2691 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 415be03d99ddafc602b260d0bfc0ea0a with result from task 2691 parsl.dataflow.dflow DEBUG: Task 2691 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ed6f8914-70c6-477b-b285-d322a18207e6_isrForCrosstalkSources_996960_42.stdout parsl.dataflow.dflow INFO: Standard error for task 2691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ed6f8914-70c6-477b-b285-d322a18207e6_isrForCrosstalkSources_996960_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:21 proc_lsst.multi:146 [INFO] found job 17074841 in provider astro proc_lsst.multi INFO: found job 17074841 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2712 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a37a86411f004c4db249c7c2e0d8e2c with result from task 2712 parsl.dataflow.dflow DEBUG: Task 2712 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5c742c96-72f0-4dff-8b4d-0f87ec2d544f_isrForCrosstalkSources_996985_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5c742c96-72f0-4dff-8b4d-0f87ec2d544f_isrForCrosstalkSources_996985_18.stderr parsl.dataflow.dflow INFO: Task 2906 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2edba1b97d4a941c528c6dd88f86ba29 with result from task 2906 parsl.dataflow.dflow DEBUG: Task 2906 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/933f54bc-0ee3-4629-8b3f-ca3ff201976c_isrForCrosstalkSources_997024_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/933f54bc-0ee3-4629-8b3f-ca3ff201976c_isrForCrosstalkSources_997024_31.stderr parsl.dataflow.dflow INFO: Task 3138 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a2127cd4c3e3d17660fc35523f81962 with result from task 3138 parsl.dataflow.dflow DEBUG: Task 3138 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6208a248-de2f-4c5b-b420-9fa95195ec11_isrForCrosstalkSources_996973_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/6208a248-de2f-4c5b-b420-9fa95195ec11_isrForCrosstalkSources_996973_54.stderr parsl.dataflow.dflow INFO: Task 3142 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d54b65c734a1f6319665379620f06d10 with result from task 3142 parsl.dataflow.dflow DEBUG: Task 3142 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/98afd34f-4e19-4c3e-bc3f-9258668b6d09_isrForCrosstalkSources_997016_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/98afd34f-4e19-4c3e-bc3f-9258668b6d09_isrForCrosstalkSources_997016_22.stderr parsl.dataflow.dflow INFO: Task 2941 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 80791cfe21f8648010695ad6fd5d9290 with result from task 2941 parsl.dataflow.dflow DEBUG: Task 2941 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a6669c99-16fa-4376-a7ca-7811bab32df5_isrForCrosstalkSources_996981_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a6669c99-16fa-4376-a7ca-7811bab32df5_isrForCrosstalkSources_996981_57.stderr parsl.dataflow.dflow INFO: Task 2706 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a07a868dee5781e4fda619a2f61e835 with result from task 2706 parsl.dataflow.dflow DEBUG: Task 2706 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a0795cdf-bc70-47f1-a47f-e3944f4a0479_isrForCrosstalkSources_996957_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a0795cdf-bc70-47f1-a47f-e3944f4a0479_isrForCrosstalkSources_996957_3.stderr parsl.dataflow.dflow INFO: Task 3030 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f4106babb5979ff50cf79ba6e175f65 with result from task 3030 parsl.dataflow.dflow DEBUG: Task 3030 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6ed15fa8-107b-4b98-92a5-31c9a81e6f39_isrForCrosstalkSources_996954_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6ed15fa8-107b-4b98-92a5-31c9a81e6f39_isrForCrosstalkSources_996954_52.stderr parsl.dataflow.dflow INFO: Task 2862 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 339af58f17a8c911ec2677e1817a20f3 with result from task 2862 parsl.dataflow.dflow DEBUG: Task 2862 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/df15e4f0-8547-4820-918f-18c7864b0685_isrForCrosstalkSources_996986_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/df15e4f0-8547-4820-918f-18c7864b0685_isrForCrosstalkSources_996986_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:22 proc_lsst.multi:146 [INFO] found job 17074842 in provider astro proc_lsst.multi INFO: found job 17074842 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2936 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f823219e002b46015f431d863d3cdaae with result from task 2936 parsl.dataflow.dflow DEBUG: Task 2936 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/494120fc-a4fc-4bd5-a764-67e54947eaf1_isrForCrosstalkSources_997022_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/494120fc-a4fc-4bd5-a764-67e54947eaf1_isrForCrosstalkSources_997022_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:23 proc_lsst.multi:146 [INFO] found job 17074843 in provider astro proc_lsst.multi INFO: found job 17074843 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2917 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea4971aa6e284a68b709ebb1847eafff with result from task 2917 parsl.dataflow.dflow DEBUG: Task 2917 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a934996c-d7cd-4178-a803-a1d2cc28bd1f_isrForCrosstalkSources_996956_43.stdout parsl.dataflow.dflow INFO: Standard error for task 2917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a934996c-d7cd-4178-a803-a1d2cc28bd1f_isrForCrosstalkSources_996956_43.stderr parsl.dataflow.dflow INFO: Task 2878 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec4e9960c51f23bd211fbdded437d6b2 with result from task 2878 parsl.dataflow.dflow DEBUG: Task 2878 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/ad322615-2246-4e8a-ae11-2144da57d3b6_isrForCrosstalkSources_997032_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/ad322615-2246-4e8a-ae11-2144da57d3b6_isrForCrosstalkSources_997032_28.stderr parsl.dataflow.dflow INFO: Task 2801 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58f190d3d1dd88081e7a7f35e84677c3 with result from task 2801 parsl.dataflow.dflow DEBUG: Task 2801 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/b9fca640-6e7b-4665-a8e5-19121e799350_isrForCrosstalkSources_996976_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/b9fca640-6e7b-4665-a8e5-19121e799350_isrForCrosstalkSources_996976_29.stderr parsl.dataflow.dflow INFO: Task 2838 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6da502a84ee129d51031aab28c11a2b with result from task 2838 parsl.dataflow.dflow DEBUG: Task 2838 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4d22b2cf-8f1b-4b2e-a998-9f61b5e46dbd_isrForCrosstalkSources_996986_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4d22b2cf-8f1b-4b2e-a998-9f61b5e46dbd_isrForCrosstalkSources_996986_31.stderr parsl.dataflow.dflow INFO: Task 2787 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba601f4f121ed54535e0a5c123195d38 with result from task 2787 parsl.dataflow.dflow DEBUG: Task 2787 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d0b65c7a-8a85-4497-8fbf-650dc87e2143_isrForCrosstalkSources_997000_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d0b65c7a-8a85-4497-8fbf-650dc87e2143_isrForCrosstalkSources_997000_28.stderr parsl.dataflow.dflow INFO: Task 2892 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a49ac471029611ae44d179b088e425c with result from task 2892 parsl.dataflow.dflow DEBUG: Task 2892 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/07c78432-d744-4dd9-9af5-d367fe07d9ad_isrForCrosstalkSources_997032_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/07c78432-d744-4dd9-9af5-d367fe07d9ad_isrForCrosstalkSources_997032_3.stderr parsl.dataflow.dflow INFO: Task 2803 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4a26fd134ffbfe2902380d6256fdf7b with result from task 2803 parsl.dataflow.dflow DEBUG: Task 2803 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cd60d8d3-9998-4128-a61b-ce1e30023d48_isrForCrosstalkSources_996957_47.stdout parsl.dataflow.dflow INFO: Standard error for task 2803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/cd60d8d3-9998-4128-a61b-ce1e30023d48_isrForCrosstalkSources_996957_47.stderr parsl.dataflow.dflow INFO: Task 2932 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a091f30ab106a544e513f2107e88274 with result from task 2932 parsl.dataflow.dflow DEBUG: Task 2932 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/010eea8c-b489-4982-bab7-ffcbef1d7d8e_isrForCrosstalkSources_996984_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/010eea8c-b489-4982-bab7-ffcbef1d7d8e_isrForCrosstalkSources_996984_45.stderr parsl.dataflow.dflow INFO: Task 2863 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c22d5313e658d40dbd5905a4f083bed with result from task 2863 parsl.dataflow.dflow DEBUG: Task 2863 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/21ffdaef-4a89-4225-9282-1464bdd9b68d_isrForCrosstalkSources_996984_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/21ffdaef-4a89-4225-9282-1464bdd9b68d_isrForCrosstalkSources_996984_55.stderr parsl.dataflow.dflow INFO: Task 2929 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0b47b2a5a710f24eb9ca661295590b53 with result from task 2929 parsl.dataflow.dflow DEBUG: Task 2929 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/5f79c0fa-f93c-4bae-a8b2-7775cd60ebff_isrForCrosstalkSources_997014_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/5f79c0fa-f93c-4bae-a8b2-7775cd60ebff_isrForCrosstalkSources_997014_55.stderr parsl.dataflow.dflow INFO: Task 3018 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 986b7a6a25701b3f659a41bf273d9388 with result from task 3018 parsl.dataflow.dflow DEBUG: Task 3018 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/163b6b2d-d170-49d3-a54a-411b5a357d28_isrForCrosstalkSources_997025_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/163b6b2d-d170-49d3-a54a-411b5a357d28_isrForCrosstalkSources_997025_40.stderr parsl.dataflow.dflow INFO: Task 2767 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fce09c6a6efde1c970de365148e59877 with result from task 2767 parsl.dataflow.dflow DEBUG: Task 2767 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f4ad5a95-c08e-44f8-a439-161f879b9dbc_isrForCrosstalkSources_996966_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f4ad5a95-c08e-44f8-a439-161f879b9dbc_isrForCrosstalkSources_996966_58.stderr parsl.dataflow.dflow INFO: Task 2848 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53f2d3cfdec586152fdda366b672c624 with result from task 2848 parsl.dataflow.dflow DEBUG: Task 2848 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/68fc4a28-1d21-4ac1-8c9d-ee02d906f14b_isrForCrosstalkSources_997021_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/68fc4a28-1d21-4ac1-8c9d-ee02d906f14b_isrForCrosstalkSources_997021_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:23 proc_lsst.multi:146 [INFO] found job 17074850 in provider astro proc_lsst.multi INFO: found job 17074850 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2833 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0222c8fb075f3ad0ca02bc810151b5ce with result from task 2833 parsl.dataflow.dflow DEBUG: Task 2833 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/3bf4ba14-50a3-4f2b-806c-e2ddbdddcdce_isrForCrosstalkSources_997022_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/3bf4ba14-50a3-4f2b-806c-e2ddbdddcdce_isrForCrosstalkSources_997022_6.stderr parsl.dataflow.dflow INFO: Task 3145 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8751070e658da5b5b40a452e9a09cb14 with result from task 3145 parsl.dataflow.dflow DEBUG: Task 3145 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f016a26b-f0c5-47cb-837d-31d7075c10a3_isrForCrosstalkSources_996995_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/f016a26b-f0c5-47cb-837d-31d7075c10a3_isrForCrosstalkSources_996995_32.stderr parsl.dataflow.dflow INFO: Task 2709 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aec4a4aca5060929e4b131310234b3b6 with result from task 2709 parsl.dataflow.dflow DEBUG: Task 2709 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/53a906f5-b673-4264-befa-456de1883116_isrForCrosstalkSources_997022_29.stdout parsl.dataflow.dflow INFO: Standard error for task 2709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/53a906f5-b673-4264-befa-456de1883116_isrForCrosstalkSources_997022_29.stderr parsl.dataflow.dflow INFO: Task 2981 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8fe9928b92c6d1661fc295946b2fc5b0 with result from task 2981 parsl.dataflow.dflow DEBUG: Task 2981 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/51c6b59f-6de9-4400-a231-a050bc85aafa_isrForCrosstalkSources_997008_50.stdout parsl.dataflow.dflow INFO: Standard error for task 2981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/51c6b59f-6de9-4400-a231-a050bc85aafa_isrForCrosstalkSources_997008_50.stderr parsl.dataflow.dflow INFO: Task 2950 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b87cbf5fce3ff75a01e1f89c2a3ba7f2 with result from task 2950 parsl.dataflow.dflow DEBUG: Task 2950 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ae6010cb-5cd3-4d1f-86f1-6b0e18c360ec_isrForCrosstalkSources_997030_23.stdout parsl.dataflow.dflow INFO: Standard error for task 2950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/ae6010cb-5cd3-4d1f-86f1-6b0e18c360ec_isrForCrosstalkSources_997030_23.stderr parsl.dataflow.dflow INFO: Task 3029 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 575db023553de509e300963ed25c8930 with result from task 3029 parsl.dataflow.dflow DEBUG: Task 3029 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/742f9bfc-45da-41ce-bc77-7737838d3d73_isrForCrosstalkSources_996970_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/742f9bfc-45da-41ce-bc77-7737838d3d73_isrForCrosstalkSources_996970_36.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 3157 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 2b46bccf2fc18ebe463c5f9a203eca4f with result from task 3157 parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:24 proc_lsst.multi:146 [INFO] found job 17074851 in provider astro proc_lsst.multi INFO: found job 17074851 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow DEBUG: Task 3157 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/60c8e9ce-83b6-43e6-84bb-20f333a3b041_isrForCrosstalkSources_996987_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/60c8e9ce-83b6-43e6-84bb-20f333a3b041_isrForCrosstalkSources_996987_3.stderr parsl.dataflow.dflow INFO: Task 2968 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4018652108056f0b2222a43d28cfafd with result from task 2968 parsl.dataflow.dflow DEBUG: Task 2968 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c1c8518a-fd2f-4416-b825-3af0ce9729af_isrForCrosstalkSources_996972_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c1c8518a-fd2f-4416-b825-3af0ce9729af_isrForCrosstalkSources_996972_17.stderr parsl.dataflow.dflow INFO: Task 2830 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 970a871a7115ad35f72619f297629c54 with result from task 2830 parsl.dataflow.dflow DEBUG: Task 2830 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6d36a060-b1e6-49fc-9ffa-fe1762def735_isrForCrosstalkSources_997006_54.stdout parsl.dataflow.dflow INFO: Standard error for task 2830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6d36a060-b1e6-49fc-9ffa-fe1762def735_isrForCrosstalkSources_997006_54.stderr parsl.dataflow.dflow INFO: Task 2933 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2793d2ab3274f1e5e055f529e3463a7b with result from task 2933 parsl.dataflow.dflow DEBUG: Task 2933 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f3e31e8b-b134-4fde-8c6f-6778e251ca74_isrForCrosstalkSources_996960_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f3e31e8b-b134-4fde-8c6f-6778e251ca74_isrForCrosstalkSources_996960_6.stderr parsl.dataflow.dflow INFO: Task 2834 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry edbd735bf22df590be923516834c731f with result from task 2834 parsl.dataflow.dflow DEBUG: Task 2834 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/fbd6fd52-9e0d-4b2a-9586-b7f637666daf_isrForCrosstalkSources_997007_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/fbd6fd52-9e0d-4b2a-9586-b7f637666daf_isrForCrosstalkSources_997007_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:25 proc_lsst.multi:146 [INFO] found job 17074852 in provider astro proc_lsst.multi INFO: found job 17074852 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2783 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6051f94995306f625138d803c0e12c9d with result from task 2783 parsl.dataflow.dflow DEBUG: Task 2783 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6a9bc977-d840-4912-a8e1-4cef99537d6b_isrForCrosstalkSources_997006_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/6a9bc977-d840-4912-a8e1-4cef99537d6b_isrForCrosstalkSources_997006_52.stderr parsl.dataflow.dflow INFO: Task 3048 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5de7e7a72462c1694826d0acebb88c8 with result from task 3048 parsl.dataflow.dflow DEBUG: Task 3048 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e30533f4-7a58-4bd9-ae38-8f0f2f4e6d79_isrForCrosstalkSources_996955_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e30533f4-7a58-4bd9-ae38-8f0f2f4e6d79_isrForCrosstalkSources_996955_59.stderr parsl.dataflow.dflow INFO: Task 3169 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba409ad8a8836e69eea5ee61e31ea48a with result from task 3169 parsl.dataflow.dflow DEBUG: Task 3169 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/1fb32f38-b0ce-4ed6-9fd2-946df923d91f_isrForCrosstalkSources_997010_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/1fb32f38-b0ce-4ed6-9fd2-946df923d91f_isrForCrosstalkSources_997010_6.stderr parsl.dataflow.dflow INFO: Task 2828 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e50d364edf51f8b0d5d8546e649ee09b with result from task 2828 parsl.dataflow.dflow DEBUG: Task 2828 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/007a4596-b88b-4c02-8895-55beaf05e71b_isrForCrosstalkSources_997019_39.stdout parsl.dataflow.dflow INFO: Standard error for task 2828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/007a4596-b88b-4c02-8895-55beaf05e71b_isrForCrosstalkSources_997019_39.stderr parsl.dataflow.dflow INFO: Task 2902 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d54607ec0bf51b544b79d623125ffba7 with result from task 2902 parsl.dataflow.dflow DEBUG: Task 2902 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/46a05853-9e64-437b-970a-9f24d3262cd4_isrForCrosstalkSources_997011_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/46a05853-9e64-437b-970a-9f24d3262cd4_isrForCrosstalkSources_997011_44.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:25 proc_lsst.multi:146 [INFO] found job 17074853 in provider astro proc_lsst.multi INFO: found job 17074853 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2977 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9756fbe22918c39da570762529784fd6 with result from task 2977 parsl.dataflow.dflow DEBUG: Task 2977 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/30bc09cc-feba-46a6-8812-47917bf415b3_isrForCrosstalkSources_997016_45.stdout parsl.dataflow.dflow INFO: Standard error for task 2977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/30bc09cc-feba-46a6-8812-47917bf415b3_isrForCrosstalkSources_997016_45.stderr parsl.dataflow.dflow INFO: Task 3090 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3aa58d1cbda89600b2b790e5aa43f2d with result from task 3090 parsl.dataflow.dflow DEBUG: Task 3090 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/389e64d5-77a8-4369-932d-7c8d66f9b244_isrForCrosstalkSources_996984_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/389e64d5-77a8-4369-932d-7c8d66f9b244_isrForCrosstalkSources_996984_26.stderr parsl.dataflow.dflow INFO: Task 3183 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 421558814e2f9b6c19102ab04820c084 with result from task 3183 parsl.dataflow.dflow DEBUG: Task 3183 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e7498fe2-e98d-42bf-b097-d0c095c8b9ea_isrForCrosstalkSources_997021_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e7498fe2-e98d-42bf-b097-d0c095c8b9ea_isrForCrosstalkSources_997021_27.stderr parsl.dataflow.dflow INFO: Task 3176 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70f700c6ef6f75887bbe4ad0f86c3dfc with result from task 3176 parsl.dataflow.dflow DEBUG: Task 3176 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/99953982-306c-4cc3-86f7-4e623d98e9be_isrForCrosstalkSources_997019_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/99953982-306c-4cc3-86f7-4e623d98e9be_isrForCrosstalkSources_997019_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:26 proc_lsst.multi:146 [INFO] found job 17074854 in provider astro proc_lsst.multi INFO: found job 17074854 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2785 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0fb769181c926dd73771729a82f3c29e with result from task 2785 parsl.dataflow.dflow DEBUG: Task 2785 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/9f8df356-5868-4678-9a63-db8f7d32123e_isrForCrosstalkSources_997033_59.stdout parsl.dataflow.dflow INFO: Standard error for task 2785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/9f8df356-5868-4678-9a63-db8f7d32123e_isrForCrosstalkSources_997033_59.stderr parsl.dataflow.dflow INFO: Task 3200 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7838c5007eee0ecedb11c737aa8900c with result from task 3200 parsl.dataflow.dflow DEBUG: Task 3200 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/136de1f6-f40b-4957-aff4-09207c2e05b6_isrForCrosstalkSources_996952_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/136de1f6-f40b-4957-aff4-09207c2e05b6_isrForCrosstalkSources_996952_32.stderr parsl.dataflow.dflow INFO: Task 3194 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 61ce7eb9838adddd2644ea67d6d4bbb5 with result from task 3194 parsl.dataflow.dflow DEBUG: Task 3194 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/cfecac5c-b504-497a-b2be-57abca298442_isrForCrosstalkSources_996952_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/cfecac5c-b504-497a-b2be-57abca298442_isrForCrosstalkSources_996952_56.stderr parsl.dataflow.dflow INFO: Task 3143 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c6e889051b4526851d450b0f63f2030e with result from task 3143 parsl.dataflow.dflow DEBUG: Task 3143 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a9b3f19f-531b-4ac7-9004-61c5fe35f8c1_isrForCrosstalkSources_997001_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a9b3f19f-531b-4ac7-9004-61c5fe35f8c1_isrForCrosstalkSources_997001_40.stderr parsl.dataflow.dflow INFO: Task 3073 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ced6d00428a59b25242b303e46c13e1 with result from task 3073 parsl.dataflow.dflow DEBUG: Task 3073 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7c2b48fc-d2d8-4dd4-9d8b-eee41d38de81_isrForCrosstalkSources_997024_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7c2b48fc-d2d8-4dd4-9d8b-eee41d38de81_isrForCrosstalkSources_997024_33.stderr parsl.dataflow.dflow INFO: Task 2707 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a718a2fe2e687ab9eba0651ee21d8f3 with result from task 2707 parsl.dataflow.dflow DEBUG: Task 2707 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/47b8723c-d6f0-4baf-98bc-532cd9b3e6a5_isrForCrosstalkSources_997017_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/47b8723c-d6f0-4baf-98bc-532cd9b3e6a5_isrForCrosstalkSources_997017_30.stderr parsl.dataflow.dflow INFO: Task 2938 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb5ced68b2cdc4ddac5e00b3a2e6faf9 with result from task 2938 parsl.dataflow.dflow DEBUG: Task 2938 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/04a00608-a6e8-4b15-942e-df1b8489f115_isrForCrosstalkSources_997006_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/04a00608-a6e8-4b15-942e-df1b8489f115_isrForCrosstalkSources_997006_55.stderr parsl.dataflow.dflow INFO: Task 3182 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a2f031b987544e15b883f2cc351bc97 with result from task 3182 parsl.dataflow.dflow DEBUG: Task 3182 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ed7b8605-f528-48bf-ab81-4b1a999e09e5_isrForCrosstalkSources_997019_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ed7b8605-f528-48bf-ab81-4b1a999e09e5_isrForCrosstalkSources_997019_10.stderr parsl.dataflow.dflow INFO: Task 2996 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1647d5406a71b994da2196f58d05e74e with result from task 2996 parsl.dataflow.dflow DEBUG: Task 2996 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/e8310b60-1d43-4921-872e-3021bece0224_isrForCrosstalkSources_997024_55.stdout parsl.dataflow.dflow INFO: Standard error for task 2996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/e8310b60-1d43-4921-872e-3021bece0224_isrForCrosstalkSources_997024_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:26 proc_lsst.multi:146 [INFO] found job 17074855 in provider astro proc_lsst.multi INFO: found job 17074855 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2985 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc761d8ada7074e444eb15b60feb273e with result from task 2985 parsl.dataflow.dflow DEBUG: Task 2985 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/8b94f7eb-bb0d-4350-bb3d-2ff2b683b086_isrForCrosstalkSources_997014_41.stdout parsl.dataflow.dflow INFO: Standard error for task 2985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/8b94f7eb-bb0d-4350-bb3d-2ff2b683b086_isrForCrosstalkSources_997014_41.stderr parsl.dataflow.dflow INFO: Task 2934 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f19e470cf7ef474ed1657bdf6fe2f326 with result from task 2934 parsl.dataflow.dflow DEBUG: Task 2934 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/7b952721-e0cd-49bd-b200-48c67fb6fb9e_isrForCrosstalkSources_996954_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/7b952721-e0cd-49bd-b200-48c67fb6fb9e_isrForCrosstalkSources_996954_2.stderr parsl.dataflow.dflow INFO: Task 2992 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c31036224c30115b3421e29b1081da1 with result from task 2992 parsl.dataflow.dflow DEBUG: Task 2992 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e2783aa9-560a-4986-8d30-22097e037ee0_isrForCrosstalkSources_997019_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/e2783aa9-560a-4986-8d30-22097e037ee0_isrForCrosstalkSources_997019_27.stderr parsl.dataflow.dflow INFO: Task 3174 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27ca5b9f854e56d4005d7f20ab057ba8 with result from task 3174 parsl.dataflow.dflow DEBUG: Task 3174 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f3fefa24-3e4c-4fcc-8afc-1b35a55e5157_isrForCrosstalkSources_997017_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/f3fefa24-3e4c-4fcc-8afc-1b35a55e5157_isrForCrosstalkSources_997017_11.stderr parsl.dataflow.dflow INFO: Task 2979 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 79bd4bea7fb2d518caa8cd2621017efd with result from task 2979 parsl.dataflow.dflow DEBUG: Task 2979 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f83c5495-d647-42fe-b58b-33d7f8e932c7_isrForCrosstalkSources_997018_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f83c5495-d647-42fe-b58b-33d7f8e932c7_isrForCrosstalkSources_997018_2.stderr parsl.dataflow.dflow INFO: Task 2685 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0786c0e16de055abb01302c078a5ace4 with result from task 2685 parsl.dataflow.dflow DEBUG: Task 2685 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3a2003ed-0231-43a8-b6b4-2a1335e9268d_isrForCrosstalkSources_997031_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/3a2003ed-0231-43a8-b6b4-2a1335e9268d_isrForCrosstalkSources_997031_52.stderr parsl.dataflow.dflow INFO: Task 2924 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0917ec1cf83f8639e2d6aabeb0788576 with result from task 2924 parsl.dataflow.dflow DEBUG: Task 2924 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9e691935-3518-4313-9d09-371b0f5d307c_isrForCrosstalkSources_996968_57.stdout parsl.dataflow.dflow INFO: Standard error for task 2924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/9e691935-3518-4313-9d09-371b0f5d307c_isrForCrosstalkSources_996968_57.stderr parsl.dataflow.dflow INFO: Task 2866 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54411aedecbe0155a52efd0c5fd2752e with result from task 2866 parsl.dataflow.dflow DEBUG: Task 2866 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/32fb4123-a4d8-4e5e-b0ac-0cf0ccb354cb_isrForCrosstalkSources_996953_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/32fb4123-a4d8-4e5e-b0ac-0cf0ccb354cb_isrForCrosstalkSources_996953_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:27 proc_lsst.multi:146 [INFO] found job 17074856 in provider astro proc_lsst.multi INFO: found job 17074856 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3049 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7f03cdee32c0bdf43925622a477bc18 with result from task 3049 parsl.dataflow.dflow DEBUG: Task 3049 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/130878b3-1575-41e6-90b3-fa68938a3d49_isrForCrosstalkSources_996958_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/130878b3-1575-41e6-90b3-fa68938a3d49_isrForCrosstalkSources_996958_30.stderr parsl.dataflow.dflow INFO: Task 2982 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5f94a86ce3e6f49ea6bf799c2532b07 with result from task 2982 parsl.dataflow.dflow DEBUG: Task 2982 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2c7d4765-92d3-487a-a15a-eee9c7940738_isrForCrosstalkSources_996990_14.stdout parsl.dataflow.dflow INFO: Standard error for task 2982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2c7d4765-92d3-487a-a15a-eee9c7940738_isrForCrosstalkSources_996990_14.stderr parsl.dataflow.dflow INFO: Task 3192 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a57dc4910e6826a595583fe932ee6e69 with result from task 3192 parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 3192 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:27 proc_lsst.multi:146 [INFO] found job 17074857 in provider astro proc_lsst.multi INFO: found job 17074857 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/fd8acc59-2c74-4276-96e9-5f89bb77bc54_isrForCrosstalkSources_997025_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/fd8acc59-2c74-4276-96e9-5f89bb77bc54_isrForCrosstalkSources_997025_23.stderr parsl.dataflow.dflow INFO: Task 3196 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 984aaf6e4368cb4a579920df3705dfd0 with result from task 3196 parsl.dataflow.dflow DEBUG: Task 3196 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/832d4c01-2894-4029-aed2-b7c94a034eec_isrForCrosstalkSources_996955_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/832d4c01-2894-4029-aed2-b7c94a034eec_isrForCrosstalkSources_996955_3.stderr parsl.dataflow.dflow INFO: Task 2983 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1b9ecc97cca9fdd5a9d0deeda77e9ea4 with result from task 2983 parsl.dataflow.dflow DEBUG: Task 2983 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/90e67a6a-0c3a-49a1-9cdc-b6780fadfde7_isrForCrosstalkSources_996961_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/90e67a6a-0c3a-49a1-9cdc-b6780fadfde7_isrForCrosstalkSources_996961_28.stderr parsl.dataflow.dflow INFO: Task 2819 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25fbd4e21f4a5fa118f42167c39d2fb3 with result from task 2819 parsl.dataflow.dflow DEBUG: Task 2819 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3eefe7dc-2ae2-4b05-bbbc-d1eb83666ed4_isrForCrosstalkSources_996969_12.stdout parsl.dataflow.dflow INFO: Standard error for task 2819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/3eefe7dc-2ae2-4b05-bbbc-d1eb83666ed4_isrForCrosstalkSources_996969_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:28 proc_lsst.multi:146 [INFO] found job 17074859 in provider astro proc_lsst.multi INFO: found job 17074859 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2931 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a158b7acfcc8072f2a2ee95d20e08cb1 with result from task 2931 parsl.dataflow.dflow DEBUG: Task 2931 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ea803b00-3faa-48f6-8b3b-ad11dc6481aa_isrForCrosstalkSources_996969_6.stdout parsl.dataflow.dflow INFO: Standard error for task 2931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ea803b00-3faa-48f6-8b3b-ad11dc6481aa_isrForCrosstalkSources_996969_6.stderr parsl.dataflow.dflow INFO: Task 3202 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c4c99170ab8b289c676a0e0b44a2ddd with result from task 3202 parsl.dataflow.dflow DEBUG: Task 3202 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/63080e83-ed69-4fcd-9b9e-c4339949d1fd_isrForCrosstalkSources_996989_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/63080e83-ed69-4fcd-9b9e-c4339949d1fd_isrForCrosstalkSources_996989_18.stderr parsl.dataflow.dflow INFO: Task 3207 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 289213aed312d6fdc007cb47cfaba8bc with result from task 3207 parsl.dataflow.dflow DEBUG: Task 3207 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/b17c6dab-10ab-4980-a8b6-eba66d4b846b_isrForCrosstalkSources_997024_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/b17c6dab-10ab-4980-a8b6-eba66d4b846b_isrForCrosstalkSources_997024_27.stderr parsl.dataflow.dflow INFO: Task 3184 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8bd3742b7a752933f5eb619cdd56cec2 with result from task 3184 parsl.dataflow.dflow DEBUG: Task 3184 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5baefdda-e37a-4609-bcbe-d4ff07371fc6_isrForCrosstalkSources_997031_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5baefdda-e37a-4609-bcbe-d4ff07371fc6_isrForCrosstalkSources_997031_2.stderr parsl.dataflow.dflow INFO: Task 3173 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c72f80f7522a34592129bc70cf1f44e9 with result from task 3173 parsl.dataflow.dflow DEBUG: Task 3173 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/461d7d06-4b7f-4609-978b-47f387289f97_isrForCrosstalkSources_997016_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/461d7d06-4b7f-4609-978b-47f387289f97_isrForCrosstalkSources_997016_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:28 proc_lsst.multi:146 [INFO] found job 17074861 in provider astro proc_lsst.multi INFO: found job 17074861 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3210 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e720632593d1eb1d493e62312f2afd70 with result from task 3210 parsl.dataflow.dflow DEBUG: Task 3210 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2c647f14-3b60-4da6-8d24-9580fe73e375_isrForCrosstalkSources_996980_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2c647f14-3b60-4da6-8d24-9580fe73e375_isrForCrosstalkSources_996980_46.stderr parsl.dataflow.dflow INFO: Task 3134 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 522f81db866844d47dcf49ae32ef2fad with result from task 3134 parsl.dataflow.dflow DEBUG: Task 3134 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0f2a29cc-7016-4ad6-807f-8983c658f848_isrForCrosstalkSources_997025_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0f2a29cc-7016-4ad6-807f-8983c658f848_isrForCrosstalkSources_997025_29.stderr parsl.dataflow.dflow INFO: Task 3206 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fef557c642f066c7df06bbeb628d64a with result from task 3206 parsl.dataflow.dflow DEBUG: Task 3206 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/6ad7d136-3349-478e-a140-7fbcf288940c_isrForCrosstalkSources_996964_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/6ad7d136-3349-478e-a140-7fbcf288940c_isrForCrosstalkSources_996964_43.stderr parsl.dataflow.dflow INFO: Task 3204 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4e27834e1a6dc1c0ac9a19e5156a006 with result from task 3204 parsl.dataflow.dflow DEBUG: Task 3204 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/9c1d3748-3da0-4251-9b5e-148c1083bde2_isrForCrosstalkSources_997011_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/9c1d3748-3da0-4251-9b5e-148c1083bde2_isrForCrosstalkSources_997011_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:29 proc_lsst.multi:146 [INFO] found job 17074863 in provider astro proc_lsst.multi INFO: found job 17074863 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2897 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5da18f3b401cf188f618e55f0ebe20d with result from task 2897 parsl.dataflow.dflow DEBUG: Task 2897 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/177edfb3-95aa-478b-ba6e-eb8df23f0c3f_isrForCrosstalkSources_997010_17.stdout parsl.dataflow.dflow INFO: Standard error for task 2897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/177edfb3-95aa-478b-ba6e-eb8df23f0c3f_isrForCrosstalkSources_997010_17.stderr parsl.dataflow.dflow INFO: Task 3198 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec8b69df9fa866c44575d2785b850ff1 with result from task 3198 parsl.dataflow.dflow DEBUG: Task 3198 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/dd134de8-0688-4481-964c-5b306078ac1d_isrForCrosstalkSources_996992_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/dd134de8-0688-4481-964c-5b306078ac1d_isrForCrosstalkSources_996992_36.stderr parsl.dataflow.dflow INFO: Task 2947 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ecff021fda3e5944f7aa343daa65183f with result from task 2947 parsl.dataflow.dflow DEBUG: Task 2947 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f44f50d9-b109-413e-9596-411dc23cbfce_isrForCrosstalkSources_996966_40.stdout parsl.dataflow.dflow INFO: Standard error for task 2947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f44f50d9-b109-413e-9596-411dc23cbfce_isrForCrosstalkSources_996966_40.stderr parsl.dataflow.dflow INFO: Task 3212 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32083de5205e4e945152f97293099521 with result from task 3212 parsl.dataflow.dflow DEBUG: Task 3212 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/11905bef-a0f5-4c3c-abd1-26584f9b6ee4_isrForCrosstalkSources_997023_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/11905bef-a0f5-4c3c-abd1-26584f9b6ee4_isrForCrosstalkSources_997023_59.stderr parsl.dataflow.dflow INFO: Task 3116 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c75f4072f02c12b265d76f27f6368cd1 with result from task 3116 parsl.dataflow.dflow DEBUG: Task 3116 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/f48792d7-f5fc-4a03-9a8e-c75d8f20fe0b_isrForCrosstalkSources_996982_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/f48792d7-f5fc-4a03-9a8e-c75d8f20fe0b_isrForCrosstalkSources_996982_47.stderr parsl.dataflow.dflow INFO: Task 2978 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ad796ae475d1f20c68de92484afa067 with result from task 2978 parsl.dataflow.dflow DEBUG: Task 2978 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/2f9cacad-7d33-4b2c-a525-70f97971f187_isrForCrosstalkSources_997029_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/2f9cacad-7d33-4b2c-a525-70f97971f187_isrForCrosstalkSources_997029_16.stderr parsl.dataflow.dflow INFO: Task 2913 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0d653fd1723abd1e7e9d2c7f8d24a338 with result from task 2913 parsl.dataflow.dflow DEBUG: Task 2913 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d2276757-709c-4f0c-929d-f0c7f13fd065_isrForCrosstalkSources_996965_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/d2276757-709c-4f0c-929d-f0c7f13fd065_isrForCrosstalkSources_996965_33.stderr parsl.dataflow.dflow INFO: Task 3181 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a44c317bca64b5741934dcf31b511a9 with result from task 3181 parsl.dataflow.dflow DEBUG: Task 3181 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5bf6f98c-2ae7-427e-92eb-bbb3e0c1bc0e_isrForCrosstalkSources_997031_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5bf6f98c-2ae7-427e-92eb-bbb3e0c1bc0e_isrForCrosstalkSources_997031_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:29 proc_lsst.multi:146 [INFO] found job 17074865 in provider astro proc_lsst.multi INFO: found job 17074865 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3222 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9856d82af50d4c8cb93832e5f6919ddb with result from task 3222 parsl.dataflow.dflow DEBUG: Task 3222 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2e6095da-83a9-4c51-a794-fc09dbc8ee6c_isrForCrosstalkSources_997003_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2e6095da-83a9-4c51-a794-fc09dbc8ee6c_isrForCrosstalkSources_997003_15.stderr parsl.dataflow.dflow INFO: Task 3211 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b1dc6c3ae1e468dfb2f89f9c74c7e1be with result from task 3211 parsl.dataflow.dflow DEBUG: Task 3211 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/bcbcc29e-e829-4bed-b604-8aadc676f129_isrForCrosstalkSources_997013_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/bcbcc29e-e829-4bed-b604-8aadc676f129_isrForCrosstalkSources_997013_32.stderr parsl.dataflow.dflow INFO: Task 2907 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2951dba98378d765e1e7206473e749b7 with result from task 2907 parsl.dataflow.dflow DEBUG: Task 2907 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ff4a8135-d2c5-40ce-a5be-46c4807fc22b_isrForCrosstalkSources_996989_2.stdout parsl.dataflow.dflow INFO: Standard error for task 2907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/ff4a8135-d2c5-40ce-a5be-46c4807fc22b_isrForCrosstalkSources_996989_2.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:30 proc_lsst.multi:146 [INFO] found job 17074869 in provider astro proc_lsst.multi INFO: found job 17074869 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2874 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2401e277dcae74c819d9d5cf1e48db6 with result from task 2874 parsl.dataflow.dflow DEBUG: Task 2874 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/460b18e5-b238-444a-b5e5-a558f72c2588_isrForCrosstalkSources_996969_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/460b18e5-b238-444a-b5e5-a558f72c2588_isrForCrosstalkSources_996969_16.stderr parsl.dataflow.dflow INFO: Task 3221 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23ec3891b820c3b04ccb98c815594998 with result from task 3221 parsl.dataflow.dflow DEBUG: Task 3221 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/340afdf2-b31c-4d58-b94d-d7c682bb662e_isrForCrosstalkSources_996976_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/340afdf2-b31c-4d58-b94d-d7c682bb662e_isrForCrosstalkSources_996976_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:30 proc_lsst.multi:146 [INFO] found job 17074870 in provider astro proc_lsst.multi INFO: found job 17074870 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3165 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53b4786b06e3203ce310e818d8fc9c64 with result from task 3165 parsl.dataflow.dflow DEBUG: Task 3165 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8ad98ed8-93c2-4b71-b642-a7b97369f422_isrForCrosstalkSources_996952_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8ad98ed8-93c2-4b71-b642-a7b97369f422_isrForCrosstalkSources_996952_40.stderr parsl.dataflow.dflow INFO: Task 2972 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 586e80af006b6bd62b1e5eb5c578269f with result from task 2972 parsl.dataflow.dflow DEBUG: Task 2972 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/edaa37de-22da-4dd2-b00f-ae171167dbe3_isrForCrosstalkSources_997016_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/edaa37de-22da-4dd2-b00f-ae171167dbe3_isrForCrosstalkSources_997016_20.stderr parsl.dataflow.dflow INFO: Task 2926 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d211c9d107a4d8083908e7bedb3078c8 with result from task 2926 parsl.dataflow.dflow DEBUG: Task 2926 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/d751e76a-c671-4f1c-8d1b-4baa46865040_isrForCrosstalkSources_997031_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/d751e76a-c671-4f1c-8d1b-4baa46865040_isrForCrosstalkSources_997031_37.stderr parsl.dataflow.dflow INFO: Task 3117 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 389083b5900eb03d84672acb0b180bde with result from task 3117 parsl.dataflow.dflow DEBUG: Task 3117 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4f5a8ca7-6a17-43fe-a1ea-f76beb932e5f_isrForCrosstalkSources_997017_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/4f5a8ca7-6a17-43fe-a1ea-f76beb932e5f_isrForCrosstalkSources_997017_26.stderr parsl.dataflow.dflow INFO: Task 2710 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fda82ec971134675ca36a31af54e604a with result from task 2710 parsl.dataflow.dflow DEBUG: Task 2710 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e6b75f4c-7c01-4876-ae61-f80ba9e1806a_isrForCrosstalkSources_996997_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e6b75f4c-7c01-4876-ae61-f80ba9e1806a_isrForCrosstalkSources_996997_53.stderr parsl.dataflow.dflow INFO: Task 3119 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e569f5ff155b716177261845369ff2c3 with result from task 3119 parsl.dataflow.dflow DEBUG: Task 3119 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/2a755af7-6a2b-48f9-a2f1-66211cd606b2_isrForCrosstalkSources_996999_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/2a755af7-6a2b-48f9-a2f1-66211cd606b2_isrForCrosstalkSources_996999_10.stderr parsl.dataflow.dflow INFO: Task 2875 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62a87642ca1c8313fd2aed199e9e53ef with result from task 2875 parsl.dataflow.dflow DEBUG: Task 2875 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/aa983a72-5840-4fbd-a5d8-45a2dd3dc9e4_isrForCrosstalkSources_996960_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/aa983a72-5840-4fbd-a5d8-45a2dd3dc9e4_isrForCrosstalkSources_996960_1.stderr parsl.dataflow.dflow INFO: Task 3187 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8277203a29961a1a2beb422f2f621232 with result from task 3187 parsl.dataflow.dflow DEBUG: Task 3187 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/989592d4-d748-4d6c-9972-8ccce586e4ae_isrForCrosstalkSources_997022_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/989592d4-d748-4d6c-9972-8ccce586e4ae_isrForCrosstalkSources_997022_37.stderr parsl.dataflow.dflow INFO: Task 2865 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d233af6b6ba67e159a9c28cd6adca48 with result from task 2865 parsl.dataflow.dflow DEBUG: Task 2865 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/6c5c5afc-c619-49a5-ad89-d73121b6fa83_isrForCrosstalkSources_997011_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/6c5c5afc-c619-49a5-ad89-d73121b6fa83_isrForCrosstalkSources_997011_32.stderr parsl.dataflow.dflow INFO: Task 3228 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a243ccc59b492874983637b9d84f7c1b with result from task 3228 parsl.dataflow.dflow DEBUG: Task 3228 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/59edb1b1-1d1f-45f6-bd87-5c13ab002150_isrForCrosstalkSources_997019_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/59edb1b1-1d1f-45f6-bd87-5c13ab002150_isrForCrosstalkSources_997019_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:31 proc_lsst.multi:146 [INFO] found job 17074871 in provider astro proc_lsst.multi INFO: found job 17074871 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2888 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b8f965610a4609d3f0ec81e26131962 with result from task 2888 parsl.dataflow.dflow DEBUG: Task 2888 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ddfd41e1-42b7-4702-9548-7b7feb6a9018_isrForCrosstalkSources_996986_34.stdout parsl.dataflow.dflow INFO: Standard error for task 2888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/ddfd41e1-42b7-4702-9548-7b7feb6a9018_isrForCrosstalkSources_996986_34.stderr parsl.dataflow.dflow INFO: Task 2937 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ecb8ca175f3abd05d4fe0848e406813b with result from task 2937 parsl.dataflow.dflow DEBUG: Task 2937 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1a217e41-6131-41cb-bf3f-a5386f353b5a_isrForCrosstalkSources_996968_27.stdout parsl.dataflow.dflow INFO: Standard error for task 2937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1a217e41-6131-41cb-bf3f-a5386f353b5a_isrForCrosstalkSources_996968_27.stderr parsl.dataflow.dflow INFO: Task 3232 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a589f78d4258f9f201b9ea44c7c14d4 with result from task 3232 parsl.dataflow.dflow DEBUG: Task 3232 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1c012e16-2d3f-47eb-8685-90768efb91d3_isrForCrosstalkSources_996968_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1c012e16-2d3f-47eb-8685-90768efb91d3_isrForCrosstalkSources_996968_15.stderr parsl.dataflow.dflow INFO: Task 2858 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5c60c2e978804978255b1fb1a10798a with result from task 2858 parsl.dataflow.dflow DEBUG: Task 2858 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/bcb85892-ef4c-4ffe-a8fc-8f2cbe694779_isrForCrosstalkSources_996993_35.stdout parsl.dataflow.dflow INFO: Standard error for task 2858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/bcb85892-ef4c-4ffe-a8fc-8f2cbe694779_isrForCrosstalkSources_996993_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 3015 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 24e90229e560960a68be2e96433464ea with result from task 3015 parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:32 proc_lsst.multi:146 [INFO] found job 17074872 in provider astro proc_lsst.multi INFO: found job 17074872 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow DEBUG: Task 3015 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ad1dc883-53d4-4c07-802f-9a2b7cd76b52_isrForCrosstalkSources_996988_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ad1dc883-53d4-4c07-802f-9a2b7cd76b52_isrForCrosstalkSources_996988_22.stderr parsl.dataflow.dflow INFO: Task 2922 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 021e3a3fec33810f843289e51e7f9236 with result from task 2922 parsl.dataflow.dflow DEBUG: Task 2922 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/46b75174-985b-471d-a143-9f8c3b731039_isrForCrosstalkSources_996984_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/46b75174-985b-471d-a143-9f8c3b731039_isrForCrosstalkSources_996984_36.stderr parsl.dataflow.dflow INFO: Task 3054 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d88d58c5a6a02df367eb41e434dcd7c with result from task 3054 parsl.dataflow.dflow DEBUG: Task 3054 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/4474889b-c5cf-42a8-8efb-830e6c40b76d_isrForCrosstalkSources_996993_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/4474889b-c5cf-42a8-8efb-830e6c40b76d_isrForCrosstalkSources_996993_22.stderr parsl.dataflow.dflow INFO: Task 3233 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e7f2e367290bffc9dc8830015f88500 with result from task 3233 parsl.dataflow.dflow DEBUG: Task 3233 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/5d53d407-0c5c-4959-8802-5db729c38656_isrForCrosstalkSources_996960_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/5d53d407-0c5c-4959-8802-5db729c38656_isrForCrosstalkSources_996960_59.stderr parsl.dataflow.dflow INFO: Task 3242 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b754a289f7912ce67d24645786e33e96 with result from task 3242 parsl.dataflow.dflow DEBUG: Task 3242 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/69ec19b2-61eb-40f9-a248-5ef03557c973_isrForCrosstalkSources_997029_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/69ec19b2-61eb-40f9-a248-5ef03557c973_isrForCrosstalkSources_997029_8.stderr parsl.dataflow.dflow INFO: Task 3066 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 785c447f3b96c406c96139de630f0c37 with result from task 3066 parsl.dataflow.dflow DEBUG: Task 3066 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/4188305a-b57c-4e4e-85ae-edacab1dda95_isrForCrosstalkSources_996965_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/4188305a-b57c-4e4e-85ae-edacab1dda95_isrForCrosstalkSources_996965_22.stderr parsl.dataflow.dflow INFO: Task 2993 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a0dd778bd18c39c787bb2d70170d7f93 with result from task 2993 parsl.dataflow.dflow DEBUG: Task 2993 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/1ac4efb5-a844-4938-b608-b89fb1358188_isrForCrosstalkSources_997012_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/1ac4efb5-a844-4938-b608-b89fb1358188_isrForCrosstalkSources_997012_18.stderr parsl.dataflow.dflow INFO: Task 2990 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf8f2471ba00367ff1195eef8ba67a8c with result from task 2990 parsl.dataflow.dflow DEBUG: Task 2990 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c523b3ce-37fc-46e8-b926-bbda289a93d2_isrForCrosstalkSources_997006_33.stdout parsl.dataflow.dflow INFO: Standard error for task 2990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c523b3ce-37fc-46e8-b926-bbda289a93d2_isrForCrosstalkSources_997006_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:32 proc_lsst.multi:146 [INFO] found job 17074873 in provider astro proc_lsst.multi INFO: found job 17074873 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3241 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cff97c3cde5ff7ca353c805f3d99d6a3 with result from task 3241 parsl.dataflow.dflow DEBUG: Task 3241 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/22b0b4ad-c9dc-4eb5-8c32-fb32e4ab5ba3_isrForCrosstalkSources_997031_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/22b0b4ad-c9dc-4eb5-8c32-fb32e4ab5ba3_isrForCrosstalkSources_997031_47.stderr parsl.dataflow.dflow INFO: Task 2942 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 913870cf0b593c02acf55505e35a762d with result from task 2942 parsl.dataflow.dflow DEBUG: Task 2942 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/ab79b855-35ef-4502-b426-023352ff219f_isrForCrosstalkSources_997022_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/ab79b855-35ef-4502-b426-023352ff219f_isrForCrosstalkSources_997022_3.stderr parsl.dataflow.dflow INFO: Task 3133 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b44e000a29ad2ec4f665a0fdca1fb72e with result from task 3133 parsl.dataflow.dflow DEBUG: Task 3133 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/be429ecb-6fae-46c9-aff1-8eb7905f187c_isrForCrosstalkSources_996984_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/be429ecb-6fae-46c9-aff1-8eb7905f187c_isrForCrosstalkSources_996984_10.stderr parsl.dataflow.dflow INFO: Task 3158 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f00399d665afe62d758db29eae6dda12 with result from task 3158 parsl.dataflow.dflow DEBUG: Task 3158 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6116f23a-3507-45bf-96f9-7a5efcce218a_isrForCrosstalkSources_996952_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6116f23a-3507-45bf-96f9-7a5efcce218a_isrForCrosstalkSources_996952_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:33 proc_lsst.multi:146 [INFO] found job 17074876 in provider astro proc_lsst.multi INFO: found job 17074876 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2948 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3eb111c4ab84a14178cab0e11ed75db9 with result from task 2948 parsl.dataflow.dflow DEBUG: Task 2948 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/800f343f-4cfa-4646-b994-048922fe17b1_isrForCrosstalkSources_996985_21.stdout parsl.dataflow.dflow INFO: Standard error for task 2948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/800f343f-4cfa-4646-b994-048922fe17b1_isrForCrosstalkSources_996985_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:33 proc_lsst.multi:146 [INFO] found job 17074877 in provider astro proc_lsst.multi INFO: found job 17074877 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3115 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b21373053c85a1ab697bc40d2569c47 with result from task 3115 parsl.dataflow.dflow DEBUG: Task 3115 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7c716cb8-4fb8-4fa0-b214-1a5a4c22d133_isrForCrosstalkSources_997015_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7c716cb8-4fb8-4fa0-b214-1a5a4c22d133_isrForCrosstalkSources_997015_29.stderr parsl.dataflow.dflow INFO: Task 2898 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 291b4f5d391ca6205ee0049117d8b4d5 with result from task 2898 parsl.dataflow.dflow DEBUG: Task 2898 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/a93a4ca0-0e11-4315-ada0-9a469480afd0_isrForCrosstalkSources_996962_16.stdout parsl.dataflow.dflow INFO: Standard error for task 2898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/a93a4ca0-0e11-4315-ada0-9a469480afd0_isrForCrosstalkSources_996962_16.stderr parsl.dataflow.dflow INFO: Task 3038 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb85e83d5b96bcfe4be2ead17b57c028 with result from task 3038 parsl.dataflow.dflow DEBUG: Task 3038 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/bd47b642-33f5-4921-9ed2-c2f07e6f030f_isrForCrosstalkSources_996979_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/bd47b642-33f5-4921-9ed2-c2f07e6f030f_isrForCrosstalkSources_996979_2.stderr parsl.dataflow.dflow INFO: Task 3217 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f99a43e745a260a83f7b5a4970e10ffd with result from task 3217 parsl.dataflow.dflow DEBUG: Task 3217 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0c01677f-8ccf-42ac-9622-860c9dffbb78_isrForCrosstalkSources_997020_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0c01677f-8ccf-42ac-9622-860c9dffbb78_isrForCrosstalkSources_997020_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:34 proc_lsst.multi:146 [INFO] found job 17074879 in provider astro proc_lsst.multi INFO: found job 17074879 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 2995 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4026ef945ccf57504440976ca8702490 with result from task 2995 parsl.dataflow.dflow DEBUG: Task 2995 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b59dddc9-c889-4d5f-9719-03e7fb23e039_isrForCrosstalkSources_997014_20.stdout parsl.dataflow.dflow INFO: Standard error for task 2995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/b59dddc9-c889-4d5f-9719-03e7fb23e039_isrForCrosstalkSources_997014_20.stderr parsl.dataflow.dflow INFO: Task 3267 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c18137992ed4c50a6efdd38fdb7f1e34 with result from task 3267 parsl.dataflow.dflow DEBUG: Task 3267 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4f6280d3-742f-4fac-8500-22dcc2b4d494_isrForCrosstalkSources_997015_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4f6280d3-742f-4fac-8500-22dcc2b4d494_isrForCrosstalkSources_997015_48.stderr parsl.dataflow.dflow INFO: Task 3009 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f2c1bd0d7e858a30d7b6ee5b68089946 with result from task 3009 parsl.dataflow.dflow DEBUG: Task 3009 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/75a8615d-e098-470a-8ba2-ceb4b6f71336_isrForCrosstalkSources_997009_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/75a8615d-e098-470a-8ba2-ceb4b6f71336_isrForCrosstalkSources_997009_12.stderr parsl.dataflow.dflow INFO: Task 2949 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 984b3aed66eb65a77460451e538b6bd3 with result from task 2949 parsl.dataflow.dflow DEBUG: Task 2949 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b21cf7f6-1ce3-4a53-bbd8-4f1c55267741_isrForCrosstalkSources_996986_47.stdout parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.dataflow.dflow INFO: Standard error for task 2949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b21cf7f6-1ce3-4a53-bbd8-4f1c55267741_isrForCrosstalkSources_996986_47.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:34 proc_lsst.multi:146 [INFO] found job 17074880 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074880 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 2994 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4e5a30978ee6573cddd3d633e9544e4 with result from task 2994 parsl.dataflow.dflow DEBUG: Task 2994 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/2ad7c6ab-eedd-4eaa-b8d5-8c22eef7d5f7_isrForCrosstalkSources_996967_56.stdout parsl.dataflow.dflow INFO: Standard error for task 2994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/2ad7c6ab-eedd-4eaa-b8d5-8c22eef7d5f7_isrForCrosstalkSources_996967_56.stderr parsl.dataflow.dflow INFO: Task 2919 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a0ace76b9cfb9d27893ac716ea2402e with result from task 2919 parsl.dataflow.dflow DEBUG: Task 2919 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/abb611a2-7362-4a40-82f9-f5fd643cf6f6_isrForCrosstalkSources_997005_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/abb611a2-7362-4a40-82f9-f5fd643cf6f6_isrForCrosstalkSources_997005_30.stderr parsl.dataflow.dflow INFO: Task 2955 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4460ed457e9708b582bd2a6b8a8b000c with result from task 2955 parsl.dataflow.dflow DEBUG: Task 2955 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/e6a2bcae-1c1a-4f9c-96f3-f2e83fc74d11_isrForCrosstalkSources_996959_18.stdout parsl.dataflow.dflow INFO: Standard error for task 2955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/e6a2bcae-1c1a-4f9c-96f3-f2e83fc74d11_isrForCrosstalkSources_996959_18.stderr parsl.dataflow.dflow INFO: Task 3035 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70a4bb55a596a9cc89cba13d6d162da5 with result from task 3035 parsl.dataflow.dflow DEBUG: Task 3035 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/ae3f09d9-7711-4199-aa1a-9927207e3f3c_isrForCrosstalkSources_996977_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/ae3f09d9-7711-4199-aa1a-9927207e3f3c_isrForCrosstalkSources_996977_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:35 proc_lsst.multi:146 [INFO] found job 17074881 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074881 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 2930 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 043da3a70c15f79dbccc3bfb4a3e4cf5 with result from task 2930 parsl.dataflow.dflow DEBUG: Task 2930 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/07a28806-fcea-4117-904f-dd6e6a8c40d9_isrForCrosstalkSources_997007_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/07a28806-fcea-4117-904f-dd6e6a8c40d9_isrForCrosstalkSources_997007_31.stderr parsl.dataflow.dflow INFO: Task 3274 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2210ff7eee818cc57f064684e451333a with result from task 3274 parsl.dataflow.dflow DEBUG: Task 3274 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ca35e9b3-b27c-4989-bf4a-379b8c7385be_isrForCrosstalkSources_996978_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ca35e9b3-b27c-4989-bf4a-379b8c7385be_isrForCrosstalkSources_996978_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:35 proc_lsst.multi:146 [INFO] found job 17074887 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074887 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 2959 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f0e09d85d921b508a2c078b5da3691e with result from task 2959 parsl.dataflow.dflow DEBUG: Task 2959 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d5b0b451-8f69-4543-a26e-7549bc59a00c_isrForCrosstalkSources_996988_25.stdout parsl.dataflow.dflow INFO: Standard error for task 2959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/d5b0b451-8f69-4543-a26e-7549bc59a00c_isrForCrosstalkSources_996988_25.stderr parsl.dataflow.dflow INFO: Task 3225 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26dd0d67199157ebc836f44ee9bba511 with result from task 3225 parsl.dataflow.dflow DEBUG: Task 3225 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a22f5aa2-608d-4edd-ad6b-47f9a4ddd9f3_isrForCrosstalkSources_996972_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a22f5aa2-608d-4edd-ad6b-47f9a4ddd9f3_isrForCrosstalkSources_996972_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:36 proc_lsst.multi:146 [INFO] found job 17074888 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074888 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 2965 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7302b38f051f3b466aad9abe526672f2 with result from task 2965 parsl.dataflow.dflow DEBUG: Task 2965 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/b67c6557-6aa0-430a-9463-2b072b6db208_isrForCrosstalkSources_997000_44.stdout parsl.dataflow.dflow INFO: Standard error for task 2965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/b67c6557-6aa0-430a-9463-2b072b6db208_isrForCrosstalkSources_997000_44.stderr parsl.dataflow.dflow INFO: Task 3026 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4421269593a4e3a122c53922eaaf1a1 with result from task 3026 parsl.dataflow.dflow DEBUG: Task 3026 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a9624c59-12f7-4a0d-aaf7-8d8b6f927b67_isrForCrosstalkSources_996970_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/a9624c59-12f7-4a0d-aaf7-8d8b6f927b67_isrForCrosstalkSources_996970_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:36 proc_lsst.multi:146 [INFO] found job 17074889 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074889 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 2891 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae27760d3d2c4c28363f401be1797f41 with result from task 2891 parsl.dataflow.dflow DEBUG: Task 2891 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/b144cdf3-8180-49b2-9196-cbd1c308faae_isrForCrosstalkSources_997020_28.stdout parsl.dataflow.dflow INFO: Standard error for task 2891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/b144cdf3-8180-49b2-9196-cbd1c308faae_isrForCrosstalkSources_997020_28.stderr parsl.dataflow.dflow INFO: Task 2975 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 742b203f0b83f64d90b8c6da2e79d7b1 with result from task 2975 parsl.dataflow.dflow DEBUG: Task 2975 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c72a0b5e-73f4-486c-8251-8585b8a30750_isrForCrosstalkSources_997019_3.stdout parsl.dataflow.dflow INFO: Standard error for task 2975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c72a0b5e-73f4-486c-8251-8585b8a30750_isrForCrosstalkSources_997019_3.stderr parsl.dataflow.dflow INFO: Task 2997 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 95f9d825d79ac23c4db16b101c755a8e with result from task 2997 parsl.dataflow.dflow DEBUG: Task 2997 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/3a7c7af3-d949-4eb1-b94b-20a61af155d9_isrForCrosstalkSources_997007_7.stdout parsl.dataflow.dflow INFO: Standard error for task 2997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/3a7c7af3-d949-4eb1-b94b-20a61af155d9_isrForCrosstalkSources_997007_7.stderr parsl.dataflow.dflow INFO: Task 3282 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50df77904d7ac6eda55b57194f548243 with result from task 3282 parsl.dataflow.dflow DEBUG: Task 3282 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/236080cc-ae97-46b8-819a-436b4f191cb3_isrForCrosstalkSources_996959_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/236080cc-ae97-46b8-819a-436b4f191cb3_isrForCrosstalkSources_996959_28.stderr parsl.dataflow.dflow INFO: Task 2998 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62bfbc81827d3ea9c83687d53de6b620 with result from task 2998 parsl.dataflow.dflow DEBUG: Task 2998 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/1a0096a0-174e-419f-a4b4-941877ebbe74_isrForCrosstalkSources_996970_37.stdout parsl.dataflow.dflow INFO: Standard error for task 2998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/1a0096a0-174e-419f-a4b4-941877ebbe74_isrForCrosstalkSources_996970_37.stderr parsl.dataflow.dflow INFO: Task 3283 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7dad7c1a40a47b2b80b66fc4ee816a6c with result from task 3283 parsl.dataflow.dflow DEBUG: Task 3283 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/6334bec6-7cdd-40f0-9a23-6ae2c6954ae2_isrForCrosstalkSources_997028_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/6334bec6-7cdd-40f0-9a23-6ae2c6954ae2_isrForCrosstalkSources_997028_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:37 proc_lsst.multi:146 [INFO] found job 17074894 in provider escience_a40 proc_lsst.multi INFO: found job 17074894 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3000 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d642e1834ee0224f0fc51da3d12af5fb with result from task 3000 parsl.dataflow.dflow DEBUG: Task 3000 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/d8e9740d-24df-4853-a004-2eea1fcd88ae_isrForCrosstalkSources_996986_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/d8e9740d-24df-4853-a004-2eea1fcd88ae_isrForCrosstalkSources_996986_60.stderr parsl.dataflow.dflow INFO: Task 2920 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9196ab1154569ac9268e6b2832d55848 with result from task 2920 parsl.dataflow.dflow DEBUG: Task 2920 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/49f24e6e-724b-4f18-9c03-02279f27d04f_isrForCrosstalkSources_996979_1.stdout parsl.dataflow.dflow INFO: Standard error for task 2920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/49f24e6e-724b-4f18-9c03-02279f27d04f_isrForCrosstalkSources_996979_1.stderr parsl.dataflow.dflow INFO: Task 3301 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 600d412999185bdf5a2d030fc464e07d with result from task 3301 parsl.dataflow.dflow DEBUG: Task 3301 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/5f57e9ee-5c7b-4583-8124-022f78a626ba_isrForCrosstalkSources_996977_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/5f57e9ee-5c7b-4583-8124-022f78a626ba_isrForCrosstalkSources_996977_13.stderr parsl.dataflow.dflow INFO: Task 3309 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18e6c1354e8dddf8a73461a6d3ac17d9 with result from task 3309 parsl.dataflow.dflow DEBUG: Task 3309 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ef2e4274-d671-478c-9cd8-bfec861316a2_isrForCrosstalkSources_996960_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/ef2e4274-d671-478c-9cd8-bfec861316a2_isrForCrosstalkSources_996960_27.stderr parsl.dataflow.dflow INFO: Task 2944 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ddf7fbe4dc7e956ec90f58c3b8c774c8 with result from task 2944 parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:37 proc_lsst.multi:146 [INFO] found job 17074895 in provider escience_a40 proc_lsst.multi INFO: found job 17074895 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow DEBUG: Task 2944 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9e2a194-c615-49cb-8682-d2693655fb5b_isrForCrosstalkSources_997009_30.stdout parsl.dataflow.dflow INFO: Standard error for task 2944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9e2a194-c615-49cb-8682-d2693655fb5b_isrForCrosstalkSources_997009_30.stderr parsl.dataflow.dflow INFO: Task 2967 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 451b943be4f43716757040dc88f5fb0f with result from task 2967 parsl.dataflow.dflow DEBUG: Task 2967 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ab031285-508e-4019-b453-edadec8b4998_isrForCrosstalkSources_997002_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ab031285-508e-4019-b453-edadec8b4998_isrForCrosstalkSources_997002_49.stderr parsl.dataflow.dflow INFO: Task 3262 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93157f2ce45130bd026afb24afb81da1 with result from task 3262 parsl.dataflow.dflow DEBUG: Task 3262 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ea18decf-2bf9-44c5-b2d4-96d535c3ab7b_isrForCrosstalkSources_997009_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ea18decf-2bf9-44c5-b2d4-96d535c3ab7b_isrForCrosstalkSources_997009_49.stderr parsl.dataflow.dflow INFO: Task 3317 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 150333336384370241acdde71eaa2429 with result from task 3317 parsl.dataflow.dflow DEBUG: Task 3317 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/1ec48d5a-259f-42a6-9fae-5144174cbe3f_isrForCrosstalkSources_997031_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/1ec48d5a-259f-42a6-9fae-5144174cbe3f_isrForCrosstalkSources_997031_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:38 proc_lsst.multi:146 [INFO] found job 17074897 in provider escience_a40 proc_lsst.multi INFO: found job 17074897 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3120 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 691d8d110f5d4316f526b3098b40e0d4 with result from task 3120 parsl.dataflow.dflow DEBUG: Task 3120 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bb5bc034-5bc5-4141-9c31-524ccc5f4461_isrForCrosstalkSources_997027_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bb5bc034-5bc5-4141-9c31-524ccc5f4461_isrForCrosstalkSources_997027_19.stderr parsl.dataflow.dflow INFO: Task 2962 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f526bbf48a3c2c912c0e4f8ce423a8b0 with result from task 2962 parsl.dataflow.dflow DEBUG: Task 2962 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/0c6b8235-683a-4d3b-8712-d57ea5f06093_isrForCrosstalkSources_996990_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/0c6b8235-683a-4d3b-8712-d57ea5f06093_isrForCrosstalkSources_996990_62.stderr parsl.dataflow.dflow INFO: Task 3074 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f92d1652de701c3c86b1540f3cf66141 with result from task 3074 parsl.dataflow.dflow DEBUG: Task 3074 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/26816f57-3aa2-405b-bd68-e5f1dad1ca62_isrForCrosstalkSources_997024_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/26816f57-3aa2-405b-bd68-e5f1dad1ca62_isrForCrosstalkSources_997024_14.stderr parsl.dataflow.dflow INFO: Task 2914 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fceca0e18408a0a20620f53a6cc24f37 with result from task 2914 parsl.dataflow.dflow DEBUG: Task 2914 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/edc974c1-fc9c-44c8-954f-9f7e3de2a281_isrForCrosstalkSources_996958_36.stdout parsl.dataflow.dflow INFO: Standard error for task 2914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/edc974c1-fc9c-44c8-954f-9f7e3de2a281_isrForCrosstalkSources_996958_36.stderr parsl.dataflow.dflow INFO: Task 3245 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63577645c79c083c98cdd9dcd02dff9f with result from task 3245 parsl.dataflow.dflow DEBUG: Task 3245 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b14cf562-4445-4dc7-9d26-cd5fa9a36309_isrForCrosstalkSources_996964_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b14cf562-4445-4dc7-9d26-cd5fa9a36309_isrForCrosstalkSources_996964_25.stderr parsl.dataflow.dflow INFO: Task 2935 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d556533580a3f38d10e465b92c84b10c with result from task 2935 parsl.dataflow.dflow DEBUG: Task 2935 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/884b6e24-1b7e-4219-a3d4-2dc2c26d38bf_isrForCrosstalkSources_996957_49.stdout parsl.dataflow.dflow INFO: Standard error for task 2935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/884b6e24-1b7e-4219-a3d4-2dc2c26d38bf_isrForCrosstalkSources_996957_49.stderr parsl.dataflow.dflow INFO: Task 2939 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2cbbddf2a4ddd1925e76f1233a93d355 with result from task 2939 parsl.dataflow.dflow DEBUG: Task 2939 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/416cc84b-2de0-46fd-b393-b03e999491c6_isrForCrosstalkSources_997017_53.stdout parsl.dataflow.dflow INFO: Standard error for task 2939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/416cc84b-2de0-46fd-b393-b03e999491c6_isrForCrosstalkSources_997017_53.stderr parsl.dataflow.dflow INFO: Task 2879 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6aa853d564a4970f8015e46f48f0463c with result from task 2879 parsl.dataflow.dflow DEBUG: Task 2879 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a898564e-bd29-490a-a84f-a732751b7fbc_isrForCrosstalkSources_996993_31.stdout parsl.dataflow.dflow INFO: Standard error for task 2879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a898564e-bd29-490a-a84f-a732751b7fbc_isrForCrosstalkSources_996993_31.stderr parsl.dataflow.dflow INFO: Task 3043 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 934fc07552a38e8c32648a4761d81fed with result from task 3043 parsl.dataflow.dflow DEBUG: Task 3043 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/73ff5190-3455-4e3f-90ea-922e4541da3a_isrForCrosstalkSources_997020_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/73ff5190-3455-4e3f-90ea-922e4541da3a_isrForCrosstalkSources_997020_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:38 proc_lsst.multi:146 [INFO] found job 17074898 in provider escience_a40 proc_lsst.multi INFO: found job 17074898 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3316 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 703868cbb295d5f13f913930b2e4e97c with result from task 3316 parsl.dataflow.dflow DEBUG: Task 3316 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/245462cd-89ba-4499-a057-98b45c6a2e96_isrForCrosstalkSources_996988_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/245462cd-89ba-4499-a057-98b45c6a2e96_isrForCrosstalkSources_996988_46.stderr parsl.dataflow.dflow INFO: Task 3064 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1798184e89d7b636eb28a1a878ccf3b5 with result from task 3064 parsl.dataflow.dflow DEBUG: Task 3064 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/420478ac-58c6-4aaf-a22c-070409b37787_isrForCrosstalkSources_996962_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/420478ac-58c6-4aaf-a22c-070409b37787_isrForCrosstalkSources_996962_17.stderr parsl.dataflow.dflow INFO: Task 3070 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aea16c65f58fed389af265bbf1f20a76 with result from task 3070 parsl.dataflow.dflow DEBUG: Task 3070 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/84a660c8-461b-4b00-a6fb-ec4d8d31750c_isrForCrosstalkSources_997031_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/84a660c8-461b-4b00-a6fb-ec4d8d31750c_isrForCrosstalkSources_997031_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:39 proc_lsst.multi:146 [INFO] found job 17074932 in provider escience_a40 proc_lsst.multi INFO: found job 17074932 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3213 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c87ae4eb968b9205f8ec3aa060e05087 with result from task 3213 parsl.dataflow.dflow DEBUG: Task 3213 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/ff5aaf99-f39d-46f5-87a0-12e6465798e5_isrForCrosstalkSources_996963_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/ff5aaf99-f39d-46f5-87a0-12e6465798e5_isrForCrosstalkSources_996963_34.stderr parsl.dataflow.dflow INFO: Task 3023 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d071794ebeb1c9eebf89fe919280b15 with result from task 3023 parsl.dataflow.dflow DEBUG: Task 3023 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/62bc8d37-203f-4fc1-82c9-9a571b534dd1_isrForCrosstalkSources_996952_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/62bc8d37-203f-4fc1-82c9-9a571b534dd1_isrForCrosstalkSources_996952_54.stderr parsl.dataflow.dflow INFO: Task 3288 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 992e9afc89cce4251bb210b26406283b with result from task 3288 parsl.dataflow.dflow DEBUG: Task 3288 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b4931aff-0ff0-446b-8994-f35bfd9b2efd_isrForCrosstalkSources_996975_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b4931aff-0ff0-446b-8994-f35bfd9b2efd_isrForCrosstalkSources_996975_17.stderr parsl.dataflow.dflow INFO: Task 3025 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7f7839af827a32677337352bf06fdf2c with result from task 3025 parsl.dataflow.dflow DEBUG: Task 3025 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/c04f563b-66f6-41b6-9166-0df8a2f43e19_isrForCrosstalkSources_996986_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/c04f563b-66f6-41b6-9166-0df8a2f43e19_isrForCrosstalkSources_996986_54.stderr parsl.dataflow.dflow INFO: Task 3151 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a385c570f209d04c2dbb6b07326601a with result from task 3151 parsl.dataflow.dflow DEBUG: Task 3151 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/19a784a4-4f66-4da4-8d2b-253bc9c9e230_isrForCrosstalkSources_996979_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/19a784a4-4f66-4da4-8d2b-253bc9c9e230_isrForCrosstalkSources_996979_17.stderr parsl.dataflow.dflow INFO: Task 3305 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5bbda126a080b829fa64fd193088fe60 with result from task 3305 parsl.dataflow.dflow DEBUG: Task 3305 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ef95ef8e-5eeb-4a8d-b25d-ba2f7f77fe7a_isrForCrosstalkSources_997005_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ef95ef8e-5eeb-4a8d-b25d-ba2f7f77fe7a_isrForCrosstalkSources_997005_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:39 proc_lsst.multi:146 [INFO] found job 17074933 in provider escience_a40 proc_lsst.multi INFO: found job 17074933 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3300 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d9c316b217bc68cff40ba7d4c4dfc5c with result from task 3300 parsl.dataflow.dflow DEBUG: Task 3300 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/70e25ea8-bc83-4603-aa97-d02d58bc22ae_isrForCrosstalkSources_996993_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/70e25ea8-bc83-4603-aa97-d02d58bc22ae_isrForCrosstalkSources_996993_19.stderr parsl.dataflow.dflow INFO: Task 3307 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84b34b3de46b2c196c8faef2c58016b8 with result from task 3307 parsl.dataflow.dflow DEBUG: Task 3307 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b46e73bb-e872-4fb9-954f-1c441d285544_isrForCrosstalkSources_996957_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b46e73bb-e872-4fb9-954f-1c441d285544_isrForCrosstalkSources_996957_8.stderr parsl.dataflow.dflow INFO: Task 3227 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19c77aa013f7dbcefe5feabe3ec2d2a2 with result from task 3227 parsl.dataflow.dflow DEBUG: Task 3227 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/eb288399-d1b8-46a2-aa8b-08a8236b9bcb_isrForCrosstalkSources_996972_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/eb288399-d1b8-46a2-aa8b-08a8236b9bcb_isrForCrosstalkSources_996972_51.stderr parsl.dataflow.dflow INFO: Task 3314 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4d400125de48dcdd517e50e1f8cb1ed1 with result from task 3314 parsl.dataflow.dflow DEBUG: Task 3314 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/509fd5b4-41d1-4871-8f6b-1a19d5eb8fc7_isrForCrosstalkSources_996979_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/509fd5b4-41d1-4871-8f6b-1a19d5eb8fc7_isrForCrosstalkSources_996979_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:40 proc_lsst.multi:146 [INFO] found job 17074934 in provider escience_a40 proc_lsst.multi INFO: found job 17074934 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3311 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8268db8a767400a899ca5bfb62f9b38c with result from task 3311 parsl.dataflow.dflow DEBUG: Task 3311 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2a590b61-1471-4bd8-aaa8-43b22c92f47e_isrForCrosstalkSources_996981_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/2a590b61-1471-4bd8-aaa8-43b22c92f47e_isrForCrosstalkSources_996981_18.stderr parsl.dataflow.dflow INFO: Task 3161 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 43e4ad3d641b8d0431f6e3aeeb56389b with result from task 3161 parsl.dataflow.dflow DEBUG: Task 3161 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/addf0e15-c7f6-4ef1-bcf0-fd979ae157fa_isrForCrosstalkSources_997032_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/addf0e15-c7f6-4ef1-bcf0-fd979ae157fa_isrForCrosstalkSources_997032_39.stderr parsl.dataflow.dflow INFO: Task 3113 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 394eea11f85f9a5d4964f09fab96666a with result from task 3113 parsl.dataflow.dflow DEBUG: Task 3113 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1da4932f-8f04-413d-9548-c2a75bb14451_isrForCrosstalkSources_996983_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1da4932f-8f04-413d-9548-c2a75bb14451_isrForCrosstalkSources_996983_61.stderr parsl.dataflow.dflow INFO: Task 3152 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33943e32b47c0d7beb06ca1fe2ede106 with result from task 3152 parsl.dataflow.dflow DEBUG: Task 3152 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f04a59e9-646a-40eb-aaf8-2e8c05a747cf_isrForCrosstalkSources_996967_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f04a59e9-646a-40eb-aaf8-2e8c05a747cf_isrForCrosstalkSources_996967_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:40 proc_lsst.multi:146 [INFO] found job 17074935 in provider escience_a40 proc_lsst.multi INFO: found job 17074935 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3279 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4dcdc60785d10c8d4307a89a1123683e with result from task 3279 parsl.dataflow.dflow DEBUG: Task 3279 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/906cebc8-5af4-4d8d-aa54-4501ca8ee5cd_isrForCrosstalkSources_996956_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/906cebc8-5af4-4d8d-aa54-4501ca8ee5cd_isrForCrosstalkSources_996956_16.stderr parsl.dataflow.dflow INFO: Task 3003 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 223853d25f6d285cabc53ec68f199eaa with result from task 3003 parsl.dataflow.dflow DEBUG: Task 3003 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/c8f9caf0-eac8-4719-a7da-252635471289_isrForCrosstalkSources_996964_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/c8f9caf0-eac8-4719-a7da-252635471289_isrForCrosstalkSources_996964_32.stderr parsl.dataflow.dflow INFO: Task 3022 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c02509749be012c06954e298074da6f with result from task 3022 parsl.dataflow.dflow DEBUG: Task 3022 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b20d992f-affc-4174-a002-f3f74316c023_isrForCrosstalkSources_997001_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b20d992f-affc-4174-a002-f3f74316c023_isrForCrosstalkSources_997001_58.stderr parsl.dataflow.dflow INFO: Task 3135 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c9f704135b70c3f7d9f478912200262 with result from task 3135 parsl.dataflow.dflow DEBUG: Task 3135 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/895104c2-d698-495e-87c7-ef5e67a883ec_isrForCrosstalkSources_996983_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/895104c2-d698-495e-87c7-ef5e67a883ec_isrForCrosstalkSources_996983_29.stderr parsl.dataflow.dflow INFO: Task 3312 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f33c14b7c0cf595b37b928c8d244186 with result from task 3312 parsl.dataflow.dflow DEBUG: Task 3312 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/6696a742-de7d-4d1d-9eb0-dffbbc06ea48_isrForCrosstalkSources_996962_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/6696a742-de7d-4d1d-9eb0-dffbbc06ea48_isrForCrosstalkSources_996962_30.stderr parsl.dataflow.dflow INFO: Task 2958 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0251facc2da8d2eb8a264dfab10238d8 with result from task 2958 parsl.dataflow.dflow DEBUG: Task 2958 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/02dcf7c9-6669-4d58-ac90-ad7962d0df8b_isrForCrosstalkSources_997022_8.stdout parsl.dataflow.dflow INFO: Standard error for task 2958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/02dcf7c9-6669-4d58-ac90-ad7962d0df8b_isrForCrosstalkSources_997022_8.stderr parsl.dataflow.dflow INFO: Task 2999 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2198311c132f0259f50b82979ae27a8b with result from task 2999 parsl.dataflow.dflow DEBUG: Task 2999 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1f29cc4f-a5cf-4062-9011-b23e01fcc860_isrForCrosstalkSources_996960_32.stdout parsl.dataflow.dflow INFO: Standard error for task 2999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1f29cc4f-a5cf-4062-9011-b23e01fcc860_isrForCrosstalkSources_996960_32.stderr parsl.dataflow.dflow INFO: Task 3075 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dc70eab1f75cc6ff20fea2ab78b9b2ad with result from task 3075 parsl.dataflow.dflow DEBUG: Task 3075 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5ced96d3-2e74-4217-8972-c490cd2c3c7a_isrForCrosstalkSources_997031_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/5ced96d3-2e74-4217-8972-c490cd2c3c7a_isrForCrosstalkSources_997031_9.stderr parsl.dataflow.dflow INFO: Task 3216 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4553aa0464171f12abee82c6432f6c9 with result from task 3216 parsl.dataflow.dflow DEBUG: Task 3216 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4032d8d8-232f-433c-9f09-6e1b30e8b6e3_isrForCrosstalkSources_997020_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/4032d8d8-232f-433c-9f09-6e1b30e8b6e3_isrForCrosstalkSources_997020_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:41 proc_lsst.multi:146 [INFO] found job 17074936 in provider escience_a40 proc_lsst.multi INFO: found job 17074936 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3280 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c170007e2f4f5d8e3e05622ecbe3ba0 with result from task 3280 parsl.dataflow.dflow DEBUG: Task 3280 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/fee0cd93-a087-4253-af6b-424797133f8a_isrForCrosstalkSources_996989_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/fee0cd93-a087-4253-af6b-424797133f8a_isrForCrosstalkSources_996989_36.stderr parsl.dataflow.dflow INFO: Task 3109 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6e20ca28c72d72794e24fdcc20c8fe12 with result from task 3109 parsl.dataflow.dflow DEBUG: Task 3109 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6de43bfc-5256-4de3-8f63-fc196048deeb_isrForCrosstalkSources_997010_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6de43bfc-5256-4de3-8f63-fc196048deeb_isrForCrosstalkSources_997010_22.stderr parsl.dataflow.dflow INFO: Task 3139 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53c26fe370bfad2c01ce4eafa99f9473 with result from task 3139 parsl.dataflow.dflow DEBUG: Task 3139 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cf36eba1-0543-4f97-bfb8-e85df080d21d_isrForCrosstalkSources_997015_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/cf36eba1-0543-4f97-bfb8-e85df080d21d_isrForCrosstalkSources_997015_59.stderr parsl.dataflow.dflow INFO: Task 3293 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a2ed3b5f561830d3bc369bbf35c310f with result from task 3293 parsl.dataflow.dflow DEBUG: Task 3293 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/746c9a11-6d8e-4576-b1fd-3f3bcfafa6e1_isrForCrosstalkSources_996976_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/746c9a11-6d8e-4576-b1fd-3f3bcfafa6e1_isrForCrosstalkSources_996976_12.stderr parsl.dataflow.dflow INFO: Task 3103 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e67ecbad58793f8b7121ae3d69f4479c with result from task 3103 parsl.dataflow.dflow DEBUG: Task 3103 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9cc8d5f3-5e08-48f0-9a15-1f8817f10633_isrForCrosstalkSources_997004_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9cc8d5f3-5e08-48f0-9a15-1f8817f10633_isrForCrosstalkSources_997004_9.stderr parsl.dataflow.dflow INFO: Task 3057 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af6893baadc9d2430bfc1704fb6c27dd with result from task 3057 parsl.dataflow.dflow DEBUG: Task 3057 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/191efaa9-c41e-4d52-a1c0-2538f57a6238_isrForCrosstalkSources_996956_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/191efaa9-c41e-4d52-a1c0-2538f57a6238_isrForCrosstalkSources_996956_8.stderr parsl.dataflow.dflow INFO: Task 3321 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91e54edf2854d79174ac990377bb91e0 with result from task 3321 parsl.dataflow.dflow DEBUG: Task 3321 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f77adb2c-8fcb-4ff1-ba05-8356979a53e8_isrForCrosstalkSources_996985_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f77adb2c-8fcb-4ff1-ba05-8356979a53e8_isrForCrosstalkSources_996985_6.stderr parsl.dataflow.dflow INFO: Task 3308 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c146010644b3dfcf83e556d1cd949d4 with result from task 3308 parsl.dataflow.dflow DEBUG: Task 3308 checkpointed parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:41 proc_lsst.multi:146 [INFO] found job 17074937 in provider escience_a40 proc_lsst.multi INFO: found job 17074937 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fbea0fbd-c2db-43ce-a598-11ad613eb48c_isrForCrosstalkSources_996990_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fbea0fbd-c2db-43ce-a598-11ad613eb48c_isrForCrosstalkSources_996990_35.stderr parsl.dataflow.dflow INFO: Task 3199 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29b17907d3d7b65af0570c3e50c47f0e with result from task 3199 parsl.dataflow.dflow DEBUG: Task 3199 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/b7728c5e-5135-4129-85df-10b1813746ec_isrForCrosstalkSources_997005_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/b7728c5e-5135-4129-85df-10b1813746ec_isrForCrosstalkSources_997005_34.stderr parsl.dataflow.dflow INFO: Task 3338 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a258aaf9eb12242663e6a52bbcfcd6e with result from task 3338 parsl.dataflow.dflow DEBUG: Task 3338 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cd4c47bd-a869-4853-ac1a-8e09079c915d_isrForCrosstalkSources_996987_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/cd4c47bd-a869-4853-ac1a-8e09079c915d_isrForCrosstalkSources_996987_33.stderr parsl.dataflow.dflow INFO: Task 3215 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4b63358512be60e00037361fa92c544 with result from task 3215 parsl.dataflow.dflow DEBUG: Task 3215 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a7503e82-2977-43b2-8867-24ec5485b316_isrForCrosstalkSources_997032_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a7503e82-2977-43b2-8867-24ec5485b316_isrForCrosstalkSources_997032_12.stderr parsl.dataflow.dflow INFO: Task 2952 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8650f08b4ec7a1d34a536009edb93b2d with result from task 2952 parsl.dataflow.dflow DEBUG: Task 2952 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/58522cbc-b35d-4f3c-b97f-e0c4230a502c_isrForCrosstalkSources_996987_9.stdout parsl.dataflow.dflow INFO: Standard error for task 2952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/58522cbc-b35d-4f3c-b97f-e0c4230a502c_isrForCrosstalkSources_996987_9.stderr parsl.dataflow.dflow INFO: Task 3132 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7eae3fb0c4478e7489ffb4b740adfd5 with result from task 3132 parsl.dataflow.dflow DEBUG: Task 3132 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/59b23405-670e-45bd-9b4a-bec2074b20f0_isrForCrosstalkSources_996979_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/59b23405-670e-45bd-9b4a-bec2074b20f0_isrForCrosstalkSources_996979_6.stderr parsl.dataflow.dflow INFO: Task 3310 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7aed51fa4eb49e1d1564578abf0ffcc with result from task 3310 parsl.dataflow.dflow DEBUG: Task 3310 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/73673270-259e-41d4-bb05-8083cc8cc483_isrForCrosstalkSources_996955_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/73673270-259e-41d4-bb05-8083cc8cc483_isrForCrosstalkSources_996955_41.stderr parsl.dataflow.dflow INFO: Task 3296 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e63a561da99bdbd979142751de24bb1 with result from task 3296 parsl.dataflow.dflow DEBUG: Task 3296 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b69c37a3-6008-4b74-988f-dfb495a5f6f0_isrForCrosstalkSources_996986_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b69c37a3-6008-4b74-988f-dfb495a5f6f0_isrForCrosstalkSources_996986_10.stderr parsl.dataflow.dflow INFO: Task 3343 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ead4e5dc29c815a60cbf34278829dccb with result from task 3343 parsl.dataflow.dflow DEBUG: Task 3343 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e383a5ef-3842-4d82-ac03-66c5bb15e805_isrForCrosstalkSources_997018_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e383a5ef-3842-4d82-ac03-66c5bb15e805_isrForCrosstalkSources_997018_8.stderr parsl.dataflow.dflow INFO: Task 3335 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63e0364df08627eab65ae4bf994aa8a7 with result from task 3335 parsl.dataflow.dflow DEBUG: Task 3335 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/6e4e2002-5fb8-4fc9-a4e0-5cfe0461d5bc_isrForCrosstalkSources_997012_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/6e4e2002-5fb8-4fc9-a4e0-5cfe0461d5bc_isrForCrosstalkSources_997012_44.stderr parsl.dataflow.dflow INFO: Task 3344 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48a57850dea74c2e507475f997f1de43 with result from task 3344 parsl.dataflow.dflow DEBUG: Task 3344 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ec180685-56d5-4af3-8a98-fcd30059fc96_isrForCrosstalkSources_996976_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ec180685-56d5-4af3-8a98-fcd30059fc96_isrForCrosstalkSources_996976_48.stderr parsl.dataflow.dflow INFO: Task 3264 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd6d783f2f8a4fa98df3029a0ee911ab with result from task 3264 parsl.dataflow.dflow DEBUG: Task 3264 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/e7fa2deb-dc2b-40f0-a3ff-30cc8193ca32_isrForCrosstalkSources_996976_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/e7fa2deb-dc2b-40f0-a3ff-30cc8193ca32_isrForCrosstalkSources_996976_3.stderr parsl.dataflow.dflow INFO: Task 3284 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86b6a54ae86725f5b23d53201c3842c1 with result from task 3284 parsl.dataflow.dflow DEBUG: Task 3284 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7cbecc84-08d9-4ba7-8595-f45e402ff526_isrForCrosstalkSources_997029_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/7cbecc84-08d9-4ba7-8595-f45e402ff526_isrForCrosstalkSources_997029_9.stderr parsl.dataflow.dflow INFO: Task 3007 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4986fd3210b61596c7a19b23262ee766 with result from task 3007 parsl.dataflow.dflow DEBUG: Task 3007 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6171fc96-2df4-4c67-8438-48dd01f0e84d_isrForCrosstalkSources_996963_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6171fc96-2df4-4c67-8438-48dd01f0e84d_isrForCrosstalkSources_996963_46.stderr parsl.dataflow.dflow INFO: Task 3325 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 399f2241d387554b14a536d94346089c with result from task 3325 parsl.dataflow.dflow DEBUG: Task 3325 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1ec8a115-f790-445e-a5f4-a552942c30cd_isrForCrosstalkSources_996998_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1ec8a115-f790-445e-a5f4-a552942c30cd_isrForCrosstalkSources_996998_43.stderr parsl.dataflow.dflow INFO: Task 3141 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f16131a29fc0b5a3503f8a358377318 with result from task 3141 parsl.dataflow.dflow DEBUG: Task 3141 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/34a10a89-8452-4b47-ba7c-2e9ef67574d3_isrForCrosstalkSources_996993_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/34a10a89-8452-4b47-ba7c-2e9ef67574d3_isrForCrosstalkSources_996993_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:43 proc_lsst.multi:146 [INFO] found job 17074938 in provider escience_a40 proc_lsst.multi INFO: found job 17074938 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3320 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf06d857c48759d28337b2da9b8fbd06 with result from task 3320 parsl.dataflow.dflow DEBUG: Task 3320 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/aab54a7d-d7ad-40a9-a8e5-11616f74ce29_isrForCrosstalkSources_997000_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/aab54a7d-d7ad-40a9-a8e5-11616f74ce29_isrForCrosstalkSources_997000_21.stderr parsl.dataflow.dflow INFO: Task 3099 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19e0f9d712357ae1078f2ff5ba1cdbfc with result from task 3099 parsl.dataflow.dflow DEBUG: Task 3099 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7730d3ff-6a12-46dc-a079-841b49c896cc_isrForCrosstalkSources_996975_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7730d3ff-6a12-46dc-a079-841b49c896cc_isrForCrosstalkSources_996975_27.stderr parsl.dataflow.dflow INFO: Task 3205 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 546f776a7e1413b85204393e24b74a41 with result from task 3205 parsl.dataflow.dflow DEBUG: Task 3205 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/595c9d4b-c952-46de-865f-47916708198f_isrForCrosstalkSources_996972_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/595c9d4b-c952-46de-865f-47916708198f_isrForCrosstalkSources_996972_22.stderr parsl.dataflow.dflow INFO: Task 3223 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d74fbc6adda3bd961a610fbb85ffc96 with result from task 3223 parsl.dataflow.dflow DEBUG: Task 3223 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/42ec242c-8f73-4020-bdb7-8e0a087bcfee_isrForCrosstalkSources_997013_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/42ec242c-8f73-4020-bdb7-8e0a087bcfee_isrForCrosstalkSources_997013_26.stderr parsl.dataflow.dflow INFO: Task 3257 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd5bdd6dd3dc4b7efc26c009c729a7a7 with result from task 3257 parsl.dataflow.dflow DEBUG: Task 3257 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/abef7e08-46f3-4906-a3fb-7e35b692096d_isrForCrosstalkSources_996979_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/abef7e08-46f3-4906-a3fb-7e35b692096d_isrForCrosstalkSources_996979_4.stderr parsl.dataflow.dflow INFO: Task 3208 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 535956d6fd936a901eb3b0cef5e8b6cf with result from task 3208 parsl.dataflow.dflow DEBUG: Task 3208 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/cf294e22-f430-4ea0-9732-dce56a91db6b_isrForCrosstalkSources_997026_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/cf294e22-f430-4ea0-9732-dce56a91db6b_isrForCrosstalkSources_997026_6.stderr parsl.dataflow.dflow INFO: Task 3246 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 541831a4fe9d34430ac4ddd4f6ff7e7d with result from task 3246 parsl.dataflow.dflow DEBUG: Task 3246 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/aac934b1-5a9b-476a-9f0e-3876951a15fa_isrForCrosstalkSources_996977_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/aac934b1-5a9b-476a-9f0e-3876951a15fa_isrForCrosstalkSources_996977_19.stderr parsl.dataflow.dflow INFO: Task 3189 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa08bdd6e501d0d8128a176f385c5e22 with result from task 3189 parsl.dataflow.dflow DEBUG: Task 3189 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/97aa5fed-56be-43a2-a6bf-65697ab0b301_isrForCrosstalkSources_996979_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/97aa5fed-56be-43a2-a6bf-65697ab0b301_isrForCrosstalkSources_996979_25.stderr parsl.dataflow.dflow INFO: Task 3188 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4348d42e43a843b57f7735c7aab4e6a2 with result from task 3188 parsl.dataflow.dflow DEBUG: Task 3188 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d834e7d4-05f9-47fc-b2a0-6c7948c6101e_isrForCrosstalkSources_996987_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d834e7d4-05f9-47fc-b2a0-6c7948c6101e_isrForCrosstalkSources_996987_44.stderr parsl.dataflow.dflow INFO: Task 2687 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a683723fb646284b64285bf2331a50ac with result from task 2687 parsl.dataflow.dflow DEBUG: Task 2687 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/0a876077-3a81-48cf-9bd2-3258a5927d72_isrForCrosstalkSources_997029_62.stdout parsl.dataflow.dflow INFO: Standard error for task 2687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/0a876077-3a81-48cf-9bd2-3258a5927d72_isrForCrosstalkSources_997029_62.stderr parsl.dataflow.dflow INFO: Task 3331 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e3ffd83702a68a9d1220cf905af3825 with result from task 3331 parsl.dataflow.dflow DEBUG: Task 3331 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/84f229fe-e4c5-422f-b320-0391b8bb4248_isrForCrosstalkSources_996978_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/84f229fe-e4c5-422f-b320-0391b8bb4248_isrForCrosstalkSources_996978_27.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:44 proc_lsst.multi:146 [INFO] found job 17074939 in provider escience_a40 proc_lsst.multi INFO: found job 17074939 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3034 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c6b07c7126c9593520e6b5b26d2b7831 with result from task 3034 parsl.dataflow.dflow DEBUG: Task 3034 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/87e92dad-93e0-4dfe-85be-989408a65c17_isrForCrosstalkSources_996954_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/87e92dad-93e0-4dfe-85be-989408a65c17_isrForCrosstalkSources_996954_59.stderr parsl.dataflow.dflow INFO: Task 3107 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26bc1987cb97ce25ee77af592f31e363 with result from task 3107 parsl.dataflow.dflow DEBUG: Task 3107 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6ad526e-b6dd-450d-9bcc-d81d9214255b_isrForCrosstalkSources_997026_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6ad526e-b6dd-450d-9bcc-d81d9214255b_isrForCrosstalkSources_997026_31.stderr parsl.dataflow.dflow INFO: Task 3348 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a334b7b9ed495ad849a2ac366d22eb17 with result from task 3348 parsl.dataflow.dflow DEBUG: Task 3348 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/11fb70b6-883e-4790-b4d2-1327ef67980f_isrForCrosstalkSources_997000_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/11fb70b6-883e-4790-b4d2-1327ef67980f_isrForCrosstalkSources_997000_42.stderr parsl.dataflow.dflow INFO: Task 3248 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd83eb5ca8a1e909b127f7ab67361fc1 with result from task 3248 parsl.dataflow.dflow DEBUG: Task 3248 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/e86c4ec8-9086-4eba-b67b-a13a4d655b86_isrForCrosstalkSources_996974_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/e86c4ec8-9086-4eba-b67b-a13a4d655b86_isrForCrosstalkSources_996974_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 3322 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry cf197dee65b4333a58fb5cf5bbff688b with result from task 3322 parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:44 proc_lsst.multi:146 [INFO] found job 17074940 in provider escience_a40 proc_lsst.multi INFO: found job 17074940 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow DEBUG: Task 3322 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6d1aaa6e-6c88-4c9b-b290-1fb390b5eebb_isrForCrosstalkSources_996952_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6d1aaa6e-6c88-4c9b-b290-1fb390b5eebb_isrForCrosstalkSources_996952_23.stderr parsl.dataflow.dflow INFO: Task 3218 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad9b17e58731dd95d73c303c23881ea2 with result from task 3218 parsl.dataflow.dflow DEBUG: Task 3218 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4d38a619-42b3-49c8-b027-0b8c3f110290_isrForCrosstalkSources_997014_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4d38a619-42b3-49c8-b027-0b8c3f110290_isrForCrosstalkSources_997014_22.stderr parsl.dataflow.dflow INFO: Task 2951 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d24c3b18bc6c19030780fb56e62639a2 with result from task 2951 parsl.dataflow.dflow DEBUG: Task 2951 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/89f12b47-6ac2-4c31-abb2-6d837180e865_isrForCrosstalkSources_996985_52.stdout parsl.dataflow.dflow INFO: Standard error for task 2951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/89f12b47-6ac2-4c31-abb2-6d837180e865_isrForCrosstalkSources_996985_52.stderr parsl.dataflow.dflow INFO: Task 3172 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7ae3bd09a498dc07663abcc2673d90b with result from task 3172 parsl.dataflow.dflow DEBUG: Task 3172 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/a6e61a35-3c57-444f-bdbd-3260b70fad49_isrForCrosstalkSources_997023_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/a6e61a35-3c57-444f-bdbd-3260b70fad49_isrForCrosstalkSources_997023_14.stderr parsl.dataflow.dflow INFO: Task 3186 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75a3aec052d32ae817a8a131e8d6c63c with result from task 3186 parsl.dataflow.dflow DEBUG: Task 3186 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ab5e1184-05bb-4108-b197-1697526cc7c5_isrForCrosstalkSources_997009_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/ab5e1184-05bb-4108-b197-1697526cc7c5_isrForCrosstalkSources_997009_51.stderr parsl.dataflow.dflow INFO: Task 3333 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 264db0278f5896390e43db38781f1218 with result from task 3333 parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.dataflow.dflow DEBUG: Task 3333 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 3333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/1ca5f5d9-3a32-4e75-9fdf-7aedace0604c_isrForCrosstalkSources_997008_39.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:45 proc_lsst.multi:146 [INFO] found job 17074942 in provider escience_a40 proc_lsst.multi INFO: found job 17074942 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Standard error for task 3333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/1ca5f5d9-3a32-4e75-9fdf-7aedace0604c_isrForCrosstalkSources_997008_39.stderr parsl.dataflow.dflow INFO: Task 3014 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9540fb540ad382d123de1cac84a5e488 with result from task 3014 parsl.dataflow.dflow DEBUG: Task 3014 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3c71d742-8ab3-4bfa-a118-bb48e06c6bd3_isrForCrosstalkSources_996967_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3c71d742-8ab3-4bfa-a118-bb48e06c6bd3_isrForCrosstalkSources_996967_18.stderr parsl.dataflow.dflow INFO: Task 3191 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 24e2cf565d6915e11541a2c446be9210 with result from task 3191 parsl.dataflow.dflow DEBUG: Task 3191 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/693f49f3-8127-431d-a3c6-eb542140b73f_isrForCrosstalkSources_996952_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/693f49f3-8127-431d-a3c6-eb542140b73f_isrForCrosstalkSources_996952_8.stderr parsl.dataflow.dflow INFO: Task 3366 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 399375e2118e7c28b52e7e766cb50569 with result from task 3366 parsl.dataflow.dflow DEBUG: Task 3366 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7c531fb4-3fa4-4dec-8556-4b744814e57b_isrForCrosstalkSources_996953_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/7c531fb4-3fa4-4dec-8556-4b744814e57b_isrForCrosstalkSources_996953_26.stderr parsl.dataflow.dflow INFO: Task 3146 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e828027cb0c8bba16f9f88488c7b4ea9 with result from task 3146 parsl.dataflow.dflow DEBUG: Task 3146 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/7339dc9d-f267-40b8-899b-696ed2eb495e_isrForCrosstalkSources_996962_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/7339dc9d-f267-40b8-899b-696ed2eb495e_isrForCrosstalkSources_996962_47.stderr parsl.dataflow.dflow INFO: Task 3170 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0cc616ea6768c3f482b5f295325b73f4 with result from task 3170 parsl.dataflow.dflow DEBUG: Task 3170 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/e7ef8337-5c35-4fd2-a32e-89dfcc0fd117_isrForCrosstalkSources_996999_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/e7ef8337-5c35-4fd2-a32e-89dfcc0fd117_isrForCrosstalkSources_996999_43.stderr parsl.dataflow.dflow INFO: Task 3358 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0598d5533319f80f1f4822f0cb7a3dbf with result from task 3358 parsl.dataflow.dflow DEBUG: Task 3358 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4332c31e-211f-4b1b-8506-7090894ffbe8_isrForCrosstalkSources_996963_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/4332c31e-211f-4b1b-8506-7090894ffbe8_isrForCrosstalkSources_996963_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:45 proc_lsst.multi:146 [INFO] found job 17074974 in provider escience_a40 proc_lsst.multi INFO: found job 17074974 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3253 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75dea96d32364a4577ed3517e8476675 with result from task 3253 parsl.dataflow.dflow DEBUG: Task 3253 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ed44ad6e-db55-4b53-b962-bd48f489f6e4_isrForCrosstalkSources_996998_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ed44ad6e-db55-4b53-b962-bd48f489f6e4_isrForCrosstalkSources_996998_19.stderr parsl.dataflow.dflow INFO: Task 3251 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fe1c9cfd0bd42ea05008bde6247973b with result from task 3251 parsl.dataflow.dflow DEBUG: Task 3251 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a7381447-183c-4ada-bf9b-4dec0aedfaaf_isrForCrosstalkSources_996996_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a7381447-183c-4ada-bf9b-4dec0aedfaaf_isrForCrosstalkSources_996996_32.stderr parsl.dataflow.dflow INFO: Task 3096 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 497208e1a19bcab4a018e85c630eb77f with result from task 3096 parsl.dataflow.dflow DEBUG: Task 3096 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/368b6bb3-c27f-48c4-a00d-025c4b678794_isrForCrosstalkSources_996963_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/368b6bb3-c27f-48c4-a00d-025c4b678794_isrForCrosstalkSources_996963_18.stderr parsl.dataflow.dflow INFO: Task 3016 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df8f7e420b748cae4e52452e086c895c with result from task 3016 parsl.dataflow.dflow DEBUG: Task 3016 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d3aba8e9-9d9a-4c87-9c0a-0a87098b674c_isrForCrosstalkSources_996991_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d3aba8e9-9d9a-4c87-9c0a-0a87098b674c_isrForCrosstalkSources_996991_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:46 proc_lsst.multi:146 [INFO] found job 17075003 in provider escience_a40 proc_lsst.multi INFO: found job 17075003 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:47 proc_lsst.multi:146 [INFO] found job 17075018 in provider escience_a40 proc_lsst.multi INFO: found job 17075018 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3180 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4eedcf5ae37c75d5c78bb7e449c314b0 with result from task 3180 parsl.dataflow.dflow DEBUG: Task 3180 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/bf2508e1-7a2e-4228-94d8-18f2b9673ea3_isrForCrosstalkSources_997018_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/bf2508e1-7a2e-4228-94d8-18f2b9673ea3_isrForCrosstalkSources_997018_59.stderr parsl.dataflow.dflow INFO: Task 3163 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 148535e55fccb5a7c0c3ba732623c8a4 with result from task 3163 parsl.dataflow.dflow DEBUG: Task 3163 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/54cf8afc-3bbe-4801-ac45-d71d794e0f9f_isrForCrosstalkSources_996969_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/54cf8afc-3bbe-4801-ac45-d71d794e0f9f_isrForCrosstalkSources_996969_46.stderr parsl.dataflow.dflow INFO: Task 3329 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21aa4c4bdb7a07222ed4f5c6cfd02027 with result from task 3329 parsl.dataflow.dflow DEBUG: Task 3329 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/6aeebf76-d2ac-44c0-8d5a-41f13f452db8_isrForCrosstalkSources_996976_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/6aeebf76-d2ac-44c0-8d5a-41f13f452db8_isrForCrosstalkSources_996976_36.stderr parsl.dataflow.dflow INFO: Task 3376 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 542ae74c985359448c38f32491420067 with result from task 3376 parsl.dataflow.dflow DEBUG: Task 3376 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/6fffe458-1d95-4fc7-a443-5b8e555856ed_isrForCrosstalkSources_996987_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/6fffe458-1d95-4fc7-a443-5b8e555856ed_isrForCrosstalkSources_996987_35.stderr parsl.dataflow.dflow INFO: Task 3350 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e41c20e7e8e191f9d5b4b52f2ee39097 with result from task 3350 parsl.dataflow.dflow DEBUG: Task 3350 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/53a1f3c4-7dc1-4a69-91bc-3af16d6a0145_isrForCrosstalkSources_996961_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/53a1f3c4-7dc1-4a69-91bc-3af16d6a0145_isrForCrosstalkSources_996961_50.stderr parsl.dataflow.dflow INFO: Task 3297 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac6a8004c9c70237a5877947e4f2e2a7 with result from task 3297 parsl.dataflow.dflow DEBUG: Task 3297 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e46034a9-74c3-47f8-ba97-a69ce97512cf_isrForCrosstalkSources_996993_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e46034a9-74c3-47f8-ba97-a69ce97512cf_isrForCrosstalkSources_996993_10.stderr parsl.dataflow.dflow INFO: Task 3374 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58cdc5405e944f6a5d9743eccc4ad318 with result from task 3374 parsl.dataflow.dflow DEBUG: Task 3374 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2d5eb532-bf59-4db7-93c5-98a1b4efe775_isrForCrosstalkSources_996961_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/2d5eb532-bf59-4db7-93c5-98a1b4efe775_isrForCrosstalkSources_996961_56.stderr parsl.dataflow.dflow INFO: Task 3356 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83b53ff5f1210ebc89d6a1d413e7b586 with result from task 3356 parsl.dataflow.dflow DEBUG: Task 3356 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/7eaaa837-a482-4a75-a09f-afa2b54d00f6_isrForCrosstalkSources_997009_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/7eaaa837-a482-4a75-a09f-afa2b54d00f6_isrForCrosstalkSources_997009_3.stderr parsl.dataflow.dflow INFO: Task 3110 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26d3f4027769e0088069529f1c84d5ea with result from task 3110 parsl.dataflow.dflow DEBUG: Task 3110 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3a4adde7-5487-4cd9-8b26-2a4ff67a6749_isrForCrosstalkSources_996999_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/3a4adde7-5487-4cd9-8b26-2a4ff67a6749_isrForCrosstalkSources_996999_17.stderr parsl.dataflow.dflow INFO: Task 3130 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86152e23a80610e31dd2760e5af9d805 with result from task 3130 parsl.dataflow.dflow DEBUG: Task 3130 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9425560b-0a13-4543-bd34-b3c392d524a5_isrForCrosstalkSources_996995_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/9425560b-0a13-4543-bd34-b3c392d524a5_isrForCrosstalkSources_996995_55.stderr parsl.dataflow.dflow INFO: Task 3379 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25ff1f7bd1987dc0b377829081bb88d8 with result from task 3379 parsl.dataflow.dflow DEBUG: Task 3379 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/ea4aa46a-b108-4336-a6e0-9d0680b93b76_isrForCrosstalkSources_996971_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/ea4aa46a-b108-4336-a6e0-9d0680b93b76_isrForCrosstalkSources_996971_13.stderr parsl.dataflow.dflow INFO: Task 3353 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de4841c6c6f13cda9a839257d994e230 with result from task 3353 parsl.dataflow.dflow DEBUG: Task 3353 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/acbc46c5-1707-4bc1-abb3-6464df5df683_isrForCrosstalkSources_996954_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/acbc46c5-1707-4bc1-abb3-6464df5df683_isrForCrosstalkSources_996954_40.stderr parsl.dataflow.dflow INFO: Task 3380 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f0fe3f1fc5b2e13ad78307fc903c8f9 with result from task 3380 parsl.dataflow.dflow DEBUG: Task 3380 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/dfe09c84-79e6-4160-b47d-bb140c90ca2b_isrForCrosstalkSources_997004_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/dfe09c84-79e6-4160-b47d-bb140c90ca2b_isrForCrosstalkSources_997004_49.stderr parsl.dataflow.dflow INFO: Task 3179 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9e7bb2bb2437719015da1351adbe8c9 with result from task 3179 parsl.dataflow.dflow DEBUG: Task 3179 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/df55b068-9a67-47b4-a055-3f65ee831236_isrForCrosstalkSources_996988_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/df55b068-9a67-47b4-a055-3f65ee831236_isrForCrosstalkSources_996988_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:49 proc_lsst.multi:146 [INFO] found job 17075019 in provider escience_a40 proc_lsst.multi INFO: found job 17075019 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3243 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63c66e5464f916c58a99f0b0d8046a91 with result from task 3243 parsl.dataflow.dflow DEBUG: Task 3243 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/14e76306-cf6c-4d3e-bcc1-dfde3c4fe763_isrForCrosstalkSources_996990_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/14e76306-cf6c-4d3e-bcc1-dfde3c4fe763_isrForCrosstalkSources_996990_7.stderr parsl.dataflow.dflow INFO: Task 3406 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59d213eac960ebf72c57a7d1873c1ba5 with result from task 3406 parsl.dataflow.dflow DEBUG: Task 3406 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7e93d477-87e6-4556-88f8-e331ae5dab14_isrForCrosstalkSources_997008_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/7e93d477-87e6-4556-88f8-e331ae5dab14_isrForCrosstalkSources_997008_34.stderr parsl.dataflow.dflow INFO: Task 3402 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c33307f9ccf020662bb91ca0eaf4df46 with result from task 3402 parsl.dataflow.dflow DEBUG: Task 3402 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/5cd2723f-4d0a-4bdd-b90e-8e029c1bf6d4_isrForCrosstalkSources_997019_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/5cd2723f-4d0a-4bdd-b90e-8e029c1bf6d4_isrForCrosstalkSources_997019_19.stderr parsl.dataflow.dflow INFO: Task 3384 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5e1411c0a9a612bcc67ad104cf5509fb with result from task 3384 parsl.dataflow.dflow DEBUG: Task 3384 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a26340fa-4881-44e5-980c-d60129f32e29_isrForCrosstalkSources_996976_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/a26340fa-4881-44e5-980c-d60129f32e29_isrForCrosstalkSources_996976_34.stderr parsl.dataflow.dflow INFO: Task 3254 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 858d78dbbd2ff6412092ecf65ac0c2be with result from task 3254 parsl.dataflow.dflow DEBUG: Task 3254 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ac4aea4e-e619-4e2d-b5be-1a55ec42e026_isrForCrosstalkSources_997016_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ac4aea4e-e619-4e2d-b5be-1a55ec42e026_isrForCrosstalkSources_997016_8.stderr parsl.dataflow.dflow INFO: Task 3006 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b3f54311e3fa2c19d49c509e5efdccb with result from task 3006 parsl.dataflow.dflow DEBUG: Task 3006 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/7960fcf0-3d7f-413d-accf-09a1d981c18e_isrForCrosstalkSources_996955_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/7960fcf0-3d7f-413d-accf-09a1d981c18e_isrForCrosstalkSources_996955_36.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:49 proc_lsst.multi:146 [INFO] found job 17075020 in provider escience_a40 proc_lsst.multi INFO: found job 17075020 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow INFO: Task 3097 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c55708fcf9b972fbeeb9be3b49051fc with result from task 3097 parsl.dataflow.dflow DEBUG: Task 3097 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/279e6f80-5e82-4457-834c-72ef418c2b02_isrForCrosstalkSources_997020_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/279e6f80-5e82-4457-834c-72ef418c2b02_isrForCrosstalkSources_997020_9.stderr parsl.dataflow.dflow INFO: Task 3002 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a81825ac63cad0e50465addab5c72904 with result from task 3002 parsl.dataflow.dflow DEBUG: Task 3002 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/508c5c8b-5da9-4478-bde0-afa4c793a7ae_isrForCrosstalkSources_996962_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/508c5c8b-5da9-4478-bde0-afa4c793a7ae_isrForCrosstalkSources_996962_23.stderr parsl.dataflow.dflow INFO: Task 3399 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 015b77a7949b858709521a061384f279 with result from task 3399 parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 3399 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 3399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/53402ed0-e9a5-4701-8865-e90e3c6f9d9f_isrForCrosstalkSources_996989_35.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 3399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/53402ed0-e9a5-4701-8865-e90e3c6f9d9f_isrForCrosstalkSources_996989_35.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:50 proc_lsst.multi:146 [INFO] found job 17075021 in provider escience_a40 proc_lsst.multi INFO: found job 17075021 in provider escience_a40 parsl.dataflow.dflow INFO: Task 3160 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4831bdc06c1432376fb89b2ef54df015 with result from task 3160 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.dataflow.dflow DEBUG: Task 3160 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a90bb34-aab8-4624-8a8e-a7721ec89e84_isrForCrosstalkSources_997004_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a90bb34-aab8-4624-8a8e-a7721ec89e84_isrForCrosstalkSources_997004_18.stderr parsl.dataflow.dflow INFO: Task 3433 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07f39a87abfcd476a746a708f05653bb with result from task 3433 parsl.dataflow.dflow DEBUG: Task 3433 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/86363ce8-5b90-4701-a0c6-86c90b179e51_isrForCrosstalkSources_996979_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/86363ce8-5b90-4701-a0c6-86c90b179e51_isrForCrosstalkSources_996979_26.stderr parsl.dataflow.dflow INFO: Task 3058 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94953f46788d3c84ee60ca6c4e59c98c with result from task 3058 parsl.dataflow.dflow DEBUG: Task 3058 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e523d38a-79d3-4c59-b931-be2703aa69f8_isrForCrosstalkSources_997022_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e523d38a-79d3-4c59-b931-be2703aa69f8_isrForCrosstalkSources_997022_40.stderr parsl.dataflow.dflow INFO: Task 3060 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97be1d7c4067d57ad90cf8e1a0acf3a4 with result from task 3060 parsl.dataflow.dflow DEBUG: Task 3060 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/5f8e2515-8a34-493e-b3b4-4427fa6c02f7_isrForCrosstalkSources_997016_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/5f8e2515-8a34-493e-b3b4-4427fa6c02f7_isrForCrosstalkSources_997016_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:51 proc_lsst.multi:146 [INFO] found job 17075022 in provider checkpoint proc_lsst.multi INFO: found job 17075022 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3100 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa7207ec20147d8fe4904e4de6940449 with result from task 3100 parsl.dataflow.dflow DEBUG: Task 3100 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a0ffffbf-321b-4899-a4c4-2eba50ec08e0_isrForCrosstalkSources_997026_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a0ffffbf-321b-4899-a4c4-2eba50ec08e0_isrForCrosstalkSources_997026_2.stderr parsl.dataflow.dflow INFO: Task 3266 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bfeeacc7ae0290e3821760e16674fc43 with result from task 3266 parsl.dataflow.dflow DEBUG: Task 3266 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f156071d-ba4c-4b79-a24c-54d7ddf5f602_isrForCrosstalkSources_997016_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/f156071d-ba4c-4b79-a24c-54d7ddf5f602_isrForCrosstalkSources_997016_34.stderr parsl.dataflow.dflow INFO: Task 3234 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 22ef5b4a538b78ce20756470c0a9c7f0 with result from task 3234 parsl.dataflow.dflow DEBUG: Task 3234 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0e70d68d-4749-4c2b-9bd4-3de6e2bb6fd9_isrForCrosstalkSources_996953_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/0e70d68d-4749-4c2b-9bd4-3de6e2bb6fd9_isrForCrosstalkSources_996953_25.stderr parsl.dataflow.dflow INFO: Task 3436 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50e094efbb700e8d42aa5a82fff6868a with result from task 3436 parsl.dataflow.dflow DEBUG: Task 3436 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/6a0163d0-3b32-4028-be7b-4c7f049f5528_isrForCrosstalkSources_996977_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/6a0163d0-3b32-4028-be7b-4c7f049f5528_isrForCrosstalkSources_996977_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:52 proc_lsst.multi:146 [INFO] found job 17075030 in provider checkpoint proc_lsst.multi INFO: found job 17075030 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3111 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 764dee5722845ef8ec8c71de72431c85 with result from task 3111 parsl.dataflow.dflow DEBUG: Task 3111 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/4bd4510a-3d84-4a85-bd77-19425eb979f8_isrForCrosstalkSources_996979_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/4bd4510a-3d84-4a85-bd77-19425eb979f8_isrForCrosstalkSources_996979_37.stderr parsl.dataflow.dflow INFO: Task 3252 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa1cfa9a3b89e472f45c48c3e6f51b10 with result from task 3252 parsl.dataflow.dflow DEBUG: Task 3252 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/449ca920-a62e-4ede-a622-77395e28dab0_isrForCrosstalkSources_996995_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/449ca920-a62e-4ede-a622-77395e28dab0_isrForCrosstalkSources_996995_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 3435 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 64c1478f0eff796fd62cbdb5d27ea385 with result from task 3435 parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3435 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f67905c3-9db9-489e-8d45-01498aa81155_isrForCrosstalkSources_997028_32.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f67905c3-9db9-489e-8d45-01498aa81155_isrForCrosstalkSources_997028_32.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3407 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 6e5fe00faa5d8517f91f4746859605f2 with result from task 3407 parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:53 proc_lsst.multi:146 [INFO] found job 17075031 in provider checkpoint proc_lsst.multi INFO: found job 17075031 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3407 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/02749f72-e999-4198-b245-58753521c92c_isrForCrosstalkSources_996968_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/02749f72-e999-4198-b245-58753521c92c_isrForCrosstalkSources_996968_36.stderr parsl.dataflow.dflow INFO: Task 3112 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 951a0793ac0518620bc6ba6f10485116 with result from task 3112 parsl.dataflow.dflow DEBUG: Task 3112 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5dddb490-a647-468d-a66b-62a2421419b7_isrForCrosstalkSources_996999_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5dddb490-a647-468d-a66b-62a2421419b7_isrForCrosstalkSources_996999_47.stderr parsl.dataflow.dflow INFO: Task 3441 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d58d401c63024d8a6c088f9b0de47780 with result from task 3441 parsl.dataflow.dflow DEBUG: Task 3441 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1202e874-d9eb-41cd-b938-dbb875924832_isrForCrosstalkSources_996998_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1202e874-d9eb-41cd-b938-dbb875924832_isrForCrosstalkSources_996998_40.stderr parsl.dataflow.dflow INFO: Task 3295 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 369ee59c955255efee832bbf4c72e9bb with result from task 3295 parsl.dataflow.dflow DEBUG: Task 3295 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1d5ecacb-5e75-4bfb-927a-b1aa8d49f2d7_isrForCrosstalkSources_997014_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/1d5ecacb-5e75-4bfb-927a-b1aa8d49f2d7_isrForCrosstalkSources_997014_46.stderr parsl.dataflow.dflow INFO: Task 3426 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 40d5833d06bea9ca176217568eae1851 with result from task 3426 parsl.dataflow.dflow DEBUG: Task 3426 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/1b539583-ce73-4777-b5c9-336f44951a95_isrForCrosstalkSources_997010_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/1b539583-ce73-4777-b5c9-336f44951a95_isrForCrosstalkSources_997010_56.stderr parsl.dataflow.dflow INFO: Task 3383 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 342a373d6b4e3573c6f636f2861be76e with result from task 3383 parsl.dataflow.dflow DEBUG: Task 3383 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e02921c5-e531-46b5-a08c-e8a605289f98_isrForCrosstalkSources_997010_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e02921c5-e531-46b5-a08c-e8a605289f98_isrForCrosstalkSources_997010_59.stderr parsl.dataflow.dflow INFO: Task 3403 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b49c07c89e0211813309a7147ea04a6 with result from task 3403 parsl.dataflow.dflow DEBUG: Task 3403 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/d3007ea3-e31e-480a-951f-f30980c39554_isrForCrosstalkSources_996999_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/d3007ea3-e31e-480a-951f-f30980c39554_isrForCrosstalkSources_996999_44.stderr parsl.dataflow.dflow INFO: Task 3255 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2cc96e7645927b04e5caea642d7e569 with result from task 3255 parsl.dataflow.dflow DEBUG: Task 3255 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/17406ab4-da21-4b7d-a4fa-af0f2e9bf5ce_isrForCrosstalkSources_997010_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/17406ab4-da21-4b7d-a4fa-af0f2e9bf5ce_isrForCrosstalkSources_997010_45.stderr parsl.dataflow.dflow INFO: Task 3411 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 161d519ac909075758d194946708317e with result from task 3411 parsl.dataflow.dflow DEBUG: Task 3411 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b7bdce11-d3a6-47bc-a18a-7be65d4e17ee_isrForCrosstalkSources_997022_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b7bdce11-d3a6-47bc-a18a-7be65d4e17ee_isrForCrosstalkSources_997022_31.stderr parsl.dataflow.dflow INFO: Task 3286 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 550ff7cc4125c83ecdc2e431f0c545b3 with result from task 3286 parsl.dataflow.dflow DEBUG: Task 3286 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/13b33dd8-d3a2-4edb-aa61-4519d6ad5ba2_isrForCrosstalkSources_996999_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/13b33dd8-d3a2-4edb-aa61-4519d6ad5ba2_isrForCrosstalkSources_996999_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3079 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 4af89d52232ba630520afc79fe3495df with result from task 3079 parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3079 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b7abc7c7-c418-4f58-874f-4a66dc54d27a_isrForCrosstalkSources_996999_26.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b7abc7c7-c418-4f58-874f-4a66dc54d27a_isrForCrosstalkSources_996999_26.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3137 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry c0ed36c1b391fa89d532d99d9ce8d15a with result from task 3137 parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3137 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d4353d73-e580-4be7-9b3e-ddd33147ef95_isrForCrosstalkSources_996995_25.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/d4353d73-e580-4be7-9b3e-ddd33147ef95_isrForCrosstalkSources_996995_25.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3276 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 4e689b9362e8a02da874d6576d4203aa with result from task 3276 parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3276 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/827e5923-9b3b-4865-aa46-355fc5f10efa_isrForCrosstalkSources_996958_45.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/827e5923-9b3b-4865-aa46-355fc5f10efa_isrForCrosstalkSources_996958_45.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3178 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 71bafc59c464a804466aa61b1381cd0c with result from task 3178 parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3178 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:55 proc_lsst.multi:146 [INFO] found job 17075032 in provider checkpoint proc_lsst.multi INFO: found job 17075032 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Standard output for task 3178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2b97f651-1d5e-4ef9-885f-30a95de95838_isrForCrosstalkSources_996974_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2b97f651-1d5e-4ef9-885f-30a95de95838_isrForCrosstalkSources_996974_26.stderr parsl.dataflow.dflow INFO: Task 3394 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 233a2196436e9fb2d2f4db56e8f2f36c with result from task 3394 parsl.dataflow.dflow DEBUG: Task 3394 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/608055f6-bd7f-4990-ba87-60a29de2890e_isrForCrosstalkSources_996952_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/608055f6-bd7f-4990-ba87-60a29de2890e_isrForCrosstalkSources_996952_5.stderr parsl.dataflow.dflow INFO: Task 3429 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51d637d11a18fd564792db983f091922 with result from task 3429 parsl.dataflow.dflow DEBUG: Task 3429 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/9a0f1848-334d-4771-9496-65b91e251aad_isrForCrosstalkSources_996956_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/9a0f1848-334d-4771-9496-65b91e251aad_isrForCrosstalkSources_996956_44.stderr parsl.dataflow.dflow INFO: Task 3437 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 026f90986f40e49e14b789c9dec943a2 with result from task 3437 parsl.dataflow.dflow DEBUG: Task 3437 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5bd51ae6-7e5f-4537-8c5f-31cf7ca52cb7_isrForCrosstalkSources_996999_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5bd51ae6-7e5f-4537-8c5f-31cf7ca52cb7_isrForCrosstalkSources_996999_24.stderr parsl.dataflow.dflow INFO: Task 3420 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7734830d06f5ea507667cb51650713d with result from task 3420 parsl.dataflow.dflow DEBUG: Task 3420 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dcbc0ec3-e999-4901-bc93-fa91d14f8df6_isrForCrosstalkSources_996971_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dcbc0ec3-e999-4901-bc93-fa91d14f8df6_isrForCrosstalkSources_996971_5.stderr parsl.dataflow.dflow INFO: Task 3020 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 679cf53c243e3cb94539bf9c066a2eb8 with result from task 3020 parsl.dataflow.dflow DEBUG: Task 3020 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/bcf42695-00e0-4eb6-986c-f0230c73d5f2_isrForCrosstalkSources_996979_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/bcf42695-00e0-4eb6-986c-f0230c73d5f2_isrForCrosstalkSources_996979_44.stderr parsl.dataflow.dflow INFO: Task 3244 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38a02fab8a83c4fe5500e7cb7edd4b0a with result from task 3244 parsl.dataflow.dflow DEBUG: Task 3244 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c79c2b08-71c5-4e0b-af45-6583d718e0db_isrForCrosstalkSources_996993_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/c79c2b08-71c5-4e0b-af45-6583d718e0db_isrForCrosstalkSources_996993_34.stderr parsl.dataflow.dflow INFO: Task 3272 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8730b2182725b4df375f518847b02778 with result from task 3272 parsl.dataflow.dflow DEBUG: Task 3272 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f5d5213f-81bc-4776-9cbb-b7a3e5569636_isrForCrosstalkSources_997024_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f5d5213f-81bc-4776-9cbb-b7a3e5569636_isrForCrosstalkSources_997024_56.stderr parsl.dataflow.dflow INFO: Task 3148 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02273af2a1732ce70cc43b730ee0999f with result from task 3148 parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:56 proc_lsst.multi:146 [INFO] found job 17075033 in provider checkpoint proc_lsst.multi INFO: found job 17075033 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3148 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9fddd4e6-59d1-49cf-9a67-e96517e280a8_isrForCrosstalkSources_996979_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/9fddd4e6-59d1-49cf-9a67-e96517e280a8_isrForCrosstalkSources_996979_29.stderr parsl.dataflow.dflow INFO: Task 3439 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c88e1c055c24db6dc76e91d3f1d102d with result from task 3439 parsl.dataflow.dflow DEBUG: Task 3439 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/a32a6610-1f28-4c98-a5b6-4ff768fe71a6_isrForCrosstalkSources_996969_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/a32a6610-1f28-4c98-a5b6-4ff768fe71a6_isrForCrosstalkSources_996969_43.stderr parsl.dataflow.dflow INFO: Task 3042 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 67b3a3f6bd0e138872a776709561f725 with result from task 3042 parsl.dataflow.dflow DEBUG: Task 3042 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/cb55b33a-6658-4147-9939-6a6eb093c1e3_isrForCrosstalkSources_997000_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/cb55b33a-6658-4147-9939-6a6eb093c1e3_isrForCrosstalkSources_997000_49.stderr parsl.dataflow.dflow INFO: Task 3260 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6fc7c59a615e550c61d42216152c6f0 with result from task 3260 parsl.dataflow.dflow DEBUG: Task 3260 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/18c76de1-d6e0-4c43-a4fa-392a5167efbb_isrForCrosstalkSources_996987_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/18c76de1-d6e0-4c43-a4fa-392a5167efbb_isrForCrosstalkSources_996987_52.stderr parsl.dataflow.dflow INFO: Task 3193 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e275cdba30e928cc44dd043623af20fd with result from task 3193 parsl.dataflow.dflow DEBUG: Task 3193 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4e0424d2-6906-4ebb-b126-15afc314da59_isrForCrosstalkSources_996986_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/4e0424d2-6906-4ebb-b126-15afc314da59_isrForCrosstalkSources_996986_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3453 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry bc5069a38deb9ce53631959da953493b with result from task 3453 parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3453 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/873ff00d-98ef-4c0c-b3a0-b90de102f67d_isrForCrosstalkSources_997019_33.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/873ff00d-98ef-4c0c-b3a0-b90de102f67d_isrForCrosstalkSources_997019_33.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3236 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 905fceff28397cb45aeab80ab95046e3 with result from task 3236 parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3236 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/368ecd73-a0e6-4b7a-8f48-bedd1b787564_isrForCrosstalkSources_997022_38.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/368ecd73-a0e6-4b7a-8f48-bedd1b787564_isrForCrosstalkSources_997022_38.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:57 proc_lsst.multi:146 [INFO] found job 17075034 in provider checkpoint proc_lsst.multi INFO: found job 17075034 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3404 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1110ce86c38861054e947ecd71141b50 with result from task 3404 parsl.dataflow.dflow DEBUG: Task 3404 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/88693045-6570-4c5f-8371-a00c0025f2c0_isrForCrosstalkSources_996973_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/88693045-6570-4c5f-8371-a00c0025f2c0_isrForCrosstalkSources_996973_53.stderr parsl.dataflow.dflow INFO: Task 3159 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eaf53363b23609f8475312c8ddcbe149 with result from task 3159 parsl.dataflow.dflow DEBUG: Task 3159 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/85034533-19fb-49e3-b22d-2037d5f40064_isrForCrosstalkSources_996952_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/85034533-19fb-49e3-b22d-2037d5f40064_isrForCrosstalkSources_996952_50.stderr parsl.dataflow.dflow INFO: Task 3287 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a301f1c1eb7db261ec8c70d83008118 with result from task 3287 parsl.dataflow.dflow DEBUG: Task 3287 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ab728d8c-98c9-4650-9e37-a2e9ddcd9fd6_isrForCrosstalkSources_996976_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ab728d8c-98c9-4650-9e37-a2e9ddcd9fd6_isrForCrosstalkSources_996976_37.stderr parsl.dataflow.dflow INFO: Task 3415 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53c9b1a68429f968ec4e51fc8d146760 with result from task 3415 parsl.dataflow.dflow DEBUG: Task 3415 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4a1ded38-d194-41e5-be36-3ef3e09f46e9_isrForCrosstalkSources_996999_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4a1ded38-d194-41e5-be36-3ef3e09f46e9_isrForCrosstalkSources_996999_48.stderr parsl.dataflow.dflow INFO: Task 3443 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5203972c3411e38cd6c1a3b5a21888b with result from task 3443 parsl.dataflow.dflow DEBUG: Task 3443 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/917e82ff-5aac-46dd-9bcd-2156ab971f9c_isrForCrosstalkSources_996983_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/917e82ff-5aac-46dd-9bcd-2156ab971f9c_isrForCrosstalkSources_996983_8.stderr parsl.dataflow.dflow INFO: Task 3175 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd16ed989bcda8511717918efd3d796a with result from task 3175 parsl.dataflow.dflow DEBUG: Task 3175 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0755dc5c-1f8b-4114-ae24-aa5505bbc756_isrForCrosstalkSources_996981_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/0755dc5c-1f8b-4114-ae24-aa5505bbc756_isrForCrosstalkSources_996981_59.stderr parsl.dataflow.dflow INFO: Task 3458 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bca0ed57bc2575720943ca341c6bac34 with result from task 3458 parsl.dataflow.dflow DEBUG: Task 3458 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1b6eea7d-722b-497a-8b8a-719466968b60_isrForCrosstalkSources_997022_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/1b6eea7d-722b-497a-8b8a-719466968b60_isrForCrosstalkSources_997022_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:58 proc_lsst.multi:146 [INFO] found job 17075035 in provider checkpoint proc_lsst.multi INFO: found job 17075035 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3153 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32772c53f8c086769e3d37a892c8f45f with result from task 3153 parsl.dataflow.dflow DEBUG: Task 3153 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/d7bb80ab-9dd9-409f-8678-05819181ca3b_isrForCrosstalkSources_997007_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/d7bb80ab-9dd9-409f-8678-05819181ca3b_isrForCrosstalkSources_997007_62.stderr parsl.dataflow.dflow INFO: Task 3089 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cba951673e6891a41c795e0bc77f3de5 with result from task 3089 parsl.dataflow.dflow DEBUG: Task 3089 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/66e037d2-fd7b-4001-bc19-5bd3b90bcd75_isrForCrosstalkSources_996975_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/66e037d2-fd7b-4001-bc19-5bd3b90bcd75_isrForCrosstalkSources_996975_16.stderr parsl.dataflow.dflow INFO: Task 3477 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f81906bc0477fdc3920ad60fe0350c6 with result from task 3477 parsl.dataflow.dflow DEBUG: Task 3477 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/84d9a0a0-9010-4c05-85e0-777bda43f74f_isrForCrosstalkSources_997001_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/84d9a0a0-9010-4c05-85e0-777bda43f74f_isrForCrosstalkSources_997001_26.stderr parsl.dataflow.dflow INFO: Task 3150 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9404d7809289e208029a76d31fadb8aa with result from task 3150 parsl.dataflow.dflow DEBUG: Task 3150 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/aa111026-4922-4292-b946-8e5048ec3a96_isrForCrosstalkSources_996970_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/aa111026-4922-4292-b946-8e5048ec3a96_isrForCrosstalkSources_996970_62.stderr parsl.dataflow.dflow INFO: Task 3456 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 524b9a6c4c4a01ce1a9eed48522c8628 with result from task 3456 parsl.dataflow.dflow DEBUG: Task 3456 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/848193c5-f066-4bd7-8d90-7c4c69387839_isrForCrosstalkSources_997002_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/848193c5-f066-4bd7-8d90-7c4c69387839_isrForCrosstalkSources_997002_36.stderr parsl.dataflow.dflow INFO: Task 3302 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2aeba134118602fd5414ee0811ea16a5 with result from task 3302 parsl.dataflow.dflow DEBUG: Task 3302 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/0e21ecd1-c0a3-4de0-af5c-9a3a3bd9e3f8_isrForCrosstalkSources_997029_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/0e21ecd1-c0a3-4de0-af5c-9a3a3bd9e3f8_isrForCrosstalkSources_997029_49.stderr parsl.dataflow.dflow INFO: Task 3423 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b59b1cf2fed0fb6f62da164450f023d4 with result from task 3423 parsl.dataflow.dflow DEBUG: Task 3423 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/15a35360-2e8c-4cca-acf6-367c1e78efee_isrForCrosstalkSources_996997_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/15a35360-2e8c-4cca-acf6-367c1e78efee_isrForCrosstalkSources_996997_47.stderr parsl.dataflow.dflow INFO: Task 3237 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c91bad7b6b00b90832cb9724a5b0c5ec with result from task 3237 parsl.dataflow.dflow DEBUG: Task 3237 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/dc447bb4-5b0b-48a2-9573-6b48c97e0bff_isrForCrosstalkSources_996959_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/dc447bb4-5b0b-48a2-9573-6b48c97e0bff_isrForCrosstalkSources_996959_52.stderr parsl.dataflow.dflow INFO: Task 3427 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7abedd7395c24718eee71f7430434f6c with result from task 3427 parsl.dataflow.dflow DEBUG: Task 3427 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/bc168d8d-20b4-40c8-bdb3-755f48c44fe8_isrForCrosstalkSources_996991_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/bc168d8d-20b4-40c8-bdb3-755f48c44fe8_isrForCrosstalkSources_996991_59.stderr parsl.dataflow.dflow INFO: Task 3185 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 28e4d2536685e96b676b62d09567785f with result from task 3185 parsl.dataflow.dflow DEBUG: Task 3185 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/33dc09a7-321f-4e7f-b6fb-f042b3bfdc25_isrForCrosstalkSources_997026_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/33dc09a7-321f-4e7f-b6fb-f042b3bfdc25_isrForCrosstalkSources_997026_62.stderr parsl.dataflow.dflow INFO: Task 3164 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4011d81fa105c3b64fa90dde2e36bcc9 with result from task 3164 parsl.dataflow.dflow DEBUG: Task 3164 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c9a449e0-666a-4457-ade1-3e522b8face0_isrForCrosstalkSources_996959_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c9a449e0-666a-4457-ade1-3e522b8face0_isrForCrosstalkSources_996959_50.stderr parsl.dataflow.dflow INFO: Task 3203 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b6afc47a2854cbf4ca130b91f9a95c4 with result from task 3203 parsl.dataflow.dflow DEBUG: Task 3203 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a109e3e1-d97f-4b12-816a-c1910c4f77ce_isrForCrosstalkSources_997026_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a109e3e1-d97f-4b12-816a-c1910c4f77ce_isrForCrosstalkSources_997026_10.stderr parsl.dataflow.dflow INFO: Task 3273 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae31d5438f4746dbce3c85bfc895503b with result from task 3273 parsl.dataflow.dflow DEBUG: Task 3273 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/ff4bfee7-7e82-432d-996d-131f8f60b886_isrForCrosstalkSources_997027_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/ff4bfee7-7e82-432d-996d-131f8f60b886_isrForCrosstalkSources_997027_57.stderr parsl.dataflow.dflow INFO: Task 3105 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6cfcf14356d35aa284df62dd8c74a70c with result from task 3105 parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:57:59 proc_lsst.multi:146 [INFO] found job 17075038 in provider checkpoint proc_lsst.multi INFO: found job 17075038 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3105 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/4c0751d9-8b30-4ee5-998b-12cda46dd6ad_isrForCrosstalkSources_997022_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/4c0751d9-8b30-4ee5-998b-12cda46dd6ad_isrForCrosstalkSources_997022_55.stderr parsl.dataflow.dflow INFO: Task 3469 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04a807705ffafa45c10bfa2794f8e506 with result from task 3469 parsl.dataflow.dflow DEBUG: Task 3469 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/af3f0835-3c40-4240-9791-fb488bd0454c_isrForCrosstalkSources_997020_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/af3f0835-3c40-4240-9791-fb488bd0454c_isrForCrosstalkSources_997020_36.stderr parsl.dataflow.dflow INFO: Task 3108 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15ef9771c71d493721d4b68e52fc311d with result from task 3108 parsl.dataflow.dflow DEBUG: Task 3108 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/b395012e-69da-4242-b61b-94c42509bbcd_isrForCrosstalkSources_997002_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/b395012e-69da-4242-b61b-94c42509bbcd_isrForCrosstalkSources_997002_44.stderr parsl.dataflow.dflow INFO: Task 3466 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00ddfd865aeb23f41a0e3b0355aa7244 with result from task 3466 parsl.dataflow.dflow DEBUG: Task 3466 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/6ba13cb1-ca2c-47cd-8d20-2a5ca7b798e9_isrForCrosstalkSources_996969_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/6ba13cb1-ca2c-47cd-8d20-2a5ca7b798e9_isrForCrosstalkSources_996969_15.stderr parsl.dataflow.dflow INFO: Task 3480 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16e5debaa2eb9de209b818a04ee94510 with result from task 3480 parsl.dataflow.dflow DEBUG: Task 3480 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/ad709dad-6e62-4876-be3f-c089383593cf_isrForCrosstalkSources_996957_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/ad709dad-6e62-4876-be3f-c089383593cf_isrForCrosstalkSources_996957_18.stderr parsl.dataflow.dflow INFO: Task 3190 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a63b388ed538c4dfca18e5bf15450c1 with result from task 3190 parsl.dataflow.dflow DEBUG: Task 3190 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c026c71b-061a-4bfe-8d8b-73732be2701c_isrForCrosstalkSources_996980_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c026c71b-061a-4bfe-8d8b-73732be2701c_isrForCrosstalkSources_996980_61.stderr parsl.dataflow.dflow INFO: Task 3118 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf2556ccdd73aa5fbdee97679c2fa7d7 with result from task 3118 parsl.dataflow.dflow DEBUG: Task 3118 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c3a7ede0-813a-4a97-882e-ae92257cf998_isrForCrosstalkSources_996997_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c3a7ede0-813a-4a97-882e-ae92257cf998_isrForCrosstalkSources_996997_50.stderr parsl.dataflow.dflow INFO: Task 3258 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 657fbebe513a188bff6e8d009ecfe893 with result from task 3258 parsl.dataflow.dflow DEBUG: Task 3258 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c981a80c-1090-4a38-941b-42787fcdffdc_isrForCrosstalkSources_996965_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c981a80c-1090-4a38-941b-42787fcdffdc_isrForCrosstalkSources_996965_10.stderr parsl.dataflow.dflow INFO: Task 3155 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aab8cdb9902c3a57eefe4b4ab30eab4c with result from task 3155 parsl.dataflow.dflow DEBUG: Task 3155 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/c0e852b8-8146-4e5f-b967-8b8c45fd5c53_isrForCrosstalkSources_996979_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/c0e852b8-8146-4e5f-b967-8b8c45fd5c53_isrForCrosstalkSources_996979_16.stderr parsl.dataflow.dflow INFO: Task 3086 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c97f0a192d9b17c8a88e093f17309551 with result from task 3086 parsl.dataflow.dflow DEBUG: Task 3086 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/f921eb32-bafd-4bf8-9e1a-408f7294ed8c_isrForCrosstalkSources_997014_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/f921eb32-bafd-4bf8-9e1a-408f7294ed8c_isrForCrosstalkSources_997014_49.stderr parsl.dataflow.dflow INFO: Task 3166 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e941e3c5a870e55f88111dc9af94500b with result from task 3166 parsl.dataflow.dflow DEBUG: Task 3166 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/325e4e20-e9fe-43a5-a936-ee2fd509b29c_isrForCrosstalkSources_997033_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/325e4e20-e9fe-43a5-a936-ee2fd509b29c_isrForCrosstalkSources_997033_41.stderr parsl.dataflow.dflow INFO: Task 3201 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a99b146e457aade6bcd753730eb4bb4 with result from task 3201 parsl.dataflow.dflow DEBUG: Task 3201 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e0281d8e-1939-4c9a-be15-1699da61eb74_isrForCrosstalkSources_997022_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e0281d8e-1939-4c9a-be15-1699da61eb74_isrForCrosstalkSources_997022_56.stderr parsl.dataflow.dflow INFO: Task 3463 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 09371ce5e028a3fc38db0a6574e998af with result from task 3463 parsl.dataflow.dflow DEBUG: Task 3463 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d9f51d59-61f0-4f15-bc0f-5cb8624a5e6c_isrForCrosstalkSources_996990_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/d9f51d59-61f0-4f15-bc0f-5cb8624a5e6c_isrForCrosstalkSources_996990_38.stderr parsl.dataflow.dflow INFO: Task 3230 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 240c4556fff2a80af4b94083082b8d29 with result from task 3230 parsl.dataflow.dflow DEBUG: Task 3230 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/1a6db936-ace2-43ce-a445-7a4aba41b2aa_isrForCrosstalkSources_996986_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/1a6db936-ace2-43ce-a445-7a4aba41b2aa_isrForCrosstalkSources_996986_56.stderr parsl.dataflow.dflow INFO: Task 3229 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8840aa387c799cbbc09ef6e0d43f5808 with result from task 3229 parsl.dataflow.dflow DEBUG: Task 3229 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e8aa1511-1b91-4821-9ac8-384037c20fd9_isrForCrosstalkSources_996957_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e8aa1511-1b91-4821-9ac8-384037c20fd9_isrForCrosstalkSources_996957_15.stderr parsl.dataflow.dflow INFO: Task 3298 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f749086426f233d295d0d90e7c13c838 with result from task 3298 parsl.dataflow.dflow DEBUG: Task 3298 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/2e8008a3-8764-4d10-ade6-bb0189c76a41_isrForCrosstalkSources_996953_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/2e8008a3-8764-4d10-ade6-bb0189c76a41_isrForCrosstalkSources_996953_31.stderr parsl.dataflow.dflow INFO: Task 3490 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 49457e87eb797cc13b1bded4a11f4de0 with result from task 3490 parsl.dataflow.dflow DEBUG: Task 3490 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/e69c9811-51a5-4152-9e7c-83c9de0dcd29_isrForCrosstalkSources_997017_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/e69c9811-51a5-4152-9e7c-83c9de0dcd29_isrForCrosstalkSources_997017_39.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:01 proc_lsst.multi:146 [INFO] found job 17075041 in provider checkpoint proc_lsst.multi INFO: found job 17075041 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3082 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b3f5efe427bf53ebb2c60a4a02ff87fa with result from task 3082 parsl.dataflow.dflow DEBUG: Task 3082 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/60c2bfe6-bdf4-4fe1-b679-a87204a74e51_isrForCrosstalkSources_996959_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/60c2bfe6-bdf4-4fe1-b679-a87204a74e51_isrForCrosstalkSources_996959_36.stderr parsl.dataflow.dflow INFO: Task 3504 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2583e4540cbae205b456d0b977de0287 with result from task 3504 parsl.dataflow.dflow DEBUG: Task 3504 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/199d9928-26d4-41e9-b282-c1b89353c8e1_isrForCrosstalkSources_997027_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/199d9928-26d4-41e9-b282-c1b89353c8e1_isrForCrosstalkSources_997027_8.stderr parsl.dataflow.dflow INFO: Task 3144 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d58189355bb3be76db2198b863f643e7 with result from task 3144 parsl.dataflow.dflow DEBUG: Task 3144 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/df32e980-cdd0-44c8-a8ee-4c6485f8a8b8_isrForCrosstalkSources_997014_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/df32e980-cdd0-44c8-a8ee-4c6485f8a8b8_isrForCrosstalkSources_997014_47.stderr parsl.dataflow.dflow INFO: Task 3484 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b544286fb9c63adb5ee47518ea71e512 with result from task 3484 parsl.dataflow.dflow DEBUG: Task 3484 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/f9611eaa-e670-4e84-92d1-eb0c1b70e372_isrForCrosstalkSources_997027_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/f9611eaa-e670-4e84-92d1-eb0c1b70e372_isrForCrosstalkSources_997027_38.stderr parsl.dataflow.dflow INFO: Task 3478 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cae9d0b270465027ef228120aa4700dc with result from task 3478 parsl.dataflow.dflow DEBUG: Task 3478 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/404cb793-1dd5-46c8-a428-a817a6b3a14e_isrForCrosstalkSources_997024_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/404cb793-1dd5-46c8-a428-a817a6b3a14e_isrForCrosstalkSources_997024_34.stderr parsl.dataflow.dflow INFO: Task 3131 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3df1d4915aca9144deb17244d2d15314 with result from task 3131 parsl.dataflow.dflow DEBUG: Task 3131 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/1c409f64-86ba-49ae-807f-9e9c87630170_isrForCrosstalkSources_996972_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/1c409f64-86ba-49ae-807f-9e9c87630170_isrForCrosstalkSources_996972_6.stderr parsl.dataflow.dflow INFO: Task 3140 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d56d4714f8f8bc18bf5f5d6a66328838 with result from task 3140 parsl.dataflow.dflow DEBUG: Task 3140 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/99c3227f-de5c-4527-b325-734793fea5b2_isrForCrosstalkSources_997003_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/99c3227f-de5c-4527-b325-734793fea5b2_isrForCrosstalkSources_997003_55.stderr parsl.dataflow.dflow INFO: Task 3214 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac8c2a18f52a0cdee322405ca4813c5a with result from task 3214 parsl.dataflow.dflow DEBUG: Task 3214 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/25bf0146-fcb1-49ab-8cbf-5db33681836f_isrForCrosstalkSources_997023_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/25bf0146-fcb1-49ab-8cbf-5db33681836f_isrForCrosstalkSources_997023_50.stderr parsl.dataflow.dflow INFO: Task 3084 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2082acaa1a69f63151b40c1e75f247ad with result from task 3084 parsl.dataflow.dflow DEBUG: Task 3084 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/5bd2a0f1-f73b-40a5-ace0-41896dde4848_isrForCrosstalkSources_996952_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/5bd2a0f1-f73b-40a5-ace0-41896dde4848_isrForCrosstalkSources_996952_3.stderr parsl.dataflow.dflow INFO: Task 3486 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3408eaa7f3474c15c760f380f47453ce with result from task 3486 parsl.dataflow.dflow DEBUG: Task 3486 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3d47a5b2-8a96-4c84-8099-47d0831cf3af_isrForCrosstalkSources_996995_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3d47a5b2-8a96-4c84-8099-47d0831cf3af_isrForCrosstalkSources_996995_43.stderr parsl.dataflow.dflow INFO: Task 3485 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7bbe8d1a4d5e7d4cb100e81b5dc50afe with result from task 3485 parsl.dataflow.dflow DEBUG: Task 3485 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/f8a4e0e9-4fbf-4f91-8107-b72b90405b0d_isrForCrosstalkSources_997014_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/f8a4e0e9-4fbf-4f91-8107-b72b90405b0d_isrForCrosstalkSources_997014_27.stderr parsl.dataflow.dflow INFO: Task 3154 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1323c9c5b405432cf9ea3a6144a6f749 with result from task 3154 parsl.dataflow.dflow DEBUG: Task 3154 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e7811ed1-e5d6-48aa-9adb-45f5486b89ef_isrForCrosstalkSources_996965_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e7811ed1-e5d6-48aa-9adb-45f5486b89ef_isrForCrosstalkSources_996965_36.stderr parsl.dataflow.dflow INFO: Task 3224 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 429cd8521d54b161ee3c3530ffe847e7 with result from task 3224 parsl.dataflow.dflow DEBUG: Task 3224 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4c57b535-8b03-42ec-8e3d-5fa17950b991_isrForCrosstalkSources_996958_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/4c57b535-8b03-42ec-8e3d-5fa17950b991_isrForCrosstalkSources_996958_1.stderr parsl.dataflow.dflow INFO: Task 3197 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f17be8c97e434fc9c84142ee53db97e with result from task 3197 parsl.dataflow.dflow DEBUG: Task 3197 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/ab35c226-fd67-4578-a157-50aa03d3a46d_isrForCrosstalkSources_997033_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/ab35c226-fd67-4578-a157-50aa03d3a46d_isrForCrosstalkSources_997033_37.stderr parsl.dataflow.dflow INFO: Task 3476 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31b055863ad2b3601dc3e3dd96b715de with result from task 3476 parsl.dataflow.dflow DEBUG: Task 3476 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/afde6ba9-853a-48b6-9771-7b05b00611b6_isrForCrosstalkSources_996964_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/afde6ba9-853a-48b6-9771-7b05b00611b6_isrForCrosstalkSources_996964_18.stderr parsl.dataflow.dflow INFO: Task 3256 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54ccbe542c4a90af27d170b557972507 with result from task 3256 parsl.dataflow.dflow DEBUG: Task 3256 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7c90b4ad-e86b-4d13-bee4-dd7f9d24e903_isrForCrosstalkSources_997024_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/7c90b4ad-e86b-4d13-bee4-dd7f9d24e903_isrForCrosstalkSources_997024_57.stderr parsl.dataflow.dflow INFO: Task 3091 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30c7176750d3955a681957a622d4e6a0 with result from task 3091 parsl.dataflow.dflow DEBUG: Task 3091 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/48df183e-0a3b-4eec-a849-947c545c5628_isrForCrosstalkSources_996956_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/48df183e-0a3b-4eec-a849-947c545c5628_isrForCrosstalkSources_996956_12.stderr parsl.dataflow.dflow INFO: Task 3508 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91104064c2baf295b424405a0daeff35 with result from task 3508 parsl.dataflow.dflow DEBUG: Task 3508 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/43467481-0a11-41a3-8f8d-5cc9d8e0cb32_isrForCrosstalkSources_997012_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/43467481-0a11-41a3-8f8d-5cc9d8e0cb32_isrForCrosstalkSources_997012_32.stderr parsl.dataflow.dflow INFO: Task 3489 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0790b75ad387735d4cf96cd4680d7a1a with result from task 3489 parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3489 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5ecf746b-d469-4710-b6ff-9f88a6399f52_isrForCrosstalkSources_997009_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/5ecf746b-d469-4710-b6ff-9f88a6399f52_isrForCrosstalkSources_997009_43.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:02 proc_lsst.multi:146 [INFO] found job 17075042 in provider checkpoint proc_lsst.multi INFO: found job 17075042 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3395 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ee468ab2a4a9f4ebb96d670c18255dc3 with result from task 3395 parsl.dataflow.dflow DEBUG: Task 3395 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f975e241-a990-4a16-86e1-60b26e6c84c2_isrForCrosstalkSources_996986_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/f975e241-a990-4a16-86e1-60b26e6c84c2_isrForCrosstalkSources_996986_27.stderr parsl.dataflow.dflow INFO: Task 3249 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3250826321c75e6ff868e22d6f977035 with result from task 3249 parsl.dataflow.dflow DEBUG: Task 3249 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/babd6167-1f35-43a8-9c44-606f251a3425_isrForCrosstalkSources_997018_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/babd6167-1f35-43a8-9c44-606f251a3425_isrForCrosstalkSources_997018_38.stderr parsl.dataflow.dflow INFO: Task 3114 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3444236807e198f35ec878a4fe817655 with result from task 3114 parsl.dataflow.dflow DEBUG: Task 3114 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c80befd4-e15b-4634-b961-a8a18cc51c7f_isrForCrosstalkSources_997025_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c80befd4-e15b-4634-b961-a8a18cc51c7f_isrForCrosstalkSources_997025_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:03 proc_lsst.multi:146 [INFO] found job 17075046 in provider checkpoint proc_lsst.multi INFO: found job 17075046 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3507 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8157bfe6498f7f7d02693414f96eb18a with result from task 3507 parsl.dataflow.dflow DEBUG: Task 3507 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/450f6c53-79b3-4543-a5f4-23033fa7ae1c_isrForCrosstalkSources_996981_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/450f6c53-79b3-4543-a5f4-23033fa7ae1c_isrForCrosstalkSources_996981_36.stderr parsl.dataflow.dflow INFO: Task 3098 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 180600c45864f6d578cfb31ae7488e43 with result from task 3098 parsl.dataflow.dflow DEBUG: Task 3098 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/e790e64c-cb47-4986-9eb4-fa42d4eb815f_isrForCrosstalkSources_996994_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/e790e64c-cb47-4986-9eb4-fa42d4eb815f_isrForCrosstalkSources_996994_31.stderr parsl.dataflow.dflow INFO: Task 3510 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 24851d79b5677bf6438c4922077a667b with result from task 3510 parsl.dataflow.dflow DEBUG: Task 3510 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/4d253210-81b6-42db-85ab-d23fe5dfc17e_isrForCrosstalkSources_997031_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/4d253210-81b6-42db-85ab-d23fe5dfc17e_isrForCrosstalkSources_997031_40.stderr parsl.dataflow.dflow INFO: Task 2960 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 819538a2a4fc75d4f1724f6cf4c1c3aa with result from task 2960 parsl.dataflow.dflow DEBUG: Task 2960 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1bea7713-906c-4477-a981-162e3488a6a5_isrForCrosstalkSources_996961_58.stdout parsl.dataflow.dflow INFO: Standard error for task 2960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/1bea7713-906c-4477-a981-162e3488a6a5_isrForCrosstalkSources_996961_58.stderr parsl.dataflow.dflow INFO: Task 3263 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fb601e505cbed81bb0380ee9d915236 with result from task 3263 parsl.dataflow.dflow DEBUG: Task 3263 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/0440f9e3-c56d-4b48-9d7d-02e38adcf7aa_isrForCrosstalkSources_997001_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/0440f9e3-c56d-4b48-9d7d-02e38adcf7aa_isrForCrosstalkSources_997001_10.stderr parsl.dataflow.dflow INFO: Task 3162 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6480b2e32f6900dded4c54ebe7a44ade with result from task 3162 parsl.dataflow.dflow DEBUG: Task 3162 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/08bd9fab-9e60-44a8-9553-277644ac2a6d_isrForCrosstalkSources_996975_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/08bd9fab-9e60-44a8-9553-277644ac2a6d_isrForCrosstalkSources_996975_2.stderr parsl.dataflow.dflow INFO: Task 3195 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56cbd41d09cc771decf0cac8a3d3f928 with result from task 3195 parsl.dataflow.dflow DEBUG: Task 3195 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/16a7ad76-6e43-41f6-a8fd-4012b050c09f_isrForCrosstalkSources_996970_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/16a7ad76-6e43-41f6-a8fd-4012b050c09f_isrForCrosstalkSources_996970_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3209 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f97f88aaf4aae4363138f4957416a7a2 with result from task 3209 parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:04 proc_lsst.multi:146 [INFO] found job 17075047 in provider checkpoint proc_lsst.multi INFO: found job 17075047 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3209 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/bbd9647a-50a9-4b3d-9f29-185799677689_isrForCrosstalkSources_997024_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/bbd9647a-50a9-4b3d-9f29-185799677689_isrForCrosstalkSources_997024_48.stderr parsl.dataflow.dflow INFO: Task 3104 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bedb35816a4e19650beaf25180f21f5a with result from task 3104 parsl.dataflow.dflow DEBUG: Task 3104 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7b1c9eba-c856-4402-b4c8-95d39c66adb0_isrForCrosstalkSources_996990_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7b1c9eba-c856-4402-b4c8-95d39c66adb0_isrForCrosstalkSources_996990_47.stderr parsl.dataflow.dflow INFO: Task 3269 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b653374c703f787ffc660e9bcdd748a with result from task 3269 parsl.dataflow.dflow DEBUG: Task 3269 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1fbd26ac-687a-4813-953e-190ff06b9178_isrForCrosstalkSources_996957_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1fbd26ac-687a-4813-953e-190ff06b9178_isrForCrosstalkSources_996957_43.stderr parsl.dataflow.dflow INFO: Task 3088 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 886241d1ef415e60ccbc87dd85eb4b65 with result from task 3088 parsl.dataflow.dflow DEBUG: Task 3088 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/39ff7f8d-2a5a-4d06-8488-64b9e6a5655a_isrForCrosstalkSources_996996_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/39ff7f8d-2a5a-4d06-8488-64b9e6a5655a_isrForCrosstalkSources_996996_41.stderr parsl.dataflow.dflow INFO: Task 3168 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96707aa4d85b40e6df5cd2a06ef7d09c with result from task 3168 parsl.dataflow.dflow DEBUG: Task 3168 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/68baef19-176a-492a-a31e-31f08e43866a_isrForCrosstalkSources_996958_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/68baef19-176a-492a-a31e-31f08e43866a_isrForCrosstalkSources_996958_60.stderr parsl.dataflow.dflow INFO: Task 3069 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a0164a1690d34b94bdcd3e463fda6477 with result from task 3069 parsl.dataflow.dflow DEBUG: Task 3069 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/521780b1-6310-4515-9363-f24b014d246e_isrForCrosstalkSources_996992_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/521780b1-6310-4515-9363-f24b014d246e_isrForCrosstalkSources_996992_37.stderr parsl.dataflow.dflow INFO: Task 3171 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 469a621e714336b43019338db0a0f22f with result from task 3171 parsl.dataflow.dflow DEBUG: Task 3171 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/437220b3-6e19-465e-a097-00b151591ce3_isrForCrosstalkSources_997008_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/437220b3-6e19-465e-a097-00b151591ce3_isrForCrosstalkSources_997008_31.stderr parsl.dataflow.dflow INFO: Task 3289 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23cbed4e8f04657fb3fe92484ca8bffc with result from task 3289 parsl.dataflow.dflow DEBUG: Task 3289 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3b31aee7-3145-4e17-a781-778abddfe1be_isrForCrosstalkSources_997014_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3b31aee7-3145-4e17-a781-778abddfe1be_isrForCrosstalkSources_997014_15.stderr parsl.dataflow.dflow INFO: Task 3177 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2642c4d5307e0c552344e583bb528232 with result from task 3177 parsl.dataflow.dflow DEBUG: Task 3177 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/d95366c6-c062-4937-a6e8-f0571254861a_isrForCrosstalkSources_997016_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/d95366c6-c062-4937-a6e8-f0571254861a_isrForCrosstalkSources_997016_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:05 proc_lsst.multi:146 [INFO] found job 17075048 in provider checkpoint proc_lsst.multi INFO: found job 17075048 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3226 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c294770f6bd9a8f3048014f32ba04299 with result from task 3226 parsl.dataflow.dflow DEBUG: Task 3226 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/13c58d9f-d80d-47e1-a3d9-1c65e3ba11ff_isrForCrosstalkSources_996972_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/13c58d9f-d80d-47e1-a3d9-1c65e3ba11ff_isrForCrosstalkSources_996972_55.stderr parsl.dataflow.dflow INFO: Task 3512 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d33d8150e76fac8ec3612d26492da7a2 with result from task 3512 parsl.dataflow.dflow DEBUG: Task 3512 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/df3b0177-59ad-44b8-8ca5-c2e40a2f3282_isrForCrosstalkSources_997007_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/df3b0177-59ad-44b8-8ca5-c2e40a2f3282_isrForCrosstalkSources_997007_61.stderr parsl.dataflow.dflow INFO: Task 3524 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab7c478b73d7dc457e58a2b06ddb67d4 with result from task 3524 parsl.dataflow.dflow DEBUG: Task 3524 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1477a841-fa28-4ffc-8f9e-c1276e22d8da_isrForCrosstalkSources_997003_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1477a841-fa28-4ffc-8f9e-c1276e22d8da_isrForCrosstalkSources_997003_17.stderr parsl.dataflow.dflow INFO: Task 2945 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11297064c1d9f9bd122d4a8eb7ab0b3d with result from task 2945 parsl.dataflow.dflow DEBUG: Task 2945 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 2945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4064624e-0cd6-4147-b713-09fa64f0bfa8_isrForCrosstalkSources_996987_61.stdout parsl.dataflow.dflow INFO: Standard error for task 2945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4064624e-0cd6-4147-b713-09fa64f0bfa8_isrForCrosstalkSources_996987_61.stderr parsl.dataflow.dflow INFO: Task 3259 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 640e361a73ca5e7cc7d63d0b2dded230 with result from task 3259 parsl.dataflow.dflow DEBUG: Task 3259 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1ac4f8d7-b0b4-444e-a353-71809351af62_isrForCrosstalkSources_996998_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1ac4f8d7-b0b4-444e-a353-71809351af62_isrForCrosstalkSources_996998_50.stderr parsl.dataflow.dflow INFO: Task 3514 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 936a051c013da4eb203363a1bf83c1cb with result from task 3514 parsl.dataflow.dflow DEBUG: Task 3514 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c0a8e29b-d29f-46ca-8d9a-bfba687ea20a_isrForCrosstalkSources_997012_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/c0a8e29b-d29f-46ca-8d9a-bfba687ea20a_isrForCrosstalkSources_997012_14.stderr parsl.dataflow.dflow INFO: Task 3068 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04cdbe6adb31abe15ffa383d5a64ede3 with result from task 3068 parsl.dataflow.dflow DEBUG: Task 3068 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c38d23e7-f619-4a33-9782-28f1a0097b54_isrForCrosstalkSources_996959_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/c38d23e7-f619-4a33-9782-28f1a0097b54_isrForCrosstalkSources_996959_44.stderr parsl.dataflow.dflow INFO: Task 3270 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a6b66538863c308b7363c9cc1f85430 with result from task 3270 parsl.dataflow.dflow DEBUG: Task 3270 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/94b8b631-a128-4520-ac76-5b0a03f4a8f2_isrForCrosstalkSources_996967_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/94b8b631-a128-4520-ac76-5b0a03f4a8f2_isrForCrosstalkSources_996967_27.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3056 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 1b890dc6706d094f64b0f3b4414046d3 with result from task 3056 parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3056 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5933159c-d0cf-483a-a641-6176e8c78ae2_isrForCrosstalkSources_997012_35.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5933159c-d0cf-483a-a641-6176e8c78ae2_isrForCrosstalkSources_997012_35.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3523 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 929d59082d75d38f7c7ef34511107727 with result from task 3523 parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3523 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/dbbb1d65-34ae-4ef0-8883-fc9649476104_isrForCrosstalkSources_997018_54.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/dbbb1d65-34ae-4ef0-8883-fc9649476104_isrForCrosstalkSources_997018_54.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3081 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 82f5b6ae7a3a8ba39da706872c2a04f2 with result from task 3081 parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3081 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/182393d4-414f-41f8-8559-13e1f91e3da9_isrForCrosstalkSources_996976_54.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/182393d4-414f-41f8-8559-13e1f91e3da9_isrForCrosstalkSources_996976_54.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3281 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 032d525a2166102d6d2847f587fda1d9 with result from task 3281 parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3281 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f34ef32e-1329-4ee0-b7ef-1f21490fdb2d_isrForCrosstalkSources_997020_44.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/f34ef32e-1329-4ee0-b7ef-1f21490fdb2d_isrForCrosstalkSources_997020_44.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3277 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 13b98b7389647af614e5064ded88a2ef with result from task 3277 parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3277 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/bd321c43-faa5-4e9d-b421-9d54b2cbdd2e_isrForCrosstalkSources_996981_34.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/bd321c43-faa5-4e9d-b421-9d54b2cbdd2e_isrForCrosstalkSources_996981_34.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:06 proc_lsst.multi:146 [INFO] found job 17075049 in provider checkpoint proc_lsst.multi INFO: found job 17075049 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3275 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6430537eb2d7199fcf9700bb5da94a9a with result from task 3275 parsl.dataflow.dflow DEBUG: Task 3275 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/618f7062-520d-48e8-a599-878d57908ae6_isrForCrosstalkSources_997007_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/618f7062-520d-48e8-a599-878d57908ae6_isrForCrosstalkSources_997007_32.stderr parsl.dataflow.dflow INFO: Task 3313 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa7e64375073b09a565cd5e74e3d5619 with result from task 3313 parsl.dataflow.dflow DEBUG: Task 3313 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/310a60ab-e0ff-4194-be9d-fd54375fc6b5_isrForCrosstalkSources_997025_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/310a60ab-e0ff-4194-be9d-fd54375fc6b5_isrForCrosstalkSources_997025_27.stderr parsl.dataflow.dflow INFO: Task 3414 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd6979ccedaf9514869d996bd8c32466 with result from task 3414 parsl.dataflow.dflow DEBUG: Task 3414 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dd02294e-bbff-489a-80ad-41030ccd9114_isrForCrosstalkSources_996971_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/dd02294e-bbff-489a-80ad-41030ccd9114_isrForCrosstalkSources_996971_11.stderr parsl.dataflow.dflow INFO: Task 3438 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry adf317de200dddd2e57fb33009353f6d with result from task 3438 parsl.dataflow.dflow DEBUG: Task 3438 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/91219d6d-f817-4221-942b-7a2890e9455a_isrForCrosstalkSources_997006_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/91219d6d-f817-4221-942b-7a2890e9455a_isrForCrosstalkSources_997006_13.stderr parsl.dataflow.dflow INFO: Task 3219 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5b0e12f27ef5ceab52c1b3ccc4d1bb7 with result from task 3219 parsl.dataflow.dflow DEBUG: Task 3219 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3bdac7ba-5aae-40dc-af5e-cd299b29c9f8_isrForCrosstalkSources_996959_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3bdac7ba-5aae-40dc-af5e-cd299b29c9f8_isrForCrosstalkSources_996959_30.stderr parsl.dataflow.dflow INFO: Task 3339 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af3639d78d5e42ecb9372a34979efe28 with result from task 3339 parsl.dataflow.dflow DEBUG: Task 3339 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/77ef21c1-98f6-4b7d-9ba8-6421392af82f_isrForCrosstalkSources_996990_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/77ef21c1-98f6-4b7d-9ba8-6421392af82f_isrForCrosstalkSources_996990_27.stderr parsl.dataflow.dflow INFO: Task 3481 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6684978cf5749e1808e87e47a6363b6 with result from task 3481 parsl.dataflow.dflow DEBUG: Task 3481 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/04b0865c-6197-422a-bf53-16c13d43c675_isrForCrosstalkSources_997018_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/04b0865c-6197-422a-bf53-16c13d43c675_isrForCrosstalkSources_997018_30.stderr parsl.dataflow.dflow INFO: Task 3303 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a2f6f324aa8cb27b7219346063a24f2 with result from task 3303 parsl.dataflow.dflow DEBUG: Task 3303 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5e317365-03ae-475b-b687-b7794f361817_isrForCrosstalkSources_997021_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/5e317365-03ae-475b-b687-b7794f361817_isrForCrosstalkSources_997021_13.stderr parsl.dataflow.dflow INFO: Task 3240 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e55480f5e37484d14598960ac1541d2 with result from task 3240 parsl.dataflow.dflow DEBUG: Task 3240 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/5917730d-c6f2-43bf-9f21-9d134b393339_isrForCrosstalkSources_997014_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/5917730d-c6f2-43bf-9f21-9d134b393339_isrForCrosstalkSources_997014_60.stderr parsl.dataflow.dflow INFO: Task 3389 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a9303cf07e899d2549da98bbdbba037c with result from task 3389 parsl.dataflow.dflow DEBUG: Task 3389 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/c442c9bf-b8f4-417f-8584-5d812d9cc969_isrForCrosstalkSources_996981_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/c442c9bf-b8f4-417f-8584-5d812d9cc969_isrForCrosstalkSources_996981_45.stderr parsl.dataflow.dflow INFO: Task 3341 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c751be7b58635c77be296302d70e5b9e with result from task 3341 parsl.dataflow.dflow DEBUG: Task 3341 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/9fedbcc1-87e8-4f4c-baea-5a18209d59b4_isrForCrosstalkSources_997031_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/9fedbcc1-87e8-4f4c-baea-5a18209d59b4_isrForCrosstalkSources_997031_39.stderr parsl.dataflow.dflow INFO: Task 3291 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c07bc559f5a37b927906b9a3a96252a0 with result from task 3291 parsl.dataflow.dflow DEBUG: Task 3291 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/855ac11b-788b-49be-b3ee-72d246680481_isrForCrosstalkSources_997022_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/855ac11b-788b-49be-b3ee-72d246680481_isrForCrosstalkSources_997022_19.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3513 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 0d830f1203c1af66038dee7d7364fd1a with result from task 3513 parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3513 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/3bc6fcb2-2f3a-4188-9c51-f222cb51cc39_isrForCrosstalkSources_996991_12.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/3bc6fcb2-2f3a-4188-9c51-f222cb51cc39_isrForCrosstalkSources_996991_12.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3271 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 41f3efa72a6cd497838c45c77a2af504 with result from task 3271 parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3271 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/4f7aad10-7c8b-46b6-b008-6a21e5d89c55_isrForCrosstalkSources_997031_22.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/4f7aad10-7c8b-46b6-b008-6a21e5d89c55_isrForCrosstalkSources_997031_22.stderr parsl.dataflow.dflow INFO: Task 3250 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7089c735da1a579adbd73667d2a2d71 with result from task 3250 parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:08 proc_lsst.multi:146 [INFO] found job 17075050 in provider checkpoint proc_lsst.multi INFO: found job 17075050 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3250 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/559b6395-6573-4880-8d9e-e8131d5b8d92_isrForCrosstalkSources_996960_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/559b6395-6573-4880-8d9e-e8131d5b8d92_isrForCrosstalkSources_996960_60.stderr parsl.dataflow.dflow INFO: Task 3080 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55e89831026d40a28ad0ea88df80421f with result from task 3080 parsl.dataflow.dflow DEBUG: Task 3080 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/79fa66d1-7c04-4c34-aac0-d5afc5866baa_isrForCrosstalkSources_996964_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/79fa66d1-7c04-4c34-aac0-d5afc5866baa_isrForCrosstalkSources_996964_55.stderr parsl.dataflow.dflow INFO: Task 3093 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bfae83f0cd735d78786510dc30ef8546 with result from task 3093 parsl.dataflow.dflow DEBUG: Task 3093 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f2d9a9d9-4686-4257-8d10-8ac7df53babd_isrForCrosstalkSources_996953_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f2d9a9d9-4686-4257-8d10-8ac7df53babd_isrForCrosstalkSources_996953_44.stderr parsl.dataflow.dflow INFO: Task 3323 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fd7f2eefaabb2c43769e58edfa032be with result from task 3323 parsl.dataflow.dflow DEBUG: Task 3323 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e379d7a5-8612-4825-bff4-7e84ce8a1257_isrForCrosstalkSources_997011_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e379d7a5-8612-4825-bff4-7e84ce8a1257_isrForCrosstalkSources_997011_45.stderr parsl.dataflow.dflow INFO: Task 3396 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb1fd83edf2f881e336db6bbece21e69 with result from task 3396 parsl.dataflow.dflow DEBUG: Task 3396 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/bb07408a-e7e5-4acd-b8fd-2918efaa7f1a_isrForCrosstalkSources_997021_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/bb07408a-e7e5-4acd-b8fd-2918efaa7f1a_isrForCrosstalkSources_997021_38.stderr parsl.dataflow.dflow INFO: Task 3434 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a628f2d7e6b9f30edb9736485b53ffc with result from task 3434 parsl.dataflow.dflow DEBUG: Task 3434 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/8a66645d-1737-4bce-a959-cb5127f9f7ec_isrForCrosstalkSources_996964_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/8a66645d-1737-4bce-a959-cb5127f9f7ec_isrForCrosstalkSources_996964_46.stderr parsl.dataflow.dflow INFO: Task 3238 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2afa13fd64c70ed8d53453d4e13df877 with result from task 3238 parsl.dataflow.dflow DEBUG: Task 3238 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/37aafbf1-aa93-461f-bc9d-6cf19c01a040_isrForCrosstalkSources_996985_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/37aafbf1-aa93-461f-bc9d-6cf19c01a040_isrForCrosstalkSources_996985_47.stderr parsl.dataflow.dflow INFO: Task 3332 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 699fad3e59cb6cdeb5aa1099becc6092 with result from task 3332 parsl.dataflow.dflow DEBUG: Task 3332 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/81cf0bc5-7028-47af-a4da-e25a9a30164c_isrForCrosstalkSources_997011_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/81cf0bc5-7028-47af-a4da-e25a9a30164c_isrForCrosstalkSources_997011_18.stderr parsl.dataflow.dflow INFO: Task 3442 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e00af5c12445827d1cd2180daef97181 with result from task 3442 parsl.dataflow.dflow DEBUG: Task 3442 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/13a8e6d6-8a3b-4ec8-b456-4447de7ee05a_isrForCrosstalkSources_996958_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/13a8e6d6-8a3b-4ec8-b456-4447de7ee05a_isrForCrosstalkSources_996958_6.stderr parsl.dataflow.dflow INFO: Task 3455 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d38763688d6c65bf2532e7f37146241e with result from task 3455 parsl.dataflow.dflow DEBUG: Task 3455 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/8bb8aa8c-c6a5-435e-b0b1-092e956a3bfc_isrForCrosstalkSources_996958_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/8bb8aa8c-c6a5-435e-b0b1-092e956a3bfc_isrForCrosstalkSources_996958_22.stderr parsl.dataflow.dflow INFO: Task 3516 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18792f8fb8c85f9e8c8a1098ff0cd845 with result from task 3516 parsl.dataflow.dflow DEBUG: Task 3516 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/588873bf-b8d3-4da3-97ef-0d9a3a1d6b6a_isrForCrosstalkSources_997030_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/588873bf-b8d3-4da3-97ef-0d9a3a1d6b6a_isrForCrosstalkSources_997030_18.stderr parsl.dataflow.dflow INFO: Task 3330 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f1d9064e7547b06e2986ead6d89c74e with result from task 3330 parsl.dataflow.dflow DEBUG: Task 3330 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/ef686f83-df77-477d-a1b2-1daf50c197ba_isrForCrosstalkSources_997003_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/ef686f83-df77-477d-a1b2-1daf50c197ba_isrForCrosstalkSources_997003_33.stderr parsl.dataflow.dflow INFO: Task 3268 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 976134703dcc78faed33550b1d47def4 with result from task 3268 parsl.dataflow.dflow DEBUG: Task 3268 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/7eec73bf-2f77-43ca-92ac-c720da21b397_isrForCrosstalkSources_996995_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/7eec73bf-2f77-43ca-92ac-c720da21b397_isrForCrosstalkSources_996995_44.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3247 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry a898838f6105f1d1332d86b0ea3b75d4 with result from task 3247 parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3247 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/62247415-14c2-493f-8bfd-6997185a0768_isrForCrosstalkSources_997024_1.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/62247415-14c2-493f-8bfd-6997185a0768_isrForCrosstalkSources_997024_1.stderr parsl.dataflow.dflow INFO: Task 3421 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50e5e1101bfc30e576ccbd60c50e50d4 with result from task 3421 parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3421 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c51e5a64-16cb-48b5-8561-a1381805905e_isrForCrosstalkSources_996995_28.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c51e5a64-16cb-48b5-8561-a1381805905e_isrForCrosstalkSources_996995_28.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3545 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 676b85437b1c1f7d36bc4b8c67438abd with result from task 3545 parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3545 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f2ffdee1-688d-45da-bb79-ea02b280aa8d_isrForCrosstalkSources_997028_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/f2ffdee1-688d-45da-bb79-ea02b280aa8d_isrForCrosstalkSources_997028_47.stderr parsl.dataflow.dflow INFO: Task 3369 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e32fe76600c979c27a90cc4f41f0f494 with result from task 3369 parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:09 proc_lsst.multi:146 [INFO] found job 17075052 in provider checkpoint proc_lsst.multi INFO: found job 17075052 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3369 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/27b09f8a-0f14-45ca-803e-f151d7b926b2_isrForCrosstalkSources_996996_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/27b09f8a-0f14-45ca-803e-f151d7b926b2_isrForCrosstalkSources_996996_35.stderr parsl.dataflow.dflow INFO: Task 3364 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1b97eedbfe7c7d3ec56d9247b734c9f1 with result from task 3364 parsl.dataflow.dflow DEBUG: Task 3364 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/8530d2b9-49be-4a20-b1f8-7a37bde303d2_isrForCrosstalkSources_997003_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/8530d2b9-49be-4a20-b1f8-7a37bde303d2_isrForCrosstalkSources_997003_11.stderr parsl.dataflow.dflow INFO: Task 3410 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57f91c529bfe72f4bb0367af80fff992 with result from task 3410 parsl.dataflow.dflow DEBUG: Task 3410 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/5c7011ea-eb04-4b6f-9fa0-3b4ef2d1fa67_isrForCrosstalkSources_996961_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/5c7011ea-eb04-4b6f-9fa0-3b4ef2d1fa67_isrForCrosstalkSources_996961_14.stderr parsl.dataflow.dflow INFO: Task 3547 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b2bea1a1e01f247efcd422b3d38093a0 with result from task 3547 parsl.dataflow.dflow DEBUG: Task 3547 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/4c73455a-44f1-4c82-a33b-c870f579f57c_isrForCrosstalkSources_997011_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/4c73455a-44f1-4c82-a33b-c870f579f57c_isrForCrosstalkSources_997011_5.stderr parsl.dataflow.dflow INFO: Task 3387 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bdbe061218724f134745eab250f2e8b6 with result from task 3387 parsl.dataflow.dflow DEBUG: Task 3387 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/2242a6ab-eb15-429c-8c0b-dec8c474409e_isrForCrosstalkSources_997024_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/2242a6ab-eb15-429c-8c0b-dec8c474409e_isrForCrosstalkSources_997024_29.stderr parsl.dataflow.dflow INFO: Task 3294 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4976fbcd4b543227ed4268c0c50e30b9 with result from task 3294 parsl.dataflow.dflow DEBUG: Task 3294 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c817db70-217e-4880-b3b1-1685ebbd8e81_isrForCrosstalkSources_996975_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c817db70-217e-4880-b3b1-1685ebbd8e81_isrForCrosstalkSources_996975_8.stderr parsl.dataflow.dflow INFO: Task 3385 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 45d35d39ca9fc853e45a95bead8bbb36 with result from task 3385 parsl.dataflow.dflow DEBUG: Task 3385 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/c50df46b-4af6-4106-8caf-ad9d3fa9cd1b_isrForCrosstalkSources_997017_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/c50df46b-4af6-4106-8caf-ad9d3fa9cd1b_isrForCrosstalkSources_997017_5.stderr parsl.dataflow.dflow INFO: Task 3401 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17c6d2b7baf241fa2a1563ca7fb61898 with result from task 3401 parsl.dataflow.dflow DEBUG: Task 3401 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ea901c4b-bfe3-4357-97d4-3305c16ebadf_isrForCrosstalkSources_996990_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/ea901c4b-bfe3-4357-97d4-3305c16ebadf_isrForCrosstalkSources_996990_28.stderr parsl.dataflow.dflow INFO: Task 3397 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2171c33be8c03a22a77cc93eda9c82f with result from task 3397 parsl.dataflow.dflow DEBUG: Task 3397 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/6fc3be7f-4296-4a60-aa43-24aaba93029f_isrForCrosstalkSources_996998_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/6fc3be7f-4296-4a60-aa43-24aaba93029f_isrForCrosstalkSources_996998_10.stderr parsl.dataflow.dflow INFO: Task 3265 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0768f8a4e924581ae4fa30f502d5287c with result from task 3265 parsl.dataflow.dflow DEBUG: Task 3265 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/913c0388-3dff-46d2-9293-02a87fbf1b39_isrForCrosstalkSources_996961_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/913c0388-3dff-46d2-9293-02a87fbf1b39_isrForCrosstalkSources_996961_43.stderr parsl.dataflow.dflow INFO: Task 3382 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c42ce1a4d1ca8f6e5e2c2fb92f7ff98 with result from task 3382 parsl.dataflow.dflow DEBUG: Task 3382 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/bdfc3769-4e30-4e1d-a736-ccbf9515e45b_isrForCrosstalkSources_997010_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/bdfc3769-4e30-4e1d-a736-ccbf9515e45b_isrForCrosstalkSources_997010_13.stderr parsl.dataflow.dflow INFO: Task 3409 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6544706719a190fa0cab04dc9d6fd6ad with result from task 3409 parsl.dataflow.dflow DEBUG: Task 3409 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/71afd1ab-7cdd-4402-8a17-a83a72e2b5c8_isrForCrosstalkSources_996982_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/71afd1ab-7cdd-4402-8a17-a83a72e2b5c8_isrForCrosstalkSources_996982_32.stderr parsl.dataflow.dflow INFO: Task 3534 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7be7965ce0608cad8c8bc69da981ab9 with result from task 3534 parsl.dataflow.dflow DEBUG: Task 3534 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/6a6fdc83-2378-4c9e-ba77-8063cadb7416_isrForCrosstalkSources_997008_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/6a6fdc83-2378-4c9e-ba77-8063cadb7416_isrForCrosstalkSources_997008_46.stderr parsl.dataflow.dflow INFO: Task 3457 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a3ad30f8ce19c538bf9c4b2fc472b5a2 with result from task 3457 parsl.dataflow.dflow DEBUG: Task 3457 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/553718d7-5f26-4030-90de-3a4089435583_isrForCrosstalkSources_996962_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/553718d7-5f26-4030-90de-3a4089435583_isrForCrosstalkSources_996962_19.stderr parsl.dataflow.dflow INFO: Task 3491 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7d7727abc1c1fb994a06bda65aab88d with result from task 3491 parsl.dataflow.dflow DEBUG: Task 3491 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/5a8fc2ae-0277-42b4-84e5-5407a86fc23f_isrForCrosstalkSources_996980_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/5a8fc2ae-0277-42b4-84e5-5407a86fc23f_isrForCrosstalkSources_996980_32.stderr parsl.dataflow.dflow INFO: Task 3557 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 225c8139a81a89e71c3504b37f824af6 with result from task 3557 parsl.dataflow.dflow DEBUG: Task 3557 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/77b7050f-af95-495a-be7e-b7cf05a78efb_isrForCrosstalkSources_996986_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/77b7050f-af95-495a-be7e-b7cf05a78efb_isrForCrosstalkSources_996986_24.stderr parsl.dataflow.dflow INFO: Task 3559 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53d850cb15ef159eb65cd4b8232ed3e9 with result from task 3559 parsl.dataflow.dflow DEBUG: Task 3559 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e0c244c9-3d33-40bf-ae6c-20ac6eeb53c8_isrForCrosstalkSources_997018_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e0c244c9-3d33-40bf-ae6c-20ac6eeb53c8_isrForCrosstalkSources_997018_56.stderr parsl.dataflow.dflow INFO: Task 3408 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 455191ba451b9bc2717cb12acd15cdb5 with result from task 3408 parsl.dataflow.dflow DEBUG: Task 3408 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a976bb45-4871-4a3b-acc8-325f0ca683f1_isrForCrosstalkSources_996995_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/a976bb45-4871-4a3b-acc8-325f0ca683f1_isrForCrosstalkSources_996995_33.stderr parsl.dataflow.dflow INFO: Task 3290 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e84230010fa86d01ab91b87185eb782 with result from task 3290 parsl.dataflow.dflow DEBUG: Task 3290 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/8e8eeae7-f312-4152-b85b-2816f54347d4_isrForCrosstalkSources_996972_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/8e8eeae7-f312-4152-b85b-2816f54347d4_isrForCrosstalkSources_996972_59.stderr parsl.dataflow.dflow INFO: Task 3299 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4b071e7a92d54392ade3b55d9b3831d with result from task 3299 parsl.dataflow.dflow DEBUG: Task 3299 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6fa383af-1ddd-46ea-a88e-f796b3101b6a_isrForCrosstalkSources_996963_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6fa383af-1ddd-46ea-a88e-f796b3101b6a_isrForCrosstalkSources_996963_15.stderr parsl.dataflow.dflow INFO: Task 3467 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 757e8fd39c4223fad66f7e2c2790c9ae with result from task 3467 parsl.dataflow.dflow DEBUG: Task 3467 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/ef667500-5712-403d-ad28-e17e613d5548_isrForCrosstalkSources_996999_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/ef667500-5712-403d-ad28-e17e613d5548_isrForCrosstalkSources_996999_34.stderr parsl.dataflow.dflow INFO: Task 3405 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53f5a580779ea783b40e38d0bc4a5dbe with result from task 3405 parsl.dataflow.dflow DEBUG: Task 3405 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/43585f3f-6469-48bb-af38-7d62fb510ff7_isrForCrosstalkSources_996987_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/43585f3f-6469-48bb-af38-7d62fb510ff7_isrForCrosstalkSources_996987_43.stderr parsl.dataflow.dflow INFO: Task 3306 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 681c5f2d8a0ac1804f34d6f5727b3a4a with result from task 3306 parsl.dataflow.dflow DEBUG: Task 3306 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/9bafbe9c-e8d4-474e-846d-f0e98506188b_isrForCrosstalkSources_996994_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/9bafbe9c-e8d4-474e-846d-f0e98506188b_isrForCrosstalkSources_996994_19.stderr parsl.dataflow.dflow INFO: Task 3560 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15e288ffbe1093705e140cb25ed098c1 with result from task 3560 parsl.dataflow.dflow DEBUG: Task 3560 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f34f7fce-ef9a-4a0a-b93b-b1fc1b2b096f_isrForCrosstalkSources_996978_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f34f7fce-ef9a-4a0a-b93b-b1fc1b2b096f_isrForCrosstalkSources_996978_17.stderr parsl.dataflow.dflow INFO: Task 3483 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 015db32225d6c4c7d2487e587fcd457b with result from task 3483 parsl.dataflow.dflow DEBUG: Task 3483 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a8a82568-1bc0-4c74-b899-98ec2364f401_isrForCrosstalkSources_997028_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/a8a82568-1bc0-4c74-b899-98ec2364f401_isrForCrosstalkSources_997028_46.stderr parsl.dataflow.dflow INFO: Task 3540 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cdf0d1920d935c90da8ae490f453c8f6 with result from task 3540 parsl.dataflow.dflow DEBUG: Task 3540 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/c7c1b434-11b5-4cc9-966c-1ae6525f72fe_isrForCrosstalkSources_996974_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/c7c1b434-11b5-4cc9-966c-1ae6525f72fe_isrForCrosstalkSources_996974_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:10 proc_lsst.multi:146 [INFO] found job 17075053 in provider checkpoint proc_lsst.multi INFO: found job 17075053 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3542 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 767dbe8557b7a71ce9813ee7c59e04a1 with result from task 3542 parsl.dataflow.dflow DEBUG: Task 3542 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/50059f01-1ddb-48f0-b05b-72d170b811ed_isrForCrosstalkSources_997033_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/50059f01-1ddb-48f0-b05b-72d170b811ed_isrForCrosstalkSources_997033_40.stderr parsl.dataflow.dflow INFO: Task 3346 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d76b233befff78d0cb57fd3c84f2a4f with result from task 3346 parsl.dataflow.dflow DEBUG: Task 3346 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f3a0e0cc-a8a8-401e-acf3-5955143e5357_isrForCrosstalkSources_996988_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/f3a0e0cc-a8a8-401e-acf3-5955143e5357_isrForCrosstalkSources_996988_23.stderr parsl.dataflow.dflow INFO: Task 3549 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry baf9cb3e504d8fe3863afe71bd84ad4b with result from task 3549 parsl.dataflow.dflow DEBUG: Task 3549 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/891effa2-dac9-4e5e-bcbe-cb9aa20ce692_isrForCrosstalkSources_997008_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/891effa2-dac9-4e5e-bcbe-cb9aa20ce692_isrForCrosstalkSources_997008_45.stderr parsl.dataflow.dflow INFO: Task 3582 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e1630c0fe46dad596cc92cfb22cbf904 with result from task 3582 parsl.dataflow.dflow DEBUG: Task 3582 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/262bfd13-e763-4610-8ce4-7c95fe45aa44_isrForCrosstalkSources_996981_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/262bfd13-e763-4610-8ce4-7c95fe45aa44_isrForCrosstalkSources_996981_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 3231 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 7cf52dbaac5accbb7554c297a19c0d43 with result from task 3231 parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3231 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a013966a-19be-41b4-8c41-5045986ac35f_isrForCrosstalkSources_997013_41.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/a013966a-19be-41b4-8c41-5045986ac35f_isrForCrosstalkSources_997013_41.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3432 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 1f399ace812ad7b42592fa2eabcaecea with result from task 3432 parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3432 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/543bf4c4-0263-4ddf-a716-8a96d8f56767_isrForCrosstalkSources_997022_47.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/543bf4c4-0263-4ddf-a716-8a96d8f56767_isrForCrosstalkSources_997022_47.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:11 proc_lsst.multi:146 [INFO] found job 17075054 in provider checkpoint proc_lsst.multi INFO: found job 17075054 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3328 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d13416d8985102c4e7785b89bb7a7ae with result from task 3328 parsl.dataflow.dflow DEBUG: Task 3328 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/e5ebf769-0558-4758-9466-8997ade498d2_isrForCrosstalkSources_997013_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/e5ebf769-0558-4758-9466-8997ade498d2_isrForCrosstalkSources_997013_35.stderr parsl.dataflow.dflow INFO: Task 3315 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e91cadd7013959b00e68ae1bae97d44 with result from task 3315 parsl.dataflow.dflow DEBUG: Task 3315 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/9736c035-d4b3-46e3-9c48-447392d39c2a_isrForCrosstalkSources_996978_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/9736c035-d4b3-46e3-9c48-447392d39c2a_isrForCrosstalkSources_996978_8.stderr parsl.dataflow.dflow INFO: Task 3539 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7f0770d1ec72a89f65fb0f5e88a8bb40 with result from task 3539 parsl.dataflow.dflow DEBUG: Task 3539 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/713d38ed-d64e-49e7-814a-722675f38a1c_isrForCrosstalkSources_997026_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/713d38ed-d64e-49e7-814a-722675f38a1c_isrForCrosstalkSources_997026_12.stderr parsl.dataflow.dflow INFO: Task 3474 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21ecc75c289c3317f66e6ff666110f20 with result from task 3474 parsl.dataflow.dflow DEBUG: Task 3474 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/261b67ca-8d97-4e11-9835-0b492d93e6fc_isrForCrosstalkSources_996962_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/261b67ca-8d97-4e11-9835-0b492d93e6fc_isrForCrosstalkSources_996962_9.stderr parsl.dataflow.dflow INFO: Task 3522 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19c39e793125c825d5ab7287d028d572 with result from task 3522 parsl.dataflow.dflow DEBUG: Task 3522 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/bfd0f07c-dd98-4238-88a1-b4b0dac846cb_isrForCrosstalkSources_996957_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/bfd0f07c-dd98-4238-88a1-b4b0dac846cb_isrForCrosstalkSources_996957_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3574 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry e813982fd92d2d0233720ad724619ee4 with result from task 3574 parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3574 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/3c471814-55d2-41a2-8a63-92005e074d60_isrForCrosstalkSources_996961_9.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/3c471814-55d2-41a2-8a63-92005e074d60_isrForCrosstalkSources_996961_9.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3386 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 53c7540c3ee6d2e4b595c76f14c96126 with result from task 3386 parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:12 proc_lsst.multi:146 [INFO] found job 17075055 in provider checkpoint proc_lsst.multi INFO: found job 17075055 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3386 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2aba0980-47f4-4d06-aa58-38716613ed43_isrForCrosstalkSources_996974_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2aba0980-47f4-4d06-aa58-38716613ed43_isrForCrosstalkSources_996974_6.stderr parsl.dataflow.dflow INFO: Task 3451 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e3decdaaf91e1f6d009e953a482e894 with result from task 3451 parsl.dataflow.dflow DEBUG: Task 3451 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f972d9e2-fee1-4b50-b664-f71b25dd7c1a_isrForCrosstalkSources_997019_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f972d9e2-fee1-4b50-b664-f71b25dd7c1a_isrForCrosstalkSources_997019_16.stderr parsl.dataflow.dflow INFO: Task 3572 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8414c074353edb3819db36af85621ec8 with result from task 3572 parsl.dataflow.dflow DEBUG: Task 3572 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/66cd768b-b7d5-4289-b1bb-7d4bc9e94611_isrForCrosstalkSources_997013_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/66cd768b-b7d5-4289-b1bb-7d4bc9e94611_isrForCrosstalkSources_997013_28.stderr parsl.dataflow.dflow INFO: Task 3418 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1321c117adac975fb604fba278fe74c2 with result from task 3418 parsl.dataflow.dflow DEBUG: Task 3418 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/4d740d17-2d7d-4495-bc1b-dd29102ab174_isrForCrosstalkSources_996967_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/4d740d17-2d7d-4495-bc1b-dd29102ab174_isrForCrosstalkSources_996967_55.stderr parsl.dataflow.dflow INFO: Task 3318 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2ac0042f30f8644b5b2cfab3e7df79e with result from task 3318 parsl.dataflow.dflow DEBUG: Task 3318 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/7605bbfc-6713-492e-afb9-33076fc809fa_isrForCrosstalkSources_996987_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/7605bbfc-6713-492e-afb9-33076fc809fa_isrForCrosstalkSources_996987_31.stderr parsl.dataflow.dflow INFO: Task 3424 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f19a8a2e85faf1921af8cb120fa2c878 with result from task 3424 parsl.dataflow.dflow DEBUG: Task 3424 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d5e6461-e67f-45cf-b571-90605e2b27bb_isrForCrosstalkSources_996995_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4d5e6461-e67f-45cf-b571-90605e2b27bb_isrForCrosstalkSources_996995_17.stderr parsl.dataflow.dflow INFO: Task 3362 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 274620d3b4aca23275cdab1fb96aa3ce with result from task 3362 parsl.dataflow.dflow DEBUG: Task 3362 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/78940c5a-e5df-4314-9b27-a8650fb63c23_isrForCrosstalkSources_996956_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/78940c5a-e5df-4314-9b27-a8650fb63c23_isrForCrosstalkSources_996956_4.stderr parsl.dataflow.dflow INFO: Task 3536 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1032370705cb3b26edfec9f6a7f0b1d8 with result from task 3536 parsl.dataflow.dflow DEBUG: Task 3536 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/8a12c7de-ac37-48fa-a23d-2e2dc9c0114f_isrForCrosstalkSources_996979_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/8a12c7de-ac37-48fa-a23d-2e2dc9c0114f_isrForCrosstalkSources_996979_41.stderr parsl.dataflow.dflow INFO: Task 3498 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 49834bbd560d7b675db6f9c392ee15d7 with result from task 3498 parsl.dataflow.dflow DEBUG: Task 3498 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9ce4c65d-f957-4bfa-ac75-46c2efd8781c_isrForCrosstalkSources_997030_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9ce4c65d-f957-4bfa-ac75-46c2efd8781c_isrForCrosstalkSources_997030_12.stderr parsl.dataflow.dflow INFO: Task 3576 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2bbe9eb3cc5d6a3ee69ca583210220a9 with result from task 3576 parsl.dataflow.dflow DEBUG: Task 3576 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/89529e65-0972-4e11-9156-5f9988d6e9dd_isrForCrosstalkSources_997009_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/89529e65-0972-4e11-9156-5f9988d6e9dd_isrForCrosstalkSources_997009_50.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:13 proc_lsst.multi:146 [INFO] found job 17075056 in provider checkpoint proc_lsst.multi INFO: found job 17075056 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3354 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30de106bdc8bc5339640bf1f76959631 with result from task 3354 parsl.dataflow.dflow DEBUG: Task 3354 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a5482471-416a-40fa-a27c-f2ead467ccbe_isrForCrosstalkSources_996957_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/a5482471-416a-40fa-a27c-f2ead467ccbe_isrForCrosstalkSources_996957_17.stderr parsl.dataflow.dflow INFO: Task 3319 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d0378419447cd3487bee17bd17934f05 with result from task 3319 parsl.dataflow.dflow DEBUG: Task 3319 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/107515ed-ae0e-4689-9671-2025e9649f4e_isrForCrosstalkSources_996952_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/107515ed-ae0e-4689-9671-2025e9649f4e_isrForCrosstalkSources_996952_13.stderr parsl.dataflow.dflow INFO: Task 3337 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e651e4297ec1be550935d00ac8cab83b with result from task 3337 parsl.dataflow.dflow DEBUG: Task 3337 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0d214ce9-2d00-4fec-9d70-e3f2b18bd4c1_isrForCrosstalkSources_997007_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0d214ce9-2d00-4fec-9d70-e3f2b18bd4c1_isrForCrosstalkSources_997007_39.stderr parsl.dataflow.dflow INFO: Task 3496 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 95d541c1ffdad9db572bf1a8218a8180 with result from task 3496 parsl.dataflow.dflow DEBUG: Task 3496 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/99779995-a737-4312-bec3-639c32003f54_isrForCrosstalkSources_996980_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/99779995-a737-4312-bec3-639c32003f54_isrForCrosstalkSources_996980_35.stderr parsl.dataflow.dflow INFO: Task 3430 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6f3f151ea9aa59372df46c3a82639cb with result from task 3430 parsl.dataflow.dflow DEBUG: Task 3430 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6a429c0-d948-4b90-ad94-603ac4bef9b1_isrForCrosstalkSources_997026_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/f6a429c0-d948-4b90-ad94-603ac4bef9b1_isrForCrosstalkSources_997026_17.stderr parsl.dataflow.dflow INFO: Task 3261 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf5ea47092edabbb1eac68cb4ca3e6ae with result from task 3261 parsl.dataflow.dflow DEBUG: Task 3261 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a91af4c3-256b-4c0a-9bd6-8b7a53b97160_isrForCrosstalkSources_997000_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a91af4c3-256b-4c0a-9bd6-8b7a53b97160_isrForCrosstalkSources_997000_54.stderr parsl.dataflow.dflow INFO: Task 3568 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d701ec350319a556faa44cf2c8f02250 with result from task 3568 parsl.dataflow.dflow DEBUG: Task 3568 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/074ce75a-919f-4d94-bc24-cd0f6d8eedab_isrForCrosstalkSources_996978_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/074ce75a-919f-4d94-bc24-cd0f6d8eedab_isrForCrosstalkSources_996978_28.stderr parsl.dataflow.dflow INFO: Task 3551 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e8b89838ab7adca08636ccb648a1fad with result from task 3551 parsl.dataflow.dflow DEBUG: Task 3551 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/18f4f06b-f499-4726-aaef-67ad1816208c_isrForCrosstalkSources_996953_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/18f4f06b-f499-4726-aaef-67ad1816208c_isrForCrosstalkSources_996953_41.stderr parsl.dataflow.dflow INFO: Task 3525 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8114984d15c80e823a6bc2ad704e4cf9 with result from task 3525 parsl.dataflow.dflow DEBUG: Task 3525 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f09a5771-e29b-42be-96dc-d9245fd3c2a0_isrForCrosstalkSources_997031_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f09a5771-e29b-42be-96dc-d9245fd3c2a0_isrForCrosstalkSources_997031_43.stderr parsl.dataflow.dflow INFO: Task 3334 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44250622eafd68cf993b3733838c0ddf with result from task 3334 parsl.dataflow.dflow DEBUG: Task 3334 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/ab8d72fb-1a36-4940-a25c-15ede7da89de_isrForCrosstalkSources_996952_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/ab8d72fb-1a36-4940-a25c-15ede7da89de_isrForCrosstalkSources_996952_18.stderr parsl.dataflow.dflow INFO: Task 3591 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46b1ecd96afa4a176bfac01815696d69 with result from task 3591 parsl.dataflow.dflow DEBUG: Task 3591 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/87254e1f-88bf-4880-a487-4139899716c8_isrForCrosstalkSources_996966_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/87254e1f-88bf-4880-a487-4139899716c8_isrForCrosstalkSources_996966_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:13 proc_lsst.multi:146 [INFO] found job 17075057 in provider checkpoint proc_lsst.multi INFO: found job 17075057 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3393 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 717c7d6692e03ae1a4471743ece28bcc with result from task 3393 parsl.dataflow.dflow DEBUG: Task 3393 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/30638c61-f947-4f02-b599-7e1998cbdd83_isrForCrosstalkSources_996979_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/30638c61-f947-4f02-b599-7e1998cbdd83_isrForCrosstalkSources_996979_8.stderr parsl.dataflow.dflow INFO: Task 3363 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d265fb55a7cdde5edf1eac08df7c746a with result from task 3363 parsl.dataflow.dflow DEBUG: Task 3363 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/628e2c84-3ef4-4a64-8f1d-5d966b1381a9_isrForCrosstalkSources_996954_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/628e2c84-3ef4-4a64-8f1d-5d966b1381a9_isrForCrosstalkSources_996954_56.stderr parsl.dataflow.dflow INFO: Task 3590 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c06ce6ceec0735f5c098facd129ba0e with result from task 3590 parsl.dataflow.dflow DEBUG: Task 3590 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a5c039cb-a8ae-4452-a558-9406ca09b830_isrForCrosstalkSources_996999_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a5c039cb-a8ae-4452-a558-9406ca09b830_isrForCrosstalkSources_996999_59.stderr parsl.dataflow.dflow INFO: Task 3388 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fbede176979991054d0399ab67322ce8 with result from task 3388 parsl.dataflow.dflow DEBUG: Task 3388 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/9e223c22-b2c6-4fee-bb9e-286b672cd5f0_isrForCrosstalkSources_996962_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/9e223c22-b2c6-4fee-bb9e-286b672cd5f0_isrForCrosstalkSources_996962_21.stderr parsl.dataflow.dflow INFO: Task 3583 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15bd1a40a72bad6a3408d44e3c9cc77d with result from task 3583 parsl.dataflow.dflow DEBUG: Task 3583 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/1da408b4-d5fc-4288-a745-fe21c420c687_isrForCrosstalkSources_996975_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/1da408b4-d5fc-4288-a745-fe21c420c687_isrForCrosstalkSources_996975_14.stderr parsl.dataflow.dflow INFO: Task 3530 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3e1187baa059238aefb93aa80d39386e with result from task 3530 parsl.dataflow.dflow DEBUG: Task 3530 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9604bd4-6226-49e2-a657-0c5410e709ac_isrForCrosstalkSources_997009_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9604bd4-6226-49e2-a657-0c5410e709ac_isrForCrosstalkSources_997009_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:14 proc_lsst.multi:146 [INFO] found job 17075058 in provider checkpoint proc_lsst.multi INFO: found job 17075058 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3381 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 864f7187d0cb9a13cd6f382c9069b933 with result from task 3381 parsl.dataflow.dflow DEBUG: Task 3381 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5aefc0bf-7f28-467f-9fbc-f68140cc2159_isrForCrosstalkSources_997003_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/5aefc0bf-7f28-467f-9fbc-f68140cc2159_isrForCrosstalkSources_997003_23.stderr parsl.dataflow.dflow INFO: Task 3487 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 402a8064aa3b9af709b702bd99a8ca21 with result from task 3487 parsl.dataflow.dflow DEBUG: Task 3487 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/fa1a5a92-5a92-4e26-9c4e-c3805e64e673_isrForCrosstalkSources_997033_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/fa1a5a92-5a92-4e26-9c4e-c3805e64e673_isrForCrosstalkSources_997033_16.stderr parsl.dataflow.dflow INFO: Task 3468 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c9db8bf5d4a12eaf26ca5752e5f9f88 with result from task 3468 parsl.dataflow.dflow DEBUG: Task 3468 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1936ac38-8749-458a-88dc-923d63422b1c_isrForCrosstalkSources_996968_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/1936ac38-8749-458a-88dc-923d63422b1c_isrForCrosstalkSources_996968_59.stderr parsl.dataflow.dflow INFO: Task 3595 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0658f057610abf387abd14ced4888aad with result from task 3595 parsl.dataflow.dflow DEBUG: Task 3595 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/34c68e4f-74b6-4f2a-8bba-0c593e04e1ab_isrForCrosstalkSources_996961_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/34c68e4f-74b6-4f2a-8bba-0c593e04e1ab_isrForCrosstalkSources_996961_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3392 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 99750f95562203f89f782d0d62ce7770 with result from task 3392 parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3392 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/f49c050b-3a7e-42e5-a0cd-54556367be34_isrForCrosstalkSources_997010_23.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/f49c050b-3a7e-42e5-a0cd-54556367be34_isrForCrosstalkSources_997010_23.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3304 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 3d750a88aa5b26e79d3edbea5533e7b2 with result from task 3304 parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3304 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/cccfbba1-186b-4574-9166-d6651374a742_isrForCrosstalkSources_996968_60.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/cccfbba1-186b-4574-9166-d6651374a742_isrForCrosstalkSources_996968_60.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3587 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 5e9347829a435955f8c06d157dc8e2c9 with result from task 3587 parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3587 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/329bfa40-c8d8-4ff5-84ea-47b804e2db33_isrForCrosstalkSources_996966_48.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/329bfa40-c8d8-4ff5-84ea-47b804e2db33_isrForCrosstalkSources_996966_48.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3577 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 287aaec04477ce086d7a547402dd06b5 with result from task 3577 parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3577 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ae2300f7-193d-4e27-9570-62aa6211e92c_isrForCrosstalkSources_997019_12.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ae2300f7-193d-4e27-9570-62aa6211e92c_isrForCrosstalkSources_997019_12.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:15 proc_lsst.multi:146 [INFO] found job 17075059 in provider checkpoint proc_lsst.multi INFO: found job 17075059 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3448 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91fc18488ebec5afc218f7d29a47e5ff with result from task 3448 parsl.dataflow.dflow DEBUG: Task 3448 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/65cb4df0-a860-4658-8b56-2bf4b7e723f0_isrForCrosstalkSources_997019_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/65cb4df0-a860-4658-8b56-2bf4b7e723f0_isrForCrosstalkSources_997019_48.stderr parsl.dataflow.dflow INFO: Task 3597 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fe2f6e0399ea60e0cd205ee42db4c61 with result from task 3597 parsl.dataflow.dflow DEBUG: Task 3597 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/f7e36e8c-98c4-4c4e-8fad-8af40d117864_isrForCrosstalkSources_997012_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/f7e36e8c-98c4-4c4e-8fad-8af40d117864_isrForCrosstalkSources_997012_38.stderr parsl.dataflow.dflow INFO: Task 3292 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 356576c0828076e7e4e9255ea0dec653 with result from task 3292 parsl.dataflow.dflow DEBUG: Task 3292 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/dcecf8aa-9a34-463d-96ce-4885c3a8a07a_isrForCrosstalkSources_996990_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/dcecf8aa-9a34-463d-96ce-4885c3a8a07a_isrForCrosstalkSources_996990_6.stderr parsl.dataflow.dflow INFO: Task 3278 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 549859a81909c448e5b00e1bc6cf3907 with result from task 3278 parsl.dataflow.dflow DEBUG: Task 3278 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2715a42f-9501-4677-b1ae-de85f57e9e8f_isrForCrosstalkSources_996963_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/2715a42f-9501-4677-b1ae-de85f57e9e8f_isrForCrosstalkSources_996963_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:15 proc_lsst.multi:146 [INFO] found job 17075060 in provider checkpoint proc_lsst.multi INFO: found job 17075060 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3352 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f1067d5feba108c90de6b37a63386576 with result from task 3352 parsl.dataflow.dflow DEBUG: Task 3352 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/04ce60a4-2d7a-4f62-8abf-d82cd32d59a1_isrForCrosstalkSources_996961_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/04ce60a4-2d7a-4f62-8abf-d82cd32d59a1_isrForCrosstalkSources_996961_49.stderr parsl.dataflow.dflow INFO: Task 3365 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f7ae1f46edd671df7e7f52b694d7a40 with result from task 3365 parsl.dataflow.dflow DEBUG: Task 3365 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/1134b7ab-9193-428a-b7c9-89f6ede4409f_isrForCrosstalkSources_996966_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/1134b7ab-9193-428a-b7c9-89f6ede4409f_isrForCrosstalkSources_996966_39.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:16 proc_lsst.multi:146 [INFO] found job 17075061 in provider checkpoint proc_lsst.multi INFO: found job 17075061 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3613 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eacde9ffc29ea3d6a85b30f31061df45 with result from task 3613 parsl.dataflow.dflow DEBUG: Task 3613 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1e392b2e-9d3a-45da-89fc-f69df52daca1_isrForCrosstalkSources_996960_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/1e392b2e-9d3a-45da-89fc-f69df52daca1_isrForCrosstalkSources_996960_54.stderr parsl.dataflow.dflow INFO: Task 3351 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8bab6d99bf4e2135332c2cc9c2dbdd66 with result from task 3351 parsl.dataflow.dflow DEBUG: Task 3351 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/e0f659b9-733d-4367-a798-5da26facb931_isrForCrosstalkSources_996979_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/e0f659b9-733d-4367-a798-5da26facb931_isrForCrosstalkSources_996979_36.stderr parsl.dataflow.dflow INFO: Task 3612 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0cebed5315b7a909a6d9ff6fb7f6f252 with result from task 3612 parsl.dataflow.dflow DEBUG: Task 3612 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/38d5455b-bae9-43a6-b393-eab9a31ebe31_isrForCrosstalkSources_996977_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/38d5455b-bae9-43a6-b393-eab9a31ebe31_isrForCrosstalkSources_996977_35.stderr parsl.dataflow.dflow INFO: Task 3373 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 273c9bab42854162aa3111dd5a0a395e with result from task 3373 parsl.dataflow.dflow DEBUG: Task 3373 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ecbaf3de-a168-4e95-8044-ac26a2a9604e_isrForCrosstalkSources_996991_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ecbaf3de-a168-4e95-8044-ac26a2a9604e_isrForCrosstalkSources_996991_23.stderr parsl.dataflow.dflow INFO: Task 3459 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3eb7f53cae66290e774793be8cfb62e with result from task 3459 parsl.dataflow.dflow DEBUG: Task 3459 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/70b048ee-da56-43bf-ad5c-9c53cbfb3923_isrForCrosstalkSources_997007_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/70b048ee-da56-43bf-ad5c-9c53cbfb3923_isrForCrosstalkSources_997007_43.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 3336 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 57ef1f2679dd71aa854417e10a92cc11 with result from task 3336 parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3336 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5becfe46-9d18-4669-bfa1-a1c8478a20c0_isrForCrosstalkSources_997010_28.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/5becfe46-9d18-4669-bfa1-a1c8478a20c0_isrForCrosstalkSources_997010_28.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3422 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry c2c21557a9d6a48883e3df85996d0072 with result from task 3422 parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3422 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d3a43845-8066-4471-b377-f4d1c6e9b280_isrForCrosstalkSources_997024_42.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d3a43845-8066-4471-b377-f4d1c6e9b280_isrForCrosstalkSources_997024_42.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:17 proc_lsst.multi:146 [INFO] found job 17075062 in provider checkpoint proc_lsst.multi INFO: found job 17075062 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3620 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a79ad3dfaa514bf9882f3deff588acf with result from task 3620 parsl.dataflow.dflow DEBUG: Task 3620 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e9097ce8-8eeb-4dc5-b89a-17313312ed6f_isrForCrosstalkSources_996961_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/e9097ce8-8eeb-4dc5-b89a-17313312ed6f_isrForCrosstalkSources_996961_30.stderr parsl.dataflow.dflow INFO: Task 3360 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9946edddfbfb9fee1925f74b4a0e5142 with result from task 3360 parsl.dataflow.dflow DEBUG: Task 3360 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e3209367-4906-45fa-9bf9-047c0802a4d2_isrForCrosstalkSources_997018_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/e3209367-4906-45fa-9bf9-047c0802a4d2_isrForCrosstalkSources_997018_6.stderr parsl.dataflow.dflow INFO: Task 3377 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9962b4f74fc54f88a2cb584f0da8a04 with result from task 3377 parsl.dataflow.dflow DEBUG: Task 3377 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d7c705f8-d3b0-40dc-9ee1-857dd65c4e33_isrForCrosstalkSources_996989_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d7c705f8-d3b0-40dc-9ee1-857dd65c4e33_isrForCrosstalkSources_996989_10.stderr parsl.dataflow.dflow INFO: Task 3543 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b62e39e7d66b2a22d07ec56a5d9ec69 with result from task 3543 parsl.dataflow.dflow DEBUG: Task 3543 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/ad588edf-1131-4be6-b969-90472daffa27_isrForCrosstalkSources_996974_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/ad588edf-1131-4be6-b969-90472daffa27_isrForCrosstalkSources_996974_24.stderr parsl.dataflow.dflow INFO: Task 3326 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a1505606c22590421a439efce2da837 with result from task 3326 parsl.dataflow.dflow DEBUG: Task 3326 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/ee36e191-dbda-4592-aae2-06e2ecf726fd_isrForCrosstalkSources_996997_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/ee36e191-dbda-4592-aae2-06e2ecf726fd_isrForCrosstalkSources_996997_7.stderr parsl.dataflow.dflow INFO: Task 3371 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8101d38150d3ca17044231661cd9b2f with result from task 3371 parsl.dataflow.dflow DEBUG: Task 3371 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/e71e36df-908e-4b65-be55-1b379992f9ba_isrForCrosstalkSources_996959_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/e71e36df-908e-4b65-be55-1b379992f9ba_isrForCrosstalkSources_996959_35.stderr parsl.dataflow.dflow INFO: Task 3447 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 890684813896afa4f000ec84587f64ea with result from task 3447 parsl.dataflow.dflow DEBUG: Task 3447 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/edcfcf67-bb0f-405a-9564-d264455753b6_isrForCrosstalkSources_997017_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/edcfcf67-bb0f-405a-9564-d264455753b6_isrForCrosstalkSources_997017_22.stderr parsl.dataflow.dflow INFO: Task 3327 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 168213b9ef246816087e3d56e3760ff7 with result from task 3327 parsl.dataflow.dflow DEBUG: Task 3327 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c3886520-c91a-4187-81fd-000ac4a6c762_isrForCrosstalkSources_996997_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/c3886520-c91a-4187-81fd-000ac4a6c762_isrForCrosstalkSources_996997_1.stderr parsl.dataflow.dflow INFO: Task 3378 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46875b0687a5cda71eda8b70e38d2fbb with result from task 3378 parsl.dataflow.dflow DEBUG: Task 3378 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/994eb883-6d32-46cf-9fd1-fb8639c39ec0_isrForCrosstalkSources_997016_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/994eb883-6d32-46cf-9fd1-fb8639c39ec0_isrForCrosstalkSources_997016_29.stderr parsl.dataflow.dflow INFO: Task 3616 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9510d50fd7c744270b616473c26e29f6 with result from task 3616 parsl.dataflow.dflow DEBUG: Task 3616 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/1d6ecadf-9368-4595-b275-e13c2d3e2ef8_isrForCrosstalkSources_997021_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/1d6ecadf-9368-4595-b275-e13c2d3e2ef8_isrForCrosstalkSources_997021_55.stderr parsl.dataflow.dflow INFO: Task 3390 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 725515e7eb684b127298814070698f92 with result from task 3390 parsl.dataflow.dflow DEBUG: Task 3390 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f8856572-3789-44de-bbb9-e2a8dc122e66_isrForCrosstalkSources_997024_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f8856572-3789-44de-bbb9-e2a8dc122e66_isrForCrosstalkSources_997024_3.stderr parsl.dataflow.dflow INFO: Task 3355 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a62646faeae1e4385ae08fbdcf3a1865 with result from task 3355 parsl.dataflow.dflow DEBUG: Task 3355 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/18b90d5a-c335-4ef6-a03e-2d8a4accdd04_isrForCrosstalkSources_996957_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/18b90d5a-c335-4ef6-a03e-2d8a4accdd04_isrForCrosstalkSources_996957_1.stderr parsl.dataflow.dflow INFO: Task 3398 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c52c2ea20128849a8baf84b26953d5c with result from task 3398 parsl.dataflow.dflow DEBUG: Task 3398 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/d0c14004-4991-481d-b8b7-98f77c6814a0_isrForCrosstalkSources_996968_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/d0c14004-4991-481d-b8b7-98f77c6814a0_isrForCrosstalkSources_996968_43.stderr parsl.dataflow.dflow INFO: Task 3570 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 281a4955f675255b6ffdd263a60a3e7b with result from task 3570 parsl.dataflow.dflow DEBUG: Task 3570 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/419648c4-580e-4354-88d8-a9dcb2e73fa4_isrForCrosstalkSources_996993_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/419648c4-580e-4354-88d8-a9dcb2e73fa4_isrForCrosstalkSources_996993_26.stderr parsl.dataflow.dflow INFO: Task 3347 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1de16c2130fb4cf082ec942f4a9f84ac with result from task 3347 parsl.dataflow.dflow DEBUG: Task 3347 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/fa8738a7-ef97-4c7f-8832-5ea1d54bf055_isrForCrosstalkSources_996963_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/fa8738a7-ef97-4c7f-8832-5ea1d54bf055_isrForCrosstalkSources_996963_60.stderr parsl.dataflow.dflow INFO: Task 3342 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2164d58952afc75208052cc61340db0 with result from task 3342 parsl.dataflow.dflow DEBUG: Task 3342 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e665c0ab-4fb2-4bd9-820a-c815c621f54b_isrForCrosstalkSources_996964_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e665c0ab-4fb2-4bd9-820a-c815c621f54b_isrForCrosstalkSources_996964_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075065 PD 17075066 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3349 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry e4f6c444b577ef79cb2a96d69bd51f2d with result from task 3349 parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3349 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/09504192-8cf6-4289-b384-abe37101c052_isrForCrosstalkSources_996998_18.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/09504192-8cf6-4289-b384-abe37101c052_isrForCrosstalkSources_996998_18.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3413 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 9ee82fd8b2b45bf2b97c3b0d5269bf82 with result from task 3413 parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3413 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/22a1a4a4-e31d-47dc-8640-6e886d08171d_isrForCrosstalkSources_997005_36.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/22a1a4a4-e31d-47dc-8640-6e886d08171d_isrForCrosstalkSources_997005_36.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3324 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry b8df22a1139fafa21f4a82620cce5ec8 with result from task 3324 parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3324 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/a4cf3674-29da-4562-991b-99eb67f2a881_isrForCrosstalkSources_996971_59.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/a4cf3674-29da-4562-991b-99eb67f2a881_isrForCrosstalkSources_996971_59.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3653 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 643463f12b31df691ecb5537dde25028 with result from task 3653 parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3653 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5ab34ba9-0a45-443d-9c16-f7f7e7fc3ae7_isrForCrosstalkSources_996999_6.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5ab34ba9-0a45-443d-9c16-f7f7e7fc3ae7_isrForCrosstalkSources_996999_6.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3391 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 5fd6a381179c818e8f088a03cd9ea929 with result from task 3391 parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 3391 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 3391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0f5283ee-6cb1-4055-9448-a573edeb52eb_isrForCrosstalkSources_996991_2.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 3391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/0f5283ee-6cb1-4055-9448-a573edeb52eb_isrForCrosstalkSources_996991_2.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 3669 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 19d187ca90a2325840f5d0f1c3d6e43e with result from task 3669 parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 3669 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 3669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/a8ddd2e1-4dd9-4fc9-b3d4-a4d107fa5ecf_isrForCrosstalkSources_997024_20.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 3669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/a8ddd2e1-4dd9-4fc9-b3d4-a4d107fa5ecf_isrForCrosstalkSources_997024_20.stderr parsl.dataflow.dflow INFO: Task 3567 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 847e6c9b33f08609081aa33ec4eb717b with result from task 3567 parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 3567 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 3567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6c2adae9-97a9-4242-b097-8173aa9b481c_isrForCrosstalkSources_996970_23.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 3567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6c2adae9-97a9-4242-b097-8173aa9b481c_isrForCrosstalkSources_996970_23.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:20 proc_lsst.multi:146 [INFO] found job 17075063 in provider checkpoint parsl.dataflow.dflow INFO: Task 3690 completed (launched -> exec_done) proc_lsst.multi INFO: found job 17075063 in provider checkpoint parsl.dataflow.memoization DEBUG: Storing app cache entry 32b3e8ab6b61ccdf976d3142f9fdecc3 with result from task 3690 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 3690 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/2c546be9-5f30-4d76-8867-a3965627f205_isrForCrosstalkSources_997031_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/2c546be9-5f30-4d76-8867-a3965627f205_isrForCrosstalkSources_997031_25.stderr parsl.dataflow.dflow INFO: Task 3657 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 45c6f96d701687bffbb3656b1781d7c7 with result from task 3657 parsl.dataflow.dflow DEBUG: Task 3657 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/44b6f5cc-5748-4fef-95f7-f9b6c05edd6f_isrForCrosstalkSources_996975_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/44b6f5cc-5748-4fef-95f7-f9b6c05edd6f_isrForCrosstalkSources_996975_30.stderr parsl.dataflow.dflow INFO: Task 3340 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d076d50f2182bd336454065a47ad4308 with result from task 3340 parsl.dataflow.dflow DEBUG: Task 3340 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/60220f04-daa0-414f-b3b6-0582b28dc9a9_isrForCrosstalkSources_996984_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/60220f04-daa0-414f-b3b6-0582b28dc9a9_isrForCrosstalkSources_996984_30.stderr parsl.dataflow.dflow INFO: Task 3561 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 327f5f47acc380fe2b22afcff6813a57 with result from task 3561 parsl.dataflow.dflow DEBUG: Task 3561 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/c6c1c150-e46a-4064-9841-5bd0d989d6ba_isrForCrosstalkSources_997023_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/c6c1c150-e46a-4064-9841-5bd0d989d6ba_isrForCrosstalkSources_997023_46.stderr parsl.dataflow.dflow INFO: Task 3648 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1487ad9d7cce7ec59f82ce27b5605a73 with result from task 3648 parsl.dataflow.dflow DEBUG: Task 3648 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/90e6167a-d74e-4468-bc31-4a91a8dfb96a_isrForCrosstalkSources_996972_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/90e6167a-d74e-4468-bc31-4a91a8dfb96a_isrForCrosstalkSources_996972_33.stderr parsl.dataflow.dflow INFO: Task 3449 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 37f5bbeece8c2f301ca115de57fab019 with result from task 3449 parsl.dataflow.dflow DEBUG: Task 3449 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/e8a57b92-16bc-45ad-b88c-e48c989ba930_isrForCrosstalkSources_996977_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/e8a57b92-16bc-45ad-b88c-e48c989ba930_isrForCrosstalkSources_996977_7.stderr parsl.dataflow.dflow INFO: Task 3368 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a7f92b487fc873930a77454e967508a5 with result from task 3368 parsl.dataflow.dflow DEBUG: Task 3368 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/89d4232d-8c16-4bb1-9347-136ebaa87759_isrForCrosstalkSources_996974_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/89d4232d-8c16-4bb1-9347-136ebaa87759_isrForCrosstalkSources_996974_54.stderr parsl.dataflow.dflow INFO: Task 3285 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38e98ac3b4f48d23a9890efcc01e30d5 with result from task 3285 parsl.dataflow.dflow DEBUG: Task 3285 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/76093ac7-8de8-488a-9c5b-93b3e2fe8e0d_isrForCrosstalkSources_996976_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/76093ac7-8de8-488a-9c5b-93b3e2fe8e0d_isrForCrosstalkSources_996976_55.stderr parsl.dataflow.dflow INFO: Task 3670 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d00a040fb61a22610b45002cb774497 with result from task 3670 parsl.dataflow.dflow DEBUG: Task 3670 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/8176f043-cda2-4028-b237-6e6044795be5_isrForCrosstalkSources_996980_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/8176f043-cda2-4028-b237-6e6044795be5_isrForCrosstalkSources_996980_39.stderr parsl.dataflow.dflow INFO: Task 3345 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c03dbcdf130bb3f734946e2495923f23 with result from task 3345 parsl.dataflow.dflow DEBUG: Task 3345 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b43841d9-063e-4347-9dfb-67ed4ad33a05_isrForCrosstalkSources_997008_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b43841d9-063e-4347-9dfb-67ed4ad33a05_isrForCrosstalkSources_997008_57.stderr parsl.dataflow.dflow INFO: Task 3580 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62e8fee018dcda6b8802a7673802a5eb with result from task 3580 parsl.dataflow.dflow DEBUG: Task 3580 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1a76a4d7-43ad-4670-9035-c9a267122720_isrForCrosstalkSources_997017_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1a76a4d7-43ad-4670-9035-c9a267122720_isrForCrosstalkSources_997017_32.stderr parsl.dataflow.dflow INFO: Task 3677 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b309da98a9c7c8755b3a281afaee898 with result from task 3677 parsl.dataflow.dflow DEBUG: Task 3677 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f2b9b519-4005-442e-b457-27a8b8a2bbbc_isrForCrosstalkSources_996967_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/f2b9b519-4005-442e-b457-27a8b8a2bbbc_isrForCrosstalkSources_996967_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3606 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 61cb73783fd704098901d8e1adb30821 with result from task 3606 parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3606 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/321b89c8-740e-4263-b858-0aec21f3c81e_isrForCrosstalkSources_997002_20.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/321b89c8-740e-4263-b858-0aec21f3c81e_isrForCrosstalkSources_997002_20.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3683 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 18f2c1bfb370178162b1319824d0ee87 with result from task 3683 parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3683 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/334ab2b7-84d0-43dc-b4ed-5de52e2a3e96_isrForCrosstalkSources_997021_9.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/334ab2b7-84d0-43dc-b4ed-5de52e2a3e96_isrForCrosstalkSources_997021_9.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:21 proc_lsst.multi:146 [INFO] found job 17075064 in provider checkpoint proc_lsst.multi INFO: found job 17075064 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3636 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6bc5413443b4edad9deb977a35fdb42 with result from task 3636 parsl.dataflow.dflow DEBUG: Task 3636 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/9df6fcda-9dda-49e9-ac24-13f261d68cda_isrForCrosstalkSources_996989_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/9df6fcda-9dda-49e9-ac24-13f261d68cda_isrForCrosstalkSources_996989_37.stderr parsl.dataflow.dflow INFO: Task 3503 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 92b39385e7b26c0b306f220825e7dc4b with result from task 3503 parsl.dataflow.dflow DEBUG: Task 3503 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/016780c3-4164-4dfc-98ab-f3a9d520684d_isrForCrosstalkSources_996957_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/016780c3-4164-4dfc-98ab-f3a9d520684d_isrForCrosstalkSources_996957_21.stderr parsl.dataflow.dflow INFO: Task 3372 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7f8247e367210e2303c5a8bbd186703f with result from task 3372 parsl.dataflow.dflow DEBUG: Task 3372 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/f153a0eb-13ef-47da-8ba4-23ecb0f49620_isrForCrosstalkSources_996963_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/f153a0eb-13ef-47da-8ba4-23ecb0f49620_isrForCrosstalkSources_996963_27.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 3688 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 050331c4d52e4ff3f335423552b457ca with result from task 3688 parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3688 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/7265c119-4e98-4a1a-8a3b-3805a8ff8cc2_isrForCrosstalkSources_997010_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/7265c119-4e98-4a1a-8a3b-3805a8ff8cc2_isrForCrosstalkSources_997010_7.stderr parsl.dataflow.dflow INFO: Task 3235 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a6072decff83a732a31bcb0f6805109 with result from task 3235 parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3235 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/488a7fa9-5740-4806-8ad7-3712178010f7_isrForCrosstalkSources_997009_60.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/488a7fa9-5740-4806-8ad7-3712178010f7_isrForCrosstalkSources_997009_60.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3691 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry e56cfad57ef80735752ee8d0189adee2 with result from task 3691 parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3691 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/62a743e9-9c91-47a6-b5a5-674bb536207f_isrForCrosstalkSources_997009_62.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/62a743e9-9c91-47a6-b5a5-674bb536207f_isrForCrosstalkSources_997009_62.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3666 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 3683bfc7b8d4bc1e61b013780dd8508e with result from task 3666 parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3666 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c5b0207d-a17e-4aa6-9c01-b761663b87d7_isrForCrosstalkSources_997021_25.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c5b0207d-a17e-4aa6-9c01-b761663b87d7_isrForCrosstalkSources_997021_25.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:22 proc_lsst.multi:146 [INFO] found job 17075065 in provider checkpoint proc_lsst.multi INFO: found job 17075065 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3535 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4adb67237f03484ab2446e788284be9a with result from task 3535 parsl.dataflow.dflow DEBUG: Task 3535 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ecdccfd9-ee16-425e-ab26-d70fa15a4616_isrForCrosstalkSources_996994_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ecdccfd9-ee16-425e-ab26-d70fa15a4616_isrForCrosstalkSources_996994_37.stderr parsl.dataflow.dflow INFO: Task 3694 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d6d1a56fab764d02b4d7bd616ab22f5 with result from task 3694 parsl.dataflow.dflow DEBUG: Task 3694 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/fbbc96b4-ed3b-4c66-a4b9-fd74a0f77a6c_isrForCrosstalkSources_997022_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/fbbc96b4-ed3b-4c66-a4b9-fd74a0f77a6c_isrForCrosstalkSources_997022_17.stderr parsl.dataflow.dflow INFO: Task 3538 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e168aa77fc2773e7c9d6d2c1ac37abb with result from task 3538 parsl.dataflow.dflow DEBUG: Task 3538 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/583648eb-bb24-4e21-8f6b-6efa0a22e842_isrForCrosstalkSources_997006_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/583648eb-bb24-4e21-8f6b-6efa0a22e842_isrForCrosstalkSources_997006_26.stderr parsl.dataflow.dflow INFO: Task 3370 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94e41ad5915f4e45594174af1b1a20ad with result from task 3370 parsl.dataflow.dflow DEBUG: Task 3370 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8aa5c0f6-bc18-42a4-b6da-4d3fb5743fcb_isrForCrosstalkSources_996976_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/8aa5c0f6-bc18-42a4-b6da-4d3fb5743fcb_isrForCrosstalkSources_996976_8.stderr parsl.dataflow.dflow INFO: Task 3473 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f527e265f409ff93230dd3b1ef555338 with result from task 3473 parsl.dataflow.dflow DEBUG: Task 3473 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3685e924-829c-44ec-8eec-9a81b863301c_isrForCrosstalkSources_996968_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3685e924-829c-44ec-8eec-9a81b863301c_isrForCrosstalkSources_996968_13.stderr parsl.dataflow.dflow INFO: Task 3679 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 551c2f3739709baaaa3f2a0aa696950a with result from task 3679 parsl.dataflow.dflow DEBUG: Task 3679 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/45a452f3-d5e6-470c-83a2-e66ad83ba501_isrForCrosstalkSources_996987_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/45a452f3-d5e6-470c-83a2-e66ad83ba501_isrForCrosstalkSources_996987_18.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3454 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 6925a09b823f1daa42553d9ad4b0f135 with result from task 3454 parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3454 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/870ff890-9133-44d8-b329-11acad383020_isrForCrosstalkSources_997000_46.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/870ff890-9133-44d8-b329-11acad383020_isrForCrosstalkSources_997000_46.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3526 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 5a0a04055dc110eeeee5b193fbf89de1 with result from task 3526 parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3526 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e2625eaf-a3b5-4f58-b514-4d2beae5ddd6_isrForCrosstalkSources_997008_40.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e2625eaf-a3b5-4f58-b514-4d2beae5ddd6_isrForCrosstalkSources_997008_40.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3575 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry e2b456f249ff0c540c095afbab9dbf51 with result from task 3575 parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3575 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/a59caae6-2065-42d1-bd15-30edaa63c3d2_isrForCrosstalkSources_996958_17.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/a59caae6-2065-42d1-bd15-30edaa63c3d2_isrForCrosstalkSources_996958_17.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3680 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry bed04b42431f534156a2cdd795c6b4c9 with result from task 3680 parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3680 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e5dc8360-1ed7-4092-96e7-312c3ad3d8aa_isrForCrosstalkSources_996995_7.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e5dc8360-1ed7-4092-96e7-312c3ad3d8aa_isrForCrosstalkSources_996995_7.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:23 proc_lsst.multi:146 [INFO] found job 17075066 in provider checkpoint proc_lsst.multi INFO: found job 17075066 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3701 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b76220e70d7a29bfcb6ac164b5ca0cf3 with result from task 3701 parsl.dataflow.dflow DEBUG: Task 3701 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/dce83fe1-58c8-4ec4-b84d-a3f904df67df_isrForCrosstalkSources_996991_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/dce83fe1-58c8-4ec4-b84d-a3f904df67df_isrForCrosstalkSources_996991_55.stderr parsl.dataflow.dflow INFO: Task 3375 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02238c9ab4af668166506a0e3d071296 with result from task 3375 parsl.dataflow.dflow DEBUG: Task 3375 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d9080eb4-5eb1-4d4e-a3ae-50c7ad2b6a95_isrForCrosstalkSources_996971_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/d9080eb4-5eb1-4d4e-a3ae-50c7ad2b6a95_isrForCrosstalkSources_996971_60.stderr parsl.dataflow.dflow INFO: Task 3502 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b91adabe8244f5257863d594cf4ad367 with result from task 3502 parsl.dataflow.dflow DEBUG: Task 3502 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/4b377648-2447-45cf-9000-5e35b81e29a3_isrForCrosstalkSources_997012_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/4b377648-2447-45cf-9000-5e35b81e29a3_isrForCrosstalkSources_997012_40.stderr parsl.dataflow.dflow INFO: Task 3531 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8a69d3893dd924c2e594169ef7aba49 with result from task 3531 parsl.dataflow.dflow DEBUG: Task 3531 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6b2ecbb5-a9fb-4a4a-9adc-c326536d06a8_isrForCrosstalkSources_996996_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6b2ecbb5-a9fb-4a4a-9adc-c326536d06a8_isrForCrosstalkSources_996996_51.stderr parsl.dataflow.dflow INFO: Task 3712 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7498b480758f618d806241ba06e32d71 with result from task 3712 parsl.dataflow.dflow DEBUG: Task 3712 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/9d16245e-8e34-4866-9094-3b59f5179b2a_isrForCrosstalkSources_997021_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/9d16245e-8e34-4866-9094-3b59f5179b2a_isrForCrosstalkSources_997021_11.stderr parsl.dataflow.dflow INFO: Task 3632 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 544f2526a60031b5e8b3fcd6736288d4 with result from task 3632 parsl.dataflow.dflow DEBUG: Task 3632 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b689fdd8-8878-439c-ae2c-fa39dfc6bf2b_isrForCrosstalkSources_996964_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/b689fdd8-8878-439c-ae2c-fa39dfc6bf2b_isrForCrosstalkSources_996964_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3541 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 7fc42594a28091963f1da043fd156269 with result from task 3541 parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3541 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e8e5196c-1c0e-4e56-9773-52b5e22a6698_isrForCrosstalkSources_996956_35.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e8e5196c-1c0e-4e56-9773-52b5e22a6698_isrForCrosstalkSources_996956_35.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:24 proc_lsst.multi:146 [INFO] found job 17075068 in provider checkpoint proc_lsst.multi INFO: found job 17075068 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3555 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 805a7704a64d690ed50228693ce9620a with result from task 3555 parsl.dataflow.dflow DEBUG: Task 3555 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/92ea1f2a-f1af-4104-b589-31653d1bd198_isrForCrosstalkSources_997032_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/92ea1f2a-f1af-4104-b589-31653d1bd198_isrForCrosstalkSources_997032_33.stderr parsl.dataflow.dflow INFO: Task 3667 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f6e5c18ab322682d434b610d23f4e4ce with result from task 3667 parsl.dataflow.dflow DEBUG: Task 3667 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2dd4b338-c8b8-4543-98e6-47e34235331a_isrForCrosstalkSources_997015_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/2dd4b338-c8b8-4543-98e6-47e34235331a_isrForCrosstalkSources_997015_8.stderr parsl.dataflow.dflow INFO: Task 3460 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9aa5deb3e0095d4515453ec930a009bf with result from task 3460 parsl.dataflow.dflow DEBUG: Task 3460 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/699c1ec7-574e-4f61-be27-fbbef3605c17_isrForCrosstalkSources_996987_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/699c1ec7-574e-4f61-be27-fbbef3605c17_isrForCrosstalkSources_996987_28.stderr parsl.dataflow.dflow INFO: Task 3564 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96d3190a4a5d969542ab53c8a3ed11e0 with result from task 3564 parsl.dataflow.dflow DEBUG: Task 3564 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/4d681975-6371-4323-a149-001514e24109_isrForCrosstalkSources_996968_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/4d681975-6371-4323-a149-001514e24109_isrForCrosstalkSources_996968_31.stderr parsl.dataflow.dflow INFO: Task 3588 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c119b8c2fc5c76558071dda5f775bcd2 with result from task 3588 parsl.dataflow.dflow DEBUG: Task 3588 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e896f523-2444-49aa-b289-b618dfa10d5d_isrForCrosstalkSources_996992_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e896f523-2444-49aa-b289-b618dfa10d5d_isrForCrosstalkSources_996992_10.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:25 proc_lsst.multi:146 [INFO] found job 17075069 in provider checkpoint proc_lsst.multi INFO: found job 17075069 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3652 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ae3a41875e6582424752b63846b6dc3 with result from task 3652 parsl.dataflow.dflow DEBUG: Task 3652 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/1d8f2348-2599-472c-8c2c-614a5587237b_isrForCrosstalkSources_997029_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/1d8f2348-2599-472c-8c2c-614a5587237b_isrForCrosstalkSources_997029_45.stderr parsl.dataflow.dflow INFO: Task 3630 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fcf30e2de78c6aa0dda01f35279b244 with result from task 3630 parsl.dataflow.dflow DEBUG: Task 3630 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/57f30ef9-a085-4b9f-8111-61f071177f26_isrForCrosstalkSources_996989_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/57f30ef9-a085-4b9f-8111-61f071177f26_isrForCrosstalkSources_996989_22.stderr parsl.dataflow.dflow INFO: Task 3633 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4e6a0a5bbb1bac0cd260f5e82fb89de with result from task 3633 parsl.dataflow.dflow DEBUG: Task 3633 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/182c2d24-5493-466f-a162-81330283a8bb_isrForCrosstalkSources_996959_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/182c2d24-5493-466f-a162-81330283a8bb_isrForCrosstalkSources_996959_4.stderr parsl.dataflow.dflow INFO: Task 3715 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 823997420231aa4053d5ffba2c8a9361 with result from task 3715 parsl.dataflow.dflow DEBUG: Task 3715 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0d1f8ebf-7b2c-40f2-8ea0-2be002955714_isrForCrosstalkSources_997028_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0d1f8ebf-7b2c-40f2-8ea0-2be002955714_isrForCrosstalkSources_997028_55.stderr parsl.dataflow.dflow INFO: Task 3643 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9828eb72809da65e3a363d4f7771dbea with result from task 3643 parsl.dataflow.dflow DEBUG: Task 3643 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/9fd0d4a8-2b9b-42eb-9151-501a14fb4514_isrForCrosstalkSources_996998_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/9fd0d4a8-2b9b-42eb-9151-501a14fb4514_isrForCrosstalkSources_996998_51.stderr parsl.dataflow.dflow INFO: Task 3357 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d1f1ee61f254494134709f5ecdd6e102 with result from task 3357 parsl.dataflow.dflow DEBUG: Task 3357 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/4a332746-74e0-4446-890c-21c4a75b327e_isrForCrosstalkSources_996953_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/4a332746-74e0-4446-890c-21c4a75b327e_isrForCrosstalkSources_996953_22.stderr parsl.dataflow.dflow INFO: Task 3610 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 272382b3f1a6e19cdaa08dbdbc70cfd5 with result from task 3610 parsl.dataflow.dflow DEBUG: Task 3610 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/7f265e9d-0d8e-4119-bd9f-3287842cd279_isrForCrosstalkSources_996956_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/7f265e9d-0d8e-4119-bd9f-3287842cd279_isrForCrosstalkSources_996956_55.stderr parsl.dataflow.dflow INFO: Task 3735 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ea9ed1f0591562849116d92dc8e4eb2 with result from task 3735 parsl.dataflow.dflow DEBUG: Task 3735 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/25a01807-d7a9-41a0-8766-aa89a9c051f6_isrForCrosstalkSources_996977_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/25a01807-d7a9-41a0-8766-aa89a9c051f6_isrForCrosstalkSources_996977_53.stderr parsl.dataflow.dflow INFO: Task 3697 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 09edf2602ed563b57177fee109dc8ae9 with result from task 3697 parsl.dataflow.dflow DEBUG: Task 3697 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/5603cc04-f8bb-474a-93fb-5df56f10a398_isrForCrosstalkSources_996992_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/5603cc04-f8bb-474a-93fb-5df56f10a398_isrForCrosstalkSources_996992_22.stderr parsl.dataflow.dflow INFO: Task 3685 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 286267a45c9efdb39ab272bc15bc1caa with result from task 3685 parsl.dataflow.dflow DEBUG: Task 3685 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4317e33e-a28a-4d8d-9a94-172adce0f11a_isrForCrosstalkSources_997018_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/4317e33e-a28a-4d8d-9a94-172adce0f11a_isrForCrosstalkSources_997018_51.stderr parsl.dataflow.dflow INFO: Task 3708 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56d030ca3b8aba396b0e859eedfa3d36 with result from task 3708 parsl.dataflow.dflow DEBUG: Task 3708 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/76f4406c-ecbc-4c9e-ab43-93797eb1f93d_isrForCrosstalkSources_996982_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/76f4406c-ecbc-4c9e-ab43-93797eb1f93d_isrForCrosstalkSources_996982_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 CG 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:26 proc_lsst.multi:146 [INFO] found job 17075072 in provider checkpoint proc_lsst.multi INFO: found job 17075072 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3627 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a0a49410a3509305ed43e3e3c8042bc3 with result from task 3627 parsl.dataflow.dflow DEBUG: Task 3627 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7da84138-18e2-44c6-9916-2c21162835c0_isrForCrosstalkSources_997027_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7da84138-18e2-44c6-9916-2c21162835c0_isrForCrosstalkSources_997027_12.stderr parsl.dataflow.dflow INFO: Task 3520 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a4bd04f178b22323d3f9d62c76dd01d with result from task 3520 parsl.dataflow.dflow DEBUG: Task 3520 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/9f9bfee8-d276-4154-a472-e83edfc44dec_isrForCrosstalkSources_996982_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/9f9bfee8-d276-4154-a472-e83edfc44dec_isrForCrosstalkSources_996982_62.stderr parsl.dataflow.dflow INFO: Task 3548 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3f34fa4a1b108f48e50eaeb0f0423de with result from task 3548 parsl.dataflow.dflow DEBUG: Task 3548 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/7e37b2da-4b3b-4123-b231-e3a508bdd068_isrForCrosstalkSources_996967_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/7e37b2da-4b3b-4123-b231-e3a508bdd068_isrForCrosstalkSources_996967_7.stderr parsl.dataflow.dflow INFO: Task 3361 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 084a2face063f59f22b1c57b8b692813 with result from task 3361 parsl.dataflow.dflow DEBUG: Task 3361 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/82b8a0c2-8e51-4674-88e0-7892005e211f_isrForCrosstalkSources_996980_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/82b8a0c2-8e51-4674-88e0-7892005e211f_isrForCrosstalkSources_996980_36.stderr parsl.dataflow.dflow INFO: Task 3400 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c58b831a1fd9c00e3c5efbaf344fba5 with result from task 3400 parsl.dataflow.dflow DEBUG: Task 3400 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0a4c5f7b-dce1-47e7-b755-51be50781688_isrForCrosstalkSources_996980_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0a4c5f7b-dce1-47e7-b755-51be50781688_isrForCrosstalkSources_996980_49.stderr parsl.dataflow.dflow INFO: Task 3722 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 69056c6367ba00e4ce0a156e28a4c061 with result from task 3722 parsl.dataflow.dflow DEBUG: Task 3722 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/7b452528-41d5-444f-b8a9-58b32a143c27_isrForCrosstalkSources_996957_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/7b452528-41d5-444f-b8a9-58b32a143c27_isrForCrosstalkSources_996957_22.stderr parsl.dataflow.dflow INFO: Task 3602 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 92bbbdc8c6cf2c8053d040ca18b642cd with result from task 3602 parsl.dataflow.dflow DEBUG: Task 3602 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/e5df4e1d-dc37-4a62-b508-069c8bb807df_isrForCrosstalkSources_997003_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/e5df4e1d-dc37-4a62-b508-069c8bb807df_isrForCrosstalkSources_997003_12.stderr parsl.dataflow.dflow INFO: Task 3718 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cbf06d0d2f314a5a555e2e9ad3ff6a7c with result from task 3718 parsl.dataflow.dflow DEBUG: Task 3718 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/932d186c-aed7-4ea6-bf5b-d9c16960e962_isrForCrosstalkSources_997012_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/932d186c-aed7-4ea6-bf5b-d9c16960e962_isrForCrosstalkSources_997012_8.stderr parsl.dataflow.dflow INFO: Task 3675 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca62b9b3ce5d6a083b8e569227d04b7c with result from task 3675 parsl.dataflow.dflow DEBUG: Task 3675 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/83083d77-1904-4602-a46b-6305113c2d2f_isrForCrosstalkSources_997016_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/83083d77-1904-4602-a46b-6305113c2d2f_isrForCrosstalkSources_997016_46.stderr parsl.dataflow.dflow INFO: Task 3635 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b500e6beabc7dc157093c5c5d7a318a with result from task 3635 parsl.dataflow.dflow DEBUG: Task 3635 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/1b47a674-d48e-4ddc-8393-3ab6f6cfed58_isrForCrosstalkSources_997018_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/1b47a674-d48e-4ddc-8393-3ab6f6cfed58_isrForCrosstalkSources_997018_27.stderr parsl.dataflow.dflow INFO: Task 3710 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31afd4611ee84c858050ee2d0c431488 with result from task 3710 parsl.dataflow.dflow DEBUG: Task 3710 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1f305f05-3e4d-41f6-a3ed-3ca45433ce9d_isrForCrosstalkSources_997019_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/1f305f05-3e4d-41f6-a3ed-3ca45433ce9d_isrForCrosstalkSources_997019_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:27 proc_lsst.multi:146 [INFO] found job 17075073 in provider checkpoint proc_lsst.multi INFO: found job 17075073 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3615 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed5abfcc18d5713f8526fe45bfc611d0 with result from task 3615 parsl.dataflow.dflow DEBUG: Task 3615 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/67db2923-b436-478c-9e33-945171411f03_isrForCrosstalkSources_997003_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/67db2923-b436-478c-9e33-945171411f03_isrForCrosstalkSources_997003_48.stderr parsl.dataflow.dflow INFO: Task 3672 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af3214e1afcaa7f59db6ecdb53aae036 with result from task 3672 parsl.dataflow.dflow DEBUG: Task 3672 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/da413efb-6f21-4035-8e7c-26678bb3db6e_isrForCrosstalkSources_996961_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/da413efb-6f21-4035-8e7c-26678bb3db6e_isrForCrosstalkSources_996961_18.stderr parsl.dataflow.dflow INFO: Task 3488 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19fffa7edf8874a280633007cd4aab40 with result from task 3488 parsl.dataflow.dflow DEBUG: Task 3488 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/6b6b9c6d-7675-418f-a277-f490fa6fcb55_isrForCrosstalkSources_997032_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/6b6b9c6d-7675-418f-a277-f490fa6fcb55_isrForCrosstalkSources_997032_50.stderr parsl.dataflow.dflow INFO: Task 3674 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 676ee5da8d217de82713cf90aa19b5b7 with result from task 3674 parsl.dataflow.dflow DEBUG: Task 3674 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f4430a0a-97d1-4fe1-9905-6db944edde0e_isrForCrosstalkSources_996980_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/f4430a0a-97d1-4fe1-9905-6db944edde0e_isrForCrosstalkSources_996980_18.stderr parsl.dataflow.dflow INFO: Task 3728 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0bafbcbf7dedc4004bee0b853ddc4d2 with result from task 3728 parsl.dataflow.dflow DEBUG: Task 3728 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/f5080978-a0f6-4589-b2ac-1282100e3252_isrForCrosstalkSources_996982_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/f5080978-a0f6-4589-b2ac-1282100e3252_isrForCrosstalkSources_996982_23.stderr parsl.dataflow.dflow INFO: Task 3733 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec700dc87ab3e9a0d5936859694a976e with result from task 3733 parsl.dataflow.dflow DEBUG: Task 3733 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/93137248-c716-40b1-a1bb-c0b01650d945_isrForCrosstalkSources_996963_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/93137248-c716-40b1-a1bb-c0b01650d945_isrForCrosstalkSources_996963_1.stderr parsl.dataflow.dflow INFO: Task 3651 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aeac6e874fa2cd0d078f9f20a6514466 with result from task 3651 parsl.dataflow.dflow DEBUG: Task 3651 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d0dff02d-082e-4aaf-bc95-9bc80354072d_isrForCrosstalkSources_997024_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d0dff02d-082e-4aaf-bc95-9bc80354072d_isrForCrosstalkSources_997024_10.stderr parsl.dataflow.dflow INFO: Task 3740 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93fdc263c7085f55bf4489ab189019b4 with result from task 3740 parsl.dataflow.dflow DEBUG: Task 3740 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7d5af3fe-f3cb-4db7-b847-361257ae7133_isrForCrosstalkSources_996993_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7d5af3fe-f3cb-4db7-b847-361257ae7133_isrForCrosstalkSources_996993_4.stderr parsl.dataflow.dflow INFO: Task 3611 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d481f55d00b3787a4da1bc50a33457a with result from task 3611 parsl.dataflow.dflow DEBUG: Task 3611 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/bd91a66c-da8f-4e60-8fdd-ef57b744e83c_isrForCrosstalkSources_996961_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/bd91a66c-da8f-4e60-8fdd-ef57b744e83c_isrForCrosstalkSources_996961_25.stderr parsl.dataflow.dflow INFO: Task 3571 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f423a0cc7a9cd5f95cb6dc3ee65f3212 with result from task 3571 parsl.dataflow.dflow DEBUG: Task 3571 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/d63b0f0b-8f57-4a37-b51c-0a137a318880_isrForCrosstalkSources_997014_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/d63b0f0b-8f57-4a37-b51c-0a137a318880_isrForCrosstalkSources_997014_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:28 proc_lsst.multi:146 [INFO] found job 17075074 in provider checkpoint proc_lsst.multi INFO: found job 17075074 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3604 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e2409e88020e917b3048517913639c6 with result from task 3604 parsl.dataflow.dflow DEBUG: Task 3604 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8671b32e-596f-46e4-81aa-e67afaefa3fc_isrForCrosstalkSources_996992_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/8671b32e-596f-46e4-81aa-e67afaefa3fc_isrForCrosstalkSources_996992_20.stderr parsl.dataflow.dflow INFO: Task 3528 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fed7c461593df09bf5c7c24bc05450e with result from task 3528 parsl.dataflow.dflow DEBUG: Task 3528 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0c79b297-d127-41f2-8e59-15835308c943_isrForCrosstalkSources_996985_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0c79b297-d127-41f2-8e59-15835308c943_isrForCrosstalkSources_996985_24.stderr parsl.dataflow.dflow INFO: Task 3563 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f1e7e008f968408b14b02dd578d6216 with result from task 3563 parsl.dataflow.dflow DEBUG: Task 3563 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/9edc8f01-dd3f-468b-83cc-9487523da59b_isrForCrosstalkSources_996974_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/9edc8f01-dd3f-468b-83cc-9487523da59b_isrForCrosstalkSources_996974_30.stderr parsl.dataflow.dflow INFO: Task 3533 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa74ea0098e999bc6320b32cfc270eef with result from task 3533 parsl.dataflow.dflow DEBUG: Task 3533 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/30905291-635d-440b-a542-ca59b0a238ba_isrForCrosstalkSources_997029_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/30905291-635d-440b-a542-ca59b0a238ba_isrForCrosstalkSources_997029_39.stderr parsl.dataflow.dflow INFO: Task 3515 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 541f2da213e309ebb54dab9d07369841 with result from task 3515 parsl.dataflow.dflow DEBUG: Task 3515 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/64bce767-b592-4941-a994-e98901910e71_isrForCrosstalkSources_997021_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/64bce767-b592-4941-a994-e98901910e71_isrForCrosstalkSources_997021_54.stderr parsl.dataflow.dflow INFO: Task 3647 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 49cb588faccd3a56c67493fd91102dac with result from task 3647 parsl.dataflow.dflow DEBUG: Task 3647 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0b3ed7b1-b92c-480d-8fac-18fba4f79c55_isrForCrosstalkSources_997028_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0b3ed7b1-b92c-480d-8fac-18fba4f79c55_isrForCrosstalkSources_997028_35.stderr parsl.dataflow.dflow INFO: Task 3566 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e85ecb01d6013852e1c5b447fc7a6503 with result from task 3566 parsl.dataflow.dflow DEBUG: Task 3566 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/5ff32d6b-eca8-4559-8f1c-f625f238af65_isrForCrosstalkSources_996974_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/5ff32d6b-eca8-4559-8f1c-f625f238af65_isrForCrosstalkSources_996974_60.stderr parsl.dataflow.dflow INFO: Task 3671 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15d435ec682d0fcfad51d018b052b4c6 with result from task 3671 parsl.dataflow.dflow DEBUG: Task 3671 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/7fc3e582-e2f2-4a94-8c60-576d84f5088d_isrForCrosstalkSources_997022_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/7fc3e582-e2f2-4a94-8c60-576d84f5088d_isrForCrosstalkSources_997022_35.stderr parsl.dataflow.dflow INFO: Task 3726 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f72fb81403036fd0f04d35b15caea64f with result from task 3726 parsl.dataflow.dflow DEBUG: Task 3726 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c930329b-c53b-4952-b898-ec49b9081403_isrForCrosstalkSources_996969_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c930329b-c53b-4952-b898-ec49b9081403_isrForCrosstalkSources_996969_33.stderr parsl.dataflow.dflow INFO: Task 3359 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 675e9a68ad0a3e587861c4494a6f135c with result from task 3359 parsl.dataflow.dflow DEBUG: Task 3359 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e877cdea-354a-4101-b11f-254839a74c95_isrForCrosstalkSources_997032_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e877cdea-354a-4101-b11f-254839a74c95_isrForCrosstalkSources_997032_54.stderr parsl.dataflow.dflow INFO: Task 3367 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fdf3172e27f37a2324e04cfb89d1c01 with result from task 3367 parsl.dataflow.dflow DEBUG: Task 3367 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6d5caa47-b85b-4b7f-96a5-806c9b6078e9_isrForCrosstalkSources_996963_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/6d5caa47-b85b-4b7f-96a5-806c9b6078e9_isrForCrosstalkSources_996963_59.stderr parsl.dataflow.dflow INFO: Task 3709 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fab7867272fe3b5d2f93f376f3e805b with result from task 3709 parsl.dataflow.dflow DEBUG: Task 3709 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/eb4fae1a-85c6-4c24-af87-6f7cf81b7cf3_isrForCrosstalkSources_997000_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/eb4fae1a-85c6-4c24-af87-6f7cf81b7cf3_isrForCrosstalkSources_997000_7.stderr parsl.dataflow.dflow INFO: Task 3662 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7df2bd4d90da64b04451e058daeca3b with result from task 3662 parsl.dataflow.dflow DEBUG: Task 3662 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d8591211-ce92-462c-af03-8eb82cf8ab7d_isrForCrosstalkSources_997003_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/d8591211-ce92-462c-af03-8eb82cf8ab7d_isrForCrosstalkSources_997003_50.stderr parsl.dataflow.dflow INFO: Task 3659 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13ef57900ca4de8b8b0a9ec8db7104d0 with result from task 3659 parsl.dataflow.dflow DEBUG: Task 3659 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5af08bc9-4bc7-419d-9c9b-c92ad1cbcffc_isrForCrosstalkSources_996964_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/5af08bc9-4bc7-419d-9c9b-c92ad1cbcffc_isrForCrosstalkSources_996964_9.stderr parsl.dataflow.dflow INFO: Task 3706 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 714f6b6b6e15650e65243a5698d83b77 with result from task 3706 parsl.dataflow.dflow DEBUG: Task 3706 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ecbfa483-dac5-4a42-a916-73e39ef20bfa_isrForCrosstalkSources_997015_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ecbfa483-dac5-4a42-a916-73e39ef20bfa_isrForCrosstalkSources_997015_28.stderr parsl.dataflow.dflow INFO: Task 3668 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 92ca56ca52ae2c6792e91430bdd0b85e with result from task 3668 parsl.dataflow.dflow DEBUG: Task 3668 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9026d6b7-d744-407c-babd-99892866593d_isrForCrosstalkSources_996959_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9026d6b7-d744-407c-babd-99892866593d_isrForCrosstalkSources_996959_17.stderr parsl.dataflow.dflow INFO: Task 3699 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7532e3d3d2fca9778f2bc85dfd8f99eb with result from task 3699 parsl.dataflow.dflow DEBUG: Task 3699 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/48bfc154-c1ea-4491-94c8-ab0444abb97d_isrForCrosstalkSources_996985_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/48bfc154-c1ea-4491-94c8-ab0444abb97d_isrForCrosstalkSources_996985_23.stderr parsl.dataflow.dflow INFO: Task 3573 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fbd61cfc1f31a638290876b58a14db2 with result from task 3573 parsl.dataflow.dflow DEBUG: Task 3573 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/208afc97-32fe-4622-b552-06e43127a962_isrForCrosstalkSources_996990_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/208afc97-32fe-4622-b552-06e43127a962_isrForCrosstalkSources_996990_13.stderr parsl.dataflow.dflow INFO: Task 3717 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a3f050919b83fd36b6c7f4ab9ec384d9 with result from task 3717 parsl.dataflow.dflow DEBUG: Task 3717 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a400f9c7-f692-41b6-bddd-e2038e3b50d2_isrForCrosstalkSources_997033_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a400f9c7-f692-41b6-bddd-e2038e3b50d2_isrForCrosstalkSources_997033_10.stderr parsl.dataflow.dflow INFO: Task 3640 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 454542a1b8ac86d21b34606fc526e663 with result from task 3640 parsl.dataflow.dflow DEBUG: Task 3640 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b7cee98a-ae23-40d4-ac2a-a478dc17d5a9_isrForCrosstalkSources_997022_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b7cee98a-ae23-40d4-ac2a-a478dc17d5a9_isrForCrosstalkSources_997022_4.stderr parsl.dataflow.dflow INFO: Task 3771 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2fe2968c772c53850d753c9395abd153 with result from task 3771 parsl.dataflow.dflow DEBUG: Task 3771 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3ea19d43-d8ff-49dc-ba7a-233c110410d5_isrForCrosstalkSources_997019_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3ea19d43-d8ff-49dc-ba7a-233c110410d5_isrForCrosstalkSources_997019_25.stderr parsl.dataflow.dflow INFO: Task 3774 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f4f6848aceb727fd7a176ca6acdfc0a with result from task 3774 parsl.dataflow.dflow DEBUG: Task 3774 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1963cef7-b361-45ba-84e7-0181b570b1fc_isrForCrosstalkSources_996977_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1963cef7-b361-45ba-84e7-0181b570b1fc_isrForCrosstalkSources_996977_26.stderr parsl.dataflow.dflow INFO: Task 3764 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6edc21787f24d925ca45e4673ba2ae92 with result from task 3764 parsl.dataflow.dflow DEBUG: Task 3764 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c82dfea5-d8ce-454a-9ceb-35ea3736b518_isrForCrosstalkSources_997001_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c82dfea5-d8ce-454a-9ceb-35ea3736b518_isrForCrosstalkSources_997001_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:30 proc_lsst.multi:146 [INFO] found job 17075075 in provider checkpoint proc_lsst.multi INFO: found job 17075075 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3600 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b2822fd28dc99bf19a8f28b6d60ad57e with result from task 3600 parsl.dataflow.dflow DEBUG: Task 3600 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/cdf1c10f-91c8-472a-aefc-3a0e4c5aa197_isrForCrosstalkSources_997009_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/cdf1c10f-91c8-472a-aefc-3a0e4c5aa197_isrForCrosstalkSources_997009_19.stderr parsl.dataflow.dflow INFO: Task 3605 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3fb0e85080dd4e0c0335a58aedc80e4 with result from task 3605 parsl.dataflow.dflow DEBUG: Task 3605 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ef38add7-517f-4cdc-8aa6-5799be8d7423_isrForCrosstalkSources_997019_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ef38add7-517f-4cdc-8aa6-5799be8d7423_isrForCrosstalkSources_997019_40.stderr parsl.dataflow.dflow INFO: Task 3725 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7274c5fec0782fe390dd69b863590ea with result from task 3725 parsl.dataflow.dflow DEBUG: Task 3725 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/659e4f22-f777-48ff-8048-9f3b0a6096b4_isrForCrosstalkSources_997006_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/659e4f22-f777-48ff-8048-9f3b0a6096b4_isrForCrosstalkSources_997006_5.stderr parsl.dataflow.dflow INFO: Task 3518 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 307637eacd4b8f1450fea4153aff052f with result from task 3518 parsl.dataflow.dflow DEBUG: Task 3518 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/dec44a4b-6544-4285-bca7-4eea9a8f443d_isrForCrosstalkSources_996998_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/dec44a4b-6544-4285-bca7-4eea9a8f443d_isrForCrosstalkSources_996998_33.stderr parsl.dataflow.dflow INFO: Task 3682 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f53cd54abe69c869ff983a0935b614fb with result from task 3682 parsl.dataflow.dflow DEBUG: Task 3682 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9709dd43-c140-4ad8-9804-afba73e7069a_isrForCrosstalkSources_997015_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9709dd43-c140-4ad8-9804-afba73e7069a_isrForCrosstalkSources_997015_11.stderr parsl.dataflow.dflow INFO: Task 3749 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2f89d47a50829af8701884d8576a82c with result from task 3749 parsl.dataflow.dflow DEBUG: Task 3749 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/eda6619a-aba7-49ef-a873-36b6956a0fa9_isrForCrosstalkSources_996960_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/eda6619a-aba7-49ef-a873-36b6956a0fa9_isrForCrosstalkSources_996960_19.stderr parsl.dataflow.dflow INFO: Task 3123 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a30ba5c20f150490d27ba325db850c22 with result from task 3123 parsl.dataflow.dflow DEBUG: Task 3123 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/73c2d3ff-bd0e-450e-a6fc-744d61452613_isrForCrosstalkSources_997007_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/73c2d3ff-bd0e-450e-a6fc-744d61452613_isrForCrosstalkSources_997007_2.stderr parsl.dataflow.dflow INFO: Task 3126 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c18a51a645470399e98f273e63b8a26 with result from task 3126 parsl.dataflow.dflow DEBUG: Task 3126 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b38e3d1f-ebca-4505-84ca-616208312679_isrForCrosstalkSources_996953_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/b38e3d1f-ebca-4505-84ca-616208312679_isrForCrosstalkSources_996953_2.stderr parsl.dataflow.dflow INFO: Task 3127 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7be184d86b7100c1f2a0f971957001b2 with result from task 3127 parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3127 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:31 proc_lsst.multi:146 [INFO] found job 17075076 in provider checkpoint proc_lsst.multi INFO: found job 17075076 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/66b0731f-c75b-4a9d-bf41-cb06cbdd79d1_isrForCrosstalkSources_996975_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/66b0731f-c75b-4a9d-bf41-cb06cbdd79d1_isrForCrosstalkSources_996975_34.stderr parsl.dataflow.dflow INFO: Task 3124 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3764dc1b25175570f789edb0ddadeee with result from task 3124 parsl.dataflow.dflow DEBUG: Task 3124 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8bebce0f-7570-49d6-96fe-017ee46effeb_isrForCrosstalkSources_996977_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8bebce0f-7570-49d6-96fe-017ee46effeb_isrForCrosstalkSources_996977_50.stderr parsl.dataflow.dflow INFO: Task 3122 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a850727c791f6fde01a2d3ecf20d00f with result from task 3122 parsl.dataflow.dflow DEBUG: Task 3122 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/339d5cc7-23d1-4ed7-83cc-cab8c579d7db_isrForCrosstalkSources_997000_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/339d5cc7-23d1-4ed7-83cc-cab8c579d7db_isrForCrosstalkSources_997000_17.stderr parsl.dataflow.dflow INFO: Task 3125 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 749eb5031b872ac119547e9ef85aaa73 with result from task 3125 parsl.dataflow.dflow DEBUG: Task 3125 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/c015e3e3-02d3-4077-879b-90c9ab42cadd_isrForCrosstalkSources_996996_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/c015e3e3-02d3-4077-879b-90c9ab42cadd_isrForCrosstalkSources_996996_57.stderr parsl.dataflow.dflow INFO: Task 3705 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc0afec8363b68467aa1f9d9bcbce1ea with result from task 3705 parsl.dataflow.dflow DEBUG: Task 3705 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/134a2c5d-04a4-4e70-a25c-bb272625eac0_isrForCrosstalkSources_996993_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/134a2c5d-04a4-4e70-a25c-bb272625eac0_isrForCrosstalkSources_996993_50.stderr parsl.dataflow.dflow INFO: Task 3128 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4030b158143fdb918a92e880184485f1 with result from task 3128 parsl.dataflow.dflow DEBUG: Task 3128 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/ede89e6e-fd7b-42d1-b261-93e11390b812_isrForCrosstalkSources_996973_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/ede89e6e-fd7b-42d1-b261-93e11390b812_isrForCrosstalkSources_996973_52.stderr parsl.dataflow.dflow INFO: Task 3121 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44c687d791b32b7b7a328db637480e22 with result from task 3121 parsl.dataflow.dflow DEBUG: Task 3121 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/04e5ceeb-74e4-48a5-84a1-2e2065138976_isrForCrosstalkSources_996982_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/04e5ceeb-74e4-48a5-84a1-2e2065138976_isrForCrosstalkSources_996982_22.stderr parsl.dataflow.dflow INFO: Task 3780 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c0661237b3637c0497d91621670abdf with result from task 3780 parsl.dataflow.dflow DEBUG: Task 3780 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/ff56f6a5-5368-4015-b891-70380f0cdf01_isrForCrosstalkSources_996968_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/ff56f6a5-5368-4015-b891-70380f0cdf01_isrForCrosstalkSources_996968_4.stderr parsl.dataflow.dflow INFO: Task 3700 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db51895b1947e4ff53c5ba3f263bd2ed with result from task 3700 parsl.dataflow.dflow DEBUG: Task 3700 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e8e27880-ee43-450d-90cc-0ba956a8b58c_isrForCrosstalkSources_997028_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/e8e27880-ee43-450d-90cc-0ba956a8b58c_isrForCrosstalkSources_997028_37.stderr parsl.dataflow.dflow INFO: Task 3663 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6fb024bcecbc6d59bae8c678293fa754 with result from task 3663 parsl.dataflow.dflow DEBUG: Task 3663 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e4456fdf-34c4-4cbc-ba12-4cd4893076fd_isrForCrosstalkSources_996986_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e4456fdf-34c4-4cbc-ba12-4cd4893076fd_isrForCrosstalkSources_996986_2.stderr parsl.dataflow.dflow INFO: Task 3569 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac10b6425373f631a90c3e7926168437 with result from task 3569 parsl.dataflow.dflow DEBUG: Task 3569 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/375fd162-f343-4d4e-b78b-0ccec21d6fd2_isrForCrosstalkSources_997023_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/375fd162-f343-4d4e-b78b-0ccec21d6fd2_isrForCrosstalkSources_997023_62.stderr parsl.dataflow.dflow INFO: Task 3720 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 049355f8feb4f17e987b0cc7a6572a6e with result from task 3720 parsl.dataflow.dflow DEBUG: Task 3720 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f5aca3f1-7cb7-40ed-b07e-e95ae096d715_isrForCrosstalkSources_996966_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f5aca3f1-7cb7-40ed-b07e-e95ae096d715_isrForCrosstalkSources_996966_34.stderr parsl.dataflow.dflow INFO: Task 3479 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30f87cc617132b9cae79459c9280a741 with result from task 3479 parsl.dataflow.dflow DEBUG: Task 3479 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/7cc6aba9-cf4e-450f-b23f-a7a97e24af12_isrForCrosstalkSources_996960_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/7cc6aba9-cf4e-450f-b23f-a7a97e24af12_isrForCrosstalkSources_996960_53.stderr parsl.dataflow.dflow INFO: Task 3761 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 134dfde1d6073078c19bec55732dc0ef with result from task 3761 parsl.dataflow.dflow DEBUG: Task 3761 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cad81c87-f1a2-4403-a833-f5ded003bf4b_isrForCrosstalkSources_997024_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/cad81c87-f1a2-4403-a833-f5ded003bf4b_isrForCrosstalkSources_997024_62.stderr parsl.dataflow.dflow INFO: Task 3727 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a7498a01275316ab230973914f7aae3 with result from task 3727 parsl.dataflow.dflow DEBUG: Task 3727 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/2f1b3777-1c8b-43b4-9dc0-2aa64c8bc122_isrForCrosstalkSources_997004_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/2f1b3777-1c8b-43b4-9dc0-2aa64c8bc122_isrForCrosstalkSources_997004_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:32 proc_lsst.multi:146 [INFO] found job 17075078 in provider checkpoint proc_lsst.multi INFO: found job 17075078 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3650 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a12f0e8eee17859866d8f795344c572 with result from task 3650 parsl.dataflow.dflow DEBUG: Task 3650 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/984b97a0-7fe9-46d9-a124-4fe80f7525e9_isrForCrosstalkSources_996979_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/984b97a0-7fe9-46d9-a124-4fe80f7525e9_isrForCrosstalkSources_996979_12.stderr parsl.dataflow.dflow INFO: Task 3239 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ee921717ec0e995762ebc4edb83e3a96 with result from task 3239 parsl.dataflow.dflow DEBUG: Task 3239 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/5372d3e2-06f5-4b33-a68a-f559fe177799_isrForCrosstalkSources_996982_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/5372d3e2-06f5-4b33-a68a-f559fe177799_isrForCrosstalkSources_996982_60.stderr parsl.dataflow.dflow INFO: Task 3500 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f9c357bf533e67f1eb88f5cd2af9ae58 with result from task 3500 parsl.dataflow.dflow DEBUG: Task 3500 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/5682efc4-08b8-4495-8f47-6b7f1b8db364_isrForCrosstalkSources_997030_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/5682efc4-08b8-4495-8f47-6b7f1b8db364_isrForCrosstalkSources_997030_39.stderr parsl.dataflow.dflow INFO: Task 3782 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4ac6dcc4ec0d2cad398769fff1f0cc9 with result from task 3782 parsl.dataflow.dflow DEBUG: Task 3782 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/7424823e-4925-4558-8b1d-b07fb8ff3518_isrForCrosstalkSources_997019_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/7424823e-4925-4558-8b1d-b07fb8ff3518_isrForCrosstalkSources_997019_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:32 proc_lsst.multi:146 [INFO] found job 17075080 in provider checkpoint proc_lsst.multi INFO: found job 17075080 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3618 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 020f579762f27abd02af57372bd5721f with result from task 3618 parsl.dataflow.dflow DEBUG: Task 3618 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/60acf02d-3de7-4760-ab38-9e26d9ff20af_isrForCrosstalkSources_997015_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/60acf02d-3de7-4760-ab38-9e26d9ff20af_isrForCrosstalkSources_997015_52.stderr parsl.dataflow.dflow INFO: Task 3739 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01fb332833704d8c29320f93023df9a7 with result from task 3739 parsl.dataflow.dflow DEBUG: Task 3739 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/035f9578-79e6-4ee4-9f23-aa7cefa167ea_isrForCrosstalkSources_997026_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/035f9578-79e6-4ee4-9f23-aa7cefa167ea_isrForCrosstalkSources_997026_45.stderr parsl.dataflow.dflow INFO: Task 3609 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96593686e1154e80d8335419af2f92a2 with result from task 3609 parsl.dataflow.dflow DEBUG: Task 3609 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/47d5f179-83a7-4f4f-a754-f088724160d5_isrForCrosstalkSources_996956_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/47d5f179-83a7-4f4f-a754-f088724160d5_isrForCrosstalkSources_996956_6.stderr parsl.dataflow.dflow INFO: Task 3759 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9aee7b37ebf59ec4a370a8acff438f8d with result from task 3759 parsl.dataflow.dflow DEBUG: Task 3759 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/44fdc416-6f56-4960-900e-5fc4fb8e947e_isrForCrosstalkSources_996954_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/44fdc416-6f56-4960-900e-5fc4fb8e947e_isrForCrosstalkSources_996954_53.stderr parsl.dataflow.dflow INFO: Task 3462 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df29b9318c5c52fc428b8a25024a67dc with result from task 3462 parsl.dataflow.dflow DEBUG: Task 3462 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/3e5e0ca1-f03c-4694-9bbf-3bed1f0c2290_isrForCrosstalkSources_997030_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/3e5e0ca1-f03c-4694-9bbf-3bed1f0c2290_isrForCrosstalkSources_997030_51.stderr parsl.dataflow.dflow INFO: Task 3737 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5927c2f1767dd7a1e1f1c05d6f83a4c with result from task 3737 parsl.dataflow.dflow DEBUG: Task 3737 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/da4a612d-f7ff-44d7-9ea6-a2a025f23815_isrForCrosstalkSources_997027_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/da4a612d-f7ff-44d7-9ea6-a2a025f23815_isrForCrosstalkSources_997027_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:33 proc_lsst.multi:146 [INFO] found job 17075081 in provider checkpoint proc_lsst.multi INFO: found job 17075081 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3767 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d9fe48f54df4ee1456afa71f987a338 with result from task 3767 parsl.dataflow.dflow DEBUG: Task 3767 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e0cdb5f4-65d2-45e6-baa8-9750dbf33044_isrForCrosstalkSources_996964_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e0cdb5f4-65d2-45e6-baa8-9750dbf33044_isrForCrosstalkSources_996964_53.stderr parsl.dataflow.dflow INFO: Task 3493 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c5249f64dfc706a658014160ee2af71 with result from task 3493 parsl.dataflow.dflow DEBUG: Task 3493 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/e68ac325-ead5-45c5-a0eb-1fea7a492996_isrForCrosstalkSources_997003_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/e68ac325-ead5-45c5-a0eb-1fea7a492996_isrForCrosstalkSources_997003_35.stderr parsl.dataflow.dflow INFO: Task 3776 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 37e6a53099ef20aca640dcdaab6ec8e8 with result from task 3776 parsl.dataflow.dflow DEBUG: Task 3776 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e06d7f00-d09a-4541-b5a7-89d50eeab592_isrForCrosstalkSources_996983_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e06d7f00-d09a-4541-b5a7-89d50eeab592_isrForCrosstalkSources_996983_47.stderr parsl.dataflow.dflow INFO: Task 3584 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dfb2a8685ec99093a5d444242e1aa204 with result from task 3584 parsl.dataflow.dflow DEBUG: Task 3584 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c43455dc-344c-42c8-9105-a6057001c465_isrForCrosstalkSources_997021_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/c43455dc-344c-42c8-9105-a6057001c465_isrForCrosstalkSources_997021_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:34 proc_lsst.multi:146 [INFO] found job 17075082 in provider checkpoint proc_lsst.multi INFO: found job 17075082 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3763 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f38a453f4c0fd14ae308e43fdd490636 with result from task 3763 parsl.dataflow.dflow DEBUG: Task 3763 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2c2cc430-62f2-49d9-b8b9-733e27cb10c5_isrForCrosstalkSources_996962_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2c2cc430-62f2-49d9-b8b9-733e27cb10c5_isrForCrosstalkSources_996962_24.stderr parsl.dataflow.dflow INFO: Task 3517 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7a3ecf5c8e7f4893e897cbda7c06039 with result from task 3517 parsl.dataflow.dflow DEBUG: Task 3517 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/23945fba-0dd8-479b-a54b-08bfd9a69bb8_isrForCrosstalkSources_996971_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/23945fba-0dd8-479b-a54b-08bfd9a69bb8_isrForCrosstalkSources_996971_17.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:34 proc_lsst.multi:146 [INFO] found job 17075083 in provider checkpoint proc_lsst.multi INFO: found job 17075083 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3532 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88f7febffb127df7e66029f58c10b47d with result from task 3532 parsl.dataflow.dflow DEBUG: Task 3532 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/003d66fe-dbd5-4ee6-aa2e-0ff93efce1e6_isrForCrosstalkSources_996953_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/003d66fe-dbd5-4ee6-aa2e-0ff93efce1e6_isrForCrosstalkSources_996953_42.stderr parsl.dataflow.dflow INFO: Task 3521 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8cc2cc4c226174d83a228de881ee8e3 with result from task 3521 parsl.dataflow.dflow DEBUG: Task 3521 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bd555028-1229-4b3b-b131-e2f712a57eb0_isrForCrosstalkSources_997026_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/bd555028-1229-4b3b-b131-e2f712a57eb0_isrForCrosstalkSources_997026_7.stderr parsl.dataflow.dflow INFO: Task 3554 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7a8813ce160f0c62f2979a89ec8ac543 with result from task 3554 parsl.dataflow.dflow DEBUG: Task 3554 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/9c45b11c-9e0e-4074-9930-469f9f35d604_isrForCrosstalkSources_996991_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/9c45b11c-9e0e-4074-9930-469f9f35d604_isrForCrosstalkSources_996991_21.stderr parsl.dataflow.dflow INFO: Task 3693 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7363d1102f5375c2f7c25287cba0c5e6 with result from task 3693 parsl.dataflow.dflow DEBUG: Task 3693 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1a146776-5280-4686-90c6-09f09c4e299e_isrForCrosstalkSources_997017_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1a146776-5280-4686-90c6-09f09c4e299e_isrForCrosstalkSources_997017_62.stderr parsl.dataflow.dflow INFO: Task 3644 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4789bfaa84c7ce844553fa36a17c993b with result from task 3644 parsl.dataflow.dflow DEBUG: Task 3644 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/2c8a2c44-22a8-4337-ae3e-aed256b5c2d9_isrForCrosstalkSources_996978_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/2c8a2c44-22a8-4337-ae3e-aed256b5c2d9_isrForCrosstalkSources_996978_45.stderr parsl.dataflow.dflow INFO: Task 3562 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d62dc554345f0178b9fec2fec60f615 with result from task 3562 parsl.dataflow.dflow DEBUG: Task 3562 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a22cd30f-f4a0-46a7-930a-c88e5609a4ad_isrForCrosstalkSources_996964_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a22cd30f-f4a0-46a7-930a-c88e5609a4ad_isrForCrosstalkSources_996964_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:35 proc_lsst.multi:146 [INFO] found job 17075085 in provider checkpoint proc_lsst.multi INFO: found job 17075085 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3431 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a9b02745a2a0efe8b604f87de8e0fcc with result from task 3431 parsl.dataflow.dflow DEBUG: Task 3431 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fbf30e7a-33ff-4d6c-a7df-cca2c9cb6ad1_isrForCrosstalkSources_997012_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fbf30e7a-33ff-4d6c-a7df-cca2c9cb6ad1_isrForCrosstalkSources_997012_3.stderr parsl.dataflow.dflow INFO: Task 3777 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e53ed00693044e81c42d3b15600f9a21 with result from task 3777 parsl.dataflow.dflow DEBUG: Task 3777 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/86a9525a-0f29-4fdb-bad9-d1182f8be0de_isrForCrosstalkSources_997002_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/86a9525a-0f29-4fdb-bad9-d1182f8be0de_isrForCrosstalkSources_997002_46.stderr parsl.dataflow.dflow INFO: Task 3809 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a21d8e1329d626c4516bb287ed40759b with result from task 3809 parsl.dataflow.dflow DEBUG: Task 3809 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/7f2d2d58-70b9-4a39-bc13-4604b7400ed0_isrForCrosstalkSources_996972_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/7f2d2d58-70b9-4a39-bc13-4604b7400ed0_isrForCrosstalkSources_996972_34.stderr parsl.dataflow.dflow INFO: Task 3785 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e437174d292f93a80f1631e652694f52 with result from task 3785 parsl.dataflow.dflow DEBUG: Task 3785 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3fa0b926-17b6-426f-b38c-6ae23d44cc99_isrForCrosstalkSources_996963_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3fa0b926-17b6-426f-b38c-6ae23d44cc99_isrForCrosstalkSources_996963_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:36 proc_lsst.multi:146 [INFO] found job 17075086 in provider checkpoint proc_lsst.multi INFO: found job 17075086 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3798 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab61efe065809faf4d58c3bc9b780f24 with result from task 3798 parsl.dataflow.dflow DEBUG: Task 3798 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/fb27cb9c-4d11-4b16-8704-25aa082307a2_isrForCrosstalkSources_996973_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/fb27cb9c-4d11-4b16-8704-25aa082307a2_isrForCrosstalkSources_996973_29.stderr parsl.dataflow.dflow INFO: Task 3601 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e0f0859f8ce06af19d6a1e3cd8fecb9 with result from task 3601 parsl.dataflow.dflow DEBUG: Task 3601 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e35254a4-549e-4710-9c0a-2cc5f7802265_isrForCrosstalkSources_997010_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/e35254a4-549e-4710-9c0a-2cc5f7802265_isrForCrosstalkSources_997010_39.stderr parsl.dataflow.dflow INFO: Task 3689 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 60ca1297efd3c7a4557a9b5febe4f1c9 with result from task 3689 parsl.dataflow.dflow DEBUG: Task 3689 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/1148f132-8353-4881-9949-e05e58e099a5_isrForCrosstalkSources_996975_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/1148f132-8353-4881-9949-e05e58e099a5_isrForCrosstalkSources_996975_15.stderr parsl.dataflow.dflow INFO: Task 3608 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ddae82b4ef9b50cc17d3f544bca534b3 with result from task 3608 parsl.dataflow.dflow DEBUG: Task 3608 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/4b87311a-8412-4dc6-816b-0446682ded0a_isrForCrosstalkSources_997029_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/4b87311a-8412-4dc6-816b-0446682ded0a_isrForCrosstalkSources_997029_24.stderr parsl.dataflow.dflow INFO: Task 3617 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de2070448298fd1ff833aa099d270d4a with result from task 3617 parsl.dataflow.dflow DEBUG: Task 3617 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/b1b9303a-082f-4987-bfd8-9ec125960f47_isrForCrosstalkSources_996967_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/b1b9303a-082f-4987-bfd8-9ec125960f47_isrForCrosstalkSources_996967_11.stderr parsl.dataflow.dflow INFO: Task 3794 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84382b9913c458c8edd2d753c4fd2ca6 with result from task 3794 parsl.dataflow.dflow DEBUG: Task 3794 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e0e04b72-8966-4cbc-b723-0f9a9963fa64_isrForCrosstalkSources_996995_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/e0e04b72-8966-4cbc-b723-0f9a9963fa64_isrForCrosstalkSources_996995_57.stderr parsl.dataflow.dflow INFO: Task 3593 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55d7e20839b25ef118098893bbc4175f with result from task 3593 parsl.dataflow.dflow DEBUG: Task 3593 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7a976478-8e9e-4dd9-8557-29b16ca1ec12_isrForCrosstalkSources_996986_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/7a976478-8e9e-4dd9-8557-29b16ca1ec12_isrForCrosstalkSources_996986_16.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:36 proc_lsst.multi:146 [INFO] found job 17075088 in provider checkpoint proc_lsst.multi INFO: found job 17075088 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3639 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a79df5a5817a3129eacffd2757f54ab3 with result from task 3639 parsl.dataflow.dflow DEBUG: Task 3639 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7b6d7d7e-c5c1-4315-8533-c92ce34843ac_isrForCrosstalkSources_996968_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/7b6d7d7e-c5c1-4315-8533-c92ce34843ac_isrForCrosstalkSources_996968_6.stderr parsl.dataflow.dflow INFO: Task 3499 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dce505dc0dd6c39754902b3ecdf8b75f with result from task 3499 parsl.dataflow.dflow DEBUG: Task 3499 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/39971b06-55e9-4655-9a10-441f9e792561_isrForCrosstalkSources_996987_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/39971b06-55e9-4655-9a10-441f9e792561_isrForCrosstalkSources_996987_5.stderr parsl.dataflow.dflow INFO: Task 3579 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff6561eb21f6770acc21e44ef71a116c with result from task 3579 parsl.dataflow.dflow DEBUG: Task 3579 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/957dfbc4-cc06-4e46-b139-b62034d06850_isrForCrosstalkSources_996968_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/957dfbc4-cc06-4e46-b139-b62034d06850_isrForCrosstalkSources_996968_1.stderr parsl.dataflow.dflow INFO: Task 3789 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 80e13eb1aeb969ab1c0fe9bedb807547 with result from task 3789 parsl.dataflow.dflow DEBUG: Task 3789 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/213ff86c-362c-41cb-a126-7c9bff43d1df_isrForCrosstalkSources_996975_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/213ff86c-362c-41cb-a126-7c9bff43d1df_isrForCrosstalkSources_996975_45.stderr parsl.dataflow.dflow INFO: Task 3822 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a60e7571dbbbf5550f9287f1831c0cc with result from task 3822 parsl.dataflow.dflow DEBUG: Task 3822 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/7e4f1cb4-2f2b-467b-a7b1-7de547b52d1b_isrForCrosstalkSources_996996_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/7e4f1cb4-2f2b-467b-a7b1-7de547b52d1b_isrForCrosstalkSources_996996_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:37 proc_lsst.multi:146 [INFO] found job 17075090 in provider checkpoint proc_lsst.multi INFO: found job 17075090 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3607 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 841100a9aa51762cedfff8fb60ab2052 with result from task 3607 parsl.dataflow.dflow DEBUG: Task 3607 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/43030cd1-d15a-434f-8e9f-9fb9ee71c0c2_isrForCrosstalkSources_996988_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/43030cd1-d15a-434f-8e9f-9fb9ee71c0c2_isrForCrosstalkSources_996988_43.stderr parsl.dataflow.dflow INFO: Task 3783 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 528189fd0053423eb32e54bd1941e3b8 with result from task 3783 parsl.dataflow.dflow DEBUG: Task 3783 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/52a697d3-2731-4039-87b6-53cf4552de25_isrForCrosstalkSources_997013_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/52a697d3-2731-4039-87b6-53cf4552de25_isrForCrosstalkSources_997013_51.stderr parsl.dataflow.dflow INFO: Task 3793 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b76012fb752f99a3c357f1f01b1187f5 with result from task 3793 parsl.dataflow.dflow DEBUG: Task 3793 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/0c930d80-4d31-4ef3-9f67-128b940f68d9_isrForCrosstalkSources_996996_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/0c930d80-4d31-4ef3-9f67-128b940f68d9_isrForCrosstalkSources_996996_40.stderr parsl.dataflow.dflow INFO: Task 3519 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6e92a77486e0c1b21e7d9ec32cf26192 with result from task 3519 parsl.dataflow.dflow DEBUG: Task 3519 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/8d9c467a-8aea-4370-a5f8-cbb0ba693d0e_isrForCrosstalkSources_997016_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/8d9c467a-8aea-4370-a5f8-cbb0ba693d0e_isrForCrosstalkSources_997016_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:38 proc_lsst.multi:146 [INFO] found job 17075091 in provider checkpoint proc_lsst.multi INFO: found job 17075091 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3546 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6243df00cea3b88651723e78bcf11cea with result from task 3546 parsl.dataflow.dflow DEBUG: Task 3546 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d2d6eb16-7fcc-46f5-bcd2-1a583ff0a1bf_isrForCrosstalkSources_997009_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d2d6eb16-7fcc-46f5-bcd2-1a583ff0a1bf_isrForCrosstalkSources_997009_6.stderr parsl.dataflow.dflow INFO: Task 3558 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry efd902f06e5d6b7c0c738aff11dc1669 with result from task 3558 parsl.dataflow.dflow DEBUG: Task 3558 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/20b46eaa-c899-4480-a5a2-6dab69b7b5e5_isrForCrosstalkSources_997016_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/20b46eaa-c899-4480-a5a2-6dab69b7b5e5_isrForCrosstalkSources_997016_37.stderr parsl.dataflow.dflow INFO: Task 3445 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e784cd9bedfaa24f751e4b33d0cc92fc with result from task 3445 parsl.dataflow.dflow DEBUG: Task 3445 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3bbd8d9b-dbdc-41d0-bdfd-13b3080cc825_isrForCrosstalkSources_996988_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3bbd8d9b-dbdc-41d0-bdfd-13b3080cc825_isrForCrosstalkSources_996988_48.stderr parsl.dataflow.dflow INFO: Task 3813 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b74d96ab8f68982d47c5effdafa93e10 with result from task 3813 parsl.dataflow.dflow DEBUG: Task 3813 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/1d2e5425-5680-4b97-a6d1-36d447448412_isrForCrosstalkSources_996985_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/1d2e5425-5680-4b97-a6d1-36d447448412_isrForCrosstalkSources_996985_9.stderr parsl.dataflow.dflow INFO: Task 3495 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1b6b98d491b6aa01893c28d15748d854 with result from task 3495 parsl.dataflow.dflow DEBUG: Task 3495 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/75e52507-f1a6-46ef-afd0-0edb2605a070_isrForCrosstalkSources_996993_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/75e52507-f1a6-46ef-afd0-0edb2605a070_isrForCrosstalkSources_996993_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:38 proc_lsst.multi:146 [INFO] found job 17075092 in provider checkpoint proc_lsst.multi INFO: found job 17075092 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3565 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 05729c0f985263d258d60b12c5e4af24 with result from task 3565 parsl.dataflow.dflow DEBUG: Task 3565 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a14c3d8e-3d64-4090-a11c-d4a770c0559c_isrForCrosstalkSources_997022_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a14c3d8e-3d64-4090-a11c-d4a770c0559c_isrForCrosstalkSources_997022_10.stderr parsl.dataflow.dflow INFO: Task 3807 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04ecd9cfd80f7cdec42011cde4ede68c with result from task 3807 parsl.dataflow.dflow DEBUG: Task 3807 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0362cb2c-d119-4051-a085-2526ae9c5e2d_isrForCrosstalkSources_997007_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0362cb2c-d119-4051-a085-2526ae9c5e2d_isrForCrosstalkSources_997007_57.stderr parsl.dataflow.dflow INFO: Task 3734 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd9cca8468a67b73048cc7faf0b27084 with result from task 3734 parsl.dataflow.dflow DEBUG: Task 3734 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/dbc8c93a-5a74-4651-a675-b8d42c2aac34_isrForCrosstalkSources_996964_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/dbc8c93a-5a74-4651-a675-b8d42c2aac34_isrForCrosstalkSources_996964_58.stderr parsl.dataflow.dflow INFO: Task 3638 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 786769ab257d52921f809d9c40edb5e6 with result from task 3638 parsl.dataflow.dflow DEBUG: Task 3638 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f1d2ea8a-13a3-4c82-bc09-a370ead4f81c_isrForCrosstalkSources_997024_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/f1d2ea8a-13a3-4c82-bc09-a370ead4f81c_isrForCrosstalkSources_997024_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:39 proc_lsst.multi:146 [INFO] found job 17075093 in provider checkpoint proc_lsst.multi INFO: found job 17075093 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3860 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33d8590ffd419926fcb186b656dccbda with result from task 3860 parsl.dataflow.dflow DEBUG: Task 3860 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/1332ebdb-bfaa-47ec-95dd-9014148e22fe_isrForCrosstalkSources_996963_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/1332ebdb-bfaa-47ec-95dd-9014148e22fe_isrForCrosstalkSources_996963_41.stderr parsl.dataflow.dflow INFO: Task 3858 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8dbfb51fe6af39e85a15e685adbfc16b with result from task 3858 parsl.dataflow.dflow DEBUG: Task 3858 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/db020087-c5f3-4dd7-a76c-b4eb5412f371_isrForCrosstalkSources_997014_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/db020087-c5f3-4dd7-a76c-b4eb5412f371_isrForCrosstalkSources_997014_32.stderr parsl.dataflow.dflow INFO: Task 3642 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 913063731b38c10c691cdaf360522440 with result from task 3642 parsl.dataflow.dflow DEBUG: Task 3642 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fb458501-9768-454f-94dd-0b712bccb1db_isrForCrosstalkSources_996990_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fb458501-9768-454f-94dd-0b712bccb1db_isrForCrosstalkSources_996990_18.stderr parsl.dataflow.dflow INFO: Task 3832 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c955743e2243800a70ac09fc77c7583f with result from task 3832 parsl.dataflow.dflow DEBUG: Task 3832 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a84dfd6f-7f3c-497b-bb41-263134819584_isrForCrosstalkSources_996972_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/a84dfd6f-7f3c-497b-bb41-263134819584_isrForCrosstalkSources_996972_54.stderr parsl.dataflow.dflow INFO: Task 3660 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6514f82c30d9bbf6f34a9da6653ed462 with result from task 3660 parsl.dataflow.dflow DEBUG: Task 3660 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9f8182e1-99bd-401a-9156-4d3c50cb5c13_isrForCrosstalkSources_996958_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/9f8182e1-99bd-401a-9156-4d3c50cb5c13_isrForCrosstalkSources_996958_8.stderr parsl.dataflow.dflow INFO: Task 3416 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 765501d10c9abfe0352f783514208b1d with result from task 3416 parsl.dataflow.dflow DEBUG: Task 3416 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/76ecd649-c5a7-4446-956e-e2436949b3d7_isrForCrosstalkSources_996978_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/76ecd649-c5a7-4446-956e-e2436949b3d7_isrForCrosstalkSources_996978_30.stderr parsl.dataflow.dflow INFO: Task 3428 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2caa10b849116fb551b93806e1e9da8a with result from task 3428 parsl.dataflow.dflow DEBUG: Task 3428 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/de450d25-eec3-497a-bcd8-90e0dfab0a53_isrForCrosstalkSources_997010_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/de450d25-eec3-497a-bcd8-90e0dfab0a53_isrForCrosstalkSources_997010_52.stderr parsl.dataflow.dflow INFO: Task 3472 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 64e64de99db88130dd801489d487f8d0 with result from task 3472 parsl.dataflow.dflow DEBUG: Task 3472 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a28ee8cf-a4ab-4c70-ab88-52a1349bf3df_isrForCrosstalkSources_997006_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a28ee8cf-a4ab-4c70-ab88-52a1349bf3df_isrForCrosstalkSources_997006_31.stderr parsl.dataflow.dflow INFO: Task 3721 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb7e2ffaf4e46e18f2639eb68c2cce50 with result from task 3721 parsl.dataflow.dflow DEBUG: Task 3721 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d2dcd442-7a96-49a4-8d73-05f113a29169_isrForCrosstalkSources_996966_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d2dcd442-7a96-49a4-8d73-05f113a29169_isrForCrosstalkSources_996966_3.stderr parsl.dataflow.dflow INFO: Task 3835 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff4442c9ba59f3a3daf5b9fdb9e3108d with result from task 3835 parsl.dataflow.dflow DEBUG: Task 3835 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3ba178bd-5aff-472a-a38b-d1b0b53b2838_isrForCrosstalkSources_996988_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3ba178bd-5aff-472a-a38b-d1b0b53b2838_isrForCrosstalkSources_996988_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:39 proc_lsst.multi:146 [INFO] found job 17075094 in provider checkpoint proc_lsst.multi INFO: found job 17075094 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:40 proc_lsst.multi:146 [INFO] found job 17075095 in provider checkpoint proc_lsst.multi INFO: found job 17075095 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3622 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56096dbc2715850a4cee5bb3b38cc5d1 with result from task 3622 parsl.dataflow.dflow DEBUG: Task 3622 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c32149ed-2163-4ee2-bf44-d997f5426496_isrForCrosstalkSources_997003_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c32149ed-2163-4ee2-bf44-d997f5426496_isrForCrosstalkSources_997003_44.stderr parsl.dataflow.dflow INFO: Task 3769 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 636c92794854fadd98a88236ec1c214e with result from task 3769 parsl.dataflow.dflow DEBUG: Task 3769 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b9c305d9-77ec-449b-be55-e91851c3babb_isrForCrosstalkSources_996986_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b9c305d9-77ec-449b-be55-e91851c3babb_isrForCrosstalkSources_996986_11.stderr parsl.dataflow.dflow INFO: Task 3678 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b3d25c690adff44b6981033c4c548c9 with result from task 3678 parsl.dataflow.dflow DEBUG: Task 3678 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/8b5e9975-8415-4261-809c-d032c9344a40_isrForCrosstalkSources_996957_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/8b5e9975-8415-4261-809c-d032c9344a40_isrForCrosstalkSources_996957_55.stderr parsl.dataflow.dflow INFO: Task 3833 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 127bb696abc32d50954980027795f0fe with result from task 3833 parsl.dataflow.dflow DEBUG: Task 3833 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/000728c9-a311-4fa4-b096-00eef2eb6467_isrForCrosstalkSources_996998_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/000728c9-a311-4fa4-b096-00eef2eb6467_isrForCrosstalkSources_996998_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 3859 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry dbedf8fee9f7593c895b81b005d2323f with result from task 3859 parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 3859 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 3859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5d56cc1b-1e07-46d5-82fe-0b78e9e82ba4_isrForCrosstalkSources_996968_45.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 3859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/5d56cc1b-1e07-46d5-82fe-0b78e9e82ba4_isrForCrosstalkSources_996968_45.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:40 proc_lsst.multi:146 [INFO] found job 17075096 in provider checkpoint proc_lsst.multi INFO: found job 17075096 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3440 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f6a3cf72e6a2b7f44c179d0e815ff0d with result from task 3440 parsl.dataflow.dflow DEBUG: Task 3440 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a22a2f17-f4e1-4c18-af5e-8d1e2bb675aa_isrForCrosstalkSources_996954_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a22a2f17-f4e1-4c18-af5e-8d1e2bb675aa_isrForCrosstalkSources_996954_31.stderr parsl.dataflow.dflow INFO: Task 3661 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0608e61585efd716ebf180d187c83109 with result from task 3661 parsl.dataflow.dflow DEBUG: Task 3661 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/bf4924e5-647c-4db9-9678-2918f71e6f17_isrForCrosstalkSources_996957_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/bf4924e5-647c-4db9-9678-2918f71e6f17_isrForCrosstalkSources_996957_10.stderr parsl.dataflow.dflow INFO: Task 3816 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c1e1fa7e9947d327d947bf7176cf992 with result from task 3816 parsl.dataflow.dflow DEBUG: Task 3816 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/f5678b82-a1fc-489b-913d-380f424e1bd6_isrForCrosstalkSources_996968_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/f5678b82-a1fc-489b-913d-380f424e1bd6_isrForCrosstalkSources_996968_54.stderr parsl.dataflow.dflow INFO: Task 3596 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ebf23dc91416628fb33767a567bfcee with result from task 3596 parsl.dataflow.dflow DEBUG: Task 3596 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ca1ebdb6-e451-4e16-ae46-93cafb72ea29_isrForCrosstalkSources_996984_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ca1ebdb6-e451-4e16-ae46-93cafb72ea29_isrForCrosstalkSources_996984_11.stderr parsl.dataflow.dflow INFO: Task 3529 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ef150a35f63f6d9785aef9a9d29a338 with result from task 3529 parsl.dataflow.dflow DEBUG: Task 3529 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d6cd2d3d-847e-4de5-a7aa-22a2c518e986_isrForCrosstalkSources_996966_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/d6cd2d3d-847e-4de5-a7aa-22a2c518e986_isrForCrosstalkSources_996966_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3656 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry f90be0dec7bfd04b891e9dac7f03a1ed with result from task 3656 parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3656 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/649c5b86-9165-490d-8cd6-fed9d01a9475_isrForCrosstalkSources_996985_30.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/649c5b86-9165-490d-8cd6-fed9d01a9475_isrForCrosstalkSources_996985_30.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:41 proc_lsst.multi:146 [INFO] found job 17075097 in provider checkpoint proc_lsst.multi INFO: found job 17075097 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3626 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b15c5404671dcf05d99992bcee63821 with result from task 3626 parsl.dataflow.dflow DEBUG: Task 3626 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/6338a5b1-7544-4084-95ae-5f9c94e8d7d8_isrForCrosstalkSources_996977_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/6338a5b1-7544-4084-95ae-5f9c94e8d7d8_isrForCrosstalkSources_996977_45.stderr parsl.dataflow.dflow INFO: Task 3506 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54d4a2a3acf75a81efbb6c358c5bafb6 with result from task 3506 parsl.dataflow.dflow DEBUG: Task 3506 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c37b4abc-242d-464f-b9a9-749c45502593_isrForCrosstalkSources_996970_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c37b4abc-242d-464f-b9a9-749c45502593_isrForCrosstalkSources_996970_5.stderr parsl.dataflow.dflow INFO: Task 3586 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4090067cc504eb3cbacf89cd53b616ca with result from task 3586 parsl.dataflow.dflow DEBUG: Task 3586 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ffb4b679-a9e0-4a44-95a2-30d2f9788886_isrForCrosstalkSources_996992_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/ffb4b679-a9e0-4a44-95a2-30d2f9788886_isrForCrosstalkSources_996992_41.stderr parsl.dataflow.dflow INFO: Task 3866 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb8397f85823b66bcbb46b4fbd7df7fc with result from task 3866 parsl.dataflow.dflow DEBUG: Task 3866 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d67d03c7-e642-4690-b9af-b8eec5452dd4_isrForCrosstalkSources_997017_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d67d03c7-e642-4690-b9af-b8eec5452dd4_isrForCrosstalkSources_997017_20.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:42 proc_lsst.multi:146 [INFO] found job 17075098 in provider checkpoint proc_lsst.multi INFO: found job 17075098 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3846 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a53c7b01cc4a58b381c70992f037667 with result from task 3846 parsl.dataflow.dflow DEBUG: Task 3846 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b03edc3e-7770-4e6e-a5d5-85114aa2ee55_isrForCrosstalkSources_996969_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/b03edc3e-7770-4e6e-a5d5-85114aa2ee55_isrForCrosstalkSources_996969_11.stderr parsl.dataflow.dflow INFO: Task 3492 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63151faec408be8e1fe735f5e9a6d78d with result from task 3492 parsl.dataflow.dflow DEBUG: Task 3492 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ff6cb520-595e-4f35-9566-4d8055d3de0e_isrForCrosstalkSources_996962_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ff6cb520-595e-4f35-9566-4d8055d3de0e_isrForCrosstalkSources_996962_1.stderr parsl.dataflow.dflow INFO: Task 3658 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 354f10c9b2529841726d15406c4dc0ab with result from task 3658 parsl.dataflow.dflow DEBUG: Task 3658 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/95a4050b-fa21-40c8-acb2-98e117165694_isrForCrosstalkSources_996957_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/95a4050b-fa21-40c8-acb2-98e117165694_isrForCrosstalkSources_996957_38.stderr parsl.dataflow.dflow INFO: Task 3589 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d7ec5cb8de12ff20e63d152cc80e449 with result from task 3589 parsl.dataflow.dflow DEBUG: Task 3589 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8b98ca34-a73f-4627-8bfa-774941f6a98d_isrForCrosstalkSources_996971_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/8b98ca34-a73f-4627-8bfa-774941f6a98d_isrForCrosstalkSources_996971_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3494 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry ff7bd414eecf0164856316f50702f01f with result from task 3494 parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3494 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d9a9b125-d697-4435-99b1-1a54fe62b620_isrForCrosstalkSources_997030_61.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d9a9b125-d697-4435-99b1-1a54fe62b620_isrForCrosstalkSources_997030_61.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:42 proc_lsst.multi:146 [INFO] found job 17075099 in provider checkpoint proc_lsst.multi INFO: found job 17075099 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3887 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 76f8312240b4f29266660cff97028351 with result from task 3887 parsl.dataflow.dflow DEBUG: Task 3887 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c68ec87e-77b7-4b97-97df-17f29b650610_isrForCrosstalkSources_997004_19.stdout parsl.dataflow.dflow INFO: Standard error for task 3887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c68ec87e-77b7-4b97-97df-17f29b650610_isrForCrosstalkSources_997004_19.stderr parsl.dataflow.dflow INFO: Task 3882 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 146318dceafb8154af10ccfe6d0d7f68 with result from task 3882 parsl.dataflow.dflow DEBUG: Task 3882 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ab602216-b333-404b-8f21-1b7fe7b8e05e_isrForCrosstalkSources_996975_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ab602216-b333-404b-8f21-1b7fe7b8e05e_isrForCrosstalkSources_996975_55.stderr parsl.dataflow.dflow INFO: Task 3891 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a2b156276f20812f8c8fe8f2bd099e5 with result from task 3891 parsl.dataflow.dflow DEBUG: Task 3891 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/953f238d-18d3-4f82-8947-44cf8853e6e1_isrForCrosstalkSources_996964_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/953f238d-18d3-4f82-8947-44cf8853e6e1_isrForCrosstalkSources_996964_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3419 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 3c2e453791448e9d279d63b6836e6c49 with result from task 3419 parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3419 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/4b3f8ff8-8bd7-4ea1-ace8-235acc09611a_isrForCrosstalkSources_996973_50.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/4b3f8ff8-8bd7-4ea1-ace8-235acc09611a_isrForCrosstalkSources_996973_50.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:43 proc_lsst.multi:146 [INFO] found job 17075100 in provider checkpoint proc_lsst.multi INFO: found job 17075100 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3465 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e908a3c1615d625bdeb9e112342ab50 with result from task 3465 parsl.dataflow.dflow DEBUG: Task 3465 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7c439e7a-bae5-4a3e-a8cc-5c3b987c054b_isrForCrosstalkSources_997025_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7c439e7a-bae5-4a3e-a8cc-5c3b987c054b_isrForCrosstalkSources_997025_60.stderr parsl.dataflow.dflow INFO: Task 3425 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13bf401c44a48ce87ab23ad52627b9ca with result from task 3425 parsl.dataflow.dflow DEBUG: Task 3425 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d700b0a4-af78-4382-80db-0dced0b43a05_isrForCrosstalkSources_997025_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d700b0a4-af78-4382-80db-0dced0b43a05_isrForCrosstalkSources_997025_48.stderr parsl.dataflow.dflow INFO: Task 3863 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6be5fe89729efd96d3ec6f8adff3cfa7 with result from task 3863 parsl.dataflow.dflow DEBUG: Task 3863 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/dbdcdb79-2da4-406a-a79e-2cc1610b5ef2_isrForCrosstalkSources_997029_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/dbdcdb79-2da4-406a-a79e-2cc1610b5ef2_isrForCrosstalkSources_997029_56.stderr parsl.dataflow.dflow INFO: Task 3527 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16d0064496f0d19d92f71d8bc3aa182f with result from task 3527 parsl.dataflow.dflow DEBUG: Task 3527 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c083ba35-6f6c-45cb-97f9-3136f5252a4d_isrForCrosstalkSources_997003_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/c083ba35-6f6c-45cb-97f9-3136f5252a4d_isrForCrosstalkSources_997003_3.stderr parsl.dataflow.dflow INFO: Task 3578 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df87bf3c518b1438db9a75a018d0a5df with result from task 3578 parsl.dataflow.dflow DEBUG: Task 3578 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/168709c3-a032-461b-a3ea-6d475f3429c4_isrForCrosstalkSources_997030_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/168709c3-a032-461b-a3ea-6d475f3429c4_isrForCrosstalkSources_997030_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:43 proc_lsst.multi:146 [INFO] found job 17075101 in provider checkpoint proc_lsst.multi INFO: found job 17075101 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3412 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 697809287c1382e9c520f83928ab6961 with result from task 3412 parsl.dataflow.dflow DEBUG: Task 3412 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1a7c52e0-ca0e-4ec1-b94c-d188e9d45ff5_isrForCrosstalkSources_996967_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1a7c52e0-ca0e-4ec1-b94c-d188e9d45ff5_isrForCrosstalkSources_996967_57.stderr parsl.dataflow.dflow INFO: Task 3695 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cae5edd3f2741217430974497bdbfc33 with result from task 3695 parsl.dataflow.dflow DEBUG: Task 3695 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2327d6c6-d374-428c-8f7f-eadb0389497b_isrForCrosstalkSources_997007_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/2327d6c6-d374-428c-8f7f-eadb0389497b_isrForCrosstalkSources_997007_42.stderr parsl.dataflow.dflow INFO: Task 3464 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6df07daeee43340deb36fc0bf8ebe52e with result from task 3464 parsl.dataflow.dflow DEBUG: Task 3464 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/197ee9f3-bd84-4ef8-a9be-6ba11992df91_isrForCrosstalkSources_997024_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/197ee9f3-bd84-4ef8-a9be-6ba11992df91_isrForCrosstalkSources_997024_60.stderr parsl.dataflow.dflow INFO: Task 3856 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 346d2f47b830e69d941cbdf12c519c8d with result from task 3856 parsl.dataflow.dflow DEBUG: Task 3856 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/46dd3084-528c-4fa5-bdbb-32eaabbce501_isrForCrosstalkSources_996989_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/46dd3084-528c-4fa5-bdbb-32eaabbce501_isrForCrosstalkSources_996989_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:44 proc_lsst.multi:146 [INFO] found job 17075102 in provider checkpoint proc_lsst.multi INFO: found job 17075102 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3592 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78f57e3ec7d1b55826a32fcc99053ccc with result from task 3592 parsl.dataflow.dflow DEBUG: Task 3592 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/cc242ef1-78ee-412e-9c03-12b01f6f5154_isrForCrosstalkSources_996998_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/cc242ef1-78ee-412e-9c03-12b01f6f5154_isrForCrosstalkSources_996998_44.stderr parsl.dataflow.dflow INFO: Task 3655 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3e80dcd5174c7be321f89f689095578f with result from task 3655 parsl.dataflow.dflow DEBUG: Task 3655 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/77c537fd-e0ae-4873-9500-5258832f1939_isrForCrosstalkSources_997020_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/77c537fd-e0ae-4873-9500-5258832f1939_isrForCrosstalkSources_997020_27.stderr parsl.dataflow.dflow INFO: Task 3501 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f4ef89e2cb2a29ccdb968a3e6e7cf21 with result from task 3501 parsl.dataflow.dflow DEBUG: Task 3501 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/46102891-0ea5-4b82-bcf4-5db0cf197c06_isrForCrosstalkSources_997007_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/46102891-0ea5-4b82-bcf4-5db0cf197c06_isrForCrosstalkSources_997007_35.stderr parsl.dataflow.dflow INFO: Task 3556 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f776dda90bfacd054f2b819e464ced90 with result from task 3556 parsl.dataflow.dflow DEBUG: Task 3556 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f98bae2d-0ef5-437f-ae87-9ab020c1bdb1_isrForCrosstalkSources_997015_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/f98bae2d-0ef5-437f-ae87-9ab020c1bdb1_isrForCrosstalkSources_997015_6.stderr parsl.dataflow.dflow INFO: Task 3444 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e13395a491da1efdda97ac2948bddb6f with result from task 3444 parsl.dataflow.dflow DEBUG: Task 3444 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/74029931-c4ca-4628-b14e-664e7c0ca552_isrForCrosstalkSources_996965_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/74029931-c4ca-4628-b14e-664e7c0ca552_isrForCrosstalkSources_996965_29.stderr parsl.dataflow.dflow INFO: Task 3482 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 398177264d407c3692a3f86d28fa167f with result from task 3482 parsl.dataflow.dflow DEBUG: Task 3482 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/5929babd-06f4-4b66-9096-d8af9fdf7c85_isrForCrosstalkSources_997005_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/5929babd-06f4-4b66-9096-d8af9fdf7c85_isrForCrosstalkSources_997005_32.stderr parsl.dataflow.dflow INFO: Task 3594 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3e8b5dd9fba18b4e69abe8f1e2ffb05c with result from task 3594 parsl.dataflow.dflow DEBUG: Task 3594 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cc774f27-a6b6-4edf-9ed2-6dfa3ae063d3_isrForCrosstalkSources_996978_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cc774f27-a6b6-4edf-9ed2-6dfa3ae063d3_isrForCrosstalkSources_996978_41.stderr parsl.dataflow.dflow INFO: Task 3461 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50fccc24b0fe1cea257ae38ff7823450 with result from task 3461 parsl.dataflow.dflow DEBUG: Task 3461 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/deea36e6-f0e8-4cb9-80a2-6ae44fb6f13d_isrForCrosstalkSources_996953_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/deea36e6-f0e8-4cb9-80a2-6ae44fb6f13d_isrForCrosstalkSources_996953_5.stderr parsl.dataflow.dflow INFO: Task 3450 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9f3da8f270677c1afb43f0bc8330889d with result from task 3450 parsl.dataflow.dflow DEBUG: Task 3450 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/210d0ada-7e18-4f6a-81e0-fb9bc08cdb81_isrForCrosstalkSources_997017_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/210d0ada-7e18-4f6a-81e0-fb9bc08cdb81_isrForCrosstalkSources_997017_23.stderr parsl.dataflow.dflow INFO: Task 3446 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07c0b5a854f398361babbd436a92b1fc with result from task 3446 parsl.dataflow.dflow DEBUG: Task 3446 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e84c5fd7-97b0-4700-afbd-5d7c1b719e7b_isrForCrosstalkSources_996957_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e84c5fd7-97b0-4700-afbd-5d7c1b719e7b_isrForCrosstalkSources_996957_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:44 proc_lsst.multi:146 [INFO] found job 17075103 in provider checkpoint proc_lsst.multi INFO: found job 17075103 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3894 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26c8635acb0bcd95e8c19f71119d8e4a with result from task 3894 parsl.dataflow.dflow DEBUG: Task 3894 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b0928436-285d-4daa-a6b4-77dddb3a75b5_isrForCrosstalkSources_997006_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b0928436-285d-4daa-a6b4-77dddb3a75b5_isrForCrosstalkSources_997006_8.stderr parsl.dataflow.dflow INFO: Task 3505 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a2a53ce7544ce7f6870b5511f09786b with result from task 3505 parsl.dataflow.dflow DEBUG: Task 3505 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a0f7df20-1751-4372-9c3f-fc52d4bcbc7b_isrForCrosstalkSources_997026_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a0f7df20-1751-4372-9c3f-fc52d4bcbc7b_isrForCrosstalkSources_997026_40.stderr parsl.dataflow.dflow INFO: Task 3452 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44911f678e4383fe6fd618872f00c191 with result from task 3452 parsl.dataflow.dflow DEBUG: Task 3452 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/6ac1f232-52cd-461a-97dc-0b8b01a8a605_isrForCrosstalkSources_996994_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/6ac1f232-52cd-461a-97dc-0b8b01a8a605_isrForCrosstalkSources_996994_61.stderr parsl.dataflow.dflow INFO: Task 3634 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af981772b1b47b58dd6c9308eba209f2 with result from task 3634 parsl.dataflow.dflow DEBUG: Task 3634 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/4a032281-d50e-4d98-b4a1-da1012f7ef67_isrForCrosstalkSources_997029_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/4a032281-d50e-4d98-b4a1-da1012f7ef67_isrForCrosstalkSources_997029_27.stderr parsl.dataflow.dflow INFO: Task 3497 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6685194f859ed4ae81a26445870671a with result from task 3497 parsl.dataflow.dflow DEBUG: Task 3497 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c6f378d2-d4bb-4da2-8e11-df7e29afead1_isrForCrosstalkSources_996977_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/c6f378d2-d4bb-4da2-8e11-df7e29afead1_isrForCrosstalkSources_996977_20.stderr parsl.dataflow.dflow INFO: Task 3553 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97a008c7b51f0ec6c5ffccae2d10ef4d with result from task 3553 parsl.dataflow.dflow DEBUG: Task 3553 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/c613ac2f-e14b-4f72-a8fc-059ffb9ede01_isrForCrosstalkSources_997026_34.stdout parsl.dataflow.dflow INFO: Standard error for task 3553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/c613ac2f-e14b-4f72-a8fc-059ffb9ede01_isrForCrosstalkSources_997026_34.stderr parsl.dataflow.dflow INFO: Task 3908 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b0f0c3ca419f6f609617cd36d0d4a35 with result from task 3908 parsl.dataflow.dflow DEBUG: Task 3908 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/986b5c43-72c5-4b0e-badc-b4a1ae9f0aec_isrForCrosstalkSources_996983_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/986b5c43-72c5-4b0e-badc-b4a1ae9f0aec_isrForCrosstalkSources_996983_14.stderr parsl.dataflow.dflow INFO: Task 3537 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9409387715b54c9788664b1135116b5f with result from task 3537 parsl.dataflow.dflow DEBUG: Task 3537 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/a2ce60f4-8fe6-44bb-a535-bce37781b350_isrForCrosstalkSources_996959_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/a2ce60f4-8fe6-44bb-a535-bce37781b350_isrForCrosstalkSources_996959_22.stderr parsl.dataflow.dflow INFO: Task 3585 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f7d072b3b66232ebfda3db220d2c4a5 with result from task 3585 parsl.dataflow.dflow DEBUG: Task 3585 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/d31eb312-5e71-40f0-8376-2da8fccbd323_isrForCrosstalkSources_996953_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/d31eb312-5e71-40f0-8376-2da8fccbd323_isrForCrosstalkSources_996953_28.stderr parsl.dataflow.dflow INFO: Task 3621 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8002e1ecc3fdf98f2a60a1fceb195bdb with result from task 3621 parsl.dataflow.dflow DEBUG: Task 3621 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/500b9fd6-4cad-475f-8a9d-734f5d3720cc_isrForCrosstalkSources_997006_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/500b9fd6-4cad-475f-8a9d-734f5d3720cc_isrForCrosstalkSources_997006_37.stderr parsl.dataflow.dflow INFO: Task 3876 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5891b0bb8f784c2c98b86e31ed3b2b0 with result from task 3876 parsl.dataflow.dflow DEBUG: Task 3876 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/a0eb375e-e6d5-4fe4-8e2d-0a141376637b_isrForCrosstalkSources_997031_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/a0eb375e-e6d5-4fe4-8e2d-0a141376637b_isrForCrosstalkSources_997031_21.stderr parsl.dataflow.dflow INFO: Task 3471 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 208cc31f54358f74d6d7af7d30a43e49 with result from task 3471 parsl.dataflow.dflow DEBUG: Task 3471 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/80cc9b78-9007-43fa-9b5e-c62d141bb795_isrForCrosstalkSources_996979_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/80cc9b78-9007-43fa-9b5e-c62d141bb795_isrForCrosstalkSources_996979_31.stderr parsl.dataflow.dflow INFO: Task 3470 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59faab8666c07d6ddf7949396b74c604 with result from task 3470 parsl.dataflow.dflow DEBUG: Task 3470 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/694e9c62-cafb-4cf1-94b5-434072166fdb_isrForCrosstalkSources_996976_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/694e9c62-cafb-4cf1-94b5-434072166fdb_isrForCrosstalkSources_996976_31.stderr parsl.dataflow.dflow INFO: Task 3896 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5da0defd4c1b90ffe20488ed1cdbd660 with result from task 3896 parsl.dataflow.dflow DEBUG: Task 3896 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/49f2faf5-6db1-4d5f-9d6b-ef0c95ce782e_isrForCrosstalkSources_997030_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/49f2faf5-6db1-4d5f-9d6b-ef0c95ce782e_isrForCrosstalkSources_997030_29.stderr parsl.dataflow.dflow INFO: Task 3646 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e00c51d98a1c19848156071101e2374 with result from task 3646 parsl.dataflow.dflow DEBUG: Task 3646 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/bce40ee1-93d4-4e10-9b4a-3a1e00726c04_isrForCrosstalkSources_996985_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/bce40ee1-93d4-4e10-9b4a-3a1e00726c04_isrForCrosstalkSources_996985_43.stderr parsl.dataflow.dflow INFO: Task 3898 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 799df569a3a56ddb923b69b8278fa38f with result from task 3898 parsl.dataflow.dflow DEBUG: Task 3898 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/98156d87-8325-4b8c-9c0d-f7f13bfa86c3_isrForCrosstalkSources_996977_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/98156d87-8325-4b8c-9c0d-f7f13bfa86c3_isrForCrosstalkSources_996977_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:45 proc_lsst.multi:146 [INFO] found job 17075109 in provider checkpoint proc_lsst.multi INFO: found job 17075109 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3654 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46c96f44fcb4a39602734a04dea750a8 with result from task 3654 parsl.dataflow.dflow DEBUG: Task 3654 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e69dc506-ea7c-4a64-9713-04bfdfd60993_isrForCrosstalkSources_997002_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e69dc506-ea7c-4a64-9713-04bfdfd60993_isrForCrosstalkSources_997002_13.stderr parsl.dataflow.dflow INFO: Task 3673 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a24e4f91e3f05d0d006283fab0e361d1 with result from task 3673 parsl.dataflow.dflow DEBUG: Task 3673 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a7ef39e-9c58-46d7-ab0a-093c83ba2676_isrForCrosstalkSources_996994_14.stdout parsl.dataflow.dflow INFO: Standard error for task 3673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a7ef39e-9c58-46d7-ab0a-093c83ba2676_isrForCrosstalkSources_996994_14.stderr parsl.dataflow.dflow INFO: Task 3696 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c9e655b94c75ce2fe97e50be9f7e606 with result from task 3696 parsl.dataflow.dflow DEBUG: Task 3696 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/f5855bb2-9e45-43d6-bdfb-c2df0db7bef8_isrForCrosstalkSources_997009_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/f5855bb2-9e45-43d6-bdfb-c2df0db7bef8_isrForCrosstalkSources_997009_56.stderr parsl.dataflow.dflow INFO: Task 3629 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f9407015c3e56ce99f1f4d02819d6b60 with result from task 3629 parsl.dataflow.dflow DEBUG: Task 3629 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/29c58776-1b31-48dd-948d-ebb7f88a153a_isrForCrosstalkSources_996983_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/29c58776-1b31-48dd-948d-ebb7f88a153a_isrForCrosstalkSources_996983_4.stderr parsl.dataflow.dflow INFO: Task 3417 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e232ff6eeacfeb1cbeaf70e0694c133e with result from task 3417 parsl.dataflow.dflow DEBUG: Task 3417 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/10e87278-2ffd-4dbc-8eea-51c30207e84a_isrForCrosstalkSources_997018_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/10e87278-2ffd-4dbc-8eea-51c30207e84a_isrForCrosstalkSources_997018_53.stderr parsl.dataflow.dflow INFO: Task 3581 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f214e5a12962b7c43043efbf8b6d8a1 with result from task 3581 parsl.dataflow.dflow DEBUG: Task 3581 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/188e5004-858a-4f53-b4d4-fcd1999fe01d_isrForCrosstalkSources_996953_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/188e5004-858a-4f53-b4d4-fcd1999fe01d_isrForCrosstalkSources_996953_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:46 proc_lsst.multi:146 [INFO] found job 17075110 in provider checkpoint proc_lsst.multi INFO: found job 17075110 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3910 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a3660c49e0b82fb1769c406ab6bad638 with result from task 3910 parsl.dataflow.dflow DEBUG: Task 3910 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2e574040-a26f-4efc-a428-3a313ae16ba5_isrForCrosstalkSources_996980_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/2e574040-a26f-4efc-a428-3a313ae16ba5_isrForCrosstalkSources_996980_43.stderr parsl.dataflow.dflow INFO: Task 3909 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd0b7f31690bbb0325197cdba974bcab with result from task 3909 parsl.dataflow.dflow DEBUG: Task 3909 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/4909aaa2-5853-42be-87b0-fc796e55cd88_isrForCrosstalkSources_996979_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/4909aaa2-5853-42be-87b0-fc796e55cd88_isrForCrosstalkSources_996979_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:46 proc_lsst.multi:146 [INFO] found job 17075111 in provider checkpoint proc_lsst.multi INFO: found job 17075111 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3552 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33ed8db261a8d04c86d51cbe3171add8 with result from task 3552 parsl.dataflow.dflow DEBUG: Task 3552 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fa9a6e44-2d50-443e-9077-62260c8eb794_isrForCrosstalkSources_996968_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fa9a6e44-2d50-443e-9077-62260c8eb794_isrForCrosstalkSources_996968_44.stderr parsl.dataflow.dflow INFO: Task 3865 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 883d3000462ee18117380d5e5b175176 with result from task 3865 parsl.dataflow.dflow DEBUG: Task 3865 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/6d0b7c7e-44ce-4631-bd54-fa1ea2d5b0e4_isrForCrosstalkSources_996962_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/6d0b7c7e-44ce-4631-bd54-fa1ea2d5b0e4_isrForCrosstalkSources_996962_39.stderr parsl.dataflow.dflow INFO: Task 3692 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3f62affc5f4aee305904fde8cfac4e7 with result from task 3692 parsl.dataflow.dflow DEBUG: Task 3692 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/da98c1c5-2705-4aa8-826d-8c967fdd8855_isrForCrosstalkSources_996977_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/da98c1c5-2705-4aa8-826d-8c967fdd8855_isrForCrosstalkSources_996977_22.stderr parsl.dataflow.dflow INFO: Task 3698 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 150421785737e417f3442aa685f56944 with result from task 3698 parsl.dataflow.dflow DEBUG: Task 3698 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/718b4f64-70b6-448e-ae53-d782692cb982_isrForCrosstalkSources_997019_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/718b4f64-70b6-448e-ae53-d782692cb982_isrForCrosstalkSources_997019_32.stderr parsl.dataflow.dflow INFO: Task 3731 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4cd6e9ec58555b29189e757f1e3ff2e2 with result from task 3731 parsl.dataflow.dflow DEBUG: Task 3731 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ea48cd38-d2fc-4cda-8fb9-3795caa7f7d3_isrForCrosstalkSources_996984_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/ea48cd38-d2fc-4cda-8fb9-3795caa7f7d3_isrForCrosstalkSources_996984_32.stderr parsl.dataflow.dflow INFO: Task 3707 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aaa7af4cf893014f3de69a7b9412e67d with result from task 3707 parsl.dataflow.dflow DEBUG: Task 3707 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/0d61aace-95c7-4e4a-8613-e7fa90acebf3_isrForCrosstalkSources_997016_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/0d61aace-95c7-4e4a-8613-e7fa90acebf3_isrForCrosstalkSources_997016_24.stderr parsl.dataflow.dflow INFO: Task 3511 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 24fc4d4c5e41ad17cf1d932f20261017 with result from task 3511 parsl.dataflow.dflow DEBUG: Task 3511 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/62516faa-fee6-4eb9-ab29-52eebaa76293_isrForCrosstalkSources_997010_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/62516faa-fee6-4eb9-ab29-52eebaa76293_isrForCrosstalkSources_997010_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:47 proc_lsst.multi:146 [INFO] found job 17075112 in provider checkpoint proc_lsst.multi INFO: found job 17075112 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3550 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a69086b5a3f9a9dfb7af797d3ab3aaf with result from task 3550 parsl.dataflow.dflow DEBUG: Task 3550 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/903eef68-8311-4f60-9737-8c1a2ee25e9e_isrForCrosstalkSources_997026_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/903eef68-8311-4f60-9737-8c1a2ee25e9e_isrForCrosstalkSources_997026_52.stderr parsl.dataflow.dflow INFO: Task 3873 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02d17c3c7e6a8bfb7ca86289f03473b4 with result from task 3873 parsl.dataflow.dflow DEBUG: Task 3873 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3e921ba9-1e13-4e08-adaa-b728d880f261_isrForCrosstalkSources_996963_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3e921ba9-1e13-4e08-adaa-b728d880f261_isrForCrosstalkSources_996963_4.stderr parsl.dataflow.dflow INFO: Task 3824 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a7f516d6c873d75d0a3271f9145ef622 with result from task 3824 parsl.dataflow.dflow DEBUG: Task 3824 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/57e027f6-d58b-4cbc-8078-4a80f54b7b31_isrForCrosstalkSources_996959_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/57e027f6-d58b-4cbc-8078-4a80f54b7b31_isrForCrosstalkSources_996959_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:47 proc_lsst.multi:146 [INFO] found job 17075113 in provider checkpoint proc_lsst.multi INFO: found job 17075113 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3915 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba38af361cb36fb05d31f0556288e14f with result from task 3915 parsl.dataflow.dflow DEBUG: Task 3915 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3afdbe06-5083-4b2d-a96f-f6ad85c58a2f_isrForCrosstalkSources_997033_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/3afdbe06-5083-4b2d-a96f-f6ad85c58a2f_isrForCrosstalkSources_997033_3.stderr parsl.dataflow.dflow INFO: Task 3603 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 266e504ad1a860684a7763abf32f9802 with result from task 3603 parsl.dataflow.dflow DEBUG: Task 3603 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f562c574-e098-4ce0-9880-405c187cf3f0_isrForCrosstalkSources_996961_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/f562c574-e098-4ce0-9880-405c187cf3f0_isrForCrosstalkSources_996961_62.stderr parsl.dataflow.dflow INFO: Task 3544 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97e3df9031eab74363831e9c453e334e with result from task 3544 parsl.dataflow.dflow DEBUG: Task 3544 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/4fc8f5d5-7d0d-4739-b115-342f63bb1fad_isrForCrosstalkSources_996982_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/4fc8f5d5-7d0d-4739-b115-342f63bb1fad_isrForCrosstalkSources_996982_61.stderr parsl.dataflow.dflow INFO: Task 3719 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b3e418b4aa875e704f8d721d5615f91a with result from task 3719 parsl.dataflow.dflow DEBUG: Task 3719 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/7e3b0276-b573-4d40-8f24-d785ab1bf499_isrForCrosstalkSources_997030_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/7e3b0276-b573-4d40-8f24-d785ab1bf499_isrForCrosstalkSources_997030_4.stderr parsl.dataflow.dflow INFO: Task 3799 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5730ad896e39d3778733ef1f65da220 with result from task 3799 parsl.dataflow.dflow DEBUG: Task 3799 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/3656be07-bbba-4d08-9581-99ce5e9d7846_isrForCrosstalkSources_997008_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/3656be07-bbba-4d08-9581-99ce5e9d7846_isrForCrosstalkSources_997008_25.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:48 proc_lsst.multi:146 [INFO] found job 17075114 in provider checkpoint proc_lsst.multi INFO: found job 17075114 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3754 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf595e8776bdb4f30fdb05ab35ff914a with result from task 3754 parsl.dataflow.dflow DEBUG: Task 3754 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e4155eb4-31bf-464a-bad9-20324c182cf0_isrForCrosstalkSources_996998_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e4155eb4-31bf-464a-bad9-20324c182cf0_isrForCrosstalkSources_996998_16.stderr parsl.dataflow.dflow INFO: Task 3649 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1bdc228b261b03c7711408fd3ad1a69b with result from task 3649 parsl.dataflow.dflow DEBUG: Task 3649 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0e59bb08-4589-4e75-800f-bc090e04692b_isrForCrosstalkSources_996956_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/0e59bb08-4589-4e75-800f-bc090e04692b_isrForCrosstalkSources_996956_18.stderr parsl.dataflow.dflow INFO: Task 3849 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f1ae735e781737e2aa2d811a14d8c226 with result from task 3849 parsl.dataflow.dflow DEBUG: Task 3849 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e144f878-be4d-419c-abca-1eb44c0454f0_isrForCrosstalkSources_996998_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/e144f878-be4d-419c-abca-1eb44c0454f0_isrForCrosstalkSources_996998_11.stderr parsl.dataflow.dflow INFO: Task 3625 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7e9b1fa0fd4b4f2a53105b556f99873 with result from task 3625 parsl.dataflow.dflow DEBUG: Task 3625 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7087d550-d870-4c11-9330-e832f3bffb6c_isrForCrosstalkSources_996974_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7087d550-d870-4c11-9330-e832f3bffb6c_isrForCrosstalkSources_996974_7.stderr parsl.dataflow.dflow INFO: Task 3762 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 24c5dc22a1982dd8dcbb8092d7caf412 with result from task 3762 parsl.dataflow.dflow DEBUG: Task 3762 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d3f4a938-ac1e-4bdb-aeb1-7519026d8089_isrForCrosstalkSources_997008_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d3f4a938-ac1e-4bdb-aeb1-7519026d8089_isrForCrosstalkSources_997008_32.stderr parsl.dataflow.dflow INFO: Task 3645 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46796cb9fbecc0e2ae7c066a7df51be2 with result from task 3645 parsl.dataflow.dflow DEBUG: Task 3645 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d58562ef-3cf2-4f1d-8cb5-357d96776384_isrForCrosstalkSources_996973_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d58562ef-3cf2-4f1d-8cb5-357d96776384_isrForCrosstalkSources_996973_16.stderr parsl.dataflow.dflow INFO: Task 3921 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd6c9adbe309cdee8f58260d9b97493a with result from task 3921 parsl.dataflow.dflow DEBUG: Task 3921 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6627218a-313b-4596-adb8-b2eb5a1c36f5_isrForCrosstalkSources_997005_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/6627218a-313b-4596-adb8-b2eb5a1c36f5_isrForCrosstalkSources_997005_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:48 proc_lsst.multi:146 [INFO] found job 17075115 in provider checkpoint proc_lsst.multi INFO: found job 17075115 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3475 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f12ba40306ff64548577a382f245c28 with result from task 3475 parsl.dataflow.dflow DEBUG: Task 3475 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/06e0bbdc-8fd1-42e1-a8db-90319fa54086_isrForCrosstalkSources_996954_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/06e0bbdc-8fd1-42e1-a8db-90319fa54086_isrForCrosstalkSources_996954_55.stderr parsl.dataflow.dflow INFO: Task 3853 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ead73f0d091576fa91a6eb37f71964d with result from task 3853 parsl.dataflow.dflow DEBUG: Task 3853 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/bae2e892-7951-4cf0-8b3a-9c959c04fa6d_isrForCrosstalkSources_997029_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/bae2e892-7951-4cf0-8b3a-9c959c04fa6d_isrForCrosstalkSources_997029_21.stderr parsl.dataflow.dflow INFO: Task 3827 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 89a0b9739e5aad5a34494a03be0372c8 with result from task 3827 parsl.dataflow.dflow DEBUG: Task 3827 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/80925e69-b017-4ca9-939a-65faf40d7be6_isrForCrosstalkSources_996964_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/80925e69-b017-4ca9-939a-65faf40d7be6_isrForCrosstalkSources_996964_38.stderr parsl.dataflow.dflow INFO: Task 3877 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d1e4d807708deba27d543c844c504844 with result from task 3877 parsl.dataflow.dflow DEBUG: Task 3877 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0893aa6c-eac4-4209-9f2a-e2720e37527f_isrForCrosstalkSources_996975_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0893aa6c-eac4-4209-9f2a-e2720e37527f_isrForCrosstalkSources_996975_35.stderr parsl.dataflow.dflow INFO: Task 3703 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 92d4ac0e4d54bfe5bb4c677614dff73c with result from task 3703 parsl.dataflow.dflow DEBUG: Task 3703 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/23d59f5c-cf14-4fa0-806f-8dbe177de133_isrForCrosstalkSources_997008_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/23d59f5c-cf14-4fa0-806f-8dbe177de133_isrForCrosstalkSources_997008_51.stderr parsl.dataflow.dflow INFO: Task 3711 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e45892eb1ee173448821f4b8bce80a7b with result from task 3711 parsl.dataflow.dflow DEBUG: Task 3711 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/52b1310d-3a59-43b9-83cd-1da29040c07d_isrForCrosstalkSources_997024_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/52b1310d-3a59-43b9-83cd-1da29040c07d_isrForCrosstalkSources_997024_4.stderr parsl.dataflow.dflow INFO: Task 3936 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c3681f5f026aa00dd71ea7e0c1dc918 with result from task 3936 parsl.dataflow.dflow DEBUG: Task 3936 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/bd9eca6d-1191-4b37-afab-966aa6220a59_isrForCrosstalkSources_996972_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/bd9eca6d-1191-4b37-afab-966aa6220a59_isrForCrosstalkSources_996972_20.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:49 proc_lsst.multi:146 [INFO] found job 17075116 in provider checkpoint proc_lsst.multi INFO: found job 17075116 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3920 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6790e3825fe095893b11d12f36bfd1b9 with result from task 3920 parsl.dataflow.dflow DEBUG: Task 3920 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/bd357125-f005-421a-a501-95b15255d2bc_isrForCrosstalkSources_997002_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/bd357125-f005-421a-a501-95b15255d2bc_isrForCrosstalkSources_997002_10.stderr parsl.dataflow.dflow INFO: Task 3907 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 867dda5b56dedc71d64a09f571f4f89e with result from task 3907 parsl.dataflow.dflow DEBUG: Task 3907 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/90fd8756-c39d-4029-ad25-54bd9a11e214_isrForCrosstalkSources_996970_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/90fd8756-c39d-4029-ad25-54bd9a11e214_isrForCrosstalkSources_996970_53.stderr parsl.dataflow.dflow INFO: Task 3684 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d3500f42be19124ce21617ad5dd4f3c9 with result from task 3684 parsl.dataflow.dflow DEBUG: Task 3684 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/4fbe933d-2663-4f0c-a68c-2b4bbf28e517_isrForCrosstalkSources_996974_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/4fbe933d-2663-4f0c-a68c-2b4bbf28e517_isrForCrosstalkSources_996974_18.stderr parsl.dataflow.dflow INFO: Task 3847 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a4b5b61e0f421701128a794ac02992f with result from task 3847 parsl.dataflow.dflow DEBUG: Task 3847 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/aea4d690-a160-4863-9fd6-e1cf91b11eb3_isrForCrosstalkSources_996973_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/aea4d690-a160-4863-9fd6-e1cf91b11eb3_isrForCrosstalkSources_996973_25.stderr parsl.dataflow.dflow INFO: Task 3509 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 990d434121d514a82fe1a37cfccefef3 with result from task 3509 parsl.dataflow.dflow DEBUG: Task 3509 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/75f0d663-7b42-44a6-82aa-a50e21853311_isrForCrosstalkSources_997028_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/75f0d663-7b42-44a6-82aa-a50e21853311_isrForCrosstalkSources_997028_52.stderr parsl.dataflow.dflow INFO: Task 3935 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50a2a839c4832e8eefda5f56079205bd with result from task 3935 parsl.dataflow.dflow DEBUG: Task 3935 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/251e940d-d9c0-45e3-bb3a-9e4c428cb50d_isrForCrosstalkSources_996955_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/251e940d-d9c0-45e3-bb3a-9e4c428cb50d_isrForCrosstalkSources_996955_4.stderr parsl.dataflow.dflow INFO: Task 3778 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ce7f251b8bb78ec241bb71c1295ccba with result from task 3778 parsl.dataflow.dflow DEBUG: Task 3778 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a11a0ec0-f0cd-473b-a507-4d06f326a5d9_isrForCrosstalkSources_997008_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a11a0ec0-f0cd-473b-a507-4d06f326a5d9_isrForCrosstalkSources_997008_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:49 proc_lsst.multi:146 [INFO] found job 17075117 in provider checkpoint proc_lsst.multi INFO: found job 17075117 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3929 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2f18110ff7de029be88d633d3c04fb7 with result from task 3929 parsl.dataflow.dflow DEBUG: Task 3929 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/26eb14b9-4ccf-4ee7-b6c7-ade3d207ecc7_isrForCrosstalkSources_997028_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/26eb14b9-4ccf-4ee7-b6c7-ade3d207ecc7_isrForCrosstalkSources_997028_42.stderr parsl.dataflow.dflow INFO: Task 3804 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b238d3377af727d892bab61a355b31ad with result from task 3804 parsl.dataflow.dflow DEBUG: Task 3804 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/591be311-4487-4480-adab-b4c239d43c92_isrForCrosstalkSources_997008_43.stdout parsl.dataflow.dflow INFO: Standard error for task 3804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/591be311-4487-4480-adab-b4c239d43c92_isrForCrosstalkSources_997008_43.stderr parsl.dataflow.dflow INFO: Task 3938 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 749d7f5bfd59b4006cd851ac0f3ca61d with result from task 3938 parsl.dataflow.dflow DEBUG: Task 3938 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/975c472a-9194-4dbe-a6e7-23f9895af637_isrForCrosstalkSources_996961_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/975c472a-9194-4dbe-a6e7-23f9895af637_isrForCrosstalkSources_996961_38.stderr parsl.dataflow.dflow INFO: Task 3664 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7457f3e9959b9158b644247813f0589d with result from task 3664 parsl.dataflow.dflow DEBUG: Task 3664 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/8f5e13c3-86e2-4e6f-97b9-3b728d87e98b_isrForCrosstalkSources_996996_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/8f5e13c3-86e2-4e6f-97b9-3b728d87e98b_isrForCrosstalkSources_996996_25.stderr parsl.dataflow.dflow INFO: Task 3818 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d8f0358eeacc2026f8658f7020190e9 with result from task 3818 parsl.dataflow.dflow DEBUG: Task 3818 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d3e5c72a-7900-4be6-9c24-759dc29b018d_isrForCrosstalkSources_996998_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/d3e5c72a-7900-4be6-9c24-759dc29b018d_isrForCrosstalkSources_996998_7.stderr parsl.dataflow.dflow INFO: Task 3738 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0795a3955f7242055e4606b198718b89 with result from task 3738 parsl.dataflow.dflow DEBUG: Task 3738 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/aa507461-fd6d-4226-ae18-6d72f1a64dac_isrForCrosstalkSources_996961_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/aa507461-fd6d-4226-ae18-6d72f1a64dac_isrForCrosstalkSources_996961_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:50 proc_lsst.multi:146 [INFO] found job 17075118 in provider checkpoint proc_lsst.multi INFO: found job 17075118 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3944 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 326ac28962f109e85e061afc141f2800 with result from task 3944 parsl.dataflow.dflow DEBUG: Task 3944 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f536e41f-6223-4f84-b04f-71a0909a6822_isrForCrosstalkSources_997019_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f536e41f-6223-4f84-b04f-71a0909a6822_isrForCrosstalkSources_997019_38.stderr parsl.dataflow.dflow INFO: Task 3843 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 475828ff7ece545c127e158ab98abda8 with result from task 3843 parsl.dataflow.dflow DEBUG: Task 3843 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1954f380-0ea4-46f1-b50d-27e3dffb2a43_isrForCrosstalkSources_997027_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/1954f380-0ea4-46f1-b50d-27e3dffb2a43_isrForCrosstalkSources_997027_26.stderr parsl.dataflow.dflow INFO: Task 3923 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b8ec39f3a92faa9e6e8d5912c178a3e with result from task 3923 parsl.dataflow.dflow DEBUG: Task 3923 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9d7406d8-75f2-416d-8229-85b381fa584a_isrForCrosstalkSources_997030_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/9d7406d8-75f2-416d-8229-85b381fa584a_isrForCrosstalkSources_997030_46.stderr parsl.dataflow.dflow INFO: Task 3854 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a7aa896cd43ddc6f353cf0b5462c563 with result from task 3854 parsl.dataflow.dflow DEBUG: Task 3854 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/cf07f2ac-e419-4ac3-ae74-8f123fee8be3_isrForCrosstalkSources_997021_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/cf07f2ac-e419-4ac3-ae74-8f123fee8be3_isrForCrosstalkSources_997021_31.stderr parsl.dataflow.dflow INFO: Task 3676 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ded71e1032a9348440353db6436ff905 with result from task 3676 parsl.dataflow.dflow DEBUG: Task 3676 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c6fe5391-d2e2-4978-9f43-860a850a1173_isrForCrosstalkSources_996980_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c6fe5391-d2e2-4978-9f43-860a850a1173_isrForCrosstalkSources_996980_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:50 proc_lsst.multi:146 [INFO] found job 17075121 in provider checkpoint proc_lsst.multi INFO: found job 17075121 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3881 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b41beefc39852e651fc5cb07ec443704 with result from task 3881 parsl.dataflow.dflow DEBUG: Task 3881 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4e095491-0579-40b6-b454-1a547285dd0e_isrForCrosstalkSources_997023_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4e095491-0579-40b6-b454-1a547285dd0e_isrForCrosstalkSources_997023_8.stderr parsl.dataflow.dflow INFO: Task 3770 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3cc6ee0908b4dd68dc096d1b022fc1f2 with result from task 3770 parsl.dataflow.dflow DEBUG: Task 3770 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/2d031450-a403-4123-97df-fbfb48bfd6a3_isrForCrosstalkSources_997013_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/2d031450-a403-4123-97df-fbfb48bfd6a3_isrForCrosstalkSources_997013_4.stderr parsl.dataflow.dflow INFO: Task 3751 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4dd724a6e2f0900dc98509e530bc565 with result from task 3751 parsl.dataflow.dflow DEBUG: Task 3751 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4387a87e-f040-4415-a0af-c692a8b1bea5_isrForCrosstalkSources_996962_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/4387a87e-f040-4415-a0af-c692a8b1bea5_isrForCrosstalkSources_996962_22.stderr parsl.dataflow.dflow INFO: Task 3924 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0d21a22e790f1a5dc4475540a796d278 with result from task 3924 parsl.dataflow.dflow DEBUG: Task 3924 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/efc2559f-e137-46c9-880c-36cbf3272d63_isrForCrosstalkSources_996997_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/efc2559f-e137-46c9-880c-36cbf3272d63_isrForCrosstalkSources_996997_56.stderr parsl.dataflow.dflow INFO: Task 3806 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b28908f601c5cf5b363b341f09530ef4 with result from task 3806 parsl.dataflow.dflow DEBUG: Task 3806 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/1ba0243b-5a7a-4352-91fc-f04c8f50461e_isrForCrosstalkSources_996962_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/1ba0243b-5a7a-4352-91fc-f04c8f50461e_isrForCrosstalkSources_996962_40.stderr parsl.dataflow.dflow INFO: Task 3743 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 167b88a450c1ae38756150a66a98ab39 with result from task 3743 parsl.dataflow.dflow DEBUG: Task 3743 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0a08623f-10ee-4367-81c7-6eafb994c3f3_isrForCrosstalkSources_996980_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0a08623f-10ee-4367-81c7-6eafb994c3f3_isrForCrosstalkSources_996980_23.stderr parsl.dataflow.dflow INFO: Task 3631 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d735972e84f7ea3362fa2b50240b24f7 with result from task 3631 parsl.dataflow.dflow DEBUG: Task 3631 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/2eb3e35d-12d9-4233-a755-bd762ddf4a94_isrForCrosstalkSources_997033_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/2eb3e35d-12d9-4233-a755-bd762ddf4a94_isrForCrosstalkSources_997033_1.stderr parsl.dataflow.dflow INFO: Task 3867 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fbe075357b740e1803d84cebf970099b with result from task 3867 parsl.dataflow.dflow DEBUG: Task 3867 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/9133372a-6fbb-43f5-9957-876ab93142c9_isrForCrosstalkSources_996974_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/9133372a-6fbb-43f5-9957-876ab93142c9_isrForCrosstalkSources_996974_46.stderr parsl.dataflow.dflow INFO: Task 3949 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51d4e25a9b5987ebee7fa7989d1521ad with result from task 3949 parsl.dataflow.dflow DEBUG: Task 3949 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5119db02-c28a-40d0-8194-81b2d4d65e00_isrForCrosstalkSources_997017_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/5119db02-c28a-40d0-8194-81b2d4d65e00_isrForCrosstalkSources_997017_21.stderr parsl.dataflow.dflow INFO: Task 3948 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2f1d291a76f6132302e2d1ced86318b with result from task 3948 parsl.dataflow.dflow DEBUG: Task 3948 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f452c756-d342-4adc-b36d-d56570e98d70_isrForCrosstalkSources_996990_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f452c756-d342-4adc-b36d-d56570e98d70_isrForCrosstalkSources_996990_3.stderr parsl.dataflow.dflow INFO: Task 3932 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78cae430ae687eeb40bfee34ca755e67 with result from task 3932 parsl.dataflow.dflow DEBUG: Task 3932 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/71018524-c372-4267-98ae-d980d8c550d3_isrForCrosstalkSources_996973_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/71018524-c372-4267-98ae-d980d8c550d3_isrForCrosstalkSources_996973_48.stderr parsl.dataflow.dflow INFO: Task 3753 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 79524ef136fe15ae45618b12b9206397 with result from task 3753 parsl.dataflow.dflow DEBUG: Task 3753 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e9df7005-33b9-43c0-a5b5-6a982406c126_isrForCrosstalkSources_996983_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e9df7005-33b9-43c0-a5b5-6a982406c126_isrForCrosstalkSources_996983_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:52 proc_lsst.multi:146 [INFO] found job 17075122 in provider checkpoint proc_lsst.multi INFO: found job 17075122 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3959 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 214022bd0462ee1e4c0860ac6f7a00c2 with result from task 3959 parsl.dataflow.dflow DEBUG: Task 3959 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/d1c0344d-6040-4345-beec-44fca68aca87_isrForCrosstalkSources_996996_21.stdout parsl.dataflow.dflow INFO: Standard error for task 3959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/d1c0344d-6040-4345-beec-44fca68aca87_isrForCrosstalkSources_996996_21.stderr parsl.dataflow.dflow INFO: Task 3903 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d1ea9c153db5a6ee5e9f61a9067d5cb2 with result from task 3903 parsl.dataflow.dflow DEBUG: Task 3903 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/c83b3a15-62f6-45b7-9cb7-923a70cdb582_isrForCrosstalkSources_996958_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/c83b3a15-62f6-45b7-9cb7-923a70cdb582_isrForCrosstalkSources_996958_33.stderr parsl.dataflow.dflow INFO: Task 3875 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry deaf354953b493fe554e78ae30469059 with result from task 3875 parsl.dataflow.dflow DEBUG: Task 3875 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/ec5d445a-05e8-4725-8e74-d67c92652957_isrForCrosstalkSources_996953_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/ec5d445a-05e8-4725-8e74-d67c92652957_isrForCrosstalkSources_996953_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:53 proc_lsst.multi:146 [INFO] found job 17075124 in provider checkpoint proc_lsst.multi INFO: found job 17075124 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3850 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac361a51b01ef0c00193d52f49645c23 with result from task 3850 parsl.dataflow.dflow DEBUG: Task 3850 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/4ecc1f8a-6555-4eae-b73a-0128d04dec8a_isrForCrosstalkSources_997027_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/4ecc1f8a-6555-4eae-b73a-0128d04dec8a_isrForCrosstalkSources_997027_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:53 proc_lsst.multi:146 [INFO] found job 17075127 in provider checkpoint proc_lsst.multi INFO: found job 17075127 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3966 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 36c19b0f86f5e05d3c3b311ff54204b5 with result from task 3966 parsl.dataflow.dflow DEBUG: Task 3966 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c5977cf7-41da-4bfd-931a-9cdc4f4404e7_isrForCrosstalkSources_997014_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c5977cf7-41da-4bfd-931a-9cdc4f4404e7_isrForCrosstalkSources_997014_8.stderr parsl.dataflow.dflow INFO: Task 3960 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8065184f8039860af66587a71e4de1ec with result from task 3960 parsl.dataflow.dflow DEBUG: Task 3960 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/7cf31f18-2403-4653-8d46-b02f4af3322f_isrForCrosstalkSources_997031_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/7cf31f18-2403-4653-8d46-b02f4af3322f_isrForCrosstalkSources_997031_20.stderr parsl.dataflow.dflow INFO: Task 3883 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9edd78024147f0a37c4d22f4ebbba00d with result from task 3883 parsl.dataflow.dflow DEBUG: Task 3883 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/46cce164-2b51-4a88-933b-b5cdff6b0e9a_isrForCrosstalkSources_996999_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/46cce164-2b51-4a88-933b-b5cdff6b0e9a_isrForCrosstalkSources_996999_51.stderr parsl.dataflow.dflow INFO: Task 3805 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58c8ce2cb50b023c5c251608d92f06ec with result from task 3805 parsl.dataflow.dflow DEBUG: Task 3805 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/323875c8-f946-49ce-9ba8-8784d8559232_isrForCrosstalkSources_997003_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/323875c8-f946-49ce-9ba8-8784d8559232_isrForCrosstalkSources_997003_58.stderr parsl.dataflow.dflow INFO: Task 3952 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry edd7ab652a98cd2328561db70f22cc9c with result from task 3952 parsl.dataflow.dflow DEBUG: Task 3952 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/4e38ca91-a5f0-4276-9571-6dcbf918d0ee_isrForCrosstalkSources_996989_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/4e38ca91-a5f0-4276-9571-6dcbf918d0ee_isrForCrosstalkSources_996989_62.stderr parsl.dataflow.dflow INFO: Task 3825 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ee3e9610bf1e93680c4f17cd6813fe0 with result from task 3825 parsl.dataflow.dflow DEBUG: Task 3825 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/827f780b-5ff7-48c0-b192-b1e11fba9b1c_isrForCrosstalkSources_996992_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/827f780b-5ff7-48c0-b192-b1e11fba9b1c_isrForCrosstalkSources_996992_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:54 proc_lsst.multi:146 [INFO] found job 17075128 in provider checkpoint proc_lsst.multi INFO: found job 17075128 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3628 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f094742a7fd9be697ccc3dd91eaa68c5 with result from task 3628 parsl.dataflow.dflow DEBUG: Task 3628 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/da8ed39a-d1cf-4414-9a1d-650e10e3bb52_isrForCrosstalkSources_997009_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/da8ed39a-d1cf-4414-9a1d-650e10e3bb52_isrForCrosstalkSources_997009_5.stderr parsl.dataflow.dflow INFO: Task 3967 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fef271f92c7b22ffe6306b9196c90944 with result from task 3967 parsl.dataflow.dflow DEBUG: Task 3967 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/582ddccc-2939-4394-a2f0-3e1c0629484f_isrForCrosstalkSources_996975_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/582ddccc-2939-4394-a2f0-3e1c0629484f_isrForCrosstalkSources_996975_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:54 proc_lsst.multi:146 [INFO] found job 17075129 in provider checkpoint proc_lsst.multi INFO: found job 17075129 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3801 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46024288cb3cae0c3124b12d55fc43b8 with result from task 3801 parsl.dataflow.dflow DEBUG: Task 3801 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/8e79ef15-110f-4eb2-a94c-973be8dd6aea_isrForCrosstalkSources_997009_45.stdout parsl.dataflow.dflow INFO: Standard error for task 3801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/8e79ef15-110f-4eb2-a94c-973be8dd6aea_isrForCrosstalkSources_997009_45.stderr parsl.dataflow.dflow INFO: Task 3741 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 548138c8ba93e19746d94ed1c272b83a with result from task 3741 parsl.dataflow.dflow DEBUG: Task 3741 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/dd2f46a9-bf73-4d25-a0ad-0bb7cf5675b2_isrForCrosstalkSources_996969_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/dd2f46a9-bf73-4d25-a0ad-0bb7cf5675b2_isrForCrosstalkSources_996969_23.stderr parsl.dataflow.dflow INFO: Task 3637 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8f1b982dfb5d176915c087b27021f02 with result from task 3637 parsl.dataflow.dflow DEBUG: Task 3637 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/e871a168-01fb-4c48-b5e8-6d67dc9af9e9_isrForCrosstalkSources_996958_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/e871a168-01fb-4c48-b5e8-6d67dc9af9e9_isrForCrosstalkSources_996958_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:55 proc_lsst.multi:146 [INFO] found job 17075131 in provider checkpoint proc_lsst.multi INFO: found job 17075131 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3786 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c2b4ccf874c6a4b399652002101b1601 with result from task 3786 parsl.dataflow.dflow DEBUG: Task 3786 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e0e6870d-fb5d-4fc6-896c-f0c2cd1ae1bc_isrForCrosstalkSources_997005_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e0e6870d-fb5d-4fc6-896c-f0c2cd1ae1bc_isrForCrosstalkSources_997005_46.stderr parsl.dataflow.dflow INFO: Task 3724 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 60a0a63f98d7e3d6a5c9fca931efbb0d with result from task 3724 parsl.dataflow.dflow DEBUG: Task 3724 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e1dda7fb-400c-44fc-b562-c4c282d1b6a9_isrForCrosstalkSources_997030_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/e1dda7fb-400c-44fc-b562-c4c282d1b6a9_isrForCrosstalkSources_997030_44.stderr parsl.dataflow.dflow INFO: Task 3808 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86771a2a85d968aa6691db052aefbab2 with result from task 3808 parsl.dataflow.dflow DEBUG: Task 3808 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/01680928-b54b-417f-9ad8-c27fa46354be_isrForCrosstalkSources_997008_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/01680928-b54b-417f-9ad8-c27fa46354be_isrForCrosstalkSources_997008_47.stderr parsl.dataflow.dflow INFO: Task 4027 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b2b72a1b17d20af5e5ab38097d7484a4 with result from task 4027 parsl.dataflow.dflow DEBUG: Task 4027 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/bd1952a7-7e37-4eb0-8bb7-f31d21a5f0d4_isrForCrosstalkSources_996975_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/bd1952a7-7e37-4eb0-8bb7-f31d21a5f0d4_isrForCrosstalkSources_996975_9.stderr parsl.dataflow.dflow INFO: Task 3965 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b473701f4b0df4b9aa476eefaf1cdd4 with result from task 3965 parsl.dataflow.dflow DEBUG: Task 3965 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f9b05847-94fd-4af7-8b4d-a3fbab3f0d09_isrForCrosstalkSources_996978_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f9b05847-94fd-4af7-8b4d-a3fbab3f0d09_isrForCrosstalkSources_996978_57.stderr parsl.dataflow.dflow INFO: Task 3714 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab28f2e8d51b6d91fe1b7063a53af01d with result from task 3714 parsl.dataflow.dflow DEBUG: Task 3714 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/8ee68f0e-f4c4-4f22-8f64-cceae620e566_isrForCrosstalkSources_996991_1.stdout parsl.dataflow.dflow INFO: Standard error for task 3714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/8ee68f0e-f4c4-4f22-8f64-cceae620e566_isrForCrosstalkSources_996991_1.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3624 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 67712dd9a07a0bbe464692932b798947 with result from task 3624 parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3624 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/feabd467-5067-4cac-9c1d-9a337a017660_isrForCrosstalkSources_996978_52.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/feabd467-5067-4cac-9c1d-9a337a017660_isrForCrosstalkSources_996978_52.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:55 proc_lsst.multi:146 [INFO] found job 17075132 in provider checkpoint proc_lsst.multi INFO: found job 17075132 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3641 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57c0063dbd46d7197607b0da12bc82ce with result from task 3641 parsl.dataflow.dflow DEBUG: Task 3641 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b2b32dae-f197-4493-8e0d-85d5ce1eb7c3_isrForCrosstalkSources_996959_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b2b32dae-f197-4493-8e0d-85d5ce1eb7c3_isrForCrosstalkSources_996959_49.stderr parsl.dataflow.dflow INFO: Task 3870 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 879686fe9f59d083d80ab4e7727bb9d0 with result from task 3870 parsl.dataflow.dflow DEBUG: Task 3870 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/37232444-cca8-4809-8053-248d4cf2224d_isrForCrosstalkSources_996952_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/37232444-cca8-4809-8053-248d4cf2224d_isrForCrosstalkSources_996952_44.stderr parsl.dataflow.dflow INFO: Task 3758 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33af6aea1808ee31ebdf5c645c84d908 with result from task 3758 parsl.dataflow.dflow DEBUG: Task 3758 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/f49a8758-02d5-4a73-90dd-563a150a8bb6_isrForCrosstalkSources_997011_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/f49a8758-02d5-4a73-90dd-563a150a8bb6_isrForCrosstalkSources_997011_12.stderr parsl.dataflow.dflow INFO: Task 3888 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11236c94c5465c6a0adc73f9cb913331 with result from task 3888 parsl.dataflow.dflow DEBUG: Task 3888 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/01708eac-5640-41aa-a7ea-4d95a3f1d5ac_isrForCrosstalkSources_996960_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/01708eac-5640-41aa-a7ea-4d95a3f1d5ac_isrForCrosstalkSources_996960_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:56 proc_lsst.multi:146 [INFO] found job 17075133 in provider checkpoint proc_lsst.multi INFO: found job 17075133 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3800 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e0d28aeea6724d3c354774b4a2f29e5d with result from task 3800 parsl.dataflow.dflow DEBUG: Task 3800 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eafc093-107a-48a0-8a63-4847c0581b97_isrForCrosstalkSources_996985_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eafc093-107a-48a0-8a63-4847c0581b97_isrForCrosstalkSources_996985_36.stderr parsl.dataflow.dflow INFO: Task 4010 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23950d90a0be4d2bd0f471d9cc21e626 with result from task 4010 parsl.dataflow.dflow DEBUG: Task 4010 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6ea2297e-348e-4972-bcfc-09bbd8201a74_isrForCrosstalkSources_996954_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6ea2297e-348e-4972-bcfc-09bbd8201a74_isrForCrosstalkSources_996954_33.stderr parsl.dataflow.dflow INFO: Task 3834 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 483f1e2818dd56d15681379acea57e54 with result from task 3834 parsl.dataflow.dflow DEBUG: Task 3834 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/56e40733-20f4-4802-9f3d-38511541216e_isrForCrosstalkSources_996956_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/56e40733-20f4-4802-9f3d-38511541216e_isrForCrosstalkSources_996956_28.stderr parsl.dataflow.dflow INFO: Task 3810 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae7e8ed0c7dc5f247c76c6d8a97657b0 with result from task 3810 parsl.dataflow.dflow DEBUG: Task 3810 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/35be3402-f11e-49c7-9801-138a518d9273_isrForCrosstalkSources_997031_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/35be3402-f11e-49c7-9801-138a518d9273_isrForCrosstalkSources_997031_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:57 proc_lsst.multi:146 [INFO] found job 17075134 in provider checkpoint proc_lsst.multi INFO: found job 17075134 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3723 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7168be07a6a94de27419af7a95ca607 with result from task 3723 parsl.dataflow.dflow DEBUG: Task 3723 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5f8a89ce-932c-4b7b-87c0-b0e216f502d0_isrForCrosstalkSources_996987_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5f8a89ce-932c-4b7b-87c0-b0e216f502d0_isrForCrosstalkSources_996987_37.stderr parsl.dataflow.dflow INFO: Task 3704 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c73ad2c0d0a630c9c672c1500397fc22 with result from task 3704 parsl.dataflow.dflow DEBUG: Task 3704 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5d675980-c86e-4eb8-91cd-d9f8edd3e86b_isrForCrosstalkSources_996999_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/5d675980-c86e-4eb8-91cd-d9f8edd3e86b_isrForCrosstalkSources_996999_15.stderr parsl.dataflow.dflow INFO: Task 3971 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a163c5e8436e4d40ad88cd5fdcec7f83 with result from task 3971 parsl.dataflow.dflow DEBUG: Task 3971 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4be43b07-0edf-4bd1-8477-166fc0e90694_isrForCrosstalkSources_997030_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4be43b07-0edf-4bd1-8477-166fc0e90694_isrForCrosstalkSources_997030_28.stderr parsl.dataflow.dflow INFO: Task 3755 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b1235a197480f0689b472feda164b21 with result from task 3755 parsl.dataflow.dflow DEBUG: Task 3755 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/d3e9d7d4-aa47-43c8-8571-826fcf554107_isrForCrosstalkSources_996983_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/d3e9d7d4-aa47-43c8-8571-826fcf554107_isrForCrosstalkSources_996983_30.stderr parsl.dataflow.dflow INFO: Task 3844 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e53ca6ec478f74bc70fd1cddbb8752c5 with result from task 3844 parsl.dataflow.dflow DEBUG: Task 3844 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/1f35c49c-46f8-4156-8200-9c3f1180d71c_isrForCrosstalkSources_996992_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/1f35c49c-46f8-4156-8200-9c3f1180d71c_isrForCrosstalkSources_996992_50.stderr parsl.dataflow.dflow INFO: Task 3772 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 374294ca90fe288d02917d42834f1fcd with result from task 3772 parsl.dataflow.dflow DEBUG: Task 3772 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f4162314-9760-47d5-93f4-4b226a9f20e4_isrForCrosstalkSources_996999_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f4162314-9760-47d5-93f4-4b226a9f20e4_isrForCrosstalkSources_996999_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:57 proc_lsst.multi:146 [INFO] found job 17075135 in provider checkpoint proc_lsst.multi INFO: found job 17075135 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3905 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d2c7a86c5ac5e265c5f0cb57a9a5d95b with result from task 3905 parsl.dataflow.dflow DEBUG: Task 3905 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/01378bc6-d3bc-4fd3-af9e-91c563d3e4d9_isrForCrosstalkSources_996992_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/01378bc6-d3bc-4fd3-af9e-91c563d3e4d9_isrForCrosstalkSources_996992_49.stderr parsl.dataflow.dflow INFO: Task 4017 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d3dd48b3777a6960d1f49d352f53aec with result from task 4017 parsl.dataflow.dflow DEBUG: Task 4017 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/308ea9fe-9b71-4b07-af6c-eca8b4190e02_isrForCrosstalkSources_997009_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/308ea9fe-9b71-4b07-af6c-eca8b4190e02_isrForCrosstalkSources_997009_4.stderr parsl.dataflow.dflow INFO: Task 3766 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bdbbfc2be277c52243daebace3bd19de with result from task 3766 parsl.dataflow.dflow DEBUG: Task 3766 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/141eef14-1a62-4c1b-82ff-7bfe9075fecf_isrForCrosstalkSources_996978_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/141eef14-1a62-4c1b-82ff-7bfe9075fecf_isrForCrosstalkSources_996978_59.stderr parsl.dataflow.dflow INFO: Task 3732 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4254d3ed2d01912a29c3ca2447ee62b with result from task 3732 parsl.dataflow.dflow DEBUG: Task 3732 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/b62bf01b-e9ba-4b12-b246-0504c4de8622_isrForCrosstalkSources_996978_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/b62bf01b-e9ba-4b12-b246-0504c4de8622_isrForCrosstalkSources_996978_29.stderr parsl.dataflow.dflow INFO: Task 3817 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c87c7e0b611a87f15285ed26ff2a6d2 with result from task 3817 parsl.dataflow.dflow DEBUG: Task 3817 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1b3164fd-931a-4685-b19f-6c4c32d3acc0_isrForCrosstalkSources_997004_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1b3164fd-931a-4685-b19f-6c4c32d3acc0_isrForCrosstalkSources_997004_11.stderr parsl.dataflow.dflow INFO: Task 3815 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f25fee91ba1f91b90a5d5bc8d736a214 with result from task 3815 parsl.dataflow.dflow DEBUG: Task 3815 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e2dfc619-3ef0-4668-8434-14aba3d1ad61_isrForCrosstalkSources_996965_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/e2dfc619-3ef0-4668-8434-14aba3d1ad61_isrForCrosstalkSources_996965_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:58 proc_lsst.multi:146 [INFO] found job 17075136 in provider checkpoint proc_lsst.multi INFO: found job 17075136 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3852 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5c127028586ad18d39ee5862438802c with result from task 3852 parsl.dataflow.dflow DEBUG: Task 3852 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/b41d886d-809c-41ad-b470-8b3010fa5484_isrForCrosstalkSources_996991_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/b41d886d-809c-41ad-b470-8b3010fa5484_isrForCrosstalkSources_996991_7.stderr parsl.dataflow.dflow INFO: Task 3768 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 09b534ad5c898c2e51700e9d5846e215 with result from task 3768 parsl.dataflow.dflow DEBUG: Task 3768 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/96b4f9bb-d6a9-47a2-bede-67492eeda27f_isrForCrosstalkSources_997028_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/96b4f9bb-d6a9-47a2-bede-67492eeda27f_isrForCrosstalkSources_997028_28.stderr parsl.dataflow.dflow INFO: Task 3796 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0f152f74278fc93ded96a5256c61838 with result from task 3796 parsl.dataflow.dflow DEBUG: Task 3796 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/41250f54-cc37-4c05-93d6-ae08c5eb004a_isrForCrosstalkSources_997027_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/41250f54-cc37-4c05-93d6-ae08c5eb004a_isrForCrosstalkSources_997027_11.stderr parsl.dataflow.dflow INFO: Task 3814 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1f8a9757ff00c799c093c20fa9400ae with result from task 3814 parsl.dataflow.dflow DEBUG: Task 3814 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/7eb8e8c6-cde1-462f-b63f-76aa1042af3e_isrForCrosstalkSources_996992_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/7eb8e8c6-cde1-462f-b63f-76aa1042af3e_isrForCrosstalkSources_996992_30.stderr parsl.dataflow.dflow INFO: Task 4026 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 30c5b789f5234ea6ed2499bf69fa030d with result from task 4026 parsl.dataflow.dflow DEBUG: Task 4026 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/3616d9dd-937b-4070-8f86-396a119ee2f5_isrForCrosstalkSources_997022_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/3616d9dd-937b-4070-8f86-396a119ee2f5_isrForCrosstalkSources_997022_50.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:58 proc_lsst.multi:146 [INFO] found job 17075137 in provider checkpoint proc_lsst.multi INFO: found job 17075137 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4056 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4c71382b5291dab118f96dad61283dd with result from task 4056 parsl.dataflow.dflow DEBUG: Task 4056 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/75a39256-c5fc-4174-ae39-6b7c37b9dba8_isrForCrosstalkSources_996958_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/75a39256-c5fc-4174-ae39-6b7c37b9dba8_isrForCrosstalkSources_996958_26.stderr parsl.dataflow.dflow INFO: Task 3802 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88cadbd5592b792d760b9909ca2d9327 with result from task 3802 parsl.dataflow.dflow DEBUG: Task 3802 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/927c031d-f18c-4195-baa7-bfcefd8653d4_isrForCrosstalkSources_996964_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/927c031d-f18c-4195-baa7-bfcefd8653d4_isrForCrosstalkSources_996964_16.stderr parsl.dataflow.dflow INFO: Task 3868 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7743f1b8c69089808a072015616d9d43 with result from task 3868 parsl.dataflow.dflow DEBUG: Task 3868 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/248ec971-67ef-414f-9444-5115a078d816_isrForCrosstalkSources_996956_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/248ec971-67ef-414f-9444-5115a078d816_isrForCrosstalkSources_996956_47.stderr parsl.dataflow.dflow INFO: Task 3686 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a4d34b75bb8032e1a0c3a270ed61b6d with result from task 3686 parsl.dataflow.dflow DEBUG: Task 3686 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/400d7374-f54b-462c-bd03-daa8a176ac24_isrForCrosstalkSources_997029_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/400d7374-f54b-462c-bd03-daa8a176ac24_isrForCrosstalkSources_997029_61.stderr parsl.dataflow.dflow INFO: Task 3665 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f587f6fd3eceab9d18e82aa37800f12 with result from task 3665 parsl.dataflow.dflow DEBUG: Task 3665 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/47cd431a-444c-48f9-b6f9-39e89644954f_isrForCrosstalkSources_997014_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/47cd431a-444c-48f9-b6f9-39e89644954f_isrForCrosstalkSources_997014_11.stderr parsl.dataflow.dflow INFO: Task 3736 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2fe464e9777bcb77eac27bc134498ce2 with result from task 3736 parsl.dataflow.dflow DEBUG: Task 3736 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/059d0ee5-81a5-4de4-b926-efbd2ff5efde_isrForCrosstalkSources_997014_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/059d0ee5-81a5-4de4-b926-efbd2ff5efde_isrForCrosstalkSources_997014_37.stderr parsl.dataflow.dflow INFO: Task 3845 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a0ae8cbea7acf2b9ae8c1029358b506 with result from task 3845 parsl.dataflow.dflow DEBUG: Task 3845 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/5b1c7bb4-8a37-4331-b56a-39eb959e4852_isrForCrosstalkSources_997007_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/5b1c7bb4-8a37-4331-b56a-39eb959e4852_isrForCrosstalkSources_997007_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:59 proc_lsst.multi:146 [INFO] found job 17075138 in provider checkpoint proc_lsst.multi INFO: found job 17075138 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3906 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 713db0491cf98eb90999d7d6046738e3 with result from task 3906 parsl.dataflow.dflow DEBUG: Task 3906 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/891956f9-676c-4f08-bc34-776ee1106b18_isrForCrosstalkSources_996973_27.stdout parsl.dataflow.dflow INFO: Standard error for task 3906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/891956f9-676c-4f08-bc34-776ee1106b18_isrForCrosstalkSources_996973_27.stderr parsl.dataflow.dflow INFO: Task 3869 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97e818ce4532e4f3c73d7d437432aae2 with result from task 3869 parsl.dataflow.dflow DEBUG: Task 3869 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/12b5e499-abed-4874-9fd6-948feed71efc_isrForCrosstalkSources_997033_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/12b5e499-abed-4874-9fd6-948feed71efc_isrForCrosstalkSources_997033_17.stderr parsl.dataflow.dflow INFO: Task 3874 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fa010d02e8ae52b873657d6450ccb65 with result from task 3874 parsl.dataflow.dflow DEBUG: Task 3874 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/147d0c21-fad3-4d19-8fb6-7a4ab018f9e6_isrForCrosstalkSources_996980_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/147d0c21-fad3-4d19-8fb6-7a4ab018f9e6_isrForCrosstalkSources_996980_50.stderr parsl.dataflow.dflow INFO: Task 3747 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7014b2d6ea6a9c95181486196eabcf8 with result from task 3747 parsl.dataflow.dflow DEBUG: Task 3747 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6399ad81-c9d3-4c30-8bd0-23efb976fdbc_isrForCrosstalkSources_996991_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/6399ad81-c9d3-4c30-8bd0-23efb976fdbc_isrForCrosstalkSources_996991_31.stderr parsl.dataflow.dflow INFO: Task 3951 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b17efd9a3ee6a8b88eb33b18fbab0d71 with result from task 3951 parsl.dataflow.dflow DEBUG: Task 3951 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/0da5816e-e956-4a9a-9eb3-aec237c20fc4_isrForCrosstalkSources_996962_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/0da5816e-e956-4a9a-9eb3-aec237c20fc4_isrForCrosstalkSources_996962_26.stderr parsl.dataflow.dflow INFO: Task 4049 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7a6480c57992b5759841b1a9dbb74e6 with result from task 4049 parsl.dataflow.dflow DEBUG: Task 4049 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/73c0a4c0-1fce-4de8-94dd-82a2589af7e1_isrForCrosstalkSources_997021_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/73c0a4c0-1fce-4de8-94dd-82a2589af7e1_isrForCrosstalkSources_997021_20.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:58:59 proc_lsst.multi:146 [INFO] found job 17075139 in provider checkpoint proc_lsst.multi INFO: found job 17075139 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3885 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58352cb0f62a5e7cdf98e8913c156e1d with result from task 3885 parsl.dataflow.dflow DEBUG: Task 3885 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9d0dd8da-e77d-4d87-89f2-8aa1b999f694_isrForCrosstalkSources_996964_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9d0dd8da-e77d-4d87-89f2-8aa1b999f694_isrForCrosstalkSources_996964_20.stderr parsl.dataflow.dflow INFO: Task 3879 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce365fec4ca9162eec03c053aba8d484 with result from task 3879 parsl.dataflow.dflow DEBUG: Task 3879 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e024921c-a29a-463c-a7dc-79bacc7c30ba_isrForCrosstalkSources_997033_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/e024921c-a29a-463c-a7dc-79bacc7c30ba_isrForCrosstalkSources_997033_52.stderr parsl.dataflow.dflow INFO: Task 3993 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1093f09c430f8e3db0d8cda6566bb216 with result from task 3993 parsl.dataflow.dflow DEBUG: Task 3993 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/78e9deaf-aa46-4995-9026-9b64690298f1_isrForCrosstalkSources_997031_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/78e9deaf-aa46-4995-9026-9b64690298f1_isrForCrosstalkSources_997031_62.stderr parsl.dataflow.dflow INFO: Task 3792 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9eace6bc6483b71f2ce423291ab4c631 with result from task 3792 parsl.dataflow.dflow DEBUG: Task 3792 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e5aa79fe-d14c-46d4-b32d-3aaa5e88a995_isrForCrosstalkSources_997008_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/e5aa79fe-d14c-46d4-b32d-3aaa5e88a995_isrForCrosstalkSources_997008_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:00 proc_lsst.multi:146 [INFO] found job 17075140 in provider checkpoint proc_lsst.multi INFO: found job 17075140 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4019 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65208478258176adc323d6c62901424f with result from task 4019 parsl.dataflow.dflow DEBUG: Task 4019 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/86afdad4-53aa-4d67-bec1-2038025f7afa_isrForCrosstalkSources_996978_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/86afdad4-53aa-4d67-bec1-2038025f7afa_isrForCrosstalkSources_996978_46.stderr parsl.dataflow.dflow INFO: Task 4064 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a4a810a9424ed422bafd773eb32643a with result from task 4064 parsl.dataflow.dflow DEBUG: Task 4064 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/39477b9e-c0ac-4d9a-8060-b5a54a0c7aa0_isrForCrosstalkSources_996969_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/39477b9e-c0ac-4d9a-8060-b5a54a0c7aa0_isrForCrosstalkSources_996969_14.stderr parsl.dataflow.dflow INFO: Task 3681 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 123ca87e96cfe9c4ec0b9c92d5d757a7 with result from task 3681 parsl.dataflow.dflow DEBUG: Task 3681 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a5d82d10-ef42-4e79-92c3-8818a4bde98c_isrForCrosstalkSources_996981_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a5d82d10-ef42-4e79-92c3-8818a4bde98c_isrForCrosstalkSources_996981_2.stderr parsl.dataflow.dflow INFO: Task 3826 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01945313d5e71ef30e0c1499927352b4 with result from task 3826 parsl.dataflow.dflow DEBUG: Task 3826 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/71d909f1-04ef-4a4b-b885-120abc4f0ecb_isrForCrosstalkSources_996976_5.stdout parsl.dataflow.dflow INFO: Standard error for task 3826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/71d909f1-04ef-4a4b-b885-120abc4f0ecb_isrForCrosstalkSources_996976_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:00 proc_lsst.multi:146 [INFO] found job 17075141 in provider checkpoint proc_lsst.multi INFO: found job 17075141 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4070 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 545ecefc098f95a14fd0613e9886e6d5 with result from task 4070 parsl.dataflow.dflow DEBUG: Task 4070 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/1292fbcc-caca-40c5-ad74-7f2624bee17b_isrForCrosstalkSources_997011_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/1292fbcc-caca-40c5-ad74-7f2624bee17b_isrForCrosstalkSources_997011_46.stderr parsl.dataflow.dflow INFO: Task 4038 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b315b5b24e560644b09e0d9b5334d89 with result from task 4038 parsl.dataflow.dflow DEBUG: Task 4038 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/fb893e15-5025-4c9f-bb30-d28c2c0d81b9_isrForCrosstalkSources_997029_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/fb893e15-5025-4c9f-bb30-d28c2c0d81b9_isrForCrosstalkSources_997029_59.stderr parsl.dataflow.dflow INFO: Task 3975 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d9cd0700bc647cc33a71cb6ae821d57 with result from task 3975 parsl.dataflow.dflow DEBUG: Task 3975 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c3b80d89-b704-481a-a3b2-1466ff7b2455_isrForCrosstalkSources_996967_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c3b80d89-b704-481a-a3b2-1466ff7b2455_isrForCrosstalkSources_996967_20.stderr parsl.dataflow.dflow INFO: Task 4014 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c126e6edaf97386a832f3a0a31d0d05 with result from task 4014 parsl.dataflow.dflow DEBUG: Task 4014 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a8ae4991-e8ef-48e7-8cd7-100137451ee5_isrForCrosstalkSources_997000_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/a8ae4991-e8ef-48e7-8cd7-100137451ee5_isrForCrosstalkSources_997000_24.stderr parsl.dataflow.dflow INFO: Task 3992 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90d05cad723f533bc8fa294e8e5cdc21 with result from task 3992 parsl.dataflow.dflow DEBUG: Task 3992 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e985fcb1-c51d-4523-814a-577800433c61_isrForCrosstalkSources_996970_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e985fcb1-c51d-4523-814a-577800433c61_isrForCrosstalkSources_996970_4.stderr parsl.dataflow.dflow INFO: Task 3871 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd0410e04d4919607a397b9ce8bdbbb6 with result from task 3871 parsl.dataflow.dflow DEBUG: Task 3871 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/799ef417-48ef-47c6-afd3-7aef6dc5a45f_isrForCrosstalkSources_996989_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/799ef417-48ef-47c6-afd3-7aef6dc5a45f_isrForCrosstalkSources_996989_31.stderr parsl.dataflow.dflow INFO: Task 3746 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 64643ffe90ed6fc13a4e0a91439ad175 with result from task 3746 parsl.dataflow.dflow DEBUG: Task 3746 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/0ea9c1ce-7137-4c46-8dfa-ee9a6b3efd13_isrForCrosstalkSources_996955_44.stdout parsl.dataflow.dflow INFO: Standard error for task 3746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/0ea9c1ce-7137-4c46-8dfa-ee9a6b3efd13_isrForCrosstalkSources_996955_44.stderr parsl.dataflow.dflow INFO: Task 3713 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1017c02e8d38787ccb1373a1fa249e9a with result from task 3713 parsl.dataflow.dflow DEBUG: Task 3713 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c40d3465-acdc-4353-ad03-d542be9dc3c0_isrForCrosstalkSources_996968_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c40d3465-acdc-4353-ad03-d542be9dc3c0_isrForCrosstalkSources_996968_3.stderr parsl.dataflow.dflow INFO: Task 3623 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd2abca199be81e8fb76c8694f98e26e with result from task 3623 parsl.dataflow.dflow DEBUG: Task 3623 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/32faea51-5282-4903-b9ba-93221fa793cc_isrForCrosstalkSources_997025_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/32faea51-5282-4903-b9ba-93221fa793cc_isrForCrosstalkSources_997025_61.stderr parsl.dataflow.dflow INFO: Task 4004 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 838707e695aaa84573402b3e7b0f6da3 with result from task 4004 parsl.dataflow.dflow DEBUG: Task 4004 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0f13cc09-985d-4fec-821a-591f2996e68e_isrForCrosstalkSources_997023_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0f13cc09-985d-4fec-821a-591f2996e68e_isrForCrosstalkSources_997023_52.stderr parsl.dataflow.dflow INFO: Task 4062 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fc5255ef6a52076c091277d96b9dc84 with result from task 4062 parsl.dataflow.dflow DEBUG: Task 4062 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/f4fc5900-9d04-4448-9e32-cad9b4a9c4f8_isrForCrosstalkSources_996963_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/f4fc5900-9d04-4448-9e32-cad9b4a9c4f8_isrForCrosstalkSources_996963_6.stderr parsl.dataflow.dflow INFO: Task 3831 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9246168935f833a29502c1b280e36cc with result from task 3831 parsl.dataflow.dflow DEBUG: Task 3831 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bc9a7ef4-ea4b-4db1-8e7d-497fd436f3c6_isrForCrosstalkSources_997014_29.stdout parsl.dataflow.dflow INFO: Standard error for task 3831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bc9a7ef4-ea4b-4db1-8e7d-497fd436f3c6_isrForCrosstalkSources_997014_29.stderr parsl.dataflow.dflow INFO: Task 3837 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0bd5edbbc00a80664b28029f9c2f6021 with result from task 3837 parsl.dataflow.dflow DEBUG: Task 3837 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/6153715b-c994-4779-a2ee-1feb40e725d5_isrForCrosstalkSources_996969_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/6153715b-c994-4779-a2ee-1feb40e725d5_isrForCrosstalkSources_996969_38.stderr parsl.dataflow.dflow INFO: Task 4057 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 496db61a9c3782ffbb7c8a1b77938943 with result from task 4057 parsl.dataflow.dflow DEBUG: Task 4057 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/bef53721-baf1-4323-9d64-86f1232a4e22_isrForCrosstalkSources_996997_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/bef53721-baf1-4323-9d64-86f1232a4e22_isrForCrosstalkSources_996997_33.stderr parsl.dataflow.dflow INFO: Task 4066 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c7dda9bbe69dd52bbc4e7561a24d2c5 with result from task 4066 parsl.dataflow.dflow DEBUG: Task 4066 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/9fb3e7ef-bcf0-46d1-94a2-47b7955d30eb_isrForCrosstalkSources_996987_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/9fb3e7ef-bcf0-46d1-94a2-47b7955d30eb_isrForCrosstalkSources_996987_30.stderr parsl.dataflow.dflow INFO: Task 3795 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c93eb64ebbf1916e955c94158d7817c8 with result from task 3795 parsl.dataflow.dflow DEBUG: Task 3795 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0d32b070-c6f8-472a-8ccd-6002be0ea72e_isrForCrosstalkSources_997023_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0d32b070-c6f8-472a-8ccd-6002be0ea72e_isrForCrosstalkSources_997023_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:02 proc_lsst.multi:146 [INFO] found job 17075142 in provider checkpoint proc_lsst.multi INFO: found job 17075142 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3900 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d634611a5d82fc31e7a73dce05fa9f4 with result from task 3900 parsl.dataflow.dflow DEBUG: Task 3900 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4ecde8fa-3fcc-44b9-8038-0f3c37b73d77_isrForCrosstalkSources_997021_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4ecde8fa-3fcc-44b9-8038-0f3c37b73d77_isrForCrosstalkSources_997021_16.stderr parsl.dataflow.dflow INFO: Task 3872 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b589d8f48d9a715631dcaddddc03152 with result from task 3872 parsl.dataflow.dflow DEBUG: Task 3872 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/36b5df50-2d6c-49b1-8e20-00ce8de5151a_isrForCrosstalkSources_997024_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/36b5df50-2d6c-49b1-8e20-00ce8de5151a_isrForCrosstalkSources_997024_49.stderr parsl.dataflow.dflow INFO: Task 3702 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4a5d7d0ce3e489995a2fd71ceb92c21 with result from task 3702 parsl.dataflow.dflow DEBUG: Task 3702 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/0081d364-559e-4141-b06d-35a30386d431_isrForCrosstalkSources_996961_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/0081d364-559e-4141-b06d-35a30386d431_isrForCrosstalkSources_996961_47.stderr parsl.dataflow.dflow INFO: Task 3893 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b129b05aa99b5a0ae825692901cfcac with result from task 3893 parsl.dataflow.dflow DEBUG: Task 3893 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/f781a00d-c5a9-4dd5-accf-c08854ca7448_isrForCrosstalkSources_997002_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/f781a00d-c5a9-4dd5-accf-c08854ca7448_isrForCrosstalkSources_997002_61.stderr parsl.dataflow.dflow INFO: Task 3760 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51834262c567eb146d1988b49c5e2443 with result from task 3760 parsl.dataflow.dflow DEBUG: Task 3760 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/97bf9157-5b87-4b47-8eff-92bb74720532_isrForCrosstalkSources_996974_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/97bf9157-5b87-4b47-8eff-92bb74720532_isrForCrosstalkSources_996974_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:03 proc_lsst.multi:146 [INFO] found job 17075143 in provider checkpoint proc_lsst.multi INFO: found job 17075143 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3748 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a96b11d6562270bb24fcc79bbce5ad6 with result from task 3748 parsl.dataflow.dflow DEBUG: Task 3748 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b8f97ad0-11b3-4d6c-acd3-1d77e7a473cd_isrForCrosstalkSources_996962_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b8f97ad0-11b3-4d6c-acd3-1d77e7a473cd_isrForCrosstalkSources_996962_61.stderr parsl.dataflow.dflow INFO: Task 3851 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96dfbd55addab7e2fc7858ac37bf3569 with result from task 3851 parsl.dataflow.dflow DEBUG: Task 3851 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/fb72a926-3db5-403e-90cf-8c548c30dfea_isrForCrosstalkSources_996994_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/fb72a926-3db5-403e-90cf-8c548c30dfea_isrForCrosstalkSources_996994_38.stderr parsl.dataflow.dflow INFO: Task 4058 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c5a4f3e2c42722f59fe8314a017d5c2c with result from task 4058 parsl.dataflow.dflow DEBUG: Task 4058 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/761825ae-6127-489e-8488-7bce5e51a25f_isrForCrosstalkSources_997027_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/761825ae-6127-489e-8488-7bce5e51a25f_isrForCrosstalkSources_997027_10.stderr parsl.dataflow.dflow INFO: Task 3730 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 76cec014d56d0ea8d35f02af791d8a4c with result from task 3730 parsl.dataflow.dflow DEBUG: Task 3730 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4ed67aef-94dc-4ae2-a09c-d1a4535407ba_isrForCrosstalkSources_997023_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4ed67aef-94dc-4ae2-a09c-d1a4535407ba_isrForCrosstalkSources_997023_3.stderr parsl.dataflow.dflow INFO: Task 4084 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7270dbd39cc5c5d53b860cdd41fad603 with result from task 4084 parsl.dataflow.dflow DEBUG: Task 4084 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7d44aca6-3d81-4d9b-9d1e-b1e5d9cefb28_isrForCrosstalkSources_996994_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4084 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7d44aca6-3d81-4d9b-9d1e-b1e5d9cefb28_isrForCrosstalkSources_996994_32.stderr parsl.dataflow.dflow INFO: Task 3729 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4395e4ca91650d3fcd3479e57baf662 with result from task 3729 parsl.dataflow.dflow DEBUG: Task 3729 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d01be70b-c25f-420d-b893-4b533bc0ce3e_isrForCrosstalkSources_996991_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/d01be70b-c25f-420d-b893-4b533bc0ce3e_isrForCrosstalkSources_996991_35.stderr parsl.dataflow.dflow INFO: Task 3599 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12f2ef811b60acdfb56bd09e4d09ebea with result from task 3599 parsl.dataflow.dflow DEBUG: Task 3599 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e8bd96ed-6a24-41c5-b20a-091f3d1bccb7_isrForCrosstalkSources_996989_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e8bd96ed-6a24-41c5-b20a-091f3d1bccb7_isrForCrosstalkSources_996989_47.stderr parsl.dataflow.dflow INFO: Task 4072 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57a9d735b3b9a96ffb4a06455fba53d3 with result from task 4072 parsl.dataflow.dflow DEBUG: Task 4072 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/654e64fc-4af7-419a-9b34-2b78b6dfdabf_isrForCrosstalkSources_996992_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/654e64fc-4af7-419a-9b34-2b78b6dfdabf_isrForCrosstalkSources_996992_4.stderr parsl.dataflow.dflow INFO: Task 3947 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f4955a43c415971fa65d011a5030c21 with result from task 3947 parsl.dataflow.dflow DEBUG: Task 3947 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/160665b5-2cff-4b4e-8210-2c2706c80a19_isrForCrosstalkSources_996994_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/160665b5-2cff-4b4e-8210-2c2706c80a19_isrForCrosstalkSources_996994_47.stderr parsl.dataflow.dflow INFO: Task 3955 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56c8bbfb66a108a703e3fae8c4d0f8e4 with result from task 3955 parsl.dataflow.dflow DEBUG: Task 3955 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/af087199-3599-477a-b2ec-d60f8cc528d4_isrForCrosstalkSources_996972_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/af087199-3599-477a-b2ec-d60f8cc528d4_isrForCrosstalkSources_996972_56.stderr parsl.dataflow.dflow INFO: Task 3946 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 099f98d1b9713f77f43a5f4852c1d528 with result from task 3946 parsl.dataflow.dflow DEBUG: Task 3946 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/91082b54-897c-4c66-a06b-af9df9aa8138_isrForCrosstalkSources_997000_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/91082b54-897c-4c66-a06b-af9df9aa8138_isrForCrosstalkSources_997000_22.stderr parsl.dataflow.dflow INFO: Task 3880 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 259fd98529f524f73f56ff0a381d4388 with result from task 3880 parsl.dataflow.dflow DEBUG: Task 3880 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9b757870-ea23-422c-9a49-458ff07a514d_isrForCrosstalkSources_996959_51.stdout parsl.dataflow.dflow INFO: Standard error for task 3880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/9b757870-ea23-422c-9a49-458ff07a514d_isrForCrosstalkSources_996959_51.stderr parsl.dataflow.dflow INFO: Task 3933 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de5cb0a728a360d7a35b8e5eb76ed9ed with result from task 3933 parsl.dataflow.dflow DEBUG: Task 3933 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/49a0cff5-98b0-46f0-a814-4dd2a02aac96_isrForCrosstalkSources_996998_32.stdout parsl.dataflow.dflow INFO: Standard error for task 3933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/49a0cff5-98b0-46f0-a814-4dd2a02aac96_isrForCrosstalkSources_996998_32.stderr parsl.dataflow.dflow INFO: Task 3889 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b27ed0ef4a2f47d0d02c8211f87c568 with result from task 3889 parsl.dataflow.dflow DEBUG: Task 3889 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4b38b924-e7ce-45df-b9a4-5941328b33ff_isrForCrosstalkSources_997023_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/4b38b924-e7ce-45df-b9a4-5941328b33ff_isrForCrosstalkSources_997023_48.stderr parsl.dataflow.dflow INFO: Task 3912 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51ff5b0a013ae158c4f1be5427c8be23 with result from task 3912 parsl.dataflow.dflow DEBUG: Task 3912 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/64bc5f7f-4344-406b-97ad-659b75f9602e_isrForCrosstalkSources_997012_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/64bc5f7f-4344-406b-97ad-659b75f9602e_isrForCrosstalkSources_997012_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:04 proc_lsst.multi:146 [INFO] found job 17075144 in provider checkpoint proc_lsst.multi INFO: found job 17075144 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3819 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf1b717bd2ec6fb323fd1a5cf1498a76 with result from task 3819 parsl.dataflow.dflow DEBUG: Task 3819 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/12d28997-a2fd-40cf-a3e5-f8fb8fcf5a77_isrForCrosstalkSources_996977_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/12d28997-a2fd-40cf-a3e5-f8fb8fcf5a77_isrForCrosstalkSources_996977_38.stderr parsl.dataflow.dflow INFO: Task 4083 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91ba152af64b4eecb666d05cb20afd17 with result from task 4083 parsl.dataflow.dflow DEBUG: Task 4083 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/fecdcd53-c143-40c5-b17f-2ecf942161ae_isrForCrosstalkSources_996981_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/fecdcd53-c143-40c5-b17f-2ecf942161ae_isrForCrosstalkSources_996981_3.stderr parsl.dataflow.dflow INFO: Task 3890 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a43a6400ff17a08230dfbd3d006f6de5 with result from task 3890 parsl.dataflow.dflow DEBUG: Task 3890 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1fb33086-b129-48c2-ab8e-c56ccc3e9cf7_isrForCrosstalkSources_997000_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/1fb33086-b129-48c2-ab8e-c56ccc3e9cf7_isrForCrosstalkSources_997000_53.stderr parsl.dataflow.dflow INFO: Task 3829 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 035615b29b8a1406a5a5646dfeb854bb with result from task 3829 parsl.dataflow.dflow DEBUG: Task 3829 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/937e1bf0-f3ea-4c2d-8b51-ba39de4b7fca_isrForCrosstalkSources_997028_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/937e1bf0-f3ea-4c2d-8b51-ba39de4b7fca_isrForCrosstalkSources_997028_6.stderr parsl.dataflow.dflow INFO: Task 3994 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6e67e00c196e8109260e90a266c55c5 with result from task 3994 parsl.dataflow.dflow DEBUG: Task 3994 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0da4e8e0-386f-4be4-ba4c-199d52620697_isrForCrosstalkSources_997028_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/0da4e8e0-386f-4be4-ba4c-199d52620697_isrForCrosstalkSources_997028_7.stderr parsl.dataflow.dflow INFO: Task 3943 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8b83b2194350090fd4a47bd08eb76a8 with result from task 3943 parsl.dataflow.dflow DEBUG: Task 3943 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f0fcf9e9-1105-44c6-878a-ee6bcc62bdf5_isrForCrosstalkSources_996993_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f0fcf9e9-1105-44c6-878a-ee6bcc62bdf5_isrForCrosstalkSources_996993_59.stderr parsl.dataflow.dflow INFO: Task 3742 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fc049ef12821a25964bf9ddfae3ae0b with result from task 3742 parsl.dataflow.dflow DEBUG: Task 3742 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ca98e3b3-2b73-4048-9663-de6993f25ca6_isrForCrosstalkSources_997019_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ca98e3b3-2b73-4048-9663-de6993f25ca6_isrForCrosstalkSources_997019_42.stderr parsl.dataflow.dflow INFO: Task 3820 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf014002684b3515ae178f25f6a002b4 with result from task 3820 parsl.dataflow.dflow DEBUG: Task 3820 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/5af224e0-3a82-44f9-bc25-dee78b83b925_isrForCrosstalkSources_996962_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/5af224e0-3a82-44f9-bc25-dee78b83b925_isrForCrosstalkSources_996962_18.stderr parsl.dataflow.dflow INFO: Task 3939 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb4a4936b2c14e0bf5970b5ac1bd77c0 with result from task 3939 parsl.dataflow.dflow DEBUG: Task 3939 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/8d590be7-8d88-4c6b-bc72-516a7ea800d3_isrForCrosstalkSources_997002_16.stdout parsl.dataflow.dflow INFO: Standard error for task 3939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/8d590be7-8d88-4c6b-bc72-516a7ea800d3_isrForCrosstalkSources_997002_16.stderr parsl.dataflow.dflow INFO: Task 3745 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa44b3260be4058d4f484b70685e8362 with result from task 3745 parsl.dataflow.dflow DEBUG: Task 3745 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/de07963c-008f-4d30-8734-7a04cc30f58f_isrForCrosstalkSources_996954_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/de07963c-008f-4d30-8734-7a04cc30f58f_isrForCrosstalkSources_996954_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 3716 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry fcb386ce35b9964b39fc6a1c2c90cda8 with result from task 3716 parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3716 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/323a7aa6-573a-4515-84de-3c6f271f35f7_isrForCrosstalkSources_997010_50.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/323a7aa6-573a-4515-84de-3c6f271f35f7_isrForCrosstalkSources_997010_50.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:04 proc_lsst.multi:146 [INFO] found job 17075145 in provider checkpoint proc_lsst.multi INFO: found job 17075145 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3756 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b40dc79f787966ed4ab74dcff4b24e93 with result from task 3756 parsl.dataflow.dflow DEBUG: Task 3756 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5b2d0ac3-5253-440c-a9b7-8209be24bf10_isrForCrosstalkSources_996972_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5b2d0ac3-5253-440c-a9b7-8209be24bf10_isrForCrosstalkSources_996972_49.stderr parsl.dataflow.dflow INFO: Task 3687 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8684b818a85220c108997cc5e0919bdd with result from task 3687 parsl.dataflow.dflow DEBUG: Task 3687 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0c8759cf-7399-4b89-9735-66a195ec014f_isrForCrosstalkSources_997007_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0c8759cf-7399-4b89-9735-66a195ec014f_isrForCrosstalkSources_997007_41.stderr parsl.dataflow.dflow INFO: Task 3750 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6404e86d24d028c843133cdf604bf9f9 with result from task 3750 parsl.dataflow.dflow DEBUG: Task 3750 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/35d3dc6d-90ba-44d5-90b6-88ffe9a138a9_isrForCrosstalkSources_997001_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/35d3dc6d-90ba-44d5-90b6-88ffe9a138a9_isrForCrosstalkSources_997001_52.stderr parsl.dataflow.dflow INFO: Task 4029 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc85f9a4c5228589cae1d631ec443125 with result from task 4029 parsl.dataflow.dflow DEBUG: Task 4029 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b7f7e401-1ad7-4856-bd9d-6cc6bdec23ac_isrForCrosstalkSources_996959_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/b7f7e401-1ad7-4856-bd9d-6cc6bdec23ac_isrForCrosstalkSources_996959_15.stderr parsl.dataflow.dflow INFO: Task 4105 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8240755dc47065f3e21c63f9b398bb5c with result from task 4105 parsl.dataflow.dflow DEBUG: Task 4105 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b1a9f1e4-1c40-4bbb-95ba-0072aa1a8a7c_isrForCrosstalkSources_997030_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4105 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b1a9f1e4-1c40-4bbb-95ba-0072aa1a8a7c_isrForCrosstalkSources_997030_25.stderr parsl.dataflow.dflow INFO: Task 4051 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 764f1b3a311159b125a9ba981e77fbcb with result from task 4051 parsl.dataflow.dflow DEBUG: Task 4051 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c819094f-a72d-4583-9e24-791b5eaabb04_isrForCrosstalkSources_997025_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c819094f-a72d-4583-9e24-791b5eaabb04_isrForCrosstalkSources_997025_25.stderr parsl.dataflow.dflow INFO: Task 3884 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa4c447dc3188f7b93033f0ebe833072 with result from task 3884 parsl.dataflow.dflow DEBUG: Task 3884 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4fcfc62d-b1e0-4d91-99c2-e3722e77f650_isrForCrosstalkSources_996998_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4fcfc62d-b1e0-4d91-99c2-e3722e77f650_isrForCrosstalkSources_996998_12.stderr parsl.dataflow.dflow INFO: Task 3842 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86644f25bbc54e36a87899a704341338 with result from task 3842 parsl.dataflow.dflow DEBUG: Task 3842 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/b7380c84-a6ff-406d-90bf-44f7d7f7de38_isrForCrosstalkSources_996982_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/b7380c84-a6ff-406d-90bf-44f7d7f7de38_isrForCrosstalkSources_996982_35.stderr parsl.dataflow.dflow INFO: Task 4085 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2abeb310bcbc1f072f0f46fde9fc4a44 with result from task 4085 parsl.dataflow.dflow DEBUG: Task 4085 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/0778fe5a-d566-4c5c-99cd-d7d777cb054a_isrForCrosstalkSources_996964_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4085 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/0778fe5a-d566-4c5c-99cd-d7d777cb054a_isrForCrosstalkSources_996964_34.stderr parsl.dataflow.dflow INFO: Task 3956 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fdf0e74466739dde5814a0b1b9a343ce with result from task 3956 parsl.dataflow.dflow DEBUG: Task 3956 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b4505b9c-7e78-467d-ac33-4cbdc241b0bc_isrForCrosstalkSources_996957_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/b4505b9c-7e78-467d-ac33-4cbdc241b0bc_isrForCrosstalkSources_996957_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:05 proc_lsst.multi:146 [INFO] found job 17075146 in provider checkpoint proc_lsst.multi INFO: found job 17075146 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3812 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48a61e06515780146909239bc2a0c80f with result from task 3812 parsl.dataflow.dflow DEBUG: Task 3812 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/e53a23d1-cc8b-41e2-af08-7e2d9e3a5187_isrForCrosstalkSources_997006_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/e53a23d1-cc8b-41e2-af08-7e2d9e3a5187_isrForCrosstalkSources_997006_46.stderr parsl.dataflow.dflow INFO: Task 3828 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 191cf0b249932b9c7fd131821143bd65 with result from task 3828 parsl.dataflow.dflow DEBUG: Task 3828 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/dced2144-9e6a-4584-8545-8cde8f6434fe_isrForCrosstalkSources_996965_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/dced2144-9e6a-4584-8545-8cde8f6434fe_isrForCrosstalkSources_996965_3.stderr parsl.dataflow.dflow INFO: Task 4078 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ccd60bad19533c4080e391e076457141 with result from task 4078 parsl.dataflow.dflow DEBUG: Task 4078 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0888f7e8-6157-48af-8777-861ef9d3a261_isrForCrosstalkSources_996973_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/0888f7e8-6157-48af-8777-861ef9d3a261_isrForCrosstalkSources_996973_24.stderr parsl.dataflow.dflow INFO: Task 4073 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8478393cbe8f06a02dc1b3ce7aa97789 with result from task 4073 parsl.dataflow.dflow DEBUG: Task 4073 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/035bd836-996a-4dfd-a4dc-f140933377d4_isrForCrosstalkSources_996998_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/035bd836-996a-4dfd-a4dc-f140933377d4_isrForCrosstalkSources_996998_6.stderr parsl.dataflow.dflow INFO: Task 3840 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4af146f70e953da2d919304203fec83d with result from task 3840 parsl.dataflow.dflow DEBUG: Task 3840 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/fd37037d-1848-4f4a-a322-37f7348fe732_isrForCrosstalkSources_996999_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/fd37037d-1848-4f4a-a322-37f7348fe732_isrForCrosstalkSources_996999_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:05 proc_lsst.multi:146 [INFO] found job 17075147 in provider checkpoint proc_lsst.multi INFO: found job 17075147 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4087 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1604a236c3fe8bfefe84ae077985d343 with result from task 4087 parsl.dataflow.dflow DEBUG: Task 4087 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/1dbcd8c9-8270-4d96-a9eb-bd3fcbe5fb7a_isrForCrosstalkSources_997016_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/1dbcd8c9-8270-4d96-a9eb-bd3fcbe5fb7a_isrForCrosstalkSources_997016_7.stderr parsl.dataflow.dflow INFO: Task 4091 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bba9150d519b6686024b51b462d1ccf4 with result from task 4091 parsl.dataflow.dflow DEBUG: Task 4091 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c27fb42f-0e0a-404b-af81-01987ce4e5b0_isrForCrosstalkSources_996992_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4091 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c27fb42f-0e0a-404b-af81-01987ce4e5b0_isrForCrosstalkSources_996992_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:06 proc_lsst.multi:146 [INFO] found job 17075148 in provider checkpoint proc_lsst.multi INFO: found job 17075148 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4053 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d622f7c99132d7ef7b4adf59b883570 with result from task 4053 parsl.dataflow.dflow DEBUG: Task 4053 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1d69ddf2-6992-4680-8b23-bbd3a728b34e_isrForCrosstalkSources_996998_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/1d69ddf2-6992-4680-8b23-bbd3a728b34e_isrForCrosstalkSources_996998_31.stderr parsl.dataflow.dflow INFO: Task 4098 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7e1ec8ff65ee5d218ac9f387c93e0e2 with result from task 4098 parsl.dataflow.dflow DEBUG: Task 4098 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/97993fa0-9760-4ecc-831c-2916da5d71da_isrForCrosstalkSources_997001_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4098 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/97993fa0-9760-4ecc-831c-2916da5d71da_isrForCrosstalkSources_997001_59.stderr parsl.dataflow.dflow INFO: Task 3830 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18bbac4383bdae16cd537a388b0032b0 with result from task 3830 parsl.dataflow.dflow DEBUG: Task 3830 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4c47c570-4681-461e-80c8-200c9858b068_isrForCrosstalkSources_997030_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/4c47c570-4681-461e-80c8-200c9858b068_isrForCrosstalkSources_997030_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:06 proc_lsst.multi:146 [INFO] found job 17075149 in provider checkpoint proc_lsst.multi INFO: found job 17075149 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3803 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cddd114aa0d78c60c2deb2f313bb76dd with result from task 3803 parsl.dataflow.dflow DEBUG: Task 3803 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fe870f27-3f76-4e69-a28d-75f4484e6d1e_isrForCrosstalkSources_997001_56.stdout parsl.dataflow.dflow INFO: Standard error for task 3803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fe870f27-3f76-4e69-a28d-75f4484e6d1e_isrForCrosstalkSources_997001_56.stderr parsl.dataflow.dflow INFO: Task 3962 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0524eb22c6a3ab550e400dbc053d1fd9 with result from task 3962 parsl.dataflow.dflow DEBUG: Task 3962 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/c6c6eaf9-1455-4734-a552-9a7822a0ab20_isrForCrosstalkSources_996971_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/c6c6eaf9-1455-4734-a552-9a7822a0ab20_isrForCrosstalkSources_996971_42.stderr parsl.dataflow.dflow INFO: Task 4037 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e27eb4489510b6c52087bdcaa5da9da4 with result from task 4037 parsl.dataflow.dflow DEBUG: Task 4037 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/bbbb6adf-6fd4-4980-8459-40fafaad9aba_isrForCrosstalkSources_996967_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/bbbb6adf-6fd4-4980-8459-40fafaad9aba_isrForCrosstalkSources_996967_4.stderr parsl.dataflow.dflow INFO: Task 4095 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f2adefbdff40e2b73a9d06921db3dac with result from task 4095 parsl.dataflow.dflow DEBUG: Task 4095 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/0fb50ef0-beba-4fc4-87c4-257a520d6a39_isrForCrosstalkSources_996967_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/0fb50ef0-beba-4fc4-87c4-257a520d6a39_isrForCrosstalkSources_996967_10.stderr parsl.dataflow.dflow INFO: Task 3784 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d0888377dc12b6a586247c1ced07647 with result from task 3784 parsl.dataflow.dflow DEBUG: Task 3784 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/39789347-3276-4af7-9cf8-a92099708856_isrForCrosstalkSources_997026_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/39789347-3276-4af7-9cf8-a92099708856_isrForCrosstalkSources_997026_35.stderr parsl.dataflow.dflow INFO: Task 4092 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d4b59cb921a8fc341af54186fa510a0 with result from task 4092 parsl.dataflow.dflow DEBUG: Task 4092 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/99f9e8b4-78f9-4554-8d0e-131520033d80_isrForCrosstalkSources_996965_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/99f9e8b4-78f9-4554-8d0e-131520033d80_isrForCrosstalkSources_996965_48.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:07 proc_lsst.multi:146 [INFO] found job 17075150 in provider checkpoint proc_lsst.multi INFO: found job 17075150 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3779 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a0b12474824eafadc3ef775f82695e70 with result from task 3779 parsl.dataflow.dflow DEBUG: Task 3779 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/26dbba2b-fb37-46ba-825a-74afaeebaa0e_isrForCrosstalkSources_997007_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/26dbba2b-fb37-46ba-825a-74afaeebaa0e_isrForCrosstalkSources_997007_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:08 proc_lsst.multi:146 [INFO] found job 17075151 in provider checkpoint proc_lsst.multi INFO: found job 17075151 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4106 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88acee77c4e5d3447afc97b9563e3ee9 with result from task 4106 parsl.dataflow.dflow DEBUG: Task 4106 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b016b1cc-8225-4855-8d60-de3a872625af_isrForCrosstalkSources_997015_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4106 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b016b1cc-8225-4855-8d60-de3a872625af_isrForCrosstalkSources_997015_47.stderr parsl.dataflow.dflow INFO: Task 3788 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1422d93f6a58b3912a47aa6a24cdc9dd with result from task 3788 parsl.dataflow.dflow DEBUG: Task 3788 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/e3abf641-c857-49d9-a9a8-62147cbc1125_isrForCrosstalkSources_997006_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/e3abf641-c857-49d9-a9a8-62147cbc1125_isrForCrosstalkSources_997006_2.stderr parsl.dataflow.dflow INFO: Task 4114 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9233cad2bcae30bce63764fe43d0d4e8 with result from task 4114 parsl.dataflow.dflow DEBUG: Task 4114 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/983d17e5-908c-4e97-aa04-14f8e105f3e1_isrForCrosstalkSources_996952_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4114 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/983d17e5-908c-4e97-aa04-14f8e105f3e1_isrForCrosstalkSources_996952_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:08 proc_lsst.multi:146 [INFO] found job 17075152 in provider checkpoint proc_lsst.multi INFO: found job 17075152 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4018 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aebb3f8ec67bd9c97a46d2650407ab50 with result from task 4018 parsl.dataflow.dflow DEBUG: Task 4018 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/fd257609-0333-418b-b8e5-55902fffd9b8_isrForCrosstalkSources_997023_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/fd257609-0333-418b-b8e5-55902fffd9b8_isrForCrosstalkSources_997023_54.stderr parsl.dataflow.dflow INFO: Task 4045 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a151312c4f0e94baa58044e9bc4ee634 with result from task 4045 parsl.dataflow.dflow DEBUG: Task 4045 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a646866e-1f60-4c69-97cd-e892a4f203af_isrForCrosstalkSources_996994_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a646866e-1f60-4c69-97cd-e892a4f203af_isrForCrosstalkSources_996994_44.stderr parsl.dataflow.dflow INFO: Task 4036 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f63fa2b6dc3f2c6c3f1b49eabe47d97c with result from task 4036 parsl.dataflow.dflow DEBUG: Task 4036 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/87f89b21-cfe5-46af-8c77-7f9a5129b14b_isrForCrosstalkSources_996994_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/87f89b21-cfe5-46af-8c77-7f9a5129b14b_isrForCrosstalkSources_996994_54.stderr parsl.dataflow.dflow INFO: Task 3928 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fc7ccba6e9c817047c187bbf09f50eea with result from task 3928 parsl.dataflow.dflow DEBUG: Task 3928 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0fa028b7-0098-4218-87f2-3990e1c95346_isrForCrosstalkSources_996982_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/0fa028b7-0098-4218-87f2-3990e1c95346_isrForCrosstalkSources_996982_42.stderr parsl.dataflow.dflow INFO: Task 4052 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44c3f7c8a800e57760e850cf3ca24945 with result from task 4052 parsl.dataflow.dflow DEBUG: Task 4052 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a81bb35b-adc9-42c3-a57d-e62abab5d110_isrForCrosstalkSources_997015_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a81bb35b-adc9-42c3-a57d-e62abab5d110_isrForCrosstalkSources_997015_61.stderr parsl.dataflow.dflow INFO: Task 3878 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d5e43e3b01ecff1c02f762fd0edd423 with result from task 3878 parsl.dataflow.dflow DEBUG: Task 3878 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e3169082-d695-42ea-8924-a3380596cb46_isrForCrosstalkSources_997022_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e3169082-d695-42ea-8924-a3380596cb46_isrForCrosstalkSources_997022_59.stderr parsl.dataflow.dflow INFO: Task 4124 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 77fd25016a0329d8b5598a280ae5fb02 with result from task 4124 parsl.dataflow.dflow DEBUG: Task 4124 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/28b7e20c-65e0-41bf-bd23-32e707725b14_isrForCrosstalkSources_996953_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4124 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/28b7e20c-65e0-41bf-bd23-32e707725b14_isrForCrosstalkSources_996953_19.stderr parsl.dataflow.dflow INFO: Task 3811 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c2b2ef4b9b906c7b70b5ec8a3a6aa13 with result from task 3811 parsl.dataflow.dflow DEBUG: Task 3811 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0b5d3bf9-857a-481a-a28a-ef0e2aa91feb_isrForCrosstalkSources_996980_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/0b5d3bf9-857a-481a-a28a-ef0e2aa91feb_isrForCrosstalkSources_996980_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:09 proc_lsst.multi:146 [INFO] found job 17075153 in provider checkpoint proc_lsst.multi INFO: found job 17075153 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4134 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 211fc6c285af33faed08edba655a3aab with result from task 4134 parsl.dataflow.dflow DEBUG: Task 4134 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ce1f7a10-772e-423a-b431-9de22abad73b_isrForCrosstalkSources_996975_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4134 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/ce1f7a10-772e-423a-b431-9de22abad73b_isrForCrosstalkSources_996975_50.stderr parsl.dataflow.dflow INFO: Task 3773 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 137ca423108cd39a146420ebddc99feb with result from task 3773 parsl.dataflow.dflow DEBUG: Task 3773 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/42bef535-11f7-4ca9-b7b7-dafd0837fa75_isrForCrosstalkSources_996999_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/42bef535-11f7-4ca9-b7b7-dafd0837fa75_isrForCrosstalkSources_996999_49.stderr parsl.dataflow.dflow INFO: Task 4119 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f94191a669cad51251c5d5ed63f78f78 with result from task 4119 parsl.dataflow.dflow DEBUG: Task 4119 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a1a44dce-1acc-4cf5-bb86-1b8f750a8123_isrForCrosstalkSources_996977_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4119 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/a1a44dce-1acc-4cf5-bb86-1b8f750a8123_isrForCrosstalkSources_996977_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:10 proc_lsst.multi:146 [INFO] found job 17075154 in provider checkpoint proc_lsst.multi INFO: found job 17075154 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4133 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c37e400a10626d0ff211dcec2ddda103 with result from task 4133 parsl.dataflow.dflow DEBUG: Task 4133 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/48bcfc13-f82b-4bb2-b3b6-898712cd8b98_isrForCrosstalkSources_997031_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4133 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/48bcfc13-f82b-4bb2-b3b6-898712cd8b98_isrForCrosstalkSources_997031_57.stderr parsl.dataflow.dflow INFO: Task 3787 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b36a78c84a60ecd1ce2f9ea42d80416 with result from task 3787 parsl.dataflow.dflow DEBUG: Task 3787 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6517670c-8319-4a01-9ad7-9d55f41e9004_isrForCrosstalkSources_996996_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/6517670c-8319-4a01-9ad7-9d55f41e9004_isrForCrosstalkSources_996996_12.stderr parsl.dataflow.dflow INFO: Task 4146 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 972c54c424ac3703e6e0da38a5ee73c7 with result from task 4146 parsl.dataflow.dflow DEBUG: Task 4146 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c6bef2d1-9842-4722-8d55-7b984e78303d_isrForCrosstalkSources_997007_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4146 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/c6bef2d1-9842-4722-8d55-7b984e78303d_isrForCrosstalkSources_997007_4.stderr parsl.dataflow.dflow INFO: Task 3848 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44bef34bc7164ffaad624f9ed18f45cb with result from task 3848 parsl.dataflow.dflow DEBUG: Task 3848 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/1000b7ce-33b7-4c2a-9d6c-9c5a866300b1_isrForCrosstalkSources_996992_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/1000b7ce-33b7-4c2a-9d6c-9c5a866300b1_isrForCrosstalkSources_996992_2.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:10 proc_lsst.multi:146 [INFO] found job 17075155 in provider checkpoint proc_lsst.multi INFO: found job 17075155 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3791 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d85169c8d37c8d812dde1c00993a5b0 with result from task 3791 parsl.dataflow.dflow DEBUG: Task 3791 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/83b26efd-57fe-4242-b5cd-e3e2f3e54e2e_isrForCrosstalkSources_997020_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/83b26efd-57fe-4242-b5cd-e3e2f3e54e2e_isrForCrosstalkSources_997020_48.stderr parsl.dataflow.dflow INFO: Task 3744 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b21a429dc9c668db746af84f8286089 with result from task 3744 parsl.dataflow.dflow DEBUG: Task 3744 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/44b17767-4c91-4f69-bd6f-d359202563e9_isrForCrosstalkSources_997012_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/44b17767-4c91-4f69-bd6f-d359202563e9_isrForCrosstalkSources_997012_59.stderr parsl.dataflow.dflow INFO: Task 3752 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 45c1170b3e77c1452af8ba1d68fcd21b with result from task 3752 parsl.dataflow.dflow DEBUG: Task 3752 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/8e64767d-bfd6-48e5-bb78-48525b3e9813_isrForCrosstalkSources_997021_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/8e64767d-bfd6-48e5-bb78-48525b3e9813_isrForCrosstalkSources_997021_60.stderr parsl.dataflow.dflow INFO: Task 3757 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70a922d3797f5ebedc8ca3fd9b4b5801 with result from task 3757 parsl.dataflow.dflow DEBUG: Task 3757 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/00e8d836-1fa1-4b20-9bfe-3a88ae9f0688_isrForCrosstalkSources_996980_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/00e8d836-1fa1-4b20-9bfe-3a88ae9f0688_isrForCrosstalkSources_996980_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:11 proc_lsst.multi:146 [INFO] found job 17075157 in provider checkpoint proc_lsst.multi INFO: found job 17075157 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3797 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f2343115503c14d9262dee5018aeb8d with result from task 3797 parsl.dataflow.dflow DEBUG: Task 3797 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/24d1276e-9655-4668-a656-ad603693ad42_isrForCrosstalkSources_997033_6.stdout parsl.dataflow.dflow INFO: Standard error for task 3797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/24d1276e-9655-4668-a656-ad603693ad42_isrForCrosstalkSources_997033_6.stderr parsl.dataflow.dflow INFO: Task 4155 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f371cbda97e9f9913896cca75240b82 with result from task 4155 parsl.dataflow.dflow DEBUG: Task 4155 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/493338f7-d670-4798-863a-3ee3df092cef_isrForCrosstalkSources_996974_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4155 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/493338f7-d670-4798-863a-3ee3df092cef_isrForCrosstalkSources_996974_25.stderr parsl.dataflow.dflow INFO: Task 3917 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fb952dbbab71c1aff6735793fced821 with result from task 3917 parsl.dataflow.dflow DEBUG: Task 3917 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/82a8eef1-4d24-473d-b01c-c5e74befa9a3_isrForCrosstalkSources_997030_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/82a8eef1-4d24-473d-b01c-c5e74befa9a3_isrForCrosstalkSources_997030_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3790 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:11 proc_lsst.multi:146 [INFO] found job 17075159 in provider checkpoint proc_lsst.multi INFO: found job 17075159 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.memoization DEBUG: Storing app cache entry e5c90e85f67e2d6f82548de25cb98fea with result from task 3790 parsl.dataflow.dflow DEBUG: Task 3790 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d6c13469-2ee2-4e3b-b3fb-6c59120dee38_isrForCrosstalkSources_996958_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/d6c13469-2ee2-4e3b-b3fb-6c59120dee38_isrForCrosstalkSources_996958_37.stderr parsl.dataflow.dflow INFO: Task 3916 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 033004d633d1956817ad928c4cb23e57 with result from task 3916 parsl.dataflow.dflow DEBUG: Task 3916 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6cfc74e1-8bdd-4f11-aa87-25caf81beb07_isrForCrosstalkSources_996982_33.stdout parsl.dataflow.dflow INFO: Standard error for task 3916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/6cfc74e1-8bdd-4f11-aa87-25caf81beb07_isrForCrosstalkSources_996982_33.stderr parsl.dataflow.dflow INFO: Task 4152 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97fc08ad3310eea7d1a8519c3f1390f4 with result from task 4152 parsl.dataflow.dflow DEBUG: Task 4152 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/1480630c-0858-4e6f-bba6-967096e7f632_isrForCrosstalkSources_997002_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4152 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/1480630c-0858-4e6f-bba6-967096e7f632_isrForCrosstalkSources_997002_60.stderr parsl.dataflow.dflow INFO: Task 3775 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b5eb75a8596d27cb0405a02c2e57f67 with result from task 3775 parsl.dataflow.dflow DEBUG: Task 3775 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/689e85b4-b3f5-45aa-a939-40dfd49d3267_isrForCrosstalkSources_996971_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/689e85b4-b3f5-45aa-a939-40dfd49d3267_isrForCrosstalkSources_996971_49.stderr parsl.dataflow.dflow INFO: Task 4025 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c79cadfa3d8edf842d466215fbf33809 with result from task 4025 parsl.dataflow.dflow DEBUG: Task 4025 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/b18a10fc-d4a4-45d7-aa9a-0c1f4b1b379d_isrForCrosstalkSources_997029_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/b18a10fc-d4a4-45d7-aa9a-0c1f4b1b379d_isrForCrosstalkSources_997029_26.stderr parsl.app.errors DEBUG: Reraising exception of type parsl.dataflow.dflow DEBUG: Task 3067 try 0 failed parsl.dataflow.dflow INFO: Task 3067 marked for retry parsl.dataflow.dflow INFO: Standard output for task 3067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e3c2a0bd-c124-40e8-b6c6-536607cd459f_isrForCrosstalkSources_997016_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e3c2a0bd-c124-40e8-b6c6-536607cd459f_isrForCrosstalkSources_997016_17.stderr parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3067 has memoization hash 003497dc6ecb95758657ef41cf23c216 parsl.dataflow.memoization INFO: Task 3067 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c864900> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3067 try 1 launched on executor multi with executor id 5085 parsl.dataflow.dflow INFO: Standard output for task 3067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e3c2a0bd-c124-40e8-b6c6-536607cd459f_isrForCrosstalkSources_997016_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e3c2a0bd-c124-40e8-b6c6-536607cd459f_isrForCrosstalkSources_997016_17.stderr parsl.app.errors DEBUG: Reraising exception of type parsl.dataflow.dflow DEBUG: Task 3087 try 0 failed parsl.dataflow.dflow INFO: Task 3087 marked for retry parsl.dataflow.dflow INFO: Standard output for task 3087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68aa2644-aebf-4fe4-b2e7-b56d61cece5b_isrForCrosstalkSources_996985_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68aa2644-aebf-4fe4-b2e7-b56d61cece5b_isrForCrosstalkSources_996985_22.stderr parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3087 has memoization hash b147eccca8403b687e3e5c44c281d0c1 parsl.dataflow.memoization INFO: Task 3087 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c865580> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3087 try 1 launched on executor multi with executor id 5086 parsl.dataflow.dflow INFO: Standard output for task 3087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68aa2644-aebf-4fe4-b2e7-b56d61cece5b_isrForCrosstalkSources_996985_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68aa2644-aebf-4fe4-b2e7-b56d61cece5b_isrForCrosstalkSources_996985_22.stderr parsl.app.errors DEBUG: Reraising exception of type parsl.dataflow.dflow DEBUG: Task 3092 try 0 failed parsl.dataflow.dflow INFO: Task 3092 marked for retry parsl.dataflow.dflow INFO: Standard output for task 3092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1ccfd1b0-f028-482b-b5f1-2e24563e8c51_isrForCrosstalkSources_996979_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1ccfd1b0-f028-482b-b5f1-2e24563e8c51_isrForCrosstalkSources_996979_48.stderr parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3092 has memoization hash cb1a2069d81441632688400a9851e131 parsl.dataflow.memoization INFO: Task 3092 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c8658a0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3092 try 1 launched on executor multi with executor id 5087 parsl.dataflow.dflow INFO: Standard output for task 3092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1ccfd1b0-f028-482b-b5f1-2e24563e8c51_isrForCrosstalkSources_996979_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1ccfd1b0-f028-482b-b5f1-2e24563e8c51_isrForCrosstalkSources_996979_48.stderr parsl.app.errors DEBUG: Reraising exception of type parsl.dataflow.dflow DEBUG: Task 3094 try 0 failed parsl.dataflow.dflow INFO: Task 3094 marked for retry parsl.dataflow.dflow INFO: Standard output for task 3094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5b03ed22-6a15-4441-aaca-4554315f2b98_isrForCrosstalkSources_996954_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5b03ed22-6a15-4441-aaca-4554315f2b98_isrForCrosstalkSources_996954_7.stderr parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3094 has memoization hash 1fa8ff7f3a7600943b4041b7e3034b50 parsl.dataflow.memoization INFO: Task 3094 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0a40> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3094 try 1 launched on executor multi with executor id 5088 parsl.dataflow.dflow INFO: Standard output for task 3094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5b03ed22-6a15-4441-aaca-4554315f2b98_isrForCrosstalkSources_996954_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5b03ed22-6a15-4441-aaca-4554315f2b98_isrForCrosstalkSources_996954_7.stderr parsl.app.errors DEBUG: Reraising exception of type parsl.dataflow.dflow DEBUG: Task 3095 try 0 failed parsl.dataflow.dflow INFO: Task 3095 marked for retry parsl.dataflow.dflow INFO: Standard output for task 3095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/285b4a39-a344-4ae3-a649-6b93608cf698_isrForCrosstalkSources_996974_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/285b4a39-a344-4ae3-a649-6b93608cf698_isrForCrosstalkSources_996974_41.stderr parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3095 has memoization hash 37fdd7bc349da7e5549d9d4591b6404b parsl.dataflow.memoization INFO: Task 3095 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7d0ae0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3095 try 1 launched on executor multi with executor id 5089 parsl.dataflow.dflow INFO: Standard output for task 3095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/285b4a39-a344-4ae3-a649-6b93608cf698_isrForCrosstalkSources_996974_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/285b4a39-a344-4ae3-a649-6b93608cf698_isrForCrosstalkSources_996974_41.stderr parsl.app.errors DEBUG: Reraising exception of type parsl.dataflow.dflow DEBUG: Task 3156 try 0 failed parsl.dataflow.dflow INFO: Task 3156 marked for retry parsl.dataflow.dflow INFO: Standard output for task 3156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dc1bfb09-a884-47f8-a8b1-e7606fe00f8a_isrForCrosstalkSources_997001_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dc1bfb09-a884-47f8-a8b1-e7606fe00f8a_isrForCrosstalkSources_997001_25.stderr parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3156 has memoization hash 8c918c8bc0f2959762072811f8264819 parsl.dataflow.memoization INFO: Task 3156 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7000e0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3156 try 1 launched on executor multi with executor id 5090 parsl.dataflow.dflow INFO: Standard output for task 3156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dc1bfb09-a884-47f8-a8b1-e7606fe00f8a_isrForCrosstalkSources_997001_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dc1bfb09-a884-47f8-a8b1-e7606fe00f8a_isrForCrosstalkSources_997001_25.stderr parsl.app.errors DEBUG: Reraising exception of type parsl.dataflow.dflow DEBUG: Task 3167 try 0 failed parsl.dataflow.dflow INFO: Task 3167 marked for retry parsl.dataflow.dflow INFO: Standard output for task 3167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/88366b76-4063-4fe2-9b3a-e6a45b5d50a3_isrForCrosstalkSources_996959_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/88366b76-4063-4fe2-9b3a-e6a45b5d50a3_isrForCrosstalkSources_996959_10.stderr parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3167 has memoization hash 2639005c90196ace37b8d59c99c096f5 parsl.dataflow.memoization INFO: Task 3167 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c7007c0> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:12 proc_lsst.multi:146 [INFO] found job 17075160 in provider checkpoint proc_lsst.multi INFO: found job 17075160 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Parsl task 3167 try 1 launched on executor multi with executor id 5091 parsl.dataflow.dflow INFO: Standard output for task 3167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/88366b76-4063-4fe2-9b3a-e6a45b5d50a3_isrForCrosstalkSources_996959_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/88366b76-4063-4fe2-9b3a-e6a45b5d50a3_isrForCrosstalkSources_996959_10.stderr parsl.app.errors DEBUG: Reraising exception of type parsl.dataflow.dflow DEBUG: Task 3220 try 0 failed parsl.dataflow.dflow INFO: Task 3220 marked for retry parsl.dataflow.dflow INFO: Standard output for task 3220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/05260c5a-fefd-4162-9220-65ecb2a345e6_isrForCrosstalkSources_997001_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/05260c5a-fefd-4162-9220-65ecb2a345e6_isrForCrosstalkSources_997001_49.stderr parsl.dataflow.memoization DEBUG: Ignoring these kwargs for checkpointing: ['stderr', 'stdout'] parsl.dataflow.memoization DEBUG: Ignoring kwarg stderr parsl.dataflow.memoization DEBUG: Ignoring kwarg stdout parsl.dataflow.memoization DEBUG: Task 3220 has memoization hash 39294c674e8523ae40f3eb849e56e7f9 parsl.dataflow.memoization INFO: Task 3220 had no result in cache parsl.executors.high_throughput.executor DEBUG: Pushing function .wrapper at 0x14a77c702840> to queue with args ("'${CTRL_MPEXEC_DIR}/bin/pipetask --long-log --log-level=VERBOSE run-qbb /mmfs1/home/stevengs/dirac/D...",) parsl.dataflow.dflow INFO: Parsl task 3220 try 1 launched on executor multi with executor id 5092 parsl.dataflow.dflow INFO: Standard output for task 3220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/05260c5a-fefd-4162-9220-65ecb2a345e6_isrForCrosstalkSources_997001_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/05260c5a-fefd-4162-9220-65ecb2a345e6_isrForCrosstalkSources_997001_49.stderr parsl.dataflow.dflow INFO: Task 3838 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f5f96e3ea7e0da78bf063e818033a3f4 with result from task 3838 parsl.dataflow.dflow DEBUG: Task 3838 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fd6d775d-86a0-4713-a715-cce6faa3a8e1_isrForCrosstalkSources_997012_26.stdout parsl.dataflow.dflow INFO: Standard error for task 3838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fd6d775d-86a0-4713-a715-cce6faa3a8e1_isrForCrosstalkSources_997012_26.stderr parsl.dataflow.dflow INFO: Task 4012 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56a5ca0661d6ced38566bac339a00a2d with result from task 4012 parsl.dataflow.dflow DEBUG: Task 4012 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0a3c32fc-4e37-4bd5-ba84-977675118796_isrForCrosstalkSources_997020_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/0a3c32fc-4e37-4bd5-ba84-977675118796_isrForCrosstalkSources_997020_25.stderr parsl.dataflow.dflow INFO: Task 3897 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d2b8dbfcf3f6d507b800e0d5341e069e with result from task 3897 parsl.dataflow.dflow DEBUG: Task 3897 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/50d8d36a-316a-419b-a1de-067e24a55655_isrForCrosstalkSources_996963_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/50d8d36a-316a-419b-a1de-067e24a55655_isrForCrosstalkSources_996963_17.stderr parsl.dataflow.dflow INFO: Task 3861 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 969cdb1c7087c216c1c9071497399a6c with result from task 3861 parsl.dataflow.dflow DEBUG: Task 3861 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9cb1968d-2c9a-419a-8d7a-25f92a65d570_isrForCrosstalkSources_997015_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/9cb1968d-2c9a-419a-8d7a-25f92a65d570_isrForCrosstalkSources_997015_24.stderr parsl.dataflow.dflow INFO: Task 4151 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3be5e972f659fad40b897b4e58ad4ce1 with result from task 4151 parsl.dataflow.dflow DEBUG: Task 4151 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7188aa70-d8a5-4e45-8b8b-f4c89d4ef759_isrForCrosstalkSources_997021_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4151 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/7188aa70-d8a5-4e45-8b8b-f4c89d4ef759_isrForCrosstalkSources_997021_2.stderr parsl.dataflow.dflow INFO: Task 3937 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f6b99a7f2cd09185ae24c01fdbf30621 with result from task 3937 parsl.dataflow.dflow DEBUG: Task 3937 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e86cb61f-9ea5-441d-9eff-7bef0211c326_isrForCrosstalkSources_997022_60.stdout parsl.dataflow.dflow INFO: Standard error for task 3937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/e86cb61f-9ea5-441d-9eff-7bef0211c326_isrForCrosstalkSources_997022_60.stderr parsl.dataflow.dflow INFO: Task 3821 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3da7838e6a6d7d1fe73cdc0bb40c6c48 with result from task 3821 parsl.dataflow.dflow DEBUG: Task 3821 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/5941fbef-6797-4286-b7c1-854125544cc6_isrForCrosstalkSources_997013_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/5941fbef-6797-4286-b7c1-854125544cc6_isrForCrosstalkSources_997013_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:13 proc_lsst.multi:146 [INFO] found job 17075161 in provider checkpoint proc_lsst.multi INFO: found job 17075161 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4156 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46252afcd820f20a9b38bc4a7ddc71a5 with result from task 4156 parsl.dataflow.dflow DEBUG: Task 4156 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/e18e5beb-78b6-4cbf-beaf-d3f169570390_isrForCrosstalkSources_997000_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/e18e5beb-78b6-4cbf-beaf-d3f169570390_isrForCrosstalkSources_997000_47.stderr parsl.dataflow.dflow INFO: Task 4149 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd4eb4cc0bfe19ded56a0e10a322064b with result from task 4149 parsl.dataflow.dflow DEBUG: Task 4149 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/88892a16-3df8-4143-8608-6f43ec305f53_isrForCrosstalkSources_996984_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4149 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/88892a16-3df8-4143-8608-6f43ec305f53_isrForCrosstalkSources_996984_44.stderr parsl.dataflow.dflow INFO: Task 3781 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6101d543b1e0c43db8dbeea67c69774a with result from task 3781 parsl.dataflow.dflow DEBUG: Task 3781 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/08d9a9a2-5c0a-4317-9655-0107f74bc733_isrForCrosstalkSources_996975_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/08d9a9a2-5c0a-4317-9655-0107f74bc733_isrForCrosstalkSources_996975_42.stderr parsl.dataflow.dflow INFO: Task 4168 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a33b8738952d490ae96f1a0859288b7 with result from task 4168 parsl.dataflow.dflow DEBUG: Task 4168 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/16ffb8f2-5ad0-4de6-b929-cf29139e7acd_isrForCrosstalkSources_997024_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4168 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/16ffb8f2-5ad0-4de6-b929-cf29139e7acd_isrForCrosstalkSources_997024_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:13 proc_lsst.multi:146 [INFO] found job 17075162 in provider checkpoint proc_lsst.multi INFO: found job 17075162 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4164 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b6332273a73719c46707d136a30469e with result from task 4164 parsl.dataflow.dflow DEBUG: Task 4164 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/65c89758-1653-41db-9358-35fd9c955afe_isrForCrosstalkSources_996972_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4164 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/65c89758-1653-41db-9358-35fd9c955afe_isrForCrosstalkSources_996972_44.stderr parsl.dataflow.dflow INFO: Task 4169 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6bab71556478705e84c4d338a27cd070 with result from task 4169 parsl.dataflow.dflow DEBUG: Task 4169 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/52f7e4e0-4381-4f4c-acae-28286a816f92_isrForCrosstalkSources_997014_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4169 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/52f7e4e0-4381-4f4c-acae-28286a816f92_isrForCrosstalkSources_997014_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:14 proc_lsst.multi:146 [INFO] found job 17075163 in provider checkpoint proc_lsst.multi INFO: found job 17075163 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4094 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d0e24e6acc45020298efd8668db5b0e with result from task 4094 parsl.dataflow.dflow DEBUG: Task 4094 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/052094d5-d50b-4cfb-b9a3-ef2fcfd4807a_isrForCrosstalkSources_996959_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/052094d5-d50b-4cfb-b9a3-ef2fcfd4807a_isrForCrosstalkSources_996959_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:14 proc_lsst.multi:146 [INFO] found job 17075166 in provider checkpoint proc_lsst.multi INFO: found job 17075166 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4165 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f48173614fc058ab1b1cc7122077cc4 with result from task 4165 parsl.dataflow.dflow DEBUG: Task 4165 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a42f1078-aac2-4d79-b0d6-0c119db67e00_isrForCrosstalkSources_996954_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4165 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a42f1078-aac2-4d79-b0d6-0c119db67e00_isrForCrosstalkSources_996954_43.stderr parsl.dataflow.dflow INFO: Task 4158 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00bbe70ef127b45d75d25b7f80bea7c8 with result from task 4158 parsl.dataflow.dflow DEBUG: Task 4158 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/eddbdf41-916e-4ec2-87dd-6ace3035f384_isrForCrosstalkSources_996977_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4158 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/eddbdf41-916e-4ec2-87dd-6ace3035f384_isrForCrosstalkSources_996977_54.stderr parsl.dataflow.dflow INFO: Task 4113 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8eb727d700be1e2d7f6178ad620a95d with result from task 4113 parsl.dataflow.dflow DEBUG: Task 4113 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/d2d3a68e-52aa-4995-b5bd-974a627a9be2_isrForCrosstalkSources_997021_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4113 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/d2d3a68e-52aa-4995-b5bd-974a627a9be2_isrForCrosstalkSources_997021_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:15 proc_lsst.multi:146 [INFO] found job 17075167 in provider checkpoint proc_lsst.multi INFO: found job 17075167 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3953 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0391875a935be92c63ac18ac3d94f6fb with result from task 3953 parsl.dataflow.dflow DEBUG: Task 3953 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0eaba4eb-d87e-4cc5-b54c-6805f50ffe9d_isrForCrosstalkSources_996970_40.stdout parsl.dataflow.dflow INFO: Standard error for task 3953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/0eaba4eb-d87e-4cc5-b54c-6805f50ffe9d_isrForCrosstalkSources_996970_40.stderr parsl.dataflow.dflow INFO: Task 4040 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry deb7b936d36330f093495532120eb977 with result from task 4040 parsl.dataflow.dflow DEBUG: Task 4040 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6bd79186-92e3-455d-aa8a-08750f46bbf6_isrForCrosstalkSources_996970_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/6bd79186-92e3-455d-aa8a-08750f46bbf6_isrForCrosstalkSources_996970_14.stderr parsl.dataflow.dflow INFO: Task 4180 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e133a9636b23dc094456f5da02741d12 with result from task 4180 parsl.dataflow.dflow DEBUG: Task 4180 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/5f4d5d13-7d12-48d9-ba25-c4631233a368_isrForCrosstalkSources_996993_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4180 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/5f4d5d13-7d12-48d9-ba25-c4631233a368_isrForCrosstalkSources_996993_30.stderr parsl.dataflow.dflow INFO: Task 4178 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d9fc446f93604534f0a47df4ae54de7 with result from task 4178 parsl.dataflow.dflow DEBUG: Task 4178 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/8d359b4b-ede0-4c3c-94f0-1347ef92883d_isrForCrosstalkSources_997003_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4178 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/8d359b4b-ede0-4c3c-94f0-1347ef92883d_isrForCrosstalkSources_997003_25.stderr parsl.dataflow.dflow INFO: Task 4161 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 80876451aeb98a2f3464bc6da205b37d with result from task 4161 parsl.dataflow.dflow DEBUG: Task 4161 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/e01626b5-494c-4e36-b550-bfec6c22f7f4_isrForCrosstalkSources_996974_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4161 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/e01626b5-494c-4e36-b550-bfec6c22f7f4_isrForCrosstalkSources_996974_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:16 proc_lsst.multi:146 [INFO] found job 17075168 in provider checkpoint proc_lsst.multi INFO: found job 17075168 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3954 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry deef4d204c82c6efd6cefbb98978e05f with result from task 3954 parsl.dataflow.dflow DEBUG: Task 3954 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0b9bcfe9-78ca-45d5-bde8-c6a96e053622_isrForCrosstalkSources_996983_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0b9bcfe9-78ca-45d5-bde8-c6a96e053622_isrForCrosstalkSources_996983_11.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:16 proc_lsst.multi:146 [INFO] found job 17075171 in provider checkpoint proc_lsst.multi INFO: found job 17075171 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4218 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4d56df95c520662773317725450bcea with result from task 4218 parsl.dataflow.dflow DEBUG: Task 4218 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/02bfc06a-d221-404c-af07-00641b74e8ab_isrForCrosstalkSources_996990_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4218 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/02bfc06a-d221-404c-af07-00641b74e8ab_isrForCrosstalkSources_996990_45.stderr parsl.dataflow.dflow INFO: Task 4042 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10c88a9faa9c9f384cbf50e13c015453 with result from task 4042 parsl.dataflow.dflow DEBUG: Task 4042 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/6d1550b6-214c-44a3-8b3c-a83515aac1e9_isrForCrosstalkSources_996985_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/6d1550b6-214c-44a3-8b3c-a83515aac1e9_isrForCrosstalkSources_996985_13.stderr parsl.dataflow.dflow INFO: Task 4183 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e70c8aa219539853a7cf026a2048290b with result from task 4183 parsl.dataflow.dflow DEBUG: Task 4183 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/6b2b9caa-5bcd-4e61-9255-7957686f7732_isrForCrosstalkSources_997021_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4183 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/6b2b9caa-5bcd-4e61-9255-7957686f7732_isrForCrosstalkSources_997021_51.stderr parsl.dataflow.dflow INFO: Task 4192 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2a49ecc37e00be8824baeb095fd75dd2 with result from task 4192 parsl.dataflow.dflow DEBUG: Task 4192 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7483437d-a0ce-4e2d-8b58-5774831bf904_isrForCrosstalkSources_997005_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4192 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7483437d-a0ce-4e2d-8b58-5774831bf904_isrForCrosstalkSources_997005_35.stderr parsl.dataflow.dflow INFO: Task 3886 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ebc156439046a149da7113234062a362 with result from task 3886 parsl.dataflow.dflow DEBUG: Task 3886 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2410b7eb-c714-42a6-b534-5b2493714dba_isrForCrosstalkSources_997002_9.stdout parsl.dataflow.dflow INFO: Standard error for task 3886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/2410b7eb-c714-42a6-b534-5b2493714dba_isrForCrosstalkSources_997002_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:17 proc_lsst.multi:146 [INFO] found job 17075172 in provider checkpoint proc_lsst.multi INFO: found job 17075172 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3895 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99e60e7a7c550372eed71def9014455e with result from task 3895 parsl.dataflow.dflow DEBUG: Task 3895 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/236b5fe0-4c79-4a06-8241-301a730f12fe_isrForCrosstalkSources_997002_23.stdout parsl.dataflow.dflow INFO: Standard error for task 3895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/236b5fe0-4c79-4a06-8241-301a730f12fe_isrForCrosstalkSources_997002_23.stderr parsl.dataflow.dflow INFO: Task 4206 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6c9cf0c67983dc095b9275462890d98 with result from task 4206 parsl.dataflow.dflow DEBUG: Task 4206 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0ffa1115-317e-4ae7-87b4-e352e125938e_isrForCrosstalkSources_997021_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4206 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0ffa1115-317e-4ae7-87b4-e352e125938e_isrForCrosstalkSources_997021_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:17 proc_lsst.multi:146 [INFO] found job 17075173 in provider checkpoint proc_lsst.multi INFO: found job 17075173 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4130 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46c485b9d67d18f30f43a38868fc187b with result from task 4130 parsl.dataflow.dflow DEBUG: Task 4130 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/85720f88-e3ca-4be5-a8c6-91b4c47889e9_isrForCrosstalkSources_997010_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4130 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/85720f88-e3ca-4be5-a8c6-91b4c47889e9_isrForCrosstalkSources_997010_14.stderr parsl.dataflow.dflow INFO: Task 3901 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 143a6e303d517ba60352037a88a6c67a with result from task 3901 parsl.dataflow.dflow DEBUG: Task 3901 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/c6be46a3-925e-42db-8810-692808c04c60_isrForCrosstalkSources_996974_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/c6be46a3-925e-42db-8810-692808c04c60_isrForCrosstalkSources_996974_11.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:18 proc_lsst.multi:146 [INFO] found job 17075179 in provider checkpoint proc_lsst.multi INFO: found job 17075179 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4015 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9071a990338ee7a21efbc6ffce013128 with result from task 4015 parsl.dataflow.dflow DEBUG: Task 4015 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/8e781840-b022-4524-8ad2-923877223d04_isrForCrosstalkSources_996972_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/8e781840-b022-4524-8ad2-923877223d04_isrForCrosstalkSources_996972_21.stderr parsl.dataflow.dflow INFO: Task 4013 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 784e9cd7b60a0fcce0572faa8b3cdceb with result from task 4013 parsl.dataflow.dflow DEBUG: Task 4013 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/dcc8270d-7943-4d9b-b00b-a0cb52b0f937_isrForCrosstalkSources_997032_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/dcc8270d-7943-4d9b-b00b-a0cb52b0f937_isrForCrosstalkSources_997032_46.stderr parsl.dataflow.dflow INFO: Task 4063 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb29d7bd0c29c2137f2cce2a0497bbcf with result from task 4063 parsl.dataflow.dflow DEBUG: Task 4063 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/391d17eb-1377-408a-abdf-c32a531b631b_isrForCrosstalkSources_997029_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/391d17eb-1377-408a-abdf-c32a531b631b_isrForCrosstalkSources_997029_15.stderr parsl.dataflow.dflow INFO: Task 4125 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6124831ab0565daa98a5edb3d5051dcc with result from task 4125 parsl.dataflow.dflow DEBUG: Task 4125 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c99ed4ec-fe96-4f2c-a90f-9566728cef85_isrForCrosstalkSources_996972_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4125 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/c99ed4ec-fe96-4f2c-a90f-9566728cef85_isrForCrosstalkSources_996972_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:19 proc_lsst.multi:146 [INFO] found job 17075180 in provider checkpoint proc_lsst.multi INFO: found job 17075180 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3922 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d5e6e0b22303e165f0bbcf31a5ee8954 with result from task 3922 parsl.dataflow.dflow DEBUG: Task 3922 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e697d78e-0a7f-4fed-86c4-bc41fb9ae429_isrForCrosstalkSources_996963_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/e697d78e-0a7f-4fed-86c4-bc41fb9ae429_isrForCrosstalkSources_996963_35.stderr parsl.dataflow.dflow INFO: Task 4227 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ec11028ef8887e5514d4c54d2d00ec0f with result from task 4227 parsl.dataflow.dflow DEBUG: Task 4227 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/7e86c72b-e108-41ad-8328-7bfc4b095dd4_isrForCrosstalkSources_997018_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4227 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/7e86c72b-e108-41ad-8328-7bfc4b095dd4_isrForCrosstalkSources_997018_25.stderr parsl.dataflow.dflow INFO: Task 3918 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5762a8361dd693ac1b4ec410b74e604 with result from task 3918 parsl.dataflow.dflow DEBUG: Task 3918 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/70ceb363-bf38-4a42-b19a-e9ff781b1184_isrForCrosstalkSources_997023_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/70ceb363-bf38-4a42-b19a-e9ff781b1184_isrForCrosstalkSources_997023_2.stderr parsl.dataflow.dflow INFO: Task 3862 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73254b1dbe67785f56e0b3c159559407 with result from task 3862 parsl.dataflow.dflow DEBUG: Task 3862 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3ad37cc5-2530-4595-8cec-5a8f696835b6_isrForCrosstalkSources_996995_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3ad37cc5-2530-4595-8cec-5a8f696835b6_isrForCrosstalkSources_996995_48.stderr parsl.dataflow.dflow INFO: Task 4214 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44e28a95604f76de255d5931a6dd32cc with result from task 4214 parsl.dataflow.dflow DEBUG: Task 4214 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/fc650015-32a3-4bee-9989-671c27ed0b55_isrForCrosstalkSources_997028_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4214 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/fc650015-32a3-4bee-9989-671c27ed0b55_isrForCrosstalkSources_997028_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:20 proc_lsst.multi:146 [INFO] found job 17075181 in provider checkpoint proc_lsst.multi INFO: found job 17075181 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3958 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0db9950bd69192adb81ae207eab993bc with result from task 3958 parsl.dataflow.dflow DEBUG: Task 3958 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/ab0a924d-d713-42c8-bdea-bea0d95cce76_isrForCrosstalkSources_996977_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/ab0a924d-d713-42c8-bdea-bea0d95cce76_isrForCrosstalkSources_996977_46.stderr parsl.dataflow.dflow INFO: Task 3913 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0897f4dffa42c11cf4f45eac6687ffa7 with result from task 3913 parsl.dataflow.dflow DEBUG: Task 3913 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/dd1e1b40-2879-43d3-809d-44a81ac444d5_isrForCrosstalkSources_996961_11.stdout parsl.dataflow.dflow INFO: Standard error for task 3913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/dd1e1b40-2879-43d3-809d-44a81ac444d5_isrForCrosstalkSources_996961_11.stderr parsl.dataflow.dflow INFO: Task 3857 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd987e8cfa0d5d99fd53b6a0d2cc24ee with result from task 3857 parsl.dataflow.dflow DEBUG: Task 3857 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/75554253-227a-42fb-aca5-ac43121fca45_isrForCrosstalkSources_996956_2.stdout parsl.dataflow.dflow INFO: Standard error for task 3857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/75554253-227a-42fb-aca5-ac43121fca45_isrForCrosstalkSources_996956_2.stderr parsl.dataflow.dflow INFO: Task 3974 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d69f34f9d019775b823a4c2f8bb68327 with result from task 3974 parsl.dataflow.dflow DEBUG: Task 3974 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/12342e66-40a8-4bc6-9bd5-9d26242b871b_isrForCrosstalkSources_996995_35.stdout parsl.dataflow.dflow INFO: Standard error for task 3974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/12342e66-40a8-4bc6-9bd5-9d26242b871b_isrForCrosstalkSources_996995_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:20 proc_lsst.multi:146 [INFO] found job 17075182 in provider checkpoint proc_lsst.multi INFO: found job 17075182 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4217 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea6bb4a181e2bb4253fa4c120295540f with result from task 4217 parsl.dataflow.dflow DEBUG: Task 4217 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9d254a22-e5f6-46aa-8ecc-614e04b8b594_isrForCrosstalkSources_997004_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4217 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/9d254a22-e5f6-46aa-8ecc-614e04b8b594_isrForCrosstalkSources_997004_38.stderr parsl.dataflow.dflow INFO: Task 4220 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00e92c2efa666c6fa8a9719d9af9bd04 with result from task 4220 parsl.dataflow.dflow DEBUG: Task 4220 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9bdfdb3b-68ed-49af-9c93-fec3829d9d76_isrForCrosstalkSources_996984_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9bdfdb3b-68ed-49af-9c93-fec3829d9d76_isrForCrosstalkSources_996984_18.stderr parsl.dataflow.dflow INFO: Task 4229 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46165493ff0de1786ba670d4be0bbcce with result from task 4229 parsl.dataflow.dflow DEBUG: Task 4229 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/48d8e5e2-0b1d-4e17-b856-cb339e9dfaa2_isrForCrosstalkSources_996974_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4229 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/48d8e5e2-0b1d-4e17-b856-cb339e9dfaa2_isrForCrosstalkSources_996974_51.stderr parsl.dataflow.dflow INFO: Task 3963 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 18d92ca7734db8190c4deaf13748ab0f with result from task 3963 parsl.dataflow.dflow DEBUG: Task 3963 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/fca94434-a89a-4d09-92f7-e51aee01fc73_isrForCrosstalkSources_997026_4.stdout parsl.dataflow.dflow INFO: Standard error for task 3963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/fca94434-a89a-4d09-92f7-e51aee01fc73_isrForCrosstalkSources_997026_4.stderr parsl.dataflow.dflow INFO: Task 4048 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97a05cd04cebf5b3f6ea5ff77b25b2f1 with result from task 4048 parsl.dataflow.dflow DEBUG: Task 4048 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/66f8c903-4626-49f8-b9a6-7449a12e975e_isrForCrosstalkSources_996999_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/66f8c903-4626-49f8-b9a6-7449a12e975e_isrForCrosstalkSources_996999_27.stderr parsl.dataflow.dflow INFO: Task 4090 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f8e99b7da06ef4c69797008dce8d418 with result from task 4090 parsl.dataflow.dflow DEBUG: Task 4090 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/f689a925-506a-4c65-8595-11a82e9284d5_isrForCrosstalkSources_997021_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4090 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/f689a925-506a-4c65-8595-11a82e9284d5_isrForCrosstalkSources_997021_24.stderr parsl.dataflow.dflow INFO: Task 4221 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 950ead4c6ef69226c439abb848da24cd with result from task 4221 parsl.dataflow.dflow DEBUG: Task 4221 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/082178dc-332c-4ce3-b17b-738713d50400_isrForCrosstalkSources_996966_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4221 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/082178dc-332c-4ce3-b17b-738713d50400_isrForCrosstalkSources_996966_36.stderr parsl.dataflow.dflow INFO: Task 4030 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d70fc5f929ba178f88e6ab90b9f933e5 with result from task 4030 parsl.dataflow.dflow DEBUG: Task 4030 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/2ac33dc2-a453-4263-bb1a-3790ff71d36a_isrForCrosstalkSources_996972_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/2ac33dc2-a453-4263-bb1a-3790ff71d36a_isrForCrosstalkSources_996972_27.stderr parsl.dataflow.dflow INFO: Task 4103 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17c897aa7e43c82575d9fae97e5274c9 with result from task 4103 parsl.dataflow.dflow DEBUG: Task 4103 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/be020954-1366-4b89-9bc7-24bca9e90bfc_isrForCrosstalkSources_997015_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4103 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/be020954-1366-4b89-9bc7-24bca9e90bfc_isrForCrosstalkSources_997015_19.stderr parsl.dataflow.dflow INFO: Task 3950 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57b9d1d7ce4c2eedfca91753e3072f51 with result from task 3950 parsl.dataflow.dflow DEBUG: Task 3950 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7058a74d-746f-4b8d-bfa8-6f25ffa10810_isrForCrosstalkSources_996990_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7058a74d-746f-4b8d-bfa8-6f25ffa10810_isrForCrosstalkSources_996990_53.stderr parsl.dataflow.dflow INFO: Task 4022 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1083539ea131d08e53507cc33524a254 with result from task 4022 parsl.dataflow.dflow DEBUG: Task 4022 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/21d0d287-1e2d-4c3a-850c-210009a18482_isrForCrosstalkSources_996959_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/21d0d287-1e2d-4c3a-850c-210009a18482_isrForCrosstalkSources_996959_14.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:21 proc_lsst.multi:146 [INFO] found job 17075183 in provider checkpoint proc_lsst.multi INFO: found job 17075183 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4225 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31b44a132b84ef66fcf4c60f994da32f with result from task 4225 parsl.dataflow.dflow DEBUG: Task 4225 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0a8eff75-dde9-4ec9-8c97-1c171015f500_isrForCrosstalkSources_997023_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4225 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/0a8eff75-dde9-4ec9-8c97-1c171015f500_isrForCrosstalkSources_997023_31.stderr parsl.dataflow.dflow INFO: Task 3904 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a0d0f1bf5263d75f9a50eeba2740bfb with result from task 3904 parsl.dataflow.dflow DEBUG: Task 3904 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/5f5b470d-d004-43fd-90fb-5613a963436e_isrForCrosstalkSources_996994_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/5f5b470d-d004-43fd-90fb-5613a963436e_isrForCrosstalkSources_996994_22.stderr parsl.dataflow.dflow INFO: Task 3914 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aba1e8339356e3f4a0a6998f61635901 with result from task 3914 parsl.dataflow.dflow DEBUG: Task 3914 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/263820f2-1a14-4bf3-8935-6babc2070403_isrForCrosstalkSources_996962_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/263820f2-1a14-4bf3-8935-6babc2070403_isrForCrosstalkSources_996962_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075068 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:22 proc_lsst.multi:146 [INFO] found job 17075185 in provider checkpoint proc_lsst.multi INFO: found job 17075185 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4231 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8dcbc0b8d5180fea10836ee75730292 with result from task 4231 parsl.dataflow.dflow DEBUG: Task 4231 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c9feeab0-3cd6-4b59-8779-83bdf26b9077_isrForCrosstalkSources_996969_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4231 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c9feeab0-3cd6-4b59-8779-83bdf26b9077_isrForCrosstalkSources_996969_57.stderr parsl.dataflow.dflow INFO: Task 4238 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 972d445bb482a9222c60bb9fabb51993 with result from task 4238 parsl.dataflow.dflow DEBUG: Task 4238 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/26e7e675-8aad-4dd3-8018-6c282b657ce7_isrForCrosstalkSources_996984_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4238 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/26e7e675-8aad-4dd3-8018-6c282b657ce7_isrForCrosstalkSources_996984_46.stderr parsl.dataflow.dflow INFO: Task 3823 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b75a500870f6064c1b0bc6dc1f01cc7 with result from task 3823 parsl.dataflow.dflow DEBUG: Task 3823 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/d8826375-7278-4f34-b754-21a584b8e748_isrForCrosstalkSources_997013_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/d8826375-7278-4f34-b754-21a584b8e748_isrForCrosstalkSources_997013_55.stderr parsl.dataflow.dflow INFO: Task 3940 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29d3474a0825a1a4b53b7ee95e9b983f with result from task 3940 parsl.dataflow.dflow DEBUG: Task 3940 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8cfe6893-5172-41b7-84c8-b59e10c1d676_isrForCrosstalkSources_997032_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8cfe6893-5172-41b7-84c8-b59e10c1d676_isrForCrosstalkSources_997032_47.stderr parsl.dataflow.dflow INFO: Task 3970 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9e9f63567d7d3203938002e02753554 with result from task 3970 parsl.dataflow.dflow DEBUG: Task 3970 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/44e8c673-4ac8-4e04-9a93-63fc5fb8f4f1_isrForCrosstalkSources_996975_24.stdout parsl.dataflow.dflow INFO: Standard error for task 3970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/44e8c673-4ac8-4e04-9a93-63fc5fb8f4f1_isrForCrosstalkSources_996975_24.stderr parsl.dataflow.dflow INFO: Task 4007 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94eaf39462f4eeec15db8354669e7089 with result from task 4007 parsl.dataflow.dflow DEBUG: Task 4007 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e3140b86-f5c6-4244-97ff-2877c37553b5_isrForCrosstalkSources_996983_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/e3140b86-f5c6-4244-97ff-2877c37553b5_isrForCrosstalkSources_996983_40.stderr parsl.dataflow.dflow INFO: Task 3925 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc3d84d1d5ed52d6af624e40c0d8065c with result from task 3925 parsl.dataflow.dflow DEBUG: Task 3925 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/734a7b6e-59f4-4e8e-9fbf-ef5d5b60ef5a_isrForCrosstalkSources_997028_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/734a7b6e-59f4-4e8e-9fbf-ef5d5b60ef5a_isrForCrosstalkSources_997028_38.stderr parsl.dataflow.dflow INFO: Task 4021 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 36fd21edce92648fa01bf4244409ed74 with result from task 4021 parsl.dataflow.dflow DEBUG: Task 4021 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/34d207bd-f400-4aef-8eda-28c9cd380ed3_isrForCrosstalkSources_996996_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/34d207bd-f400-4aef-8eda-28c9cd380ed3_isrForCrosstalkSources_996996_46.stderr parsl.dataflow.dflow INFO: Task 3864 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38fdcc08f7be1fefed5d9ea987ac8339 with result from task 3864 parsl.dataflow.dflow DEBUG: Task 3864 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/5bd1e177-4100-41df-880f-0053664cd375_isrForCrosstalkSources_996990_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/5bd1e177-4100-41df-880f-0053664cd375_isrForCrosstalkSources_996990_42.stderr parsl.dataflow.dflow INFO: Task 4240 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50d86dabceecdb3ca08d5897bc3f6156 with result from task 4240 parsl.dataflow.dflow DEBUG: Task 4240 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3409e5ae-da2c-43b3-ac0f-078da9174de3_isrForCrosstalkSources_996955_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4240 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/3409e5ae-da2c-43b3-ac0f-078da9174de3_isrForCrosstalkSources_996955_45.stderr parsl.dataflow.dflow INFO: Task 4123 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bbd7c121e0246682d6b9ace4f1961649 with result from task 4123 parsl.dataflow.dflow DEBUG: Task 4123 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/61e7d61c-9385-4259-8082-82c0dc22d634_isrForCrosstalkSources_996987_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4123 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/61e7d61c-9385-4259-8082-82c0dc22d634_isrForCrosstalkSources_996987_11.stderr parsl.dataflow.dflow INFO: Task 4242 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 437e0e7e587503bbd94760724ff756f6 with result from task 4242 parsl.dataflow.dflow DEBUG: Task 4242 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/44012aad-b28f-4a26-91cb-628f10999a4c_isrForCrosstalkSources_996980_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4242 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/44012aad-b28f-4a26-91cb-628f10999a4c_isrForCrosstalkSources_996980_2.stderr parsl.dataflow.dflow INFO: Task 3934 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fde1c5e1fdfbf1e1b5a7f89e2df67a62 with result from task 3934 parsl.dataflow.dflow DEBUG: Task 3934 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/8b6bfe4c-5a5f-4b0a-8a96-370aa3180bbc_isrForCrosstalkSources_996982_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/8b6bfe4c-5a5f-4b0a-8a96-370aa3180bbc_isrForCrosstalkSources_996982_17.stderr parsl.dataflow.dflow INFO: Task 4075 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9ef7a1f03c1e6d4c16826a36c8af84e with result from task 4075 parsl.dataflow.dflow DEBUG: Task 4075 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/aae1b624-2ac6-46de-ae86-26a7f0d130af_isrForCrosstalkSources_996980_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/aae1b624-2ac6-46de-ae86-26a7f0d130af_isrForCrosstalkSources_996980_55.stderr parsl.dataflow.dflow INFO: Task 3855 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0be91296a5a993834b81f202b407e09 with result from task 3855 parsl.dataflow.dflow DEBUG: Task 3855 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9c495159-46d0-4fba-a11e-b8de65bf6443_isrForCrosstalkSources_996960_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9c495159-46d0-4fba-a11e-b8de65bf6443_isrForCrosstalkSources_996960_41.stderr parsl.dataflow.dflow INFO: Task 4080 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b4912986cb23e609e9ffd55022652a0e with result from task 4080 parsl.dataflow.dflow DEBUG: Task 4080 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/57b0d630-bbc0-4672-897b-cff5da1fb44d_isrForCrosstalkSources_997019_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/57b0d630-bbc0-4672-897b-cff5da1fb44d_isrForCrosstalkSources_997019_35.stderr parsl.dataflow.dflow INFO: Task 4108 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0cc456fb810ef1db86b67190a9895a88 with result from task 4108 parsl.dataflow.dflow DEBUG: Task 4108 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e1be73b5-213d-4744-b30c-4a582aa42e3d_isrForCrosstalkSources_997026_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4108 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e1be73b5-213d-4744-b30c-4a582aa42e3d_isrForCrosstalkSources_997026_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:24 proc_lsst.multi:146 [INFO] found job 17075187 in provider checkpoint proc_lsst.multi INFO: found job 17075187 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3973 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5b460570d1a61d968efd7a1ef5faf220 with result from task 3973 parsl.dataflow.dflow DEBUG: Task 3973 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/eea46093-9b38-4039-932b-a8cd5a69e33d_isrForCrosstalkSources_996967_13.stdout parsl.dataflow.dflow INFO: Standard error for task 3973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/eea46093-9b38-4039-932b-a8cd5a69e33d_isrForCrosstalkSources_996967_13.stderr parsl.dataflow.dflow INFO: Task 3930 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0435f941f3dc4fcc453c10f3bc552979 with result from task 3930 parsl.dataflow.dflow DEBUG: Task 3930 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/724f7fb1-5948-49b3-9b3d-0a691427966a_isrForCrosstalkSources_996973_42.stdout parsl.dataflow.dflow INFO: Standard error for task 3930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/724f7fb1-5948-49b3-9b3d-0a691427966a_isrForCrosstalkSources_996973_42.stderr parsl.dataflow.dflow INFO: Task 4210 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ed3227749662040cd6fdbacb68abb81 with result from task 4210 parsl.dataflow.dflow DEBUG: Task 4210 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/2c047c19-37f3-4294-8be5-2f7a54525633_isrForCrosstalkSources_996988_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4210 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/2c047c19-37f3-4294-8be5-2f7a54525633_isrForCrosstalkSources_996988_29.stderr parsl.dataflow.dflow INFO: Task 4233 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 177c7bb29a25d6821300f712edef49b7 with result from task 4233 parsl.dataflow.dflow DEBUG: Task 4233 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/1993d646-0316-4509-b624-bd2f3063c322_isrForCrosstalkSources_996970_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4233 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/1993d646-0316-4509-b624-bd2f3063c322_isrForCrosstalkSources_996970_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:24 proc_lsst.multi:146 [INFO] found job 17075188 in provider checkpoint proc_lsst.multi INFO: found job 17075188 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4236 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56a513eb6619a56b419f7fde85db2ed9 with result from task 4236 parsl.dataflow.dflow DEBUG: Task 4236 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/28217a73-f482-4f94-b5d6-7c160031eed3_isrForCrosstalkSources_997033_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4236 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/28217a73-f482-4f94-b5d6-7c160031eed3_isrForCrosstalkSources_997033_23.stderr parsl.dataflow.dflow INFO: Task 4252 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4682730096ab90eb5fd34da01e427ff9 with result from task 4252 parsl.dataflow.dflow DEBUG: Task 4252 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/8627908c-d843-474d-8e8e-8884d86d3176_isrForCrosstalkSources_997030_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4252 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/8627908c-d843-474d-8e8e-8884d86d3176_isrForCrosstalkSources_997030_21.stderr parsl.dataflow.dflow INFO: Task 4127 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd5cb134507183e1040e2cc310960847 with result from task 4127 parsl.dataflow.dflow DEBUG: Task 4127 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/05b1c6ad-3799-4dda-919e-e4612f55fa5d_isrForCrosstalkSources_997006_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4127 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/05b1c6ad-3799-4dda-919e-e4612f55fa5d_isrForCrosstalkSources_997006_18.stderr parsl.dataflow.dflow INFO: Task 4219 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1dc4bb633a6e409eb4d4acef1a56558b with result from task 4219 parsl.dataflow.dflow DEBUG: Task 4219 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/b39560b2-c54a-4aa6-a5c9-a1a552e915b6_isrForCrosstalkSources_996993_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4219 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/b39560b2-c54a-4aa6-a5c9-a1a552e915b6_isrForCrosstalkSources_996993_37.stderr parsl.dataflow.dflow INFO: Task 3902 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94983bd1f4e98face99ab0a15a6bd5d8 with result from task 3902 parsl.dataflow.dflow DEBUG: Task 3902 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2025360-472f-4900-8272-7d1577929b30_isrForCrosstalkSources_996996_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/f2025360-472f-4900-8272-7d1577929b30_isrForCrosstalkSources_996996_15.stderr parsl.dataflow.dflow INFO: Task 4079 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 08b66b2865ac2d840a7f3ec8f4fbde29 with result from task 4079 parsl.dataflow.dflow DEBUG: Task 4079 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/7ddf6eff-1278-4a7c-9ac6-ef7f2b2013e6_isrForCrosstalkSources_996970_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/7ddf6eff-1278-4a7c-9ac6-ef7f2b2013e6_isrForCrosstalkSources_996970_41.stderr parsl.dataflow.dflow INFO: Task 3765 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38917fce167c22f244f8a125cf74a5f0 with result from task 3765 parsl.dataflow.dflow DEBUG: Task 3765 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/bf3e4baf-bcb7-46fd-8fe2-85836409a792_isrForCrosstalkSources_997000_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/bf3e4baf-bcb7-46fd-8fe2-85836409a792_isrForCrosstalkSources_997000_3.stderr parsl.dataflow.dflow INFO: Task 3614 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 175e96bfc66f87dc17028dbaa1361dd9 with result from task 3614 parsl.dataflow.dflow DEBUG: Task 3614 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/36be853a-11a1-4ea9-b475-240def996622_isrForCrosstalkSources_996975_53.stdout parsl.dataflow.dflow INFO: Standard error for task 3614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/36be853a-11a1-4ea9-b475-240def996622_isrForCrosstalkSources_996975_53.stderr parsl.dataflow.dflow INFO: Task 4207 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 063c14af82c731875ec6e4752676279b with result from task 4207 parsl.dataflow.dflow DEBUG: Task 4207 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e84c3c63-c558-4217-8914-0533b95ad9e2_isrForCrosstalkSources_996970_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4207 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e84c3c63-c558-4217-8914-0533b95ad9e2_isrForCrosstalkSources_996970_15.stderr parsl.dataflow.dflow INFO: Task 3619 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d78245ed56e36f6b146c472f37ef5cd with result from task 3619 parsl.dataflow.dflow DEBUG: Task 3619 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b4c10945-e679-4d34-83ce-3f7f732971fe_isrForCrosstalkSources_996998_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b4c10945-e679-4d34-83ce-3f7f732971fe_isrForCrosstalkSources_996998_52.stderr parsl.dataflow.dflow INFO: Task 3598 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6db9dce2c9418d6ec0d27e7b889f73a0 with result from task 3598 parsl.dataflow.dflow DEBUG: Task 3598 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/11a9f932-4ea5-46f3-a0ef-594f4892237b_isrForCrosstalkSources_996963_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/11a9f932-4ea5-46f3-a0ef-594f4892237b_isrForCrosstalkSources_996963_52.stderr parsl.dataflow.dflow INFO: Task 4269 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae7fa99b748092269d337d3411d8bc13 with result from task 4269 parsl.dataflow.dflow DEBUG: Task 4269 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/96e834d1-1e7d-4ddf-9970-236eed987d4f_isrForCrosstalkSources_996970_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4269 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/96e834d1-1e7d-4ddf-9970-236eed987d4f_isrForCrosstalkSources_996970_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:25 proc_lsst.multi:146 [INFO] found job 17075190 in provider checkpoint proc_lsst.multi INFO: found job 17075190 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4071 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3310cfca9a90f4a14e10649a56920874 with result from task 4071 parsl.dataflow.dflow DEBUG: Task 4071 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bad13cbb-0c2c-4a50-b274-ffc7188b14fa_isrForCrosstalkSources_996960_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/bad13cbb-0c2c-4a50-b274-ffc7188b14fa_isrForCrosstalkSources_996960_36.stderr parsl.dataflow.dflow INFO: Task 4256 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ff483c4c0dfe10927a63048dc52afa5f with result from task 4256 parsl.dataflow.dflow DEBUG: Task 4256 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/99688dd0-5794-452b-8e0c-4671d60842a6_isrForCrosstalkSources_997013_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4256 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/99688dd0-5794-452b-8e0c-4671d60842a6_isrForCrosstalkSources_997013_39.stderr parsl.dataflow.dflow INFO: Task 3919 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aae9172ba3bba187497dfe767f5b47d0 with result from task 3919 parsl.dataflow.dflow DEBUG: Task 3919 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d303f3d6-012e-428e-83f9-6287eb90cf0e_isrForCrosstalkSources_997008_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/d303f3d6-012e-428e-83f9-6287eb90cf0e_isrForCrosstalkSources_997008_61.stderr parsl.dataflow.dflow INFO: Task 4245 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53557ac54657271588f8f55af6648937 with result from task 4245 parsl.dataflow.dflow DEBUG: Task 4245 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1264c6fe-2eef-4ee0-a7c5-dde25ea5f4a7_isrForCrosstalkSources_996957_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4245 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/1264c6fe-2eef-4ee0-a7c5-dde25ea5f4a7_isrForCrosstalkSources_996957_52.stderr parsl.dataflow.dflow INFO: Task 4205 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8170337b154278dc10ad252eb881a114 with result from task 4205 parsl.dataflow.dflow DEBUG: Task 4205 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/eab6b0f3-31d3-4b9b-b5d9-696a20c38db3_isrForCrosstalkSources_996976_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4205 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/eab6b0f3-31d3-4b9b-b5d9-696a20c38db3_isrForCrosstalkSources_996976_41.stderr parsl.dataflow.dflow INFO: Task 4234 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9c2a80ad8cd3b49855746cbcc413b1e with result from task 4234 parsl.dataflow.dflow DEBUG: Task 4234 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/ea015206-a2b1-496f-95d6-a8e229501fc1_isrForCrosstalkSources_996966_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4234 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/ea015206-a2b1-496f-95d6-a8e229501fc1_isrForCrosstalkSources_996966_49.stderr parsl.dataflow.dflow INFO: Task 4243 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15b4c8e94669ae26dd13ecd9c2d6c1fe with result from task 4243 parsl.dataflow.dflow DEBUG: Task 4243 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/48bb62a2-6d2f-4c1f-a6e2-83ab31958a8a_isrForCrosstalkSources_997010_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4243 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/48bb62a2-6d2f-4c1f-a6e2-83ab31958a8a_isrForCrosstalkSources_997010_38.stderr parsl.dataflow.dflow INFO: Task 4215 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d77d9062a6bc9cc830d6cadb315fc8c with result from task 4215 parsl.dataflow.dflow DEBUG: Task 4215 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/ccabb9bc-2720-4f8b-a1d2-05ebb8966e7a_isrForCrosstalkSources_997023_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4215 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/ccabb9bc-2720-4f8b-a1d2-05ebb8966e7a_isrForCrosstalkSources_997023_60.stderr parsl.dataflow.dflow INFO: Task 3841 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e4259350ccf8e27935e1788982608c4 with result from task 3841 parsl.dataflow.dflow DEBUG: Task 3841 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/93e9586b-9d77-4c19-8964-c3fc5d3600d9_isrForCrosstalkSources_997021_52.stdout parsl.dataflow.dflow INFO: Standard error for task 3841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/93e9586b-9d77-4c19-8964-c3fc5d3600d9_isrForCrosstalkSources_997021_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:26 proc_lsst.multi:146 [INFO] found job 17075191 in provider checkpoint proc_lsst.multi INFO: found job 17075191 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3899 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ea2569cde6707ad465818ce5e1bfc15 with result from task 3899 parsl.dataflow.dflow DEBUG: Task 3899 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/d7896fb8-cfc7-41a8-ab35-a03ebd2b55d5_isrForCrosstalkSources_997011_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/d7896fb8-cfc7-41a8-ab35-a03ebd2b55d5_isrForCrosstalkSources_997011_50.stderr parsl.dataflow.dflow INFO: Task 3839 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a61dac0cf5b210a456a56eb3a2ff6e6 with result from task 3839 parsl.dataflow.dflow DEBUG: Task 3839 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/11768f6c-dcdf-4c95-8eee-f1e943d17d12_isrForCrosstalkSources_997008_3.stdout parsl.dataflow.dflow INFO: Standard error for task 3839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/11768f6c-dcdf-4c95-8eee-f1e943d17d12_isrForCrosstalkSources_997008_3.stderr parsl.dataflow.dflow INFO: Task 3836 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d3b0dcd1edcafdcce020ab4705e70826 with result from task 3836 parsl.dataflow.dflow DEBUG: Task 3836 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/0886d232-6538-4a8e-b215-8c69d5390d43_isrForCrosstalkSources_996968_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/0886d232-6538-4a8e-b215-8c69d5390d43_isrForCrosstalkSources_996968_38.stderr parsl.dataflow.dflow INFO: Task 4137 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82e339374e5ca8221610b4f33a161a6c with result from task 4137 parsl.dataflow.dflow DEBUG: Task 4137 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/46c16a19-1ea9-407d-919f-fca018e0e555_isrForCrosstalkSources_996969_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4137 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/46c16a19-1ea9-407d-919f-fca018e0e555_isrForCrosstalkSources_996969_31.stderr parsl.dataflow.dflow INFO: Task 4065 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 46222b58e46d56846d8e04434366eb7e with result from task 4065 parsl.dataflow.dflow DEBUG: Task 4065 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/db947eb1-caf1-4f86-939a-082c850b6b27_isrForCrosstalkSources_996953_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/db947eb1-caf1-4f86-939a-082c850b6b27_isrForCrosstalkSources_996953_57.stderr parsl.dataflow.dflow INFO: Task 4047 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ca80818b6a5530cee5fac820a07a4dc with result from task 4047 parsl.dataflow.dflow DEBUG: Task 4047 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/9c3b144f-372c-4205-a765-1d7c43cfd6e2_isrForCrosstalkSources_996970_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/9c3b144f-372c-4205-a765-1d7c43cfd6e2_isrForCrosstalkSources_996970_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:26 proc_lsst.multi:146 [INFO] found job 17075197 in provider checkpoint proc_lsst.multi INFO: found job 17075197 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4121 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cff1bf60b1e013c232844c30573f2852 with result from task 4121 parsl.dataflow.dflow DEBUG: Task 4121 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/73217616-2029-4563-be0f-d6e91c1d3dca_isrForCrosstalkSources_997011_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4121 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/73217616-2029-4563-be0f-d6e91c1d3dca_isrForCrosstalkSources_997011_49.stderr parsl.dataflow.dflow INFO: Task 4096 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8889b563ad2808d64a0dbfecb2e15b70 with result from task 4096 parsl.dataflow.dflow DEBUG: Task 4096 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4f9103da-4728-4480-b5cf-def4ae47444a_isrForCrosstalkSources_996954_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4096 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/4f9103da-4728-4480-b5cf-def4ae47444a_isrForCrosstalkSources_996954_44.stderr parsl.dataflow.dflow INFO: Task 4033 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85784e9203d9cfaf0cae257f7b1fbc04 with result from task 4033 parsl.dataflow.dflow DEBUG: Task 4033 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2549b6c1-f1f9-436a-b8bd-5b9be9c64b5b_isrForCrosstalkSources_997003_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/2549b6c1-f1f9-436a-b8bd-5b9be9c64b5b_isrForCrosstalkSources_997003_31.stderr parsl.dataflow.dflow INFO: Task 4282 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5dc5a7f2c87ed72938fb0aa4f0b288d with result from task 4282 parsl.dataflow.dflow DEBUG: Task 4282 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/d06a860c-e816-4778-a4c3-0b7e43f030b7_isrForCrosstalkSources_996955_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4282 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/d06a860c-e816-4778-a4c3-0b7e43f030b7_isrForCrosstalkSources_996955_8.stderr parsl.dataflow.dflow INFO: Task 4273 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f60457ad69b6c214fad25fe49a134bf with result from task 4273 parsl.dataflow.dflow DEBUG: Task 4273 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/799e61bf-8a1c-4405-9070-fb3a5ea3cc33_isrForCrosstalkSources_997007_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4273 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/799e61bf-8a1c-4405-9070-fb3a5ea3cc33_isrForCrosstalkSources_997007_20.stderr parsl.dataflow.dflow INFO: Task 4224 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 163feeffdb5a2e1070cf4a94aa0a9973 with result from task 4224 parsl.dataflow.dflow DEBUG: Task 4224 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/201cd0a5-32e9-453b-8ca1-568613a74163_isrForCrosstalkSources_997004_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4224 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/201cd0a5-32e9-453b-8ca1-568613a74163_isrForCrosstalkSources_997004_60.stderr parsl.dataflow.dflow INFO: Task 4046 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26ab9dfad4a51900c8d6edc22c194253 with result from task 4046 parsl.dataflow.dflow DEBUG: Task 4046 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/de7d3066-6ac1-4a4b-9bfe-1f7d5c29949c_isrForCrosstalkSources_997013_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/de7d3066-6ac1-4a4b-9bfe-1f7d5c29949c_isrForCrosstalkSources_997013_23.stderr parsl.dataflow.dflow INFO: Task 3957 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eff353b0c125e1610a2eb67dab5c24b1 with result from task 3957 parsl.dataflow.dflow DEBUG: Task 3957 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/5a3e8b94-747a-4d82-9dcb-98fc68aa78b1_isrForCrosstalkSources_997001_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/5a3e8b94-747a-4d82-9dcb-98fc68aa78b1_isrForCrosstalkSources_997001_36.stderr parsl.dataflow.dflow INFO: Task 3892 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d05a63d179adfef1a1a27e430df62e1d with result from task 3892 parsl.dataflow.dflow DEBUG: Task 3892 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4b4c13b1-a5e5-450a-aef1-4f243392bab6_isrForCrosstalkSources_996997_31.stdout parsl.dataflow.dflow INFO: Standard error for task 3892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/4b4c13b1-a5e5-450a-aef1-4f243392bab6_isrForCrosstalkSources_996997_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:27 proc_lsst.multi:146 [INFO] found job 17075200 in provider checkpoint proc_lsst.multi INFO: found job 17075200 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3964 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d66200386d5cfe038111ec732da5958 with result from task 3964 parsl.dataflow.dflow DEBUG: Task 3964 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/cb38b111-ce66-4dfa-a058-8ff0bfb274fa_isrForCrosstalkSources_996989_30.stdout parsl.dataflow.dflow INFO: Standard error for task 3964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/cb38b111-ce66-4dfa-a058-8ff0bfb274fa_isrForCrosstalkSources_996989_30.stderr parsl.dataflow.dflow INFO: Task 4267 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d5b089b7e67c5bd58945dcf81ace5fc0 with result from task 4267 parsl.dataflow.dflow DEBUG: Task 4267 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a261f3fe-33cb-43e3-979f-e09db6d7cd27_isrForCrosstalkSources_996964_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4267 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/a261f3fe-33cb-43e3-979f-e09db6d7cd27_isrForCrosstalkSources_996964_12.stderr parsl.dataflow.dflow INFO: Task 4128 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4acaf04d63048f70c944d432806ab47d with result from task 4128 parsl.dataflow.dflow DEBUG: Task 4128 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/3ecbd8a5-15d8-4c0f-9cca-6cc2d59108a8_isrForCrosstalkSources_997002_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4128 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/3ecbd8a5-15d8-4c0f-9cca-6cc2d59108a8_isrForCrosstalkSources_997002_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:27 proc_lsst.multi:146 [INFO] found job 17075201 in provider checkpoint proc_lsst.multi INFO: found job 17075201 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4086 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15ef7fabf5bbfb837dd652685fa5ac70 with result from task 4086 parsl.dataflow.dflow DEBUG: Task 4086 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/d910ca4d-a629-4d38-adb8-0aa9fbd9fe3a_isrForCrosstalkSources_997015_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4086 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/d910ca4d-a629-4d38-adb8-0aa9fbd9fe3a_isrForCrosstalkSources_997015_53.stderr parsl.dataflow.dflow INFO: Task 4266 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fda24defd9e9f55e4ba04a3321a323f5 with result from task 4266 parsl.dataflow.dflow DEBUG: Task 4266 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/41f95b41-bbe9-412e-bc8d-a4c0360ce2df_isrForCrosstalkSources_997033_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4266 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/41f95b41-bbe9-412e-bc8d-a4c0360ce2df_isrForCrosstalkSources_997033_53.stderr parsl.dataflow.dflow INFO: Task 3961 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 263b0ede77ee98883e83b6cb8ecc136b with result from task 3961 parsl.dataflow.dflow DEBUG: Task 3961 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/30519971-2f04-4776-a2c8-15d7d400a935_isrForCrosstalkSources_996967_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/30519971-2f04-4776-a2c8-15d7d400a935_isrForCrosstalkSources_996967_28.stderr parsl.dataflow.dflow INFO: Task 3968 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c33e2ae27b6f2a5565f5231adab906cc with result from task 3968 parsl.dataflow.dflow DEBUG: Task 3968 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8e81859a-e401-4ca0-bda8-ee7fa61e7fe4_isrForCrosstalkSources_996963_55.stdout parsl.dataflow.dflow INFO: Standard error for task 3968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/8e81859a-e401-4ca0-bda8-ee7fa61e7fe4_isrForCrosstalkSources_996963_55.stderr parsl.dataflow.dflow INFO: Task 3969 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 35aab918d30db6b32b87fa48f0b79267 with result from task 3969 parsl.dataflow.dflow DEBUG: Task 3969 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/771755d8-c768-47e5-9c8f-6acfc850a5cf_isrForCrosstalkSources_996999_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/771755d8-c768-47e5-9c8f-6acfc850a5cf_isrForCrosstalkSources_996999_58.stderr parsl.dataflow.dflow INFO: Task 4101 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6bb51c057d1ecfdc3d78b9e005f51372 with result from task 4101 parsl.dataflow.dflow DEBUG: Task 4101 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/fd11f259-48e9-41ea-b00c-3aae4708d8e0_isrForCrosstalkSources_996979_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4101 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/fd11f259-48e9-41ea-b00c-3aae4708d8e0_isrForCrosstalkSources_996979_15.stderr parsl.dataflow.dflow INFO: Task 4097 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c44a0a8ee057fc37893d98487bbf60b7 with result from task 4097 parsl.dataflow.dflow DEBUG: Task 4097 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0f41d76f-7fe9-498b-a1f1-0682bcf99c03_isrForCrosstalkSources_997013_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4097 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0f41d76f-7fe9-498b-a1f1-0682bcf99c03_isrForCrosstalkSources_997013_17.stderr parsl.dataflow.dflow INFO: Task 4089 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c25acac3c462f7aebc12b6e6078c81de with result from task 4089 parsl.dataflow.dflow DEBUG: Task 4089 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/2242a595-3027-40da-af6e-530cef73615f_isrForCrosstalkSources_997021_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4089 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/2242a595-3027-40da-af6e-530cef73615f_isrForCrosstalkSources_997021_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:28 proc_lsst.multi:146 [INFO] found job 17075202 in provider checkpoint proc_lsst.multi INFO: found job 17075202 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4116 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3d5529cf4ad38b212ffc253dc380ef87 with result from task 4116 parsl.dataflow.dflow DEBUG: Task 4116 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/6a54ff00-6b18-41c8-abd7-8c0028d2da73_isrForCrosstalkSources_996986_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4116 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/6a54ff00-6b18-41c8-abd7-8c0028d2da73_isrForCrosstalkSources_996986_59.stderr parsl.dataflow.dflow INFO: Task 4271 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b6c005683595b7aaa6d2b4805ed4f30 with result from task 4271 parsl.dataflow.dflow DEBUG: Task 4271 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/367761cd-8464-4cdc-b993-5da0987f0583_isrForCrosstalkSources_996967_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4271 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/367761cd-8464-4cdc-b993-5da0987f0583_isrForCrosstalkSources_996967_52.stderr parsl.dataflow.dflow INFO: Task 4032 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9dd10217c3e2c71db3e173d6c6d1bd6b with result from task 4032 parsl.dataflow.dflow DEBUG: Task 4032 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/1b481612-8b66-44b1-9b9f-020824dfc101_isrForCrosstalkSources_997012_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/1b481612-8b66-44b1-9b9f-020824dfc101_isrForCrosstalkSources_997012_23.stderr parsl.dataflow.dflow INFO: Task 4031 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c626a4628d474226823572538a73a923 with result from task 4031 parsl.dataflow.dflow DEBUG: Task 4031 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/34e01d0c-37be-455a-bcd4-e16d7ba53baf_isrForCrosstalkSources_996988_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/34e01d0c-37be-455a-bcd4-e16d7ba53baf_isrForCrosstalkSources_996988_14.stderr parsl.dataflow.dflow INFO: Task 4088 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a420eb001da1cc43434e80966d6f1fac with result from task 4088 parsl.dataflow.dflow DEBUG: Task 4088 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f1004d21-25f8-4ce6-bffa-84d0b19380c3_isrForCrosstalkSources_996989_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4088 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f1004d21-25f8-4ce6-bffa-84d0b19380c3_isrForCrosstalkSources_996989_27.stderr parsl.dataflow.dflow INFO: Task 4129 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5b512c2ef5fc7fa46f5d612d27a5cd5 with result from task 4129 parsl.dataflow.dflow DEBUG: Task 4129 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/cd22cccb-b455-404a-92ac-3234361266ae_isrForCrosstalkSources_996959_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4129 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/cd22cccb-b455-404a-92ac-3234361266ae_isrForCrosstalkSources_996959_24.stderr parsl.dataflow.dflow INFO: Task 4068 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8523e2605dea846aa53ed946fb539c43 with result from task 4068 parsl.dataflow.dflow DEBUG: Task 4068 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ad7915a6-7cdc-4266-a387-6c044c195442_isrForCrosstalkSources_996962_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/ad7915a6-7cdc-4266-a387-6c044c195442_isrForCrosstalkSources_996962_53.stderr parsl.dataflow.dflow INFO: Task 4093 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3680740c3af364d7337bdbff9bb78106 with result from task 4093 parsl.dataflow.dflow DEBUG: Task 4093 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/2d8579b6-5036-41ec-9cfc-4e94bb0a58dd_isrForCrosstalkSources_996957_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4093 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/2d8579b6-5036-41ec-9cfc-4e94bb0a58dd_isrForCrosstalkSources_996957_58.stderr parsl.dataflow.dflow INFO: Task 4135 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 996b353404ec61630a4b9a156f6d2e04 with result from task 4135 parsl.dataflow.dflow DEBUG: Task 4135 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6bbd4c91-bcf9-4abe-ac27-b8515dc2933c_isrForCrosstalkSources_996952_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4135 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/6bbd4c91-bcf9-4abe-ac27-b8515dc2933c_isrForCrosstalkSources_996952_46.stderr parsl.dataflow.dflow INFO: Task 4163 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bcdaecea48c4efeb630a38e097d098f2 with result from task 4163 parsl.dataflow.dflow DEBUG: Task 4163 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1ebc5081-d79d-448c-a81a-afdba4b6c588_isrForCrosstalkSources_996964_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4163 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/1ebc5081-d79d-448c-a81a-afdba4b6c588_isrForCrosstalkSources_996964_10.stderr parsl.dataflow.dflow INFO: Task 4020 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e50d1c54e747309cfe64f2ffc7c7fba5 with result from task 4020 parsl.dataflow.dflow DEBUG: Task 4020 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/83cd4117-1b9d-4054-81f0-947b13556397_isrForCrosstalkSources_996964_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/83cd4117-1b9d-4054-81f0-947b13556397_isrForCrosstalkSources_996964_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:29 proc_lsst.multi:146 [INFO] found job 17075203 in provider checkpoint proc_lsst.multi INFO: found job 17075203 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4082 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c98392077edbcb6295b3767c67e0c6b8 with result from task 4082 parsl.dataflow.dflow DEBUG: Task 4082 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2b554c3b-2571-418f-ad52-70a986195b82_isrForCrosstalkSources_996989_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2b554c3b-2571-418f-ad52-70a986195b82_isrForCrosstalkSources_996989_6.stderr parsl.dataflow.dflow INFO: Task 4034 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a585474cc07ba9fa2c707767e5d920ad with result from task 4034 parsl.dataflow.dflow DEBUG: Task 4034 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/900ffbcf-aaf6-4781-9b75-47ec2a882a93_isrForCrosstalkSources_996980_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/900ffbcf-aaf6-4781-9b75-47ec2a882a93_isrForCrosstalkSources_996980_44.stderr parsl.dataflow.dflow INFO: Task 4104 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 454a96ebb935118411045a885e7b9a48 with result from task 4104 parsl.dataflow.dflow DEBUG: Task 4104 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/5c6c9dd1-118a-4207-9a1a-c1a57f14d82b_isrForCrosstalkSources_997028_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4104 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/5c6c9dd1-118a-4207-9a1a-c1a57f14d82b_isrForCrosstalkSources_997028_16.stderr parsl.dataflow.dflow INFO: Task 4043 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1143f4fa636d82bab88fa19c6d32d99b with result from task 4043 parsl.dataflow.dflow DEBUG: Task 4043 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/5ca1d33b-36b2-4c40-a98f-bed0499b0505_isrForCrosstalkSources_996994_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/5ca1d33b-36b2-4c40-a98f-bed0499b0505_isrForCrosstalkSources_996994_5.stderr parsl.dataflow.dflow INFO: Task 3911 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b2e8b6c9ce99f9dfd9059be2e49682d with result from task 3911 parsl.dataflow.dflow DEBUG: Task 3911 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/6af6e820-fd6f-4564-83f9-10fb6926b859_isrForCrosstalkSources_996981_12.stdout parsl.dataflow.dflow INFO: Standard error for task 3911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/6af6e820-fd6f-4564-83f9-10fb6926b859_isrForCrosstalkSources_996981_12.stderr parsl.dataflow.dflow INFO: Task 3931 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b05cfbc29dc429564de9ff6257d2fd2d with result from task 3931 parsl.dataflow.dflow DEBUG: Task 3931 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/59c43751-9d6f-4e36-b133-2b998463f261_isrForCrosstalkSources_996984_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/59c43751-9d6f-4e36-b133-2b998463f261_isrForCrosstalkSources_996984_57.stderr parsl.dataflow.dflow INFO: Task 4193 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0abc7cb9f118d6a144af3418ddcb0e6b with result from task 4193 parsl.dataflow.dflow DEBUG: Task 4193 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/94f6be8e-2c65-4c42-861b-69808fcc8856_isrForCrosstalkSources_996980_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4193 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/94f6be8e-2c65-4c42-861b-69808fcc8856_isrForCrosstalkSources_996980_27.stderr parsl.dataflow.dflow INFO: Task 4278 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4de27e1de8c50c253dad2f631837b3a with result from task 4278 parsl.dataflow.dflow DEBUG: Task 4278 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/c239d8bf-f7b1-466a-9cac-9613f68f3909_isrForCrosstalkSources_997008_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4278 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/c239d8bf-f7b1-466a-9cac-9613f68f3909_isrForCrosstalkSources_997008_11.stderr parsl.dataflow.dflow INFO: Task 4154 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9f46fef031e37772c075aafc0cc1fd9 with result from task 4154 parsl.dataflow.dflow DEBUG: Task 4154 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/93d35b35-83ff-4e99-a97c-c62a8097f4c8_isrForCrosstalkSources_996998_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4154 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/93d35b35-83ff-4e99-a97c-c62a8097f4c8_isrForCrosstalkSources_996998_22.stderr parsl.dataflow.dflow INFO: Task 4136 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e98c8410fa94e019df273f3fa0c16407 with result from task 4136 parsl.dataflow.dflow DEBUG: Task 4136 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/229f64af-8b93-4e8b-acd4-8a81e10e74e9_isrForCrosstalkSources_996987_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4136 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/229f64af-8b93-4e8b-acd4-8a81e10e74e9_isrForCrosstalkSources_996987_21.stderr parsl.dataflow.dflow INFO: Task 4142 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e080372eda0519595855048b5f606fed with result from task 4142 parsl.dataflow.dflow DEBUG: Task 4142 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4ae80b42-1e06-4f43-a4f7-7c8fbb5aef41_isrForCrosstalkSources_997021_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4142 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/4ae80b42-1e06-4f43-a4f7-7c8fbb5aef41_isrForCrosstalkSources_997021_18.stderr parsl.dataflow.dflow INFO: Task 4166 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 701fe1bf71f647d9d73925f2dd3d2f9e with result from task 4166 parsl.dataflow.dflow DEBUG: Task 4166 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/15fa2398-ac49-4c1f-beee-9bbd15dc0900_isrForCrosstalkSources_997009_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4166 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/15fa2398-ac49-4c1f-beee-9bbd15dc0900_isrForCrosstalkSources_997009_21.stderr parsl.dataflow.dflow INFO: Task 4081 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4aeab56b2e1ed904b93f4f7fd879e567 with result from task 4081 parsl.dataflow.dflow DEBUG: Task 4081 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/9e99e92c-05d1-47ed-baad-7ac666d42868_isrForCrosstalkSources_997024_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/9e99e92c-05d1-47ed-baad-7ac666d42868_isrForCrosstalkSources_997024_35.stderr parsl.dataflow.dflow INFO: Task 4016 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b01122924d65f64a337a679af117b27 with result from task 4016 parsl.dataflow.dflow DEBUG: Task 4016 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5a7da13c-d0a4-42f6-8b0b-f83c4ee24e82_isrForCrosstalkSources_997018_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5a7da13c-d0a4-42f6-8b0b-f83c4ee24e82_isrForCrosstalkSources_997018_42.stderr parsl.dataflow.dflow INFO: Task 4140 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eda9bfb1c7f14e1dcc0a64ceeb526f5d with result from task 4140 parsl.dataflow.dflow DEBUG: Task 4140 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b969646b-a8d7-4bd4-8be8-ad29ddd80a97_isrForCrosstalkSources_996979_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4140 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/b969646b-a8d7-4bd4-8be8-ad29ddd80a97_isrForCrosstalkSources_996979_54.stderr parsl.dataflow.dflow INFO: Task 4288 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db3925ac404ba1c8166074368fd96401 with result from task 4288 parsl.dataflow.dflow DEBUG: Task 4288 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/6bde540a-5896-40ef-bbe9-4ee70a2ce528_isrForCrosstalkSources_996992_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4288 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/6bde540a-5896-40ef-bbe9-4ee70a2ce528_isrForCrosstalkSources_996992_21.stderr parsl.dataflow.dflow INFO: Task 4285 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dec0628b4a6adbdea6ba64d76ef053f0 with result from task 4285 parsl.dataflow.dflow DEBUG: Task 4285 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/102a8930-afa3-4714-beea-5245e916340f_isrForCrosstalkSources_996995_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4285 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/102a8930-afa3-4714-beea-5245e916340f_isrForCrosstalkSources_996995_58.stderr parsl.dataflow.dflow INFO: Task 3945 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7060687ccf021dbf0c72424d1ac81bcd with result from task 3945 parsl.dataflow.dflow DEBUG: Task 3945 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b1d3c471-1328-4dba-97c7-e92d653ff687_isrForCrosstalkSources_997004_50.stdout parsl.dataflow.dflow INFO: Standard error for task 3945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b1d3c471-1328-4dba-97c7-e92d653ff687_isrForCrosstalkSources_997004_50.stderr parsl.dataflow.dflow INFO: Task 4160 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 636d624dde0a992e932a4ac66af4c7f4 with result from task 4160 parsl.dataflow.dflow DEBUG: Task 4160 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/950b1801-fa98-4751-9de6-748af10f2f2d_isrForCrosstalkSources_997021_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4160 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/950b1801-fa98-4751-9de6-748af10f2f2d_isrForCrosstalkSources_997021_47.stderr parsl.dataflow.dflow INFO: Task 4023 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 05ea44a20fadfb2316fcbc355b73e903 with result from task 4023 parsl.dataflow.dflow DEBUG: Task 4023 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a06daf9-ec8a-4473-986e-731bdd7a42a7_isrForCrosstalkSources_996994_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/8a06daf9-ec8a-4473-986e-731bdd7a42a7_isrForCrosstalkSources_996994_3.stderr parsl.dataflow.dflow INFO: Task 4006 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93773d002c459ad07ee38e57a7a8406d with result from task 4006 parsl.dataflow.dflow DEBUG: Task 4006 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/700823b4-21ab-44cb-b37b-e34a0614bf8e_isrForCrosstalkSources_996976_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/700823b4-21ab-44cb-b37b-e34a0614bf8e_isrForCrosstalkSources_996976_38.stderr parsl.dataflow.dflow INFO: Task 4144 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 12bd9523caceb1c51fbb2aa0a741d9ca with result from task 4144 parsl.dataflow.dflow DEBUG: Task 4144 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/9acb5344-5013-44f2-95be-8288833bc1be_isrForCrosstalkSources_997026_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4144 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/9acb5344-5013-44f2-95be-8288833bc1be_isrForCrosstalkSources_997026_26.stderr parsl.dataflow.dflow INFO: Task 4182 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07cdbc1bfdd3d534243b7e08e20b4373 with result from task 4182 parsl.dataflow.dflow DEBUG: Task 4182 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/8ae844d7-37db-49c8-8b59-e1d230021bee_isrForCrosstalkSources_997023_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4182 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/8ae844d7-37db-49c8-8b59-e1d230021bee_isrForCrosstalkSources_997023_22.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:30 proc_lsst.multi:146 [INFO] found job 17075204 in provider checkpoint proc_lsst.multi INFO: found job 17075204 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4024 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 22f06e5dc0eac090022abf6d934816fe with result from task 4024 parsl.dataflow.dflow DEBUG: Task 4024 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/bb523ac4-df53-44d5-8326-dd5221423faa_isrForCrosstalkSources_996983_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/bb523ac4-df53-44d5-8326-dd5221423faa_isrForCrosstalkSources_996983_54.stderr parsl.dataflow.dflow INFO: Task 4132 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 440b5fbe3ae3eebb4e941c3b6d892064 with result from task 4132 parsl.dataflow.dflow DEBUG: Task 4132 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2e628b3c-7ae3-410c-a64d-c33b99f460cd_isrForCrosstalkSources_996970_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4132 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2e628b3c-7ae3-410c-a64d-c33b99f460cd_isrForCrosstalkSources_996970_28.stderr parsl.dataflow.dflow INFO: Task 4060 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b7ba401a07cf91027a026e346d99bb5 with result from task 4060 parsl.dataflow.dflow DEBUG: Task 4060 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/23b8fbac-ef02-417b-95fb-6a8a9853db92_isrForCrosstalkSources_996959_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/23b8fbac-ef02-417b-95fb-6a8a9853db92_isrForCrosstalkSources_996959_26.stderr parsl.dataflow.dflow INFO: Task 4296 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 899e5400af82135e4aeef7c99c3e42d2 with result from task 4296 parsl.dataflow.dflow DEBUG: Task 4296 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/31adb687-30d6-48f0-bfd8-d18a506ea554_isrForCrosstalkSources_997024_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4296 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/31adb687-30d6-48f0-bfd8-d18a506ea554_isrForCrosstalkSources_997024_26.stderr parsl.dataflow.dflow INFO: Task 4237 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fc05727e97c769d3f20f092cab8f93c with result from task 4237 parsl.dataflow.dflow DEBUG: Task 4237 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/81af871e-2e6c-488b-937d-a9e4a237a70c_isrForCrosstalkSources_996975_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4237 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/81af871e-2e6c-488b-937d-a9e4a237a70c_isrForCrosstalkSources_996975_20.stderr parsl.dataflow.dflow INFO: Task 4143 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 677cd5421f0ed8e0b74dbe0e8f74bed1 with result from task 4143 parsl.dataflow.dflow DEBUG: Task 4143 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ce5ec4db-1076-41ec-92ad-c61630bcd9da_isrForCrosstalkSources_996969_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4143 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ce5ec4db-1076-41ec-92ad-c61630bcd9da_isrForCrosstalkSources_996969_18.stderr parsl.dataflow.dflow INFO: Task 4279 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6c1312adeca1548b1d714f1fa9d32c9 with result from task 4279 parsl.dataflow.dflow DEBUG: Task 4279 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/74a9e3d4-d0ad-449d-bd70-6efad244c794_isrForCrosstalkSources_996985_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4279 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/74a9e3d4-d0ad-449d-bd70-6efad244c794_isrForCrosstalkSources_996985_31.stderr parsl.dataflow.dflow INFO: Task 4050 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 662dfaf5c187a6a84d9bd57f4defaffe with result from task 4050 parsl.dataflow.dflow DEBUG: Task 4050 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/dc8290e3-74f3-48ad-a197-df467499a1a7_isrForCrosstalkSources_996965_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/dc8290e3-74f3-48ad-a197-df467499a1a7_isrForCrosstalkSources_996965_25.stderr parsl.dataflow.dflow INFO: Task 4284 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce3bd09cd975f354adf6ecb8fa86004c with result from task 4284 parsl.dataflow.dflow DEBUG: Task 4284 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/486ac73b-8f24-4a9f-aaa8-4fa8888897aa_isrForCrosstalkSources_996995_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4284 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/486ac73b-8f24-4a9f-aaa8-4fa8888897aa_isrForCrosstalkSources_996995_42.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:31 proc_lsst.multi:146 [INFO] found job 17075205 in provider checkpoint proc_lsst.multi INFO: found job 17075205 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4109 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70e46b4728c2812d409481215fd77369 with result from task 4109 parsl.dataflow.dflow DEBUG: Task 4109 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a49cf293-ca5e-4a7a-9244-23710d1414a5_isrForCrosstalkSources_996984_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4109 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/a49cf293-ca5e-4a7a-9244-23710d1414a5_isrForCrosstalkSources_996984_42.stderr parsl.dataflow.dflow INFO: Task 4059 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e1a4e0e0c5781aff89ee7cc426a2703d with result from task 4059 parsl.dataflow.dflow DEBUG: Task 4059 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/34b12def-fa76-42fd-88b5-a59953be4f7d_isrForCrosstalkSources_997021_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/34b12def-fa76-42fd-88b5-a59953be4f7d_isrForCrosstalkSources_997021_15.stderr parsl.dataflow.dflow INFO: Task 4173 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 458df32076ef2c1331df08be4d4e5f63 with result from task 4173 parsl.dataflow.dflow DEBUG: Task 4173 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/11eb4d01-d653-460c-b9ad-a16526e6b9ad_isrForCrosstalkSources_996997_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4173 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/11eb4d01-d653-460c-b9ad-a16526e6b9ad_isrForCrosstalkSources_996997_32.stderr parsl.dataflow.dflow INFO: Task 4201 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d25c6dac8d35d36f896554e249f0caf3 with result from task 4201 parsl.dataflow.dflow DEBUG: Task 4201 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/adea6b4f-f785-4b03-9ded-cf415524126f_isrForCrosstalkSources_997007_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4201 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/adea6b4f-f785-4b03-9ded-cf415524126f_isrForCrosstalkSources_997007_13.stderr parsl.dataflow.dflow INFO: Task 4174 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9141211f8fce8fad1b1da80f47254b9 with result from task 4174 parsl.dataflow.dflow DEBUG: Task 4174 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/697a0adc-f57b-4405-9507-4569a572c9e9_isrForCrosstalkSources_997001_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4174 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/697a0adc-f57b-4405-9507-4569a572c9e9_isrForCrosstalkSources_997001_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:31 proc_lsst.multi:146 [INFO] found job 17075206 in provider checkpoint proc_lsst.multi INFO: found job 17075206 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4209 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d853b94a94362011ed94c77651c7694 with result from task 4209 parsl.dataflow.dflow DEBUG: Task 4209 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/344cb7ff-6064-4722-bc6d-7f1c935431bd_isrForCrosstalkSources_996955_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4209 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/344cb7ff-6064-4722-bc6d-7f1c935431bd_isrForCrosstalkSources_996955_34.stderr parsl.dataflow.dflow INFO: Task 4172 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c412d6dc9402c7951fbfa21ddc405094 with result from task 4172 parsl.dataflow.dflow DEBUG: Task 4172 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/fc58541b-8063-4872-b290-94edd79ca642_isrForCrosstalkSources_997009_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4172 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/fc58541b-8063-4872-b290-94edd79ca642_isrForCrosstalkSources_997009_15.stderr parsl.dataflow.dflow INFO: Task 4112 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7afe6279b87e2ba493654ac83c21f554 with result from task 4112 parsl.dataflow.dflow DEBUG: Task 4112 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/49c6eac4-2169-4fd6-a226-da8c832169f9_isrForCrosstalkSources_997024_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4112 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/49c6eac4-2169-4fd6-a226-da8c832169f9_isrForCrosstalkSources_997024_39.stderr parsl.dataflow.dflow INFO: Task 4131 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e07ee9c5e793d69969bd30f3ecd8f702 with result from task 4131 parsl.dataflow.dflow DEBUG: Task 4131 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/630a00ea-cd76-42f0-92e1-c313d10fd19b_isrForCrosstalkSources_997028_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4131 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/630a00ea-cd76-42f0-92e1-c313d10fd19b_isrForCrosstalkSources_997028_20.stderr parsl.dataflow.dflow INFO: Task 4001 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f09405b0453f19f77eb7b6687323f868 with result from task 4001 parsl.dataflow.dflow DEBUG: Task 4001 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/64fda880-f143-4025-85be-6cdf594be4a7_isrForCrosstalkSources_996974_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/64fda880-f143-4025-85be-6cdf594be4a7_isrForCrosstalkSources_996974_27.stderr parsl.dataflow.dflow INFO: Task 3942 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 17a5011689c7b25447ca84ee3fb46ac9 with result from task 3942 parsl.dataflow.dflow DEBUG: Task 3942 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/65c5f76b-f7cb-456e-ba08-1abfde7938ce_isrForCrosstalkSources_996952_36.stdout parsl.dataflow.dflow INFO: Standard error for task 3942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/65c5f76b-f7cb-456e-ba08-1abfde7938ce_isrForCrosstalkSources_996952_36.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:32 proc_lsst.multi:146 [INFO] found job 17075207 in provider checkpoint proc_lsst.multi INFO: found job 17075207 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4175 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7032366b7359504ac634f34dd8a76614 with result from task 4175 parsl.dataflow.dflow DEBUG: Task 4175 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/cd319707-7a2b-495c-8736-4f528ab81f1a_isrForCrosstalkSources_996990_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4175 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/cd319707-7a2b-495c-8736-4f528ab81f1a_isrForCrosstalkSources_996990_20.stderr parsl.dataflow.dflow INFO: Task 4039 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a0386efd37f1d2963e859c3ee65014e with result from task 4039 parsl.dataflow.dflow DEBUG: Task 4039 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d438732d-de89-4707-b813-759f97022e85_isrForCrosstalkSources_997024_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/d438732d-de89-4707-b813-759f97022e85_isrForCrosstalkSources_997024_54.stderr parsl.dataflow.dflow INFO: Task 4110 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 534daafa5ad567e45262bba0f464acce with result from task 4110 parsl.dataflow.dflow DEBUG: Task 4110 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f3eefc6c-b517-4d73-aab9-a54af69cfcdc_isrForCrosstalkSources_997018_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4110 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f3eefc6c-b517-4d73-aab9-a54af69cfcdc_isrForCrosstalkSources_997018_48.stderr parsl.dataflow.dflow INFO: Task 4162 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e5c01f224003a911056870216bd0d934 with result from task 4162 parsl.dataflow.dflow DEBUG: Task 4162 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/aadee31f-5e25-4038-bffc-65e18cb5e7ff_isrForCrosstalkSources_996957_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4162 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/aadee31f-5e25-4038-bffc-65e18cb5e7ff_isrForCrosstalkSources_996957_32.stderr parsl.dataflow.dflow INFO: Task 4044 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84d7c44e56ccb9465d03e969cde334e3 with result from task 4044 parsl.dataflow.dflow DEBUG: Task 4044 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/984e59a4-3b51-42b1-a091-cf3028f65170_isrForCrosstalkSources_996998_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/984e59a4-3b51-42b1-a091-cf3028f65170_isrForCrosstalkSources_996998_45.stderr parsl.dataflow.dflow INFO: Task 4318 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63e5f6c04308ee940c577591806c3a18 with result from task 4318 parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4318 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/80bd226e-13de-420a-b9da-f99a4c35dce1_isrForCrosstalkSources_996952_17.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4318 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/80bd226e-13de-420a-b9da-f99a4c35dce1_isrForCrosstalkSources_996952_17.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:32 proc_lsst.multi:146 [INFO] found job 17075208 in provider checkpoint proc_lsst.multi INFO: found job 17075208 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4177 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1e771711cd4043482b3efe54d961b923 with result from task 4177 parsl.dataflow.dflow DEBUG: Task 4177 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4a1c33eb-f2f4-4210-9677-6edbe1b0d9ac_isrForCrosstalkSources_996959_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4177 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/4a1c33eb-f2f4-4210-9677-6edbe1b0d9ac_isrForCrosstalkSources_996959_46.stderr parsl.dataflow.dflow INFO: Task 4194 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04f690b83da5c977ca75ea103f44c32c with result from task 4194 parsl.dataflow.dflow DEBUG: Task 4194 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b16f4e70-4090-4b03-8162-1d681f041250_isrForCrosstalkSources_997031_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4194 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b16f4e70-4090-4b03-8162-1d681f041250_isrForCrosstalkSources_997031_17.stderr parsl.dataflow.dflow INFO: Task 4115 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e5d28b7a2a25d115863996e9e003023 with result from task 4115 parsl.dataflow.dflow DEBUG: Task 4115 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/2cddb7f2-36d7-406c-bf19-b6562f46655b_isrForCrosstalkSources_997019_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4115 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/2cddb7f2-36d7-406c-bf19-b6562f46655b_isrForCrosstalkSources_997019_9.stderr parsl.dataflow.dflow INFO: Task 4299 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d6d7ed399fc8b2c6180d204aad3f912a with result from task 4299 parsl.dataflow.dflow DEBUG: Task 4299 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1dc5f539-8158-4a90-8a7d-7675c7191867_isrForCrosstalkSources_997032_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4299 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1dc5f539-8158-4a90-8a7d-7675c7191867_isrForCrosstalkSources_997032_4.stderr parsl.dataflow.dflow INFO: Task 4118 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b2020b95be520557e6d9d09c5be015cb with result from task 4118 parsl.dataflow.dflow DEBUG: Task 4118 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1f2a995c-ddc4-4d95-b38c-273b65887c27_isrForCrosstalkSources_997032_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4118 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1f2a995c-ddc4-4d95-b38c-273b65887c27_isrForCrosstalkSources_997032_14.stderr parsl.dataflow.dflow INFO: Task 4263 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9911e1efd231b6b05bb3cc2f56ac8e15 with result from task 4263 parsl.dataflow.dflow DEBUG: Task 4263 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2bac6407-c81b-4161-94aa-5679e4756140_isrForCrosstalkSources_996977_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4263 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2bac6407-c81b-4161-94aa-5679e4756140_isrForCrosstalkSources_996977_29.stderr parsl.dataflow.dflow INFO: Task 4216 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c9476d09b918b1ac9edc411c79e9fe6 with result from task 4216 parsl.dataflow.dflow DEBUG: Task 4216 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e426e024-7f79-48e2-9e5b-3c06344323c0_isrForCrosstalkSources_997011_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4216 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/e426e024-7f79-48e2-9e5b-3c06344323c0_isrForCrosstalkSources_997011_14.stderr parsl.dataflow.dflow INFO: Task 4319 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9530657eb9b576ca21fddc598db9e81e with result from task 4319 parsl.dataflow.dflow DEBUG: Task 4319 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d4433bb5-b5ff-4e93-8c1f-39a3f569788f_isrForCrosstalkSources_997000_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4319 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/d4433bb5-b5ff-4e93-8c1f-39a3f569788f_isrForCrosstalkSources_997000_14.stderr parsl.dataflow.dflow INFO: Task 3941 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ee212a61b1918520dcdd3fbf7bb69b3 with result from task 3941 parsl.dataflow.dflow DEBUG: Task 3941 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1d4c218b-69db-4ca8-b383-7485fe4b7f0d_isrForCrosstalkSources_996967_38.stdout parsl.dataflow.dflow INFO: Standard error for task 3941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1d4c218b-69db-4ca8-b383-7485fe4b7f0d_isrForCrosstalkSources_996967_38.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:33 proc_lsst.multi:146 [INFO] found job 17075209 in provider checkpoint proc_lsst.multi INFO: found job 17075209 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4107 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d4e9d0aba76b99905df4e269063ae38 with result from task 4107 parsl.dataflow.dflow DEBUG: Task 4107 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7f335333-a87a-4a8a-aab7-e197ee221418_isrForCrosstalkSources_997006_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4107 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/7f335333-a87a-4a8a-aab7-e197ee221418_isrForCrosstalkSources_997006_32.stderr parsl.dataflow.dflow INFO: Task 3926 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e90885e2131d2f0b555f75323d53c58 with result from task 3926 parsl.dataflow.dflow DEBUG: Task 3926 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/00d2fb39-c654-4db2-9266-3e6143c07670_isrForCrosstalkSources_997012_58.stdout parsl.dataflow.dflow INFO: Standard error for task 3926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/00d2fb39-c654-4db2-9266-3e6143c07670_isrForCrosstalkSources_997012_58.stderr parsl.dataflow.dflow INFO: Task 4167 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 448887ef484a7930fb621e192ac35f56 with result from task 4167 parsl.dataflow.dflow DEBUG: Task 4167 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/400487b6-78f1-4033-80b8-5d32f010685a_isrForCrosstalkSources_996955_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/400487b6-78f1-4033-80b8-5d32f010685a_isrForCrosstalkSources_996955_10.stderr parsl.dataflow.dflow INFO: Task 3927 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c043a23792ad2c2a4da0f767db920779 with result from task 3927 parsl.dataflow.dflow DEBUG: Task 3927 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a09296f2-6b8c-4a3b-bdab-80cdb37c9201_isrForCrosstalkSources_996978_54.stdout parsl.dataflow.dflow INFO: Standard error for task 3927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/a09296f2-6b8c-4a3b-bdab-80cdb37c9201_isrForCrosstalkSources_996978_54.stderr parsl.dataflow.dflow INFO: Task 4102 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc9121eb30a0fe163a631e59ebd56690 with result from task 4102 parsl.dataflow.dflow DEBUG: Task 4102 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/454b18b1-6eef-456f-a963-2ea477843cea_isrForCrosstalkSources_997004_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4102 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/454b18b1-6eef-456f-a963-2ea477843cea_isrForCrosstalkSources_997004_51.stderr parsl.dataflow.dflow INFO: Task 4197 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af8c889aac2f113b9fac7366838256ef with result from task 4197 parsl.dataflow.dflow DEBUG: Task 4197 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/233d8bf0-da34-4623-a7ec-501f4e0db985_isrForCrosstalkSources_996976_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4197 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/233d8bf0-da34-4623-a7ec-501f4e0db985_isrForCrosstalkSources_996976_17.stderr parsl.dataflow.dflow INFO: Task 4000 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7645d8de582da5ca2afd817defce2ba with result from task 4000 parsl.dataflow.dflow DEBUG: Task 4000 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d34017b4-770b-48f4-bbd8-c5757dc41e06_isrForCrosstalkSources_997025_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/d34017b4-770b-48f4-bbd8-c5757dc41e06_isrForCrosstalkSources_997025_8.stderr parsl.dataflow.dflow INFO: Task 3996 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2de3ca65b8fbfe4c9335f71a74909e64 with result from task 3996 parsl.dataflow.dflow DEBUG: Task 3996 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/8388030f-bd9c-438a-a710-b4936bfe4952_isrForCrosstalkSources_997025_46.stdout parsl.dataflow.dflow INFO: Standard error for task 3996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/8388030f-bd9c-438a-a710-b4936bfe4952_isrForCrosstalkSources_997025_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:33 proc_lsst.multi:146 [INFO] found job 17075210 in provider checkpoint proc_lsst.multi INFO: found job 17075210 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4148 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b0640846d3183372fff7343d4d9100f with result from task 4148 parsl.dataflow.dflow DEBUG: Task 4148 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/78ddabd4-c910-4c02-b97b-47af64ab8951_isrForCrosstalkSources_996981_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4148 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/78ddabd4-c910-4c02-b97b-47af64ab8951_isrForCrosstalkSources_996981_40.stderr parsl.dataflow.dflow INFO: Task 4117 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f611fee5cfc429f51f47a45eb86ba3b3 with result from task 4117 parsl.dataflow.dflow DEBUG: Task 4117 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/ef581af9-ac04-4a51-95ad-40735c28960d_isrForCrosstalkSources_996973_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4117 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/ef581af9-ac04-4a51-95ad-40735c28960d_isrForCrosstalkSources_996973_23.stderr parsl.dataflow.dflow INFO: Task 4002 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 24dc5b58be966fa5463fe9cf547c623e with result from task 4002 parsl.dataflow.dflow DEBUG: Task 4002 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc5b58f7-1cda-43c1-a255-7544822d47b6_isrForCrosstalkSources_997025_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/bc5b58f7-1cda-43c1-a255-7544822d47b6_isrForCrosstalkSources_997025_17.stderr parsl.dataflow.dflow INFO: Task 4204 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c81b8e127e4873fa3cf5ab2b704972c with result from task 4204 parsl.dataflow.dflow DEBUG: Task 4204 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ade87412-3a01-4716-95a4-34b2468eb1db_isrForCrosstalkSources_997002_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4204 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ade87412-3a01-4716-95a4-34b2468eb1db_isrForCrosstalkSources_997002_22.stderr parsl.dataflow.dflow INFO: Task 4306 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf033904b0e18221f7eb004fd2636e65 with result from task 4306 parsl.dataflow.dflow DEBUG: Task 4306 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/8725a853-6c5a-4a74-8e17-60daad879948_isrForCrosstalkSources_996999_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4306 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/8725a853-6c5a-4a74-8e17-60daad879948_isrForCrosstalkSources_996999_57.stderr parsl.dataflow.dflow INFO: Task 4232 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7834fda28d75359b1bec7b40584c1830 with result from task 4232 parsl.dataflow.dflow DEBUG: Task 4232 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/8c73f837-f92c-4bbe-99ca-2c35be2a8d5f_isrForCrosstalkSources_996985_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4232 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/8c73f837-f92c-4bbe-99ca-2c35be2a8d5f_isrForCrosstalkSources_996985_25.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4315 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 0c343e6b970cfc9c5443a1ad23d2b3ae with result from task 4315 parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4315 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 4315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/678518ed-aa2f-4c76-8b9b-a502205d8979_isrForCrosstalkSources_997025_51.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 4315 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/678518ed-aa2f-4c76-8b9b-a502205d8979_isrForCrosstalkSources_997025_51.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:34 proc_lsst.multi:146 [INFO] found job 17075211 in provider checkpoint proc_lsst.multi INFO: found job 17075211 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4305 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry db9d94e9ee3dec490b3a50be6158a41a with result from task 4305 parsl.dataflow.dflow DEBUG: Task 4305 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/47e303e2-ba8b-4593-8319-384246584060_isrForCrosstalkSources_997003_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4305 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/47e303e2-ba8b-4593-8319-384246584060_isrForCrosstalkSources_997003_41.stderr parsl.dataflow.dflow INFO: Task 4304 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 624f2aff452cd3ec47ad2a848f48f94c with result from task 4304 parsl.dataflow.dflow DEBUG: Task 4304 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/499d1a4f-1bd5-4874-a303-f3960c1da952_isrForCrosstalkSources_997012_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4304 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/499d1a4f-1bd5-4874-a303-f3960c1da952_isrForCrosstalkSources_997012_6.stderr parsl.dataflow.dflow INFO: Task 4264 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39a1e6cdf8966313bf1e82971a3a5bfb with result from task 4264 parsl.dataflow.dflow DEBUG: Task 4264 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/405a5e43-3502-4d7e-8672-cbc063248059_isrForCrosstalkSources_997026_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4264 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/405a5e43-3502-4d7e-8672-cbc063248059_isrForCrosstalkSources_997026_33.stderr parsl.dataflow.dflow INFO: Task 4170 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 757f96cafda15301c436d56fe0b3272f with result from task 4170 parsl.dataflow.dflow DEBUG: Task 4170 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f5af03d1-535c-427f-8ec2-9f05cc83d5d7_isrForCrosstalkSources_996953_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4170 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/f5af03d1-535c-427f-8ec2-9f05cc83d5d7_isrForCrosstalkSources_996953_12.stderr parsl.dataflow.dflow INFO: Task 4005 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e83499a06aa287232fdd41281b6e8c4 with result from task 4005 parsl.dataflow.dflow DEBUG: Task 4005 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5d42ff7e-cd65-4451-bc78-aea3294d4361_isrForCrosstalkSources_996978_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5d42ff7e-cd65-4451-bc78-aea3294d4361_isrForCrosstalkSources_996978_56.stderr parsl.dataflow.dflow INFO: Task 4035 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2aacd5467a2733bfdb57c88df7ac87f8 with result from task 4035 parsl.dataflow.dflow DEBUG: Task 4035 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2ee63fc2-13f2-4f18-b0cc-f8bfa3fcedff_isrForCrosstalkSources_997006_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/2ee63fc2-13f2-4f18-b0cc-f8bfa3fcedff_isrForCrosstalkSources_997006_12.stderr parsl.dataflow.dflow INFO: Task 3995 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 834d9ece1fc3d9ac5cb880eb6c2f2726 with result from task 3995 parsl.dataflow.dflow DEBUG: Task 3995 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/d0c95ecc-c21e-48b5-9ba1-9a1625311fe6_isrForCrosstalkSources_996976_62.stdout parsl.dataflow.dflow INFO: Standard error for task 3995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/d0c95ecc-c21e-48b5-9ba1-9a1625311fe6_isrForCrosstalkSources_996976_62.stderr parsl.dataflow.dflow INFO: Task 3999 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d92029fd74e019edc4085b02ec43acc with result from task 3999 parsl.dataflow.dflow DEBUG: Task 3999 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/72666815-174e-4608-a618-a362e2bdb576_isrForCrosstalkSources_997012_39.stdout parsl.dataflow.dflow INFO: Standard error for task 3999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/72666815-174e-4608-a618-a362e2bdb576_isrForCrosstalkSources_997012_39.stderr parsl.dataflow.dflow INFO: Task 4274 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a140711d5f1010b3230858c39de2f1c4 with result from task 4274 parsl.dataflow.dflow DEBUG: Task 4274 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/52a7b3c4-ea41-405d-84e2-34bfc7a8697e_isrForCrosstalkSources_997019_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4274 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/52a7b3c4-ea41-405d-84e2-34bfc7a8697e_isrForCrosstalkSources_997019_21.stderr parsl.dataflow.dflow INFO: Task 4270 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5a168c785cadff7515cdc4bab343d3f5 with result from task 4270 parsl.dataflow.dflow DEBUG: Task 4270 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/974c50ca-6c36-4764-9943-daac8a0f312c_isrForCrosstalkSources_997011_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4270 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/974c50ca-6c36-4764-9943-daac8a0f312c_isrForCrosstalkSources_997011_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:34 proc_lsst.multi:146 [INFO] found job 17075212 in provider checkpoint proc_lsst.multi INFO: found job 17075212 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4208 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abc2e9bb9fca8e084d4bdf9d83fb735c with result from task 4208 parsl.dataflow.dflow DEBUG: Task 4208 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/92f3bf55-ab8b-4572-a732-aa1a6666bf24_isrForCrosstalkSources_997016_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4208 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/92f3bf55-ab8b-4572-a732-aa1a6666bf24_isrForCrosstalkSources_997016_48.stderr parsl.dataflow.dflow INFO: Task 4003 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0de5bb89de6799e43f9ec230258e26a6 with result from task 4003 parsl.dataflow.dflow DEBUG: Task 4003 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/e23a6c09-55fe-49a2-8177-d6b32d7898a7_isrForCrosstalkSources_996991_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/e23a6c09-55fe-49a2-8177-d6b32d7898a7_isrForCrosstalkSources_996991_17.stderr parsl.dataflow.dflow INFO: Task 4191 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6de01ae3c5455a1a21a759e4a0c7a4f8 with result from task 4191 parsl.dataflow.dflow DEBUG: Task 4191 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/958fc4b5-bc1e-4154-81a9-32214722dfc2_isrForCrosstalkSources_997016_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4191 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/958fc4b5-bc1e-4154-81a9-32214722dfc2_isrForCrosstalkSources_997016_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4120 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry aac9a54b80768beb8927b61897105e2b with result from task 4120 parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:35 proc_lsst.multi:146 [INFO] found job 17075213 in provider checkpoint proc_lsst.multi INFO: found job 17075213 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4120 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/649d483a-824c-4f29-ba8f-f4774380ef97_isrForCrosstalkSources_997000_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4120 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/649d483a-824c-4f29-ba8f-f4774380ef97_isrForCrosstalkSources_997000_52.stderr parsl.dataflow.dflow INFO: Task 4327 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26dcbbb7964f9f2e961edfc36795fcf4 with result from task 4327 parsl.dataflow.dflow DEBUG: Task 4327 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e8f03c71-8714-41c8-9079-302b53dedf0f_isrForCrosstalkSources_996964_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4327 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/e8f03c71-8714-41c8-9079-302b53dedf0f_isrForCrosstalkSources_996964_22.stderr parsl.dataflow.dflow INFO: Task 4099 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 582182aaf1739491f885800ba3171295 with result from task 4099 parsl.dataflow.dflow DEBUG: Task 4099 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/c9c249fc-fec1-415b-be3e-fc9d01de5d57_isrForCrosstalkSources_996963_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4099 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/c9c249fc-fec1-415b-be3e-fc9d01de5d57_isrForCrosstalkSources_996963_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4329 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 753debb6963060f04b11da3edfdf31bc with result from task 4329 parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 4329 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 4329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/e425102b-c957-449e-b083-adf24bd4c86d_isrForCrosstalkSources_996975_32.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 4329 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/e425102b-c957-449e-b083-adf24bd4c86d_isrForCrosstalkSources_996975_32.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:35 proc_lsst.multi:146 [INFO] found job 17075214 in provider checkpoint proc_lsst.multi INFO: found job 17075214 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4195 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba9f750195624b941e7b6eb722d464b8 with result from task 4195 parsl.dataflow.dflow DEBUG: Task 4195 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d9824884-dfec-40aa-b495-77253f5ee51c_isrForCrosstalkSources_996985_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4195 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/d9824884-dfec-40aa-b495-77253f5ee51c_isrForCrosstalkSources_996985_32.stderr parsl.dataflow.dflow INFO: Task 4199 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9808f9fb31b10872bed2a58cb0457950 with result from task 4199 parsl.dataflow.dflow DEBUG: Task 4199 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/678cbcd2-4d30-4d52-b03c-147019160006_isrForCrosstalkSources_996988_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4199 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/678cbcd2-4d30-4d52-b03c-147019160006_isrForCrosstalkSources_996988_4.stderr parsl.dataflow.dflow INFO: Task 4011 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27a8d8659691be87c167e53f94a9b043 with result from task 4011 parsl.dataflow.dflow DEBUG: Task 4011 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/4ad0bfa6-f471-4a56-ace0-c4712f6e936b_isrForCrosstalkSources_997002_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/4ad0bfa6-f471-4a56-ace0-c4712f6e936b_isrForCrosstalkSources_997002_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:36 proc_lsst.multi:146 [INFO] found job 17075216 in provider checkpoint proc_lsst.multi INFO: found job 17075216 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4054 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c49f057d302260f2b1254a24c8f78169 with result from task 4054 parsl.dataflow.dflow DEBUG: Task 4054 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b07814cb-7dfe-4453-ba4e-cec96dafc499_isrForCrosstalkSources_997021_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b07814cb-7dfe-4453-ba4e-cec96dafc499_isrForCrosstalkSources_997021_5.stderr parsl.dataflow.dflow INFO: Task 4111 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry deb421054e4c5d38b02005af715e463a with result from task 4111 parsl.dataflow.dflow DEBUG: Task 4111 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b578b5e5-57fb-457e-9f2f-fa25bd163c9d_isrForCrosstalkSources_997021_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4111 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/b578b5e5-57fb-457e-9f2f-fa25bd163c9d_isrForCrosstalkSources_997021_1.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:36 proc_lsst.multi:146 [INFO] found job 17075217 in provider checkpoint proc_lsst.multi INFO: found job 17075217 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 3998 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 670f8c6fe7fccfcf487fa6d2708efb0a with result from task 3998 parsl.dataflow.dflow DEBUG: Task 3998 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/17141304-cfc4-40cf-a072-3a824b2fdf4b_isrForCrosstalkSources_996968_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/17141304-cfc4-40cf-a072-3a824b2fdf4b_isrForCrosstalkSources_996968_37.stderr parsl.dataflow.dflow INFO: Task 4340 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7f945b9fcee0b62dfde5f6a929eb5bb with result from task 4340 parsl.dataflow.dflow DEBUG: Task 4340 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/55d3e3bc-6c63-460b-b64f-d181b0723d62_isrForCrosstalkSources_997023_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4340 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/55d3e3bc-6c63-460b-b64f-d181b0723d62_isrForCrosstalkSources_997023_21.stderr parsl.dataflow.dflow INFO: Task 4357 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2987a9aeed41d31923da644b747174f0 with result from task 4357 parsl.dataflow.dflow DEBUG: Task 4357 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/978d8e9e-6c27-4a55-9177-5afc92de1b7c_isrForCrosstalkSources_996967_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4357 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/978d8e9e-6c27-4a55-9177-5afc92de1b7c_isrForCrosstalkSources_996967_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:36 proc_lsst.multi:146 [INFO] found job 17075218 in provider checkpoint proc_lsst.multi INFO: found job 17075218 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4055 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07a7195377904b9b49668bc7ddc2cabe with result from task 4055 parsl.dataflow.dflow DEBUG: Task 4055 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cc260ada-d804-4aed-962a-248eae8f0fc5_isrForCrosstalkSources_996988_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/cc260ada-d804-4aed-962a-248eae8f0fc5_isrForCrosstalkSources_996988_38.stderr parsl.dataflow.dflow INFO: Task 4069 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c6ababd825e6bee829decac7658952bd with result from task 4069 parsl.dataflow.dflow DEBUG: Task 4069 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fba0d047-a55a-4ed0-8451-6c8e07ac70e9_isrForCrosstalkSources_996968_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/fba0d047-a55a-4ed0-8451-6c8e07ac70e9_isrForCrosstalkSources_996968_49.stderr parsl.dataflow.dflow INFO: Task 4145 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42e5a43d7df4822a2d3b7f32a819c68b with result from task 4145 parsl.dataflow.dflow DEBUG: Task 4145 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8eb7f91c-4277-4ef9-937d-6569aaf9718c_isrForCrosstalkSources_996973_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4145 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/8eb7f91c-4277-4ef9-937d-6569aaf9718c_isrForCrosstalkSources_996973_15.stderr parsl.dataflow.dflow INFO: Task 4354 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 597a86690d7e488889632a944c7f85f1 with result from task 4354 parsl.dataflow.dflow DEBUG: Task 4354 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/27b44e8c-5bc0-4ffd-bd26-2bb62031ee46_isrForCrosstalkSources_996980_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4354 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/27b44e8c-5bc0-4ffd-bd26-2bb62031ee46_isrForCrosstalkSources_996980_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:37 proc_lsst.multi:146 [INFO] found job 17075219 in provider checkpoint proc_lsst.multi INFO: found job 17075219 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4352 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be7a4a519abf5e448457bae0a4b9b250 with result from task 4352 parsl.dataflow.dflow DEBUG: Task 4352 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c203052d-4f78-411e-8545-bf286e89e515_isrForCrosstalkSources_996965_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4352 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c203052d-4f78-411e-8545-bf286e89e515_isrForCrosstalkSources_996965_23.stderr parsl.dataflow.dflow INFO: Task 4369 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dea01d6010f563b1168de0de92d3959f with result from task 4369 parsl.dataflow.dflow DEBUG: Task 4369 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/ac2d9dff-ac28-4cfe-ad8d-4ab8976b0668_isrForCrosstalkSources_996961_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4369 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/ac2d9dff-ac28-4cfe-ad8d-4ab8976b0668_isrForCrosstalkSources_996961_20.stderr parsl.dataflow.dflow INFO: Task 4370 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3cd1b847b669038ad049036646304a9f with result from task 4370 parsl.dataflow.dflow DEBUG: Task 4370 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d61da60d-36ee-4dbb-a011-9eb26b748081_isrForCrosstalkSources_996979_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4370 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/d61da60d-36ee-4dbb-a011-9eb26b748081_isrForCrosstalkSources_996979_19.stderr parsl.dataflow.dflow INFO: Task 4200 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3125b00dd2fe58cd55536eeeba3d70c with result from task 4200 parsl.dataflow.dflow DEBUG: Task 4200 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/75804330-87f1-4b53-bd30-73d301948ffb_isrForCrosstalkSources_996981_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4200 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/75804330-87f1-4b53-bd30-73d301948ffb_isrForCrosstalkSources_996981_29.stderr parsl.dataflow.dflow INFO: Task 4342 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 747abb568d5faf8c729eb2c2624d394e with result from task 4342 parsl.dataflow.dflow DEBUG: Task 4342 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bcc2f85e-79f4-4038-9f5d-133121a56bf9_isrForCrosstalkSources_997027_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4342 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/bcc2f85e-79f4-4038-9f5d-133121a56bf9_isrForCrosstalkSources_997027_25.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:38 proc_lsst.multi:146 [INFO] found job 17075220 in provider checkpoint proc_lsst.multi INFO: found job 17075220 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4184 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31b285c1d55ff6b29a2d1b2481b3d93a with result from task 4184 parsl.dataflow.dflow DEBUG: Task 4184 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/10fb6ade-fdd5-415d-a827-c097385e5f80_isrForCrosstalkSources_996960_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4184 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/10fb6ade-fdd5-415d-a827-c097385e5f80_isrForCrosstalkSources_996960_10.stderr parsl.dataflow.dflow INFO: Task 4157 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb2aa342f64bf1559e8730b3b2730361 with result from task 4157 parsl.dataflow.dflow DEBUG: Task 4157 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/1c89dd15-e892-4e87-8635-c3a097cef58b_isrForCrosstalkSources_997008_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4157 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/1c89dd15-e892-4e87-8635-c3a097cef58b_isrForCrosstalkSources_997008_53.stderr parsl.dataflow.dflow INFO: Task 4338 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f931a4e1c7a6b1bc3c32a090fa30c60 with result from task 4338 parsl.dataflow.dflow DEBUG: Task 4338 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/584add02-456f-4486-9efa-2c99804b4987_isrForCrosstalkSources_997010_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4338 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/584add02-456f-4486-9efa-2c99804b4987_isrForCrosstalkSources_997010_8.stderr parsl.dataflow.dflow INFO: Task 4211 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bfc1f43e72f07a5a95f3a95d008b927a with result from task 4211 parsl.dataflow.dflow DEBUG: Task 4211 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/90ea80fe-bc2b-4121-8467-df28e28861b3_isrForCrosstalkSources_997024_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4211 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/90ea80fe-bc2b-4121-8467-df28e28861b3_isrForCrosstalkSources_997024_28.stderr parsl.dataflow.dflow INFO: Task 4186 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 181509bd0c5355637232c832dc3f1223 with result from task 4186 parsl.dataflow.dflow DEBUG: Task 4186 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/0977445e-9929-432d-a449-d9f22793d10b_isrForCrosstalkSources_996954_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4186 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/0977445e-9929-432d-a449-d9f22793d10b_isrForCrosstalkSources_996954_18.stderr parsl.dataflow.dflow INFO: Task 4350 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d672c97a70dd86bd77611adae3f8cf44 with result from task 4350 parsl.dataflow.dflow DEBUG: Task 4350 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/abdfb6af-abb0-41f3-ac8b-7349e0b75e54_isrForCrosstalkSources_997028_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4350 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/abdfb6af-abb0-41f3-ac8b-7349e0b75e54_isrForCrosstalkSources_997028_54.stderr parsl.dataflow.dflow INFO: Task 4223 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2c9d814d3b90f148568137c859a97165 with result from task 4223 parsl.dataflow.dflow DEBUG: Task 4223 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7bb5177a-7e81-40bc-8945-3bc3fe4b6539_isrForCrosstalkSources_997015_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4223 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/7bb5177a-7e81-40bc-8945-3bc3fe4b6539_isrForCrosstalkSources_997015_38.stderr parsl.dataflow.dflow INFO: Task 4337 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce8d49c2c0441b5726e5e6e61c31a5f5 with result from task 4337 parsl.dataflow.dflow DEBUG: Task 4337 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d55519a6-0f53-463e-930e-83ab65d3bba5_isrForCrosstalkSources_997026_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4337 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/d55519a6-0f53-463e-930e-83ab65d3bba5_isrForCrosstalkSources_997026_32.stderr parsl.dataflow.dflow INFO: Task 4187 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e13d6be369dd7016660ebf491e211772 with result from task 4187 parsl.dataflow.dflow DEBUG: Task 4187 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ef5a9c98-786d-4a90-90ea-5cd804f795be_isrForCrosstalkSources_996954_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4187 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ef5a9c98-786d-4a90-90ea-5cd804f795be_isrForCrosstalkSources_996954_12.stderr parsl.dataflow.dflow INFO: Task 4371 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 98971daccfebc68e14c0e2cfe9acf547 with result from task 4371 parsl.dataflow.dflow DEBUG: Task 4371 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/0cfb53af-ebd2-4413-92d5-406f496bcdf4_isrForCrosstalkSources_996972_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4371 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/0cfb53af-ebd2-4413-92d5-406f496bcdf4_isrForCrosstalkSources_996972_23.stderr parsl.dataflow.dflow INFO: Task 4009 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 09fe73f266a88f19b5f1e59b3955f648 with result from task 4009 parsl.dataflow.dflow DEBUG: Task 4009 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/05fcd1f4-08ed-4d0f-ba6b-1bad6c59fda3_isrForCrosstalkSources_996953_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/05fcd1f4-08ed-4d0f-ba6b-1bad6c59fda3_isrForCrosstalkSources_996953_36.stderr parsl.dataflow.dflow INFO: Task 4376 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab37de05d90188027a3a45f809f45d0d with result from task 4376 parsl.dataflow.dflow DEBUG: Task 4376 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/7a12d075-d2f1-4e0b-911c-b09623e97e2f_isrForCrosstalkSources_997004_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4376 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/7a12d075-d2f1-4e0b-911c-b09623e97e2f_isrForCrosstalkSources_997004_25.stderr parsl.dataflow.dflow INFO: Task 4008 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8947e40a44157747733fcd700f9e0e6 with result from task 4008 parsl.dataflow.dflow DEBUG: Task 4008 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/60b48a7b-f112-429d-b47d-ff79de0dd112_isrForCrosstalkSources_996970_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/60b48a7b-f112-429d-b47d-ff79de0dd112_isrForCrosstalkSources_996970_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:39 proc_lsst.multi:146 [INFO] found job 17075221 in provider checkpoint proc_lsst.multi INFO: found job 17075221 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4041 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b01aa8a113265cc0124410b9f42dd74 with result from task 4041 parsl.dataflow.dflow DEBUG: Task 4041 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b8104e4d-161c-46d9-be4f-4d610e2bb012_isrForCrosstalkSources_996983_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b8104e4d-161c-46d9-be4f-4d610e2bb012_isrForCrosstalkSources_996983_59.stderr parsl.dataflow.dflow INFO: Task 4379 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1ae59fdb1009284bad2448903c62878d with result from task 4379 parsl.dataflow.dflow DEBUG: Task 4379 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1129c0a9-760e-4ac9-9177-e35a12329866_isrForCrosstalkSources_996979_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4379 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1129c0a9-760e-4ac9-9177-e35a12329866_isrForCrosstalkSources_996979_30.stderr parsl.dataflow.dflow INFO: Task 4176 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef9e4b5d11739e095a63f8a7c4214b8e with result from task 4176 parsl.dataflow.dflow DEBUG: Task 4176 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/cf8db234-4486-4a7e-8b43-e0f4f3531613_isrForCrosstalkSources_996962_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4176 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/cf8db234-4486-4a7e-8b43-e0f4f3531613_isrForCrosstalkSources_996962_62.stderr parsl.dataflow.dflow INFO: Task 4061 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3523f45429f850ec37cae5faacf3d14f with result from task 4061 parsl.dataflow.dflow DEBUG: Task 4061 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/0051d814-0ff5-4e34-bdc1-8f551a67f5d3_isrForCrosstalkSources_996960_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/0051d814-0ff5-4e34-bdc1-8f551a67f5d3_isrForCrosstalkSources_996960_37.stderr parsl.dataflow.dflow INFO: Task 4126 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0bedc8bb88a3b518cf5a6de07e1abb54 with result from task 4126 parsl.dataflow.dflow DEBUG: Task 4126 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8cb7c2d0-9edf-404b-91fc-48a4b7e266b2_isrForCrosstalkSources_997027_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4126 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8cb7c2d0-9edf-404b-91fc-48a4b7e266b2_isrForCrosstalkSources_997027_62.stderr parsl.dataflow.dflow INFO: Task 4339 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bce6726d809752b241e6408418015a70 with result from task 4339 parsl.dataflow.dflow DEBUG: Task 4339 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/4562e5f6-4601-405a-b3d4-7b4367116aee_isrForCrosstalkSources_997003_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4339 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/4562e5f6-4601-405a-b3d4-7b4367116aee_isrForCrosstalkSources_997003_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:40 proc_lsst.multi:146 [INFO] found job 17075222 in provider checkpoint proc_lsst.multi INFO: found job 17075222 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4347 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6d922be2025983479d29009218ccf443 with result from task 4347 parsl.dataflow.dflow DEBUG: Task 4347 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/a28a0386-87a3-4c1b-8eba-2f5b69fa0d0a_isrForCrosstalkSources_996998_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4347 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/a28a0386-87a3-4c1b-8eba-2f5b69fa0d0a_isrForCrosstalkSources_996998_1.stderr parsl.dataflow.dflow INFO: Task 4358 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3694d74bfa62bcfdb97a302a8edc90ab with result from task 4358 parsl.dataflow.dflow DEBUG: Task 4358 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/02e962ad-47c1-498b-8eb9-c51859c1a1f1_isrForCrosstalkSources_997025_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4358 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/02e962ad-47c1-498b-8eb9-c51859c1a1f1_isrForCrosstalkSources_997025_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:40 proc_lsst.multi:146 [INFO] found job 17075223 in provider checkpoint proc_lsst.multi INFO: found job 17075223 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4356 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7d0834fca80be07a81fb434ad992b6c7 with result from task 4356 parsl.dataflow.dflow DEBUG: Task 4356 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/df17f5d8-8617-4776-a41f-5482a30bb261_isrForCrosstalkSources_996978_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4356 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/df17f5d8-8617-4776-a41f-5482a30bb261_isrForCrosstalkSources_996978_49.stderr parsl.dataflow.dflow INFO: Task 4387 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 877f720ce769fa28bc7baccb18d035c7 with result from task 4387 parsl.dataflow.dflow DEBUG: Task 4387 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/251a336f-0326-40c7-8fa8-b3046655e8a7_isrForCrosstalkSources_996971_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4387 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/251a336f-0326-40c7-8fa8-b3046655e8a7_isrForCrosstalkSources_996971_53.stderr parsl.dataflow.dflow INFO: Task 4028 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21f829b4f4048877f5e2319ff8015559 with result from task 4028 parsl.dataflow.dflow DEBUG: Task 4028 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a2592c3d-80f8-4610-9b17-0416632635da_isrForCrosstalkSources_997032_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/a2592c3d-80f8-4610-9b17-0416632635da_isrForCrosstalkSources_997032_60.stderr parsl.dataflow.dflow INFO: Task 4355 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 43b56288bb12e6c62a4c5941b504c6fe with result from task 4355 parsl.dataflow.dflow DEBUG: Task 4355 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e253e809-139d-4c68-a894-908902788ac7_isrForCrosstalkSources_996978_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4355 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e253e809-139d-4c68-a894-908902788ac7_isrForCrosstalkSources_996978_38.stderr parsl.dataflow.dflow INFO: Task 4404 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 007fa2ec7f4fefe185c0c7e472b86aba with result from task 4404 parsl.dataflow.dflow DEBUG: Task 4404 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/d7bf6875-e2c8-4d87-9fda-ab91fdcb82ce_isrForCrosstalkSources_997020_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4404 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/d7bf6875-e2c8-4d87-9fda-ab91fdcb82ce_isrForCrosstalkSources_997020_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:41 proc_lsst.multi:146 [INFO] found job 17075224 in provider checkpoint proc_lsst.multi INFO: found job 17075224 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4412 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b30e4bf322c423062ed30665ce93889 with result from task 4412 parsl.dataflow.dflow DEBUG: Task 4412 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4b14cfb7-ba81-4887-abe9-3b6ccf652b40_isrForCrosstalkSources_996976_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4412 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4b14cfb7-ba81-4887-abe9-3b6ccf652b40_isrForCrosstalkSources_996976_40.stderr parsl.dataflow.dflow INFO: Task 4188 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e7871db5d09a59a0124e97371955900 with result from task 4188 parsl.dataflow.dflow DEBUG: Task 4188 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/ffeef2ca-8554-464c-93b5-a431112337dc_isrForCrosstalkSources_996965_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4188 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/ffeef2ca-8554-464c-93b5-a431112337dc_isrForCrosstalkSources_996965_35.stderr parsl.dataflow.dflow INFO: Task 3997 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fbf590f4cb378bfd300c6fdaeea775d with result from task 3997 parsl.dataflow.dflow DEBUG: Task 3997 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c94adbc0-37c1-420a-b610-7b1e19bb5542_isrForCrosstalkSources_996994_59.stdout parsl.dataflow.dflow INFO: Standard error for task 3997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c94adbc0-37c1-420a-b610-7b1e19bb5542_isrForCrosstalkSources_996994_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:41 proc_lsst.multi:146 [INFO] found job 17075225 in provider checkpoint proc_lsst.multi INFO: found job 17075225 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4185 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5404062efe545d900c16a94d6125d378 with result from task 4185 parsl.dataflow.dflow DEBUG: Task 4185 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4c02647f-3e2d-459c-9f61-4ccbd8943d81_isrForCrosstalkSources_996995_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4185 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4c02647f-3e2d-459c-9f61-4ccbd8943d81_isrForCrosstalkSources_996995_31.stderr parsl.dataflow.dflow INFO: Task 4413 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5dc17fc8b7c708aed96cb4c418499c6c with result from task 4413 parsl.dataflow.dflow DEBUG: Task 4413 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a53f999a-76e9-4851-8245-b26653382cdc_isrForCrosstalkSources_996993_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4413 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/a53f999a-76e9-4851-8245-b26653382cdc_isrForCrosstalkSources_996993_1.stderr parsl.dataflow.dflow INFO: Task 4181 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c0a504bb63cbe7503135aa0973d6140 with result from task 4181 parsl.dataflow.dflow DEBUG: Task 4181 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/2fe2a9b5-ce0f-4225-b371-2bdee8c6591c_isrForCrosstalkSources_996982_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4181 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/2fe2a9b5-ce0f-4225-b371-2bdee8c6591c_isrForCrosstalkSources_996982_19.stderr parsl.dataflow.dflow INFO: Task 4179 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 75f1bd820d756adf61149194b99ead44 with result from task 4179 parsl.dataflow.dflow DEBUG: Task 4179 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/47d5b8dd-115e-4fa9-8d3c-5d56169fecef_isrForCrosstalkSources_997009_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4179 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/47d5b8dd-115e-4fa9-8d3c-5d56169fecef_isrForCrosstalkSources_997009_34.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:42 proc_lsst.multi:146 [INFO] found job 17075226 in provider checkpoint proc_lsst.multi INFO: found job 17075226 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4202 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e33d906e1296f9634ee107000141dc97 with result from task 4202 parsl.dataflow.dflow DEBUG: Task 4202 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/19fa080b-d55e-4074-8186-e4f524ffe70f_isrForCrosstalkSources_997010_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4202 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/19fa080b-d55e-4074-8186-e4f524ffe70f_isrForCrosstalkSources_997010_19.stderr parsl.dataflow.dflow INFO: Task 4196 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fc30c8dfb06da806024ca98221b85c6 with result from task 4196 parsl.dataflow.dflow DEBUG: Task 4196 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/fb9ffdb7-fd2d-4d6e-a729-8b73f87dfef5_isrForCrosstalkSources_996956_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4196 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/fb9ffdb7-fd2d-4d6e-a729-8b73f87dfef5_isrForCrosstalkSources_996956_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4189 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry a4e988f86033d9c6bdabb1c22361e299 with result from task 4189 parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4189 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/5705dbd0-dc0f-45e8-b60c-4ccc33adbb2f_isrForCrosstalkSources_996961_29.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4189 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/5705dbd0-dc0f-45e8-b60c-4ccc33adbb2f_isrForCrosstalkSources_996961_29.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:42 proc_lsst.multi:146 [INFO] found job 17075227 in provider checkpoint proc_lsst.multi INFO: found job 17075227 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4428 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d06ff40e882c48de448b314031f3b44a with result from task 4428 parsl.dataflow.dflow DEBUG: Task 4428 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/50850fbf-9573-4427-9dbe-917d4e97ce8e_isrForCrosstalkSources_997005_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4428 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/50850fbf-9573-4427-9dbe-917d4e97ce8e_isrForCrosstalkSources_997005_9.stderr parsl.dataflow.dflow INFO: Task 4453 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 711938829902876d3ff6f7ad2e331b37 with result from task 4453 parsl.dataflow.dflow DEBUG: Task 4453 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9665953b-2216-490b-8d12-15d0f242a834_isrForCrosstalkSources_996984_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4453 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/9665953b-2216-490b-8d12-15d0f242a834_isrForCrosstalkSources_996984_19.stderr parsl.dataflow.dflow INFO: Task 4366 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7e0a30b18364e695ebb809c3c4c08de with result from task 4366 parsl.dataflow.dflow DEBUG: Task 4366 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/176a68d9-4f07-4751-b852-8b65b9ad33e5_isrForCrosstalkSources_996993_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4366 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/176a68d9-4f07-4751-b852-8b65b9ad33e5_isrForCrosstalkSources_996993_14.stderr parsl.dataflow.dflow INFO: Task 4190 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 87aeb187ed7197a99cc8f6663da67cf9 with result from task 4190 parsl.dataflow.dflow DEBUG: Task 4190 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1f180f24-235c-4764-8fe1-0a41feab2f5f_isrForCrosstalkSources_997032_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4190 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1f180f24-235c-4764-8fe1-0a41feab2f5f_isrForCrosstalkSources_997032_49.stderr parsl.dataflow.dflow INFO: Task 4426 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.memoization DEBUG: Storing app cache entry 48cca45f81d22db0efe4935b46605b3f with result from task 4426 parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4426 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/28475fdc-93e5-4b9a-8aea-38609c8492f4_isrForCrosstalkSources_996972_10.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4426 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/28475fdc-93e5-4b9a-8aea-38609c8492f4_isrForCrosstalkSources_996972_10.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:43 proc_lsst.multi:146 [INFO] found job 17075228 in provider checkpoint proc_lsst.multi INFO: found job 17075228 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4423 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3edf1fdde22d9982babdb24899a7eebe with result from task 4423 parsl.dataflow.dflow DEBUG: Task 4423 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/82705e31-463b-489b-8b28-6efbb2662acb_isrForCrosstalkSources_996998_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4423 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/82705e31-463b-489b-8b28-6efbb2662acb_isrForCrosstalkSources_996998_17.stderr parsl.dataflow.dflow INFO: Task 4280 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a6c892875b4ca6819500059a20117dd with result from task 4280 parsl.dataflow.dflow DEBUG: Task 4280 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/122d889f-28b4-4b8d-95bc-818775159030_isrForCrosstalkSources_996955_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4280 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/122d889f-28b4-4b8d-95bc-818775159030_isrForCrosstalkSources_996955_20.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:44 proc_lsst.multi:146 [INFO] found job 17075229 in provider checkpoint proc_lsst.multi INFO: found job 17075229 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4289 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81ab25946243c566bfff2a3ea56f3931 with result from task 4289 parsl.dataflow.dflow DEBUG: Task 4289 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fc002d14-77bd-4ef7-a948-624c45cff4c7_isrForCrosstalkSources_997002_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4289 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/fc002d14-77bd-4ef7-a948-624c45cff4c7_isrForCrosstalkSources_997002_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:44 proc_lsst.multi:146 [INFO] found job 17075230 in provider checkpoint proc_lsst.multi INFO: found job 17075230 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4250 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 558b0180805e634092f1b7896ce13e1e with result from task 4250 parsl.dataflow.dflow DEBUG: Task 4250 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/467c8c24-d776-4595-b3b3-66c3a5cbbf24_isrForCrosstalkSources_996969_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4250 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/467c8c24-d776-4595-b3b3-66c3a5cbbf24_isrForCrosstalkSources_996969_21.stderr parsl.dataflow.dflow INFO: Task 4139 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c14722bf0140f47741bd6ae0e5241f65 with result from task 4139 parsl.dataflow.dflow DEBUG: Task 4139 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ab4b06a6-4388-46d4-b036-41fb8903b29a_isrForCrosstalkSources_997000_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4139 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/ab4b06a6-4388-46d4-b036-41fb8903b29a_isrForCrosstalkSources_997000_18.stderr parsl.dataflow.dflow INFO: Task 4445 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8e0760272f4d0cfaa66736ef79866b34 with result from task 4445 parsl.dataflow.dflow DEBUG: Task 4445 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/670dbd10-ce23-4a12-8d72-7fcca8f31fb9_isrForCrosstalkSources_996991_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4445 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/670dbd10-ce23-4a12-8d72-7fcca8f31fb9_isrForCrosstalkSources_996991_57.stderr parsl.dataflow.dflow INFO: Task 4449 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 385837a53009ef04ffa706e63bb1b340 with result from task 4449 parsl.dataflow.dflow DEBUG: Task 4449 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5b44acfb-57a7-4f65-a491-15928938f554_isrForCrosstalkSources_996955_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4449 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/5b44acfb-57a7-4f65-a491-15928938f554_isrForCrosstalkSources_996955_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:45 proc_lsst.multi:146 [INFO] found job 17075231 in provider checkpoint proc_lsst.multi INFO: found job 17075231 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4468 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5cb4b4e432e7b002b91fcf438353bfad with result from task 4468 parsl.dataflow.dflow DEBUG: Task 4468 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/2df5ef37-616f-4e5a-a4df-d414ebd27e3f_isrForCrosstalkSources_997008_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4468 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/2df5ef37-616f-4e5a-a4df-d414ebd27e3f_isrForCrosstalkSources_997008_30.stderr parsl.dataflow.dflow INFO: Task 4275 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01f453a99603e60af078b07af409e148 with result from task 4275 parsl.dataflow.dflow DEBUG: Task 4275 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/496493d3-6a96-4ad9-926e-1baf2f7f95ab_isrForCrosstalkSources_996960_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4275 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/496493d3-6a96-4ad9-926e-1baf2f7f95ab_isrForCrosstalkSources_996960_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:46 proc_lsst.multi:146 [INFO] found job 17075232 in provider checkpoint proc_lsst.multi INFO: found job 17075232 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4448 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53931c7020a1100e49c9d7a408b1977c with result from task 4448 parsl.dataflow.dflow DEBUG: Task 4448 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4d4066da-d051-431e-adf6-e27755ae5785_isrForCrosstalkSources_996987_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4448 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/4d4066da-d051-431e-adf6-e27755ae5785_isrForCrosstalkSources_996987_26.stderr parsl.dataflow.dflow INFO: Task 4403 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03d4ee7fb26bc9124b3a9be6d0fbff5e with result from task 4403 parsl.dataflow.dflow DEBUG: Task 4403 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/90f7c3a5-3893-4fb0-b06d-05b695fb7195_isrForCrosstalkSources_997032_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4403 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/90f7c3a5-3893-4fb0-b06d-05b695fb7195_isrForCrosstalkSources_997032_26.stderr parsl.dataflow.dflow INFO: Task 4467 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 70d4d804820b978bfa00c51862701003 with result from task 4467 parsl.dataflow.dflow DEBUG: Task 4467 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/614645f8-38f3-428c-90c1-97983a115c9d_isrForCrosstalkSources_996989_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4467 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/614645f8-38f3-428c-90c1-97983a115c9d_isrForCrosstalkSources_996989_40.stderr parsl.dataflow.dflow INFO: Task 4331 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fdf7cd606567462a8208493d3b9b770f with result from task 4331 parsl.dataflow.dflow DEBUG: Task 4331 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/eb923953-063b-47ef-b8ac-5cd9fc482df7_isrForCrosstalkSources_996967_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4331 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/eb923953-063b-47ef-b8ac-5cd9fc482df7_isrForCrosstalkSources_996967_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:46 proc_lsst.multi:146 [INFO] found job 17075233 in provider checkpoint proc_lsst.multi INFO: found job 17075233 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4478 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b87a4d521dcb09eacc7ef7d1da9e7946 with result from task 4478 parsl.dataflow.dflow DEBUG: Task 4478 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/0e28ff92-3366-47a0-8496-92a9c4f73449_isrForCrosstalkSources_997017_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4478 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/0e28ff92-3366-47a0-8496-92a9c4f73449_isrForCrosstalkSources_997017_43.stderr parsl.dataflow.dflow INFO: Task 4469 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bdb3da532cf93bbdf0ca5e823381c648 with result from task 4469 parsl.dataflow.dflow DEBUG: Task 4469 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/9994882d-6655-4f36-8863-b05fa4b0f747_isrForCrosstalkSources_996956_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4469 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/9994882d-6655-4f36-8863-b05fa4b0f747_isrForCrosstalkSources_996956_51.stderr parsl.dataflow.dflow INFO: Task 4484 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e779fd1733d2a6e30eb0c02a87223d10 with result from task 4484 parsl.dataflow.dflow DEBUG: Task 4484 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bf082dc5-22b5-45a6-b842-d268760d43fc_isrForCrosstalkSources_997030_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4484 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bf082dc5-22b5-45a6-b842-d268760d43fc_isrForCrosstalkSources_997030_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:47 proc_lsst.multi:146 [INFO] found job 17075234 in provider checkpoint proc_lsst.multi INFO: found job 17075234 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4470 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 32803cc98e0ae2c484711f07414c7057 with result from task 4470 parsl.dataflow.dflow DEBUG: Task 4470 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/646c251c-814e-4e98-93c6-52097ac47c4f_isrForCrosstalkSources_997016_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4470 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/646c251c-814e-4e98-93c6-52097ac47c4f_isrForCrosstalkSources_997016_62.stderr parsl.dataflow.dflow INFO: Task 4471 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9154e14faee6483c3d517ad4e6bc4a34 with result from task 4471 parsl.dataflow.dflow DEBUG: Task 4471 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ac2b0119-231c-4f8f-b993-e7b85a70f487_isrForCrosstalkSources_997019_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4471 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/ac2b0119-231c-4f8f-b993-e7b85a70f487_isrForCrosstalkSources_997019_55.stderr parsl.dataflow.dflow INFO: Task 4451 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9960b1cbe30c8a72f148b9a3dac4f024 with result from task 4451 parsl.dataflow.dflow DEBUG: Task 4451 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/c27c8358-a018-4109-a200-f038cf976825_isrForCrosstalkSources_996983_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4451 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/c27c8358-a018-4109-a200-f038cf976825_isrForCrosstalkSources_996983_31.stderr parsl.dataflow.dflow INFO: Task 4424 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 74530eebc695a7e297237b60b09b2a1c with result from task 4424 parsl.dataflow.dflow DEBUG: Task 4424 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/dd51df0e-303d-44d3-bc8e-8ed700ec48f7_isrForCrosstalkSources_996987_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4424 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/dd51df0e-303d-44d3-bc8e-8ed700ec48f7_isrForCrosstalkSources_996987_22.stderr parsl.dataflow.dflow INFO: Task 4310 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b2c6db41bc61dc8b7e943c6a904c46b5 with result from task 4310 parsl.dataflow.dflow DEBUG: Task 4310 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/8a9883a7-2c12-42ad-988a-dc17cda4578d_isrForCrosstalkSources_996989_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4310 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/8a9883a7-2c12-42ad-988a-dc17cda4578d_isrForCrosstalkSources_996989_7.stderr parsl.dataflow.dflow INFO: Task 4493 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb0aef91e7d9c1d00346a73767b8f9f6 with result from task 4493 parsl.dataflow.dflow DEBUG: Task 4493 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/19bf4753-9b53-4ce9-9fd3-8509cc388a5a_isrForCrosstalkSources_996965_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4493 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/19bf4753-9b53-4ce9-9fd3-8509cc388a5a_isrForCrosstalkSources_996965_40.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:47 proc_lsst.multi:146 [INFO] found job 17075238 in provider checkpoint proc_lsst.multi INFO: found job 17075238 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4074 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a05a0e9ab4a82f459fc47617a500038e with result from task 4074 parsl.dataflow.dflow DEBUG: Task 4074 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/b09c66ae-f054-4f5b-a780-07c991e96cc1_isrForCrosstalkSources_996956_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/b09c66ae-f054-4f5b-a780-07c991e96cc1_isrForCrosstalkSources_996956_11.stderr parsl.dataflow.dflow INFO: Task 4328 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8250afb0826531393188095e4ff50623 with result from task 4328 parsl.dataflow.dflow DEBUG: Task 4328 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/fffccc60-2aa5-4d76-8e34-f23e78ef819c_isrForCrosstalkSources_996986_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4328 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/fffccc60-2aa5-4d76-8e34-f23e78ef819c_isrForCrosstalkSources_996986_48.stderr parsl.dataflow.dflow INFO: Task 3972 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99b3f835b517e641c89d164ee3bfebd4 with result from task 3972 parsl.dataflow.dflow DEBUG: Task 3972 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ce36c364-9258-4472-a7b9-b679ba63416f_isrForCrosstalkSources_996988_47.stdout parsl.dataflow.dflow INFO: Standard error for task 3972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ce36c364-9258-4472-a7b9-b679ba63416f_isrForCrosstalkSources_996988_47.stderr parsl.dataflow.dflow INFO: Task 4479 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 562404e858a2d9bdd83b6cfa70654fa3 with result from task 4479 parsl.dataflow.dflow DEBUG: Task 4479 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1084a3ce-7e1d-4a69-9ec2-b760cdce1e01_isrForCrosstalkSources_997032_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4479 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/1084a3ce-7e1d-4a69-9ec2-b760cdce1e01_isrForCrosstalkSources_997032_53.stderr parsl.dataflow.dflow INFO: Task 4077 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7edcacd2e21baadc3382b9d0948bc611 with result from task 4077 parsl.dataflow.dflow DEBUG: Task 4077 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/651e9a2c-0b47-459d-ae50-e3c2ba0fbad1_isrForCrosstalkSources_997029_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/651e9a2c-0b47-459d-ae50-e3c2ba0fbad1_isrForCrosstalkSources_997029_11.stderr parsl.dataflow.dflow INFO: Task 4498 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8ef2059031708159cd68ea1a371e766 with result from task 4498 parsl.dataflow.dflow DEBUG: Task 4498 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9601ddd9-4911-4fc3-b1db-96962c0d7386_isrForCrosstalkSources_997008_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4498 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/9601ddd9-4911-4fc3-b1db-96962c0d7386_isrForCrosstalkSources_997008_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:48 proc_lsst.multi:146 [INFO] found job 17075242 in provider checkpoint proc_lsst.multi INFO: found job 17075242 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4249 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f13c4dc550aae900df88e494bfec5c71 with result from task 4249 parsl.dataflow.dflow DEBUG: Task 4249 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/349c138b-222d-4978-a36c-2a13e0642006_isrForCrosstalkSources_997012_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4249 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/349c138b-222d-4978-a36c-2a13e0642006_isrForCrosstalkSources_997012_11.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:48 proc_lsst.multi:146 [INFO] found job 17075245 in provider checkpoint proc_lsst.multi INFO: found job 17075245 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4333 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a282609833c679cf6a12f832bc2f68a with result from task 4333 parsl.dataflow.dflow DEBUG: Task 4333 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/f120c409-aed0-4c53-9d2e-73ccd1370adb_isrForCrosstalkSources_997000_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4333 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/f120c409-aed0-4c53-9d2e-73ccd1370adb_isrForCrosstalkSources_997000_55.stderr parsl.dataflow.dflow INFO: Task 4292 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d78a599382a1177787e592ce6f505dde with result from task 4292 parsl.dataflow.dflow DEBUG: Task 4292 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7fdc5c7a-a8fb-47fb-9cfb-54d83afc2cdc_isrForCrosstalkSources_996988_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4292 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/7fdc5c7a-a8fb-47fb-9cfb-54d83afc2cdc_isrForCrosstalkSources_996988_26.stderr parsl.dataflow.dflow INFO: Task 4505 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c188c6a5f3432a1819ad22371884328 with result from task 4505 parsl.dataflow.dflow DEBUG: Task 4505 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/4c7aa05e-3bb3-43bb-bc24-b0443504540d_isrForCrosstalkSources_996978_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4505 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/4c7aa05e-3bb3-43bb-bc24-b0443504540d_isrForCrosstalkSources_996978_19.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:49 proc_lsst.multi:146 [INFO] found job 17075246 in provider checkpoint proc_lsst.multi INFO: found job 17075246 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4486 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c04cd14c071860abd477a73e6fa156b7 with result from task 4486 parsl.dataflow.dflow DEBUG: Task 4486 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b20cb7cd-2653-4aa2-b260-80afbff91c40_isrForCrosstalkSources_997004_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4486 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b20cb7cd-2653-4aa2-b260-80afbff91c40_isrForCrosstalkSources_997004_23.stderr parsl.dataflow.dflow INFO: Task 4268 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 40fe4287a28e7915b12ce2a47810f3a7 with result from task 4268 parsl.dataflow.dflow DEBUG: Task 4268 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a703a434-2797-4794-ac9c-7f1872af2f9e_isrForCrosstalkSources_997021_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4268 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/a703a434-2797-4794-ac9c-7f1872af2f9e_isrForCrosstalkSources_997021_19.stderr parsl.dataflow.dflow INFO: Task 4494 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88c89e57b1bce02cb4d66d0cfc1d84e7 with result from task 4494 parsl.dataflow.dflow DEBUG: Task 4494 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/2d9c5c30-4bee-44c7-8a60-bc70898bcb6b_isrForCrosstalkSources_996954_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4494 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/2d9c5c30-4bee-44c7-8a60-bc70898bcb6b_isrForCrosstalkSources_996954_17.stderr parsl.dataflow.dflow INFO: Task 4212 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb4fcaab49e893a007a48c285fa9cd6b with result from task 4212 parsl.dataflow.dflow DEBUG: Task 4212 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/25c9ccb1-c506-49f3-aa40-7a215c601b3f_isrForCrosstalkSources_997032_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4212 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/25c9ccb1-c506-49f3-aa40-7a215c601b3f_isrForCrosstalkSources_997032_40.stderr parsl.dataflow.dflow INFO: Task 4226 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4dcf7c4ac5b7792b07756c92c96762a8 with result from task 4226 parsl.dataflow.dflow DEBUG: Task 4226 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/438c10d4-1b17-47b9-9114-2bf0de49ad79_isrForCrosstalkSources_996974_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4226 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/438c10d4-1b17-47b9-9114-2bf0de49ad79_isrForCrosstalkSources_996974_1.stderr parsl.dataflow.dflow INFO: Task 4235 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 73fb0771be2690b891ff5cb0fbd47057 with result from task 4235 parsl.dataflow.dflow DEBUG: Task 4235 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c8315153-9cb1-4914-940d-8d306e92a2a3_isrForCrosstalkSources_996968_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4235 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/c8315153-9cb1-4914-940d-8d306e92a2a3_isrForCrosstalkSources_996968_14.stderr parsl.dataflow.dflow INFO: Task 4150 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 83b495377604e4517aab428608c66e58 with result from task 4150 parsl.dataflow.dflow DEBUG: Task 4150 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/47dc76cc-36fc-4df9-92a7-6f0dca8f79be_isrForCrosstalkSources_997031_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4150 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/47dc76cc-36fc-4df9-92a7-6f0dca8f79be_isrForCrosstalkSources_997031_3.stderr parsl.dataflow.dflow INFO: Task 4171 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c80df2a6b06b0cf4bdcb0155bf485d3e with result from task 4171 parsl.dataflow.dflow DEBUG: Task 4171 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/1900199a-1006-4fc7-b09a-c9fdfd3febd0_isrForCrosstalkSources_996989_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4171 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/1900199a-1006-4fc7-b09a-c9fdfd3febd0_isrForCrosstalkSources_996989_15.stderr parsl.dataflow.dflow INFO: Task 4257 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 57db0fc08cab6bc0ffea26adffc131be with result from task 4257 parsl.dataflow.dflow DEBUG: Task 4257 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/2bfa982a-2895-482f-b4d8-2e126e2caaaf_isrForCrosstalkSources_996966_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4257 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/2bfa982a-2895-482f-b4d8-2e126e2caaaf_isrForCrosstalkSources_996966_53.stderr parsl.dataflow.dflow INFO: Task 4213 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 793f354206b62e114ceb07b3e84b3a21 with result from task 4213 parsl.dataflow.dflow DEBUG: Task 4213 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/8376a5a4-4364-406f-9af6-02000f22e757_isrForCrosstalkSources_997011_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4213 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/8376a5a4-4364-406f-9af6-02000f22e757_isrForCrosstalkSources_997011_10.stderr parsl.dataflow.dflow INFO: Task 4241 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e79bfe92b24d4944c8876b026a7b026f with result from task 4241 parsl.dataflow.dflow DEBUG: Task 4241 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7dd66e7e-c356-4cc3-9ace-db2a940c8555_isrForCrosstalkSources_996989_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4241 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/7dd66e7e-c356-4cc3-9ace-db2a940c8555_isrForCrosstalkSources_996989_34.stderr parsl.dataflow.dflow INFO: Task 4500 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b3153c73937013c52c74a72e14223d91 with result from task 4500 parsl.dataflow.dflow DEBUG: Task 4500 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/74d00b04-1f50-4dd5-91dd-c2bb212e8322_isrForCrosstalkSources_996998_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4500 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/74d00b04-1f50-4dd5-91dd-c2bb212e8322_isrForCrosstalkSources_996998_8.stderr parsl.dataflow.dflow INFO: Task 4198 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f7fc79fd09515fc4d60b26c38d67686 with result from task 4198 parsl.dataflow.dflow DEBUG: Task 4198 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1ed29ae9-055a-4db2-8878-d9302182fd17_isrForCrosstalkSources_996967_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4198 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1ed29ae9-055a-4db2-8878-d9302182fd17_isrForCrosstalkSources_996967_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:50 proc_lsst.multi:146 [INFO] found job 17075247 in provider checkpoint proc_lsst.multi INFO: found job 17075247 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4228 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa0574ef2b1b79400fcbffd6978ed578 with result from task 4228 parsl.dataflow.dflow DEBUG: Task 4228 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ed79a39d-6b44-4198-88ed-5e3b0094de61_isrForCrosstalkSources_996998_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4228 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/ed79a39d-6b44-4198-88ed-5e3b0094de61_isrForCrosstalkSources_996998_25.stderr parsl.dataflow.dflow INFO: Task 4147 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd01c33084e7d9b654998debd03c21e8 with result from task 4147 parsl.dataflow.dflow DEBUG: Task 4147 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f98650ef-5f71-49c5-af02-246b42ce9b1a_isrForCrosstalkSources_997032_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4147 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/f98650ef-5f71-49c5-af02-246b42ce9b1a_isrForCrosstalkSources_997032_57.stderr parsl.dataflow.dflow INFO: Task 4472 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c82ad6539ecc1736501e167cfa5caa8b with result from task 4472 parsl.dataflow.dflow DEBUG: Task 4472 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/96477658-95e1-4e60-b1f6-640780467e0e_isrForCrosstalkSources_997026_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4472 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/96477658-95e1-4e60-b1f6-640780467e0e_isrForCrosstalkSources_997026_39.stderr parsl.dataflow.dflow INFO: Task 4425 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91378317d989ebcdb7cf658ee4720dfe with result from task 4425 parsl.dataflow.dflow DEBUG: Task 4425 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/36ae0023-58e7-4e11-bac7-f799099fff33_isrForCrosstalkSources_997006_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4425 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/36ae0023-58e7-4e11-bac7-f799099fff33_isrForCrosstalkSources_997006_27.stderr parsl.dataflow.dflow INFO: Task 4501 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9933211df6dba447484a19722926e5a with result from task 4501 parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:50 proc_lsst.multi:146 [INFO] found job 17075250 in provider checkpoint proc_lsst.multi INFO: found job 17075250 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4501 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/74c6e314-84ae-43a9-b24d-63ac1d665483_isrForCrosstalkSources_996982_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4501 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/74c6e314-84ae-43a9-b24d-63ac1d665483_isrForCrosstalkSources_996982_40.stderr parsl.dataflow.dflow INFO: Task 4122 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3baedfc3a73d29ace10d9df36bf363fe with result from task 4122 parsl.dataflow.dflow DEBUG: Task 4122 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d1df3b93-9477-48e9-867c-ab05e801a187_isrForCrosstalkSources_996987_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4122 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/d1df3b93-9477-48e9-867c-ab05e801a187_isrForCrosstalkSources_996987_49.stderr parsl.dataflow.dflow INFO: Task 4392 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b156930a8c6cb64d8fea165a1e4d81b3 with result from task 4392 parsl.dataflow.dflow DEBUG: Task 4392 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0b9627d5-8c57-4aa0-9b4e-244ee2b699fa_isrForCrosstalkSources_997012_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4392 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0b9627d5-8c57-4aa0-9b4e-244ee2b699fa_isrForCrosstalkSources_997012_13.stderr parsl.dataflow.dflow INFO: Task 4353 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7f1637a32225b7b904cbe52fc7a1b81e with result from task 4353 parsl.dataflow.dflow DEBUG: Task 4353 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/29342b58-221d-4960-916f-64c9ada8232e_isrForCrosstalkSources_997016_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4353 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/29342b58-221d-4960-916f-64c9ada8232e_isrForCrosstalkSources_997016_19.stderr parsl.dataflow.dflow INFO: Task 4248 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bcc5473212ccdf75c764021b7545045b with result from task 4248 parsl.dataflow.dflow DEBUG: Task 4248 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5a519c36-4ec3-4c28-ae2f-9642f9cc8a7c_isrForCrosstalkSources_996987_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4248 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/5a519c36-4ec3-4c28-ae2f-9642f9cc8a7c_isrForCrosstalkSources_996987_29.stderr parsl.dataflow.dflow INFO: Task 4441 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4d2dbe2d50c072eefd198b0ea5dd35e9 with result from task 4441 parsl.dataflow.dflow DEBUG: Task 4441 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/54e86e31-e60f-438d-9aea-c677ebc853b1_isrForCrosstalkSources_997007_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4441 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/54e86e31-e60f-438d-9aea-c677ebc853b1_isrForCrosstalkSources_997007_34.stderr parsl.dataflow.dflow INFO: Task 4365 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dcefdd9f87210cca781065cee7117324 with result from task 4365 parsl.dataflow.dflow DEBUG: Task 4365 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/563c1e8a-0042-4d75-b6c8-0b4778b502a7_isrForCrosstalkSources_996966_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4365 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/563c1e8a-0042-4d75-b6c8-0b4778b502a7_isrForCrosstalkSources_996966_33.stderr parsl.dataflow.dflow INFO: Task 4447 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d0c7441d9341b6b12115bdaf0018ff7 with result from task 4447 parsl.dataflow.dflow DEBUG: Task 4447 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/37add00c-bc0c-42c5-b257-ef8264e95a41_isrForCrosstalkSources_997010_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4447 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/37add00c-bc0c-42c5-b257-ef8264e95a41_isrForCrosstalkSources_997010_35.stderr parsl.dataflow.dflow INFO: Task 4258 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0590655d8824f771212ec7db471fb494 with result from task 4258 parsl.dataflow.dflow DEBUG: Task 4258 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/5a1b9e49-1591-4ddd-8b57-0c19704a7d36_isrForCrosstalkSources_996986_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4258 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/5a1b9e49-1591-4ddd-8b57-0c19704a7d36_isrForCrosstalkSources_996986_32.stderr parsl.dataflow.dflow INFO: Task 4293 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 33b0c288e8dbf619e232b259837103e0 with result from task 4293 parsl.dataflow.dflow DEBUG: Task 4293 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/58daa97e-70c9-4910-980a-f9ddead62443_isrForCrosstalkSources_997012_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4293 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/58daa97e-70c9-4910-980a-f9ddead62443_isrForCrosstalkSources_997012_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:51 proc_lsst.multi:146 [INFO] found job 17075254 in provider checkpoint proc_lsst.multi INFO: found job 17075254 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4261 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d31f094ffb2864b736856dc3d73fbf81 with result from task 4261 parsl.dataflow.dflow DEBUG: Task 4261 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/cff37941-d50d-49c4-bf0f-023177e91dea_isrForCrosstalkSources_997011_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4261 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/cff37941-d50d-49c4-bf0f-023177e91dea_isrForCrosstalkSources_997011_24.stderr parsl.dataflow.dflow INFO: Task 4509 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f286d7f4401b6cfbfac400b89cd878a0 with result from task 4509 parsl.dataflow.dflow DEBUG: Task 4509 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f4c05ffe-7c89-481e-bd46-64f9aca430e5_isrForCrosstalkSources_997031_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4509 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f4c05ffe-7c89-481e-bd46-64f9aca430e5_isrForCrosstalkSources_997031_14.stderr parsl.dataflow.dflow INFO: Task 4446 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a21e4bc21d0975b3aa398b2560f11680 with result from task 4446 parsl.dataflow.dflow DEBUG: Task 4446 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/f5b300ef-288e-4858-a702-f98c337b2d79_isrForCrosstalkSources_996954_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4446 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/f5b300ef-288e-4858-a702-f98c337b2d79_isrForCrosstalkSources_996954_30.stderr parsl.dataflow.dflow INFO: Task 4499 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f87035d772fce6c6d55b510d604828c with result from task 4499 parsl.dataflow.dflow DEBUG: Task 4499 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/01ee553a-0093-44fc-b315-fe59de80c9ed_isrForCrosstalkSources_996953_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4499 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/01ee553a-0093-44fc-b315-fe59de80c9ed_isrForCrosstalkSources_996953_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:52 proc_lsst.multi:146 [INFO] found job 17075255 in provider checkpoint proc_lsst.multi INFO: found job 17075255 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4457 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c4f99d65e682651b67f75b242db06b42 with result from task 4457 parsl.dataflow.dflow DEBUG: Task 4457 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/fbe8dfb2-da21-48e8-9e45-98e9c62a2955_isrForCrosstalkSources_997018_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4457 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/fbe8dfb2-da21-48e8-9e45-98e9c62a2955_isrForCrosstalkSources_997018_20.stderr parsl.dataflow.dflow INFO: Task 4335 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10bad021730db682e65a1f9c3b34e04a with result from task 4335 parsl.dataflow.dflow DEBUG: Task 4335 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/f499209e-42d1-4c83-8634-a20640918968_isrForCrosstalkSources_997023_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4335 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/f499209e-42d1-4c83-8634-a20640918968_isrForCrosstalkSources_997023_9.stderr parsl.dataflow.dflow INFO: Task 4487 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 463d45b64af6e3d7529d5d6aa35b13ed with result from task 4487 parsl.dataflow.dflow DEBUG: Task 4487 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/4971d94b-517f-4fbf-9128-20b06cd1f375_isrForCrosstalkSources_996981_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4487 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/4971d94b-517f-4fbf-9128-20b06cd1f375_isrForCrosstalkSources_996981_38.stderr parsl.dataflow.dflow INFO: Task 4253 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1e5c4e957283ab93782ba756b26483f with result from task 4253 parsl.dataflow.dflow DEBUG: Task 4253 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1cf65b55-2d5d-45c0-9444-6d5c73193e08_isrForCrosstalkSources_997003_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4253 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/1cf65b55-2d5d-45c0-9444-6d5c73193e08_isrForCrosstalkSources_997003_18.stderr parsl.dataflow.dflow INFO: Task 4436 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d1dea7de5dc562d6338bc8443d55945c with result from task 4436 parsl.dataflow.dflow DEBUG: Task 4436 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/db38b9e4-e7e1-4a47-b4f1-6e0dca2b1d95_isrForCrosstalkSources_997017_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4436 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/db38b9e4-e7e1-4a47-b4f1-6e0dca2b1d95_isrForCrosstalkSources_997017_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:53 proc_lsst.multi:146 [INFO] found job 17075256 in provider checkpoint proc_lsst.multi INFO: found job 17075256 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4222 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 680783e25fd63149b9ceeccd68c85109 with result from task 4222 parsl.dataflow.dflow DEBUG: Task 4222 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/6c75c8bc-26b9-4ecc-a996-eb8da8faf4a4_isrForCrosstalkSources_997001_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4222 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/6c75c8bc-26b9-4ecc-a996-eb8da8faf4a4_isrForCrosstalkSources_997001_45.stderr parsl.dataflow.dflow INFO: Task 4100 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a2c396244b9fa04b099884a147e1dcc with result from task 4100 parsl.dataflow.dflow DEBUG: Task 4100 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d99e00bf-109f-47ae-9d74-6b12c1632abf_isrForCrosstalkSources_997004_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4100 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/d99e00bf-109f-47ae-9d74-6b12c1632abf_isrForCrosstalkSources_997004_37.stderr parsl.dataflow.dflow INFO: Task 4530 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ed30302a851e5117fc1cec11f96be6f1 with result from task 4530 parsl.dataflow.dflow DEBUG: Task 4530 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c77911a3-ff58-4836-9ede-04a6b06030af_isrForCrosstalkSources_997014_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4530 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c77911a3-ff58-4836-9ede-04a6b06030af_isrForCrosstalkSources_997014_14.stderr parsl.dataflow.dflow INFO: Task 4518 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c43407f679169bd50a3e1a4cc3cede7c with result from task 4518 parsl.dataflow.dflow DEBUG: Task 4518 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d85622f4-b947-4340-be4a-54dd37c2d7f7_isrForCrosstalkSources_996989_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4518 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/d85622f4-b947-4340-be4a-54dd37c2d7f7_isrForCrosstalkSources_996989_49.stderr parsl.dataflow.dflow INFO: Task 4312 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 268d1639a388ff90af787dee8bc8eb20 with result from task 4312 parsl.dataflow.dflow DEBUG: Task 4312 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/129db217-7d1e-43ba-949a-707409ae4f4a_isrForCrosstalkSources_997019_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4312 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/129db217-7d1e-43ba-949a-707409ae4f4a_isrForCrosstalkSources_997019_46.stderr parsl.dataflow.dflow INFO: Task 4323 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4f324c2d7c47540691d0c27ca5f93715 with result from task 4323 parsl.dataflow.dflow DEBUG: Task 4323 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/aa8319c4-d2c2-4aed-afbe-b6a474b195db_isrForCrosstalkSources_996983_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4323 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/aa8319c4-d2c2-4aed-afbe-b6a474b195db_isrForCrosstalkSources_996983_33.stderr parsl.dataflow.dflow INFO: Task 4409 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a0006fbb7bb529a184e461e88a199fb with result from task 4409 parsl.dataflow.dflow DEBUG: Task 4409 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b53a540e-3b5d-4125-9734-4ca7e0fc660b_isrForCrosstalkSources_997022_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4409 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/b53a540e-3b5d-4125-9734-4ca7e0fc660b_isrForCrosstalkSources_997022_27.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:53 proc_lsst.multi:146 [INFO] found job 17075257 in provider checkpoint proc_lsst.multi INFO: found job 17075257 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4396 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f19f0e9a6ed5bc2a72caa7ad6708943f with result from task 4396 parsl.dataflow.dflow DEBUG: Task 4396 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/f5d12716-b830-4c68-9c59-8a322cea0b8e_isrForCrosstalkSources_997002_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4396 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/f5d12716-b830-4c68-9c59-8a322cea0b8e_isrForCrosstalkSources_997002_33.stderr parsl.dataflow.dflow INFO: Task 4395 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25b00943589566006f4b672d7080c6e3 with result from task 4395 parsl.dataflow.dflow DEBUG: Task 4395 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/32ceae19-a55f-42d6-939c-10581b49d84e_isrForCrosstalkSources_997007_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4395 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/32ceae19-a55f-42d6-939c-10581b49d84e_isrForCrosstalkSources_997007_27.stderr parsl.dataflow.dflow INFO: Task 4488 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c1a374f2ef27f6174920571cde46f234 with result from task 4488 parsl.dataflow.dflow DEBUG: Task 4488 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/e3528487-b9c2-4b97-954a-51df8df41494_isrForCrosstalkSources_996981_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4488 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/e3528487-b9c2-4b97-954a-51df8df41494_isrForCrosstalkSources_996981_49.stderr parsl.dataflow.dflow INFO: Task 4076 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ebadfa49cecf1ba6a0aa1502a88dc294 with result from task 4076 parsl.dataflow.dflow DEBUG: Task 4076 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1acadc24-de64-43be-9160-fa266740c9d8_isrForCrosstalkSources_996977_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1acadc24-de64-43be-9160-fa266740c9d8_isrForCrosstalkSources_996977_5.stderr parsl.dataflow.dflow INFO: Task 4405 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26922c1ad5ec88551577f9f523db06d2 with result from task 4405 parsl.dataflow.dflow DEBUG: Task 4405 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/ec32f988-0a5c-4dda-9278-1f02e3990036_isrForCrosstalkSources_996955_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4405 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/ec32f988-0a5c-4dda-9278-1f02e3990036_isrForCrosstalkSources_996955_2.stderr parsl.dataflow.dflow INFO: Task 4480 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7809e5dfb5d6ca904bc7aaecd3f41f2 with result from task 4480 parsl.dataflow.dflow DEBUG: Task 4480 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3b863b40-a1b3-42fe-9d37-c3d258205951_isrForCrosstalkSources_996967_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4480 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/3b863b40-a1b3-42fe-9d37-c3d258205951_isrForCrosstalkSources_996967_30.stderr parsl.dataflow.dflow INFO: Task 4515 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 506e05e597780455dd083ccc9063f5b4 with result from task 4515 parsl.dataflow.dflow DEBUG: Task 4515 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/54d1db09-f92b-42bb-ba38-f06938f0dc3e_isrForCrosstalkSources_997011_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4515 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/54d1db09-f92b-42bb-ba38-f06938f0dc3e_isrForCrosstalkSources_997011_6.stderr parsl.dataflow.dflow INFO: Task 4407 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd25b9e46d978394b5cc0e2f07f22176 with result from task 4407 parsl.dataflow.dflow DEBUG: Task 4407 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/49681749-34fd-4cc8-9939-09afbf5d88fa_isrForCrosstalkSources_996992_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4407 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/49681749-34fd-4cc8-9939-09afbf5d88fa_isrForCrosstalkSources_996992_14.stderr parsl.dataflow.dflow INFO: Task 4459 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ed7b890a4876489b2455cabf382df68 with result from task 4459 parsl.dataflow.dflow DEBUG: Task 4459 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/016725d3-bad2-47f9-a8bf-7be53077b111_isrForCrosstalkSources_996954_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4459 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/016725d3-bad2-47f9-a8bf-7be53077b111_isrForCrosstalkSources_996954_4.stderr parsl.dataflow.dflow INFO: Task 4529 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 030131a86a68c06be68a7d8521de0e8e with result from task 4529 parsl.dataflow.dflow DEBUG: Task 4529 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/664ffb9a-4fbf-4a7d-b5c9-00cac9c19e88_isrForCrosstalkSources_996983_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4529 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/664ffb9a-4fbf-4a7d-b5c9-00cac9c19e88_isrForCrosstalkSources_996983_24.stderr parsl.dataflow.dflow INFO: Task 4511 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e694a6dc2cb9363bc7af4580b0118a01 with result from task 4511 parsl.dataflow.dflow DEBUG: Task 4511 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/14ef5dde-e26b-4565-8dec-fefe4ca146ec_isrForCrosstalkSources_996999_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4511 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/14ef5dde-e26b-4565-8dec-fefe4ca146ec_isrForCrosstalkSources_996999_2.stderr parsl.dataflow.dflow INFO: Task 4507 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f5f92d8d49b245ae4d30188117f986b with result from task 4507 parsl.dataflow.dflow DEBUG: Task 4507 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/99ea1653-9e19-4653-b29a-2a7ecbd13f45_isrForCrosstalkSources_997029_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4507 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/99ea1653-9e19-4653-b29a-2a7ecbd13f45_isrForCrosstalkSources_997029_30.stderr parsl.dataflow.dflow INFO: Task 4434 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c80d149dced7219666ac114f8fa73282 with result from task 4434 parsl.dataflow.dflow DEBUG: Task 4434 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/ddede68c-cc46-46ce-a430-9abe9e40b956_isrForCrosstalkSources_996985_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4434 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/ddede68c-cc46-46ce-a430-9abe9e40b956_isrForCrosstalkSources_996985_20.stderr parsl.dataflow.dflow INFO: Task 4344 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da2c66875458ea4245e96f703ebd7a6f with result from task 4344 parsl.dataflow.dflow DEBUG: Task 4344 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/a0a470ad-08f1-4011-bdc9-abcfcc066d3e_isrForCrosstalkSources_996966_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4344 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/a0a470ad-08f1-4011-bdc9-abcfcc066d3e_isrForCrosstalkSources_996966_26.stderr parsl.dataflow.dflow INFO: Task 4437 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a3885d64143b708c6ff181ac625fd5d7 with result from task 4437 parsl.dataflow.dflow DEBUG: Task 4437 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/66b3e468-8ae2-4876-8569-ca47fa82ac3d_isrForCrosstalkSources_997026_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4437 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/66b3e468-8ae2-4876-8569-ca47fa82ac3d_isrForCrosstalkSources_997026_18.stderr parsl.dataflow.dflow INFO: Task 4386 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae55b70f84c5f1099054ac948f543755 with result from task 4386 parsl.dataflow.dflow DEBUG: Task 4386 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0f38e91a-8779-4786-a307-009c6fafb2ad_isrForCrosstalkSources_997021_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4386 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/0f38e91a-8779-4786-a307-009c6fafb2ad_isrForCrosstalkSources_997021_39.stderr parsl.dataflow.dflow INFO: Task 4341 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 025af1e120f4a6c462b770347b031698 with result from task 4341 parsl.dataflow.dflow DEBUG: Task 4341 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/060cb5b1-9717-4eb8-8872-45412d697828_isrForCrosstalkSources_996960_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4341 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/060cb5b1-9717-4eb8-8872-45412d697828_isrForCrosstalkSources_996960_39.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:56 proc_lsst.multi:146 [INFO] found job 17075258 in provider checkpoint proc_lsst.multi INFO: found job 17075258 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4542 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4274fd115cae6589c2b80eb5c1865481 with result from task 4542 parsl.dataflow.dflow DEBUG: Task 4542 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/cd684355-0e3c-475e-a3c1-340064b1379b_isrForCrosstalkSources_996962_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4542 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/cd684355-0e3c-475e-a3c1-340064b1379b_isrForCrosstalkSources_996962_2.stderr parsl.dataflow.dflow INFO: Task 4265 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e0a7972a28f2a469bb9ab5ec86aaea78 with result from task 4265 parsl.dataflow.dflow DEBUG: Task 4265 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/eb38b678-1907-4a12-bd9a-0a5d1ecf1b90_isrForCrosstalkSources_997027_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4265 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/eb38b678-1907-4a12-bd9a-0a5d1ecf1b90_isrForCrosstalkSources_997027_43.stderr parsl.dataflow.dflow INFO: Task 4302 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 28531514f8c3341e0b9c102abc926568 with result from task 4302 parsl.dataflow.dflow DEBUG: Task 4302 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/dd4ce919-4530-4d70-9173-d70ffe830143_isrForCrosstalkSources_996993_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4302 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/dd4ce919-4530-4d70-9173-d70ffe830143_isrForCrosstalkSources_996993_51.stderr parsl.dataflow.dflow INFO: Task 4360 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d6d308be608325171355b32b83bbbee with result from task 4360 parsl.dataflow.dflow DEBUG: Task 4360 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fb9777d8-838e-4773-98bb-762efb35b279_isrForCrosstalkSources_996976_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4360 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fb9777d8-838e-4773-98bb-762efb35b279_isrForCrosstalkSources_996976_24.stderr parsl.dataflow.dflow INFO: Task 4526 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f619c80ea3f1e673bf2b69668f15df50 with result from task 4526 parsl.dataflow.dflow DEBUG: Task 4526 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/56263901-1e69-4056-862d-1ae1ad736fa6_isrForCrosstalkSources_997015_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4526 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/56263901-1e69-4056-862d-1ae1ad736fa6_isrForCrosstalkSources_997015_21.stderr parsl.dataflow.dflow INFO: Task 4239 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 98d8dffc25be4bd219b936851f6739ef with result from task 4239 parsl.dataflow.dflow DEBUG: Task 4239 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/403f446a-af80-46cb-bbce-d1af4713b520_isrForCrosstalkSources_996998_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4239 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/403f446a-af80-46cb-bbce-d1af4713b520_isrForCrosstalkSources_996998_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:56 proc_lsst.multi:146 [INFO] found job 17075264 in provider checkpoint proc_lsst.multi INFO: found job 17075264 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4497 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa3e134f961192251c28c1c8cfb4b49e with result from task 4497 parsl.dataflow.dflow DEBUG: Task 4497 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/29750e47-42d5-4ea8-972b-7251ca5e7c74_isrForCrosstalkSources_997006_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4497 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/29750e47-42d5-4ea8-972b-7251ca5e7c74_isrForCrosstalkSources_997006_1.stderr parsl.dataflow.dflow INFO: Task 4397 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 92985a837ab004024fc6d819373a4e58 with result from task 4397 parsl.dataflow.dflow DEBUG: Task 4397 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/89babaae-eff8-4779-afa8-e37636543322_isrForCrosstalkSources_997014_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4397 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/89babaae-eff8-4779-afa8-e37636543322_isrForCrosstalkSources_997014_56.stderr parsl.dataflow.dflow INFO: Task 4244 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe8dd61343f3f6f52508fb8d5549c653 with result from task 4244 parsl.dataflow.dflow DEBUG: Task 4244 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/c09fb3e9-f663-4c0c-b577-ffffb1dc8b11_isrForCrosstalkSources_996988_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4244 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/c09fb3e9-f663-4c0c-b577-ffffb1dc8b11_isrForCrosstalkSources_996988_21.stderr parsl.dataflow.dflow INFO: Task 4067 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef87a944d6fe1dc64e7299d7854de843 with result from task 4067 parsl.dataflow.dflow DEBUG: Task 4067 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/6125ebd7-f2fb-4a55-b9e4-a252051294db_isrForCrosstalkSources_997033_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/6125ebd7-f2fb-4a55-b9e4-a252051294db_isrForCrosstalkSources_997033_50.stderr parsl.dataflow.dflow INFO: Task 4548 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53e4862f87000b3bee8a4fc6df24ab0b with result from task 4548 parsl.dataflow.dflow DEBUG: Task 4548 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a10e2f9d-c1e1-4c71-82b8-f6ad37ee0f38_isrForCrosstalkSources_996983_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4548 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/a10e2f9d-c1e1-4c71-82b8-f6ad37ee0f38_isrForCrosstalkSources_996983_22.stderr parsl.dataflow.dflow INFO: Task 4262 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3011fff44afaab9bd49e627de67eec00 with result from task 4262 parsl.dataflow.dflow DEBUG: Task 4262 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a98710d7-d1d6-41b0-bccf-535dc0fe5e95_isrForCrosstalkSources_997026_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4262 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/a98710d7-d1d6-41b0-bccf-535dc0fe5e95_isrForCrosstalkSources_997026_9.stderr parsl.dataflow.dflow INFO: Task 4317 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd1944ff0871283512674a38089c8716 with result from task 4317 parsl.dataflow.dflow DEBUG: Task 4317 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/4e93dc14-a76d-4405-ba35-5979e1eccd0c_isrForCrosstalkSources_997010_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4317 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/4e93dc14-a76d-4405-ba35-5979e1eccd0c_isrForCrosstalkSources_997010_10.stderr parsl.dataflow.dflow INFO: Task 4283 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2409fd723b34cb842fbd79bd0897b3a7 with result from task 4283 parsl.dataflow.dflow DEBUG: Task 4283 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f0e5fe4f-5d6b-4efe-b903-ba83eeae7484_isrForCrosstalkSources_996975_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4283 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/f0e5fe4f-5d6b-4efe-b903-ba83eeae7484_isrForCrosstalkSources_996975_36.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:57 proc_lsst.multi:146 [INFO] found job 17075265 in provider checkpoint proc_lsst.multi INFO: found job 17075265 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4452 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fcb2f0a5fa64e5b76e6c288307031e1 with result from task 4452 parsl.dataflow.dflow DEBUG: Task 4452 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ba485dc9-07a9-4ad3-ac79-96a2d39e99e7_isrForCrosstalkSources_996981_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4452 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ba485dc9-07a9-4ad3-ac79-96a2d39e99e7_isrForCrosstalkSources_996981_43.stderr parsl.dataflow.dflow INFO: Task 4351 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry edb8e8b71c40f2efce66b99ab8d63fad with result from task 4351 parsl.dataflow.dflow DEBUG: Task 4351 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/97cdb5c9-de15-438d-b5d0-c6245e51d19b_isrForCrosstalkSources_997012_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4351 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/97cdb5c9-de15-438d-b5d0-c6245e51d19b_isrForCrosstalkSources_997012_9.stderr parsl.dataflow.dflow INFO: Task 4415 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58c4014c5f80d97ffd3044477378d016 with result from task 4415 parsl.dataflow.dflow DEBUG: Task 4415 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/5b6233d9-1043-456e-94df-24c6d6e84427_isrForCrosstalkSources_996984_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4415 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/5b6233d9-1043-456e-94df-24c6d6e84427_isrForCrosstalkSources_996984_17.stderr parsl.dataflow.dflow INFO: Task 4272 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1f71834ee8526fac7e0b731ce25b01ad with result from task 4272 parsl.dataflow.dflow DEBUG: Task 4272 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1e4d37c6-f9fc-43e4-8f23-801482c14d4f_isrForCrosstalkSources_996967_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4272 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/1e4d37c6-f9fc-43e4-8f23-801482c14d4f_isrForCrosstalkSources_996967_21.stderr parsl.dataflow.dflow INFO: Task 4559 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ac2ae1ae073e7d79b2233d39edcda35 with result from task 4559 parsl.dataflow.dflow DEBUG: Task 4559 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/00249d29-ead9-43de-9ab7-8de9816cd442_isrForCrosstalkSources_997016_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4559 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/00249d29-ead9-43de-9ab7-8de9816cd442_isrForCrosstalkSources_997016_4.stderr parsl.dataflow.dflow INFO: Task 4580 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48b183829b3d7b4514f60cbfa10ca007 with result from task 4580 parsl.dataflow.dflow DEBUG: Task 4580 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3a2c8306-4be0-45b4-9bf2-7a68b701c62e_isrForCrosstalkSources_997001_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4580 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3a2c8306-4be0-45b4-9bf2-7a68b701c62e_isrForCrosstalkSources_997001_27.stderr parsl.dataflow.dflow INFO: Task 4549 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 266e47ac374a216e75205bac80f9368d with result from task 4549 parsl.dataflow.dflow DEBUG: Task 4549 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/52d3ef4c-52f5-4085-b478-20d703d2eeb6_isrForCrosstalkSources_996969_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4549 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/52d3ef4c-52f5-4085-b478-20d703d2eeb6_isrForCrosstalkSources_996969_10.stderr parsl.dataflow.dflow INFO: Task 4510 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16b040a0197b8c316552d249f7c09061 with result from task 4510 parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:58 proc_lsst.multi:146 [INFO] found job 17075273 in provider checkpoint proc_lsst.multi INFO: found job 17075273 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4510 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/38068b8c-d7eb-4938-9136-ac984f9ea849_isrForCrosstalkSources_997011_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4510 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/38068b8c-d7eb-4938-9136-ac984f9ea849_isrForCrosstalkSources_997011_61.stderr parsl.dataflow.dflow INFO: Task 4153 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ce872ec3301e4cddbbd69cf3087b2bd with result from task 4153 parsl.dataflow.dflow DEBUG: Task 4153 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8c2d38ca-17cd-4f58-be55-bacdfa856ed1_isrForCrosstalkSources_996956_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4153 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/8c2d38ca-17cd-4f58-be55-bacdfa856ed1_isrForCrosstalkSources_996956_54.stderr parsl.dataflow.dflow INFO: Task 4378 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a391bb080436efd8ad9219fe8bde4d29 with result from task 4378 parsl.dataflow.dflow DEBUG: Task 4378 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f325644-ca42-4ca4-8cb9-37801a9a896b_isrForCrosstalkSources_996994_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4378 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/7f325644-ca42-4ca4-8cb9-37801a9a896b_isrForCrosstalkSources_996994_53.stderr parsl.dataflow.dflow INFO: Task 4364 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 97c7989638b5fa1b4c52c98e3cab6f66 with result from task 4364 parsl.dataflow.dflow DEBUG: Task 4364 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ab2c92c9-5338-44cf-82b0-2d354679c7c2_isrForCrosstalkSources_996980_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4364 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/ab2c92c9-5338-44cf-82b0-2d354679c7c2_isrForCrosstalkSources_996980_37.stderr parsl.dataflow.dflow INFO: Task 4568 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5d936306ae0c72008da98a7c9cac564 with result from task 4568 parsl.dataflow.dflow DEBUG: Task 4568 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/a6027c55-988b-4ddb-adf9-8754c79f049a_isrForCrosstalkSources_996963_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4568 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/a6027c55-988b-4ddb-adf9-8754c79f049a_isrForCrosstalkSources_996963_8.stderr parsl.dataflow.dflow INFO: Task 4570 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c4cddb4177cf16a2cb5cc55218d8268 with result from task 4570 parsl.dataflow.dflow DEBUG: Task 4570 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/b1657f4d-4ea2-4727-9619-04384a563a3c_isrForCrosstalkSources_996987_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4570 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/b1657f4d-4ea2-4727-9619-04384a563a3c_isrForCrosstalkSources_996987_27.stderr parsl.dataflow.dflow INFO: Task 4569 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ea1f1b53992bb2de95b0f877ab2ab97 with result from task 4569 parsl.dataflow.dflow DEBUG: Task 4569 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/49cf20ee-ac31-463d-84d7-08b43d2dc1a4_isrForCrosstalkSources_996970_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4569 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/49cf20ee-ac31-463d-84d7-08b43d2dc1a4_isrForCrosstalkSources_996970_46.stderr parsl.dataflow.dflow INFO: Task 4553 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f70699e4962fe8edb63af6ac169157c1 with result from task 4553 parsl.dataflow.dflow DEBUG: Task 4553 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/511aacff-c034-4b3f-b93b-b58fb1ed18fd_isrForCrosstalkSources_997010_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4553 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/511aacff-c034-4b3f-b93b-b58fb1ed18fd_isrForCrosstalkSources_997010_53.stderr parsl.dataflow.dflow INFO: Task 4485 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c56c06a1096323d2707a52e3100f929 with result from task 4485 parsl.dataflow.dflow DEBUG: Task 4485 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/15b197d4-6133-471a-bb83-7b71ce32389d_isrForCrosstalkSources_996962_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4485 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/15b197d4-6133-471a-bb83-7b71ce32389d_isrForCrosstalkSources_996962_50.stderr parsl.dataflow.dflow INFO: Task 4482 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 138bbb29a5b240fc8434f3fbb5ed1117 with result from task 4482 parsl.dataflow.dflow DEBUG: Task 4482 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/cf1bb616-06fa-4d3e-91c6-3e0fcc1e0731_isrForCrosstalkSources_997026_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4482 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/cf1bb616-06fa-4d3e-91c6-3e0fcc1e0731_isrForCrosstalkSources_997026_57.stderr parsl.dataflow.dflow INFO: Task 4435 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a10dc3f5d28f028f0db63930f0b509c8 with result from task 4435 parsl.dataflow.dflow DEBUG: Task 4435 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7080564a-8094-4048-b5f9-c682d01b6da8_isrForCrosstalkSources_996973_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4435 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/7080564a-8094-4048-b5f9-c682d01b6da8_isrForCrosstalkSources_996973_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 13:59:59 proc_lsst.multi:146 [INFO] found job 17075274 in provider checkpoint proc_lsst.multi INFO: found job 17075274 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4381 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90c4d6c39467d647fdd0375e62f8b21c with result from task 4381 parsl.dataflow.dflow DEBUG: Task 4381 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fa8a4834-5113-45f7-b870-8501cba7becd_isrForCrosstalkSources_996976_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4381 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/fa8a4834-5113-45f7-b870-8501cba7becd_isrForCrosstalkSources_996976_33.stderr parsl.dataflow.dflow INFO: Task 4367 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df094b900aebd0a3de1cfa01f7978b3c with result from task 4367 parsl.dataflow.dflow DEBUG: Task 4367 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/13f92840-d2cd-4aee-bc2f-296af3e79704_isrForCrosstalkSources_997022_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4367 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/13f92840-d2cd-4aee-bc2f-296af3e79704_isrForCrosstalkSources_997022_11.stderr parsl.dataflow.dflow INFO: Task 4203 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 652dabd4e8d4fe43135bda91b7fd75d1 with result from task 4203 parsl.dataflow.dflow DEBUG: Task 4203 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/cacb8d7c-6750-4b3e-bd61-b49aac149092_isrForCrosstalkSources_996982_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4203 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/cacb8d7c-6750-4b3e-bd61-b49aac149092_isrForCrosstalkSources_996982_48.stderr parsl.dataflow.dflow INFO: Task 4552 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a7f9723650c759f0f245d65e89395122 with result from task 4552 parsl.dataflow.dflow DEBUG: Task 4552 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/37b0a483-ef97-46f0-9cb3-8933a6aa3b76_isrForCrosstalkSources_997018_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4552 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/37b0a483-ef97-46f0-9cb3-8933a6aa3b76_isrForCrosstalkSources_997018_7.stderr parsl.dataflow.dflow INFO: Task 4138 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef5bdd0041ae955fde9a9998839af27a with result from task 4138 parsl.dataflow.dflow DEBUG: Task 4138 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/caddeb77-0558-40b9-abf7-6312963ae369_isrForCrosstalkSources_996961_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4138 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/caddeb77-0558-40b9-abf7-6312963ae369_isrForCrosstalkSources_996961_31.stderr parsl.dataflow.dflow INFO: Task 4574 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ae2e421c8ce2cde5a4243426d37df008 with result from task 4574 parsl.dataflow.dflow DEBUG: Task 4574 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/48a309b6-8602-45e0-8f4a-3bd661a7449b_isrForCrosstalkSources_997015_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4574 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/48a309b6-8602-45e0-8f4a-3bd661a7449b_isrForCrosstalkSources_997015_43.stderr parsl.dataflow.dflow INFO: Task 4463 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abacb36069f6494742da607c5c4e54cf with result from task 4463 parsl.dataflow.dflow DEBUG: Task 4463 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/b2232f25-04a5-4b03-bb00-d2c4c51c98ac_isrForCrosstalkSources_996956_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4463 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/b2232f25-04a5-4b03-bb00-d2c4c51c98ac_isrForCrosstalkSources_996956_38.stderr parsl.dataflow.dflow INFO: Task 4334 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 72b7a77d0591350c75ac51c3e4af4641 with result from task 4334 parsl.dataflow.dflow DEBUG: Task 4334 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/65f9cb3e-b55e-45ee-9533-44cf52c9be2d_isrForCrosstalkSources_996993_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4334 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/65f9cb3e-b55e-45ee-9533-44cf52c9be2d_isrForCrosstalkSources_996993_45.stderr parsl.dataflow.dflow INFO: Task 4564 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d205e9e7e201aae43d2f9aac2300005b with result from task 4564 parsl.dataflow.dflow DEBUG: Task 4564 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8cde139f-f98f-4c37-a207-8c1bcbd69183_isrForCrosstalkSources_996997_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4564 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8cde139f-f98f-4c37-a207-8c1bcbd69183_isrForCrosstalkSources_996997_22.stderr parsl.dataflow.dflow INFO: Task 4406 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1edcfdaa6c1e4e01c34c7356cc1e8fd5 with result from task 4406 parsl.dataflow.dflow DEBUG: Task 4406 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1cca67a3-c358-4eec-a559-48b58987f7e2_isrForCrosstalkSources_996983_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4406 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/1cca67a3-c358-4eec-a559-48b58987f7e2_isrForCrosstalkSources_996983_49.stderr parsl.dataflow.dflow INFO: Task 4419 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f9ec08f96b265f68c0444b22d5920690 with result from task 4419 parsl.dataflow.dflow DEBUG: Task 4419 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2770f4fa-f399-4757-9687-a317ded134f1_isrForCrosstalkSources_996990_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4419 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/2770f4fa-f399-4757-9687-a317ded134f1_isrForCrosstalkSources_996990_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:00 proc_lsst.multi:146 [INFO] found job 17075275 in provider checkpoint proc_lsst.multi INFO: found job 17075275 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4246 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0d847fd6ecffaa9dd9238ef69e8aee83 with result from task 4246 parsl.dataflow.dflow DEBUG: Task 4246 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f307728e-2925-41fc-aed9-c02fc0e4db8f_isrForCrosstalkSources_996966_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4246 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/f307728e-2925-41fc-aed9-c02fc0e4db8f_isrForCrosstalkSources_996966_35.stderr parsl.dataflow.dflow INFO: Task 4325 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 93b9497f2cd3d2696635fb13a17fc7f2 with result from task 4325 parsl.dataflow.dflow DEBUG: Task 4325 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1f5a6f1e-879c-44f2-ae22-3b9a562f45e0_isrForCrosstalkSources_996977_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4325 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/1f5a6f1e-879c-44f2-ae22-3b9a562f45e0_isrForCrosstalkSources_996977_9.stderr parsl.dataflow.dflow INFO: Task 4602 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c42e8cc43ebaa061c9506edabdb9ce21 with result from task 4602 parsl.dataflow.dflow DEBUG: Task 4602 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/210667da-4692-4534-94ce-9b6257aa4abc_isrForCrosstalkSources_997023_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4602 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/210667da-4692-4534-94ce-9b6257aa4abc_isrForCrosstalkSources_997023_20.stderr parsl.dataflow.dflow INFO: Task 4298 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0452f80d6e39ab38f595970f1f43a3dc with result from task 4298 parsl.dataflow.dflow DEBUG: Task 4298 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/74f68542-35be-4ba6-930b-289c65d08f9c_isrForCrosstalkSources_996966_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4298 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/74f68542-35be-4ba6-930b-289c65d08f9c_isrForCrosstalkSources_996966_10.stderr parsl.dataflow.dflow INFO: Task 4324 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f822a7afa193d7f32059cdb5ac645e3d with result from task 4324 parsl.dataflow.dflow DEBUG: Task 4324 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/586dd823-efaa-4f84-8147-257eb15c713c_isrForCrosstalkSources_996962_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4324 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/586dd823-efaa-4f84-8147-257eb15c713c_isrForCrosstalkSources_996962_41.stderr parsl.dataflow.dflow INFO: Task 4295 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2fbff6e844b844064171ee427b89464d with result from task 4295 parsl.dataflow.dflow DEBUG: Task 4295 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1099ac38-0317-41d4-a200-63d2d9845e3c_isrForCrosstalkSources_996958_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4295 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1099ac38-0317-41d4-a200-63d2d9845e3c_isrForCrosstalkSources_996958_51.stderr parsl.dataflow.dflow INFO: Task 4442 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e784af375d6328cba2ce752e6ad4d587 with result from task 4442 parsl.dataflow.dflow DEBUG: Task 4442 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d46eccf9-35a6-4137-aa9b-27297f8be649_isrForCrosstalkSources_997030_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4442 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d46eccf9-35a6-4137-aa9b-27297f8be649_isrForCrosstalkSources_997030_62.stderr parsl.dataflow.dflow INFO: Task 4377 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 43af7fcd07347fd1887823859133ba13 with result from task 4377 parsl.dataflow.dflow DEBUG: Task 4377 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/c434cab6-f6b5-4205-8bc6-ec92caf9de4f_isrForCrosstalkSources_996957_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4377 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/c434cab6-f6b5-4205-8bc6-ec92caf9de4f_isrForCrosstalkSources_996957_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:01 proc_lsst.multi:146 [INFO] found job 17075276 in provider checkpoint proc_lsst.multi INFO: found job 17075276 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4385 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5af068a1741ef426f54640456a74bd20 with result from task 4385 parsl.dataflow.dflow DEBUG: Task 4385 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/324fa248-8458-46d6-b1a4-44094af0381c_isrForCrosstalkSources_996972_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4385 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/324fa248-8458-46d6-b1a4-44094af0381c_isrForCrosstalkSources_996972_15.stderr parsl.dataflow.dflow INFO: Task 4294 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29344119ece6c036c7d2bcd095d0b415 with result from task 4294 parsl.dataflow.dflow DEBUG: Task 4294 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/90659ea9-a39e-489c-95cb-3cfcf92227da_isrForCrosstalkSources_997012_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4294 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/90659ea9-a39e-489c-95cb-3cfcf92227da_isrForCrosstalkSources_997012_60.stderr parsl.dataflow.dflow INFO: Task 4587 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c0ffe22e59a29306d6ce13a52147767d with result from task 4587 parsl.dataflow.dflow DEBUG: Task 4587 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8b68af6a-c1b2-4e72-88d2-e426b74e005a_isrForCrosstalkSources_996967_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4587 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8b68af6a-c1b2-4e72-88d2-e426b74e005a_isrForCrosstalkSources_996967_17.stderr parsl.dataflow.dflow INFO: Task 4476 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bff739b3f3c22e660a815a57264f0aad with result from task 4476 parsl.dataflow.dflow DEBUG: Task 4476 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/43340da4-fd97-45a9-9ec5-ea4a703443ed_isrForCrosstalkSources_997026_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4476 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/43340da4-fd97-45a9-9ec5-ea4a703443ed_isrForCrosstalkSources_997026_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:02 proc_lsst.multi:146 [INFO] found job 17075278 in provider checkpoint proc_lsst.multi INFO: found job 17075278 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4399 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e996717b7526ff70e4234a41f7bb3d02 with result from task 4399 parsl.dataflow.dflow DEBUG: Task 4399 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/4885c83d-bce3-4865-bfcc-a61f09ebabd0_isrForCrosstalkSources_996961_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4399 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/4885c83d-bce3-4865-bfcc-a61f09ebabd0_isrForCrosstalkSources_996961_41.stderr parsl.dataflow.dflow INFO: Task 4558 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb904b7ec5e8000859f3c7d0406d7933 with result from task 4558 parsl.dataflow.dflow DEBUG: Task 4558 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ba3976dd-b02f-42fb-9b08-99f854b0db2b_isrForCrosstalkSources_996994_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4558 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ba3976dd-b02f-42fb-9b08-99f854b0db2b_isrForCrosstalkSources_996994_35.stderr parsl.dataflow.dflow INFO: Task 4326 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea8566fbce0408e78fadb0e2e5baf4e0 with result from task 4326 parsl.dataflow.dflow DEBUG: Task 4326 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/41b752bd-81ba-48b1-902f-83acda1b27e2_isrForCrosstalkSources_997031_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4326 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/41b752bd-81ba-48b1-902f-83acda1b27e2_isrForCrosstalkSources_997031_16.stderr parsl.dataflow.dflow INFO: Task 4290 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90efc874228356e68179b57407cead8f with result from task 4290 parsl.dataflow.dflow DEBUG: Task 4290 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/7c682adf-b85d-4e87-a011-0015a248f869_isrForCrosstalkSources_996960_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4290 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/7c682adf-b85d-4e87-a011-0015a248f869_isrForCrosstalkSources_996960_44.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4503 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 25f1baecc5c52017fde72e608d8d6f87 with result from task 4503 parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4503 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/52e22a4d-446b-41de-b0e2-7c39ab10e2eb_isrForCrosstalkSources_997010_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4503 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/52e22a4d-446b-41de-b0e2-7c39ab10e2eb_isrForCrosstalkSources_997010_40.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:02 proc_lsst.multi:146 [INFO] found job 17075279 in provider checkpoint proc_lsst.multi INFO: found job 17075279 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:03 proc_lsst.multi:146 [INFO] found job 17075280 in provider checkpoint proc_lsst.multi INFO: found job 17075280 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4523 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b1040ee0bbddf4ce72af7f0bae97f60c with result from task 4523 parsl.dataflow.dflow DEBUG: Task 4523 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/408e36d9-dc10-4d32-96ed-d3aa294588bd_isrForCrosstalkSources_996996_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4523 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/408e36d9-dc10-4d32-96ed-d3aa294588bd_isrForCrosstalkSources_996996_19.stderr parsl.dataflow.dflow INFO: Task 4567 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91db5a4814bfd89fa41ce15851b84b1a with result from task 4567 parsl.dataflow.dflow DEBUG: Task 4567 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f5a45dc2-992d-4dc6-8171-8b88896f052e_isrForCrosstalkSources_996990_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4567 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/f5a45dc2-992d-4dc6-8171-8b88896f052e_isrForCrosstalkSources_996990_11.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:03 proc_lsst.multi:146 [INFO] found job 17075281 in provider checkpoint proc_lsst.multi INFO: found job 17075281 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4368 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f3c6fe43b121364144f2634f50d83ed with result from task 4368 parsl.dataflow.dflow DEBUG: Task 4368 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/ea76a04c-35af-42a7-916f-ac32a4435908_isrForCrosstalkSources_996952_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4368 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/ea76a04c-35af-42a7-916f-ac32a4435908_isrForCrosstalkSources_996952_37.stderr parsl.dataflow.dflow INFO: Task 4596 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e5c4756b0d014e3bee1a8b55e5b71001 with result from task 4596 parsl.dataflow.dflow DEBUG: Task 4596 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cac4b099-ab73-46a6-a406-37fc67623edb_isrForCrosstalkSources_996978_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4596 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cac4b099-ab73-46a6-a406-37fc67623edb_isrForCrosstalkSources_996978_7.stderr parsl.dataflow.dflow INFO: Task 4585 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8060b1819d90fec219647dcf16ceaea with result from task 4585 parsl.dataflow.dflow DEBUG: Task 4585 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/7a2090e3-62a3-4e19-a987-c59ed6c4030c_isrForCrosstalkSources_996958_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4585 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/7a2090e3-62a3-4e19-a987-c59ed6c4030c_isrForCrosstalkSources_996958_55.stderr parsl.dataflow.dflow INFO: Task 4513 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d5c85610c750b97badc8ada6565dcf2 with result from task 4513 parsl.dataflow.dflow DEBUG: Task 4513 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/78474556-93db-4e91-b494-46c210507d7d_isrForCrosstalkSources_996958_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4513 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/78474556-93db-4e91-b494-46c210507d7d_isrForCrosstalkSources_996958_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:04 proc_lsst.multi:146 [INFO] found job 17075282 in provider checkpoint proc_lsst.multi INFO: found job 17075282 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4524 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f29976a957ff354efcfdbed7da9f36af with result from task 4524 parsl.dataflow.dflow DEBUG: Task 4524 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/efc1d32c-0ffd-47f8-a954-8dde0c843264_isrForCrosstalkSources_997004_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4524 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/efc1d32c-0ffd-47f8-a954-8dde0c843264_isrForCrosstalkSources_997004_2.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Task 4454 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 4ab0837481677506691408438277c02c with result from task 4454 parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4454 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/c7aec28c-003a-4b63-a6b8-a05c47381574_isrForCrosstalkSources_996990_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4454 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/c7aec28c-003a-4b63-a6b8-a05c47381574_isrForCrosstalkSources_996990_26.stderr parsl.dataflow.dflow INFO: Task 4309 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 503f850a7ec4ad0dc2f3097ee5346609 with result from task 4309 parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:05 proc_lsst.multi:146 [INFO] found job 17075283 in provider checkpoint proc_lsst.multi INFO: found job 17075283 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4309 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/bdb9ec38-9df2-42cc-b879-b8c8e1265685_isrForCrosstalkSources_996965_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4309 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/bdb9ec38-9df2-42cc-b879-b8c8e1265685_isrForCrosstalkSources_996965_62.stderr parsl.dataflow.dflow INFO: Task 4439 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dd478dda5998caa7732830b26702d013 with result from task 4439 parsl.dataflow.dflow DEBUG: Task 4439 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/be44508b-1d91-479f-bc54-702ad721b2fb_isrForCrosstalkSources_996990_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4439 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/be44508b-1d91-479f-bc54-702ad721b2fb_isrForCrosstalkSources_996990_32.stderr parsl.dataflow.dflow INFO: Task 4300 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9930c98ff16b355acea28f44ebdd24da with result from task 4300 parsl.dataflow.dflow DEBUG: Task 4300 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/4541df1e-75a2-4d2f-8d69-143f20620366_isrForCrosstalkSources_996952_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4300 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/4541df1e-75a2-4d2f-8d69-143f20620366_isrForCrosstalkSources_996952_9.stderr parsl.dataflow.dflow INFO: Task 4254 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fe54e1f94f684581b6ea5e4b2407a444 with result from task 4254 parsl.dataflow.dflow DEBUG: Task 4254 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/27788f04-3a6f-47e4-896f-f1f821422ced_isrForCrosstalkSources_996975_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4254 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/27788f04-3a6f-47e4-896f-f1f821422ced_isrForCrosstalkSources_996975_60.stderr parsl.dataflow.dflow INFO: Task 4281 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e941d61b3a314260c828cd600ae106ee with result from task 4281 parsl.dataflow.dflow DEBUG: Task 4281 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/939ea313-3f4a-4603-b8aa-1f69979abc66_isrForCrosstalkSources_997029_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4281 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/939ea313-3f4a-4603-b8aa-1f69979abc66_isrForCrosstalkSources_997029_17.stderr parsl.dataflow.dflow INFO: Task 4361 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c5c20a856479802368de999e18d0999e with result from task 4361 parsl.dataflow.dflow DEBUG: Task 4361 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/942f9dc9-4b2d-49f2-a16f-33c3913b1e9b_isrForCrosstalkSources_996955_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4361 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/942f9dc9-4b2d-49f2-a16f-33c3913b1e9b_isrForCrosstalkSources_996955_12.stderr parsl.dataflow.dflow INFO: Task 4615 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 41b2149b455a1a9d270b165321c0c6bf with result from task 4615 parsl.dataflow.dflow DEBUG: Task 4615 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/fc9496e8-e998-4bec-a0f0-9ac0fa4414fa_isrForCrosstalkSources_997022_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4615 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/fc9496e8-e998-4bec-a0f0-9ac0fa4414fa_isrForCrosstalkSources_997022_5.stderr parsl.dataflow.dflow INFO: Task 4383 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 85259dd9fb5fc405c0145a05cab4089e with result from task 4383 parsl.dataflow.dflow DEBUG: Task 4383 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/82217c1f-ff80-4f7e-80d6-670629e8f2bf_isrForCrosstalkSources_997025_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4383 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/82217c1f-ff80-4f7e-80d6-670629e8f2bf_isrForCrosstalkSources_997025_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:05 proc_lsst.multi:146 [INFO] found job 17075284 in provider checkpoint proc_lsst.multi INFO: found job 17075284 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4398 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 71c3b978fe3a64d9642e142bc6ecca02 with result from task 4398 parsl.dataflow.dflow DEBUG: Task 4398 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/914a8c2b-ba8b-4b85-b905-cf42fa3a12c0_isrForCrosstalkSources_997024_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4398 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/914a8c2b-ba8b-4b85-b905-cf42fa3a12c0_isrForCrosstalkSources_997024_12.stderr parsl.dataflow.dflow INFO: Task 4301 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3787bf645a3da4f1ebcd4632bf25a866 with result from task 4301 parsl.dataflow.dflow DEBUG: Task 4301 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/839f1c1b-4ef3-4c06-845b-0962c7e38ca3_isrForCrosstalkSources_997027_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4301 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/839f1c1b-4ef3-4c06-845b-0962c7e38ca3_isrForCrosstalkSources_997027_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:06 proc_lsst.multi:146 [INFO] found job 17075285 in provider checkpoint proc_lsst.multi INFO: found job 17075285 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4634 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 056efd6dee6b5e19461878116d625a9e with result from task 4634 parsl.dataflow.dflow DEBUG: Task 4634 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/3a484663-3377-402f-aed5-5f5791a9f37e_isrForCrosstalkSources_997011_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4634 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/3a484663-3377-402f-aed5-5f5791a9f37e_isrForCrosstalkSources_997011_36.stderr parsl.dataflow.dflow INFO: Task 4375 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2217e0d18563b23ffe271ba682c3c404 with result from task 4375 parsl.dataflow.dflow DEBUG: Task 4375 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/6b09887d-7a43-4435-a4ef-598ef1824cdc_isrForCrosstalkSources_997028_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4375 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/6b09887d-7a43-4435-a4ef-598ef1824cdc_isrForCrosstalkSources_997028_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4408 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 9e181d3bdc9051194cf9dba8cd1f78aa with result from task 4408 parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4408 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/2fa59036-3c1e-4f4a-a9fd-80c2a1647502_isrForCrosstalkSources_997011_16.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4408 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/2fa59036-3c1e-4f4a-a9fd-80c2a1647502_isrForCrosstalkSources_997011_16.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4613 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry fc6e04a7fd480a9d44093fb1998eb914 with result from task 4613 parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4613 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3b221d3d-5568-4ebe-be85-f69538a87e94_isrForCrosstalkSources_996963_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4613 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/3b221d3d-5568-4ebe-be85-f69538a87e94_isrForCrosstalkSources_996963_21.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:07 proc_lsst.multi:146 [INFO] found job 17075286 in provider checkpoint proc_lsst.multi INFO: found job 17075286 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4362 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 55cf2dd98f359d0f577206be291d6781 with result from task 4362 parsl.dataflow.dflow DEBUG: Task 4362 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/f32b5a3b-9e95-4af5-9e11-60ad194f967d_isrForCrosstalkSources_996991_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4362 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/f32b5a3b-9e95-4af5-9e11-60ad194f967d_isrForCrosstalkSources_996991_11.stderr parsl.dataflow.dflow INFO: Task 4316 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6128804be3295e35df5eca1752ea7a8 with result from task 4316 parsl.dataflow.dflow DEBUG: Task 4316 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3d04cd7b-f3a4-492b-8954-4f38142923de_isrForCrosstalkSources_996959_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4316 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3d04cd7b-f3a4-492b-8954-4f38142923de_isrForCrosstalkSources_996959_6.stderr parsl.dataflow.dflow INFO: Task 4624 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af0358652febefc3a3ea7394b1b45569 with result from task 4624 parsl.dataflow.dflow DEBUG: Task 4624 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/756c07dc-8cb7-4d96-b903-c61dd7c807ea_isrForCrosstalkSources_996988_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4624 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/756c07dc-8cb7-4d96-b903-c61dd7c807ea_isrForCrosstalkSources_996988_31.stderr parsl.dataflow.dflow INFO: Task 4461 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b97b284b559546ea0797221dea32b87d with result from task 4461 parsl.dataflow.dflow DEBUG: Task 4461 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7c8f9dbc-37b2-4a52-9864-9008a0a49e33_isrForCrosstalkSources_997026_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4461 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7c8f9dbc-37b2-4a52-9864-9008a0a49e33_isrForCrosstalkSources_997026_27.stderr parsl.dataflow.dflow INFO: Task 4506 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31b19316e707a33b4ebc83350b88caec with result from task 4506 parsl.dataflow.dflow DEBUG: Task 4506 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/eed28bb0-9cb7-4f7b-92ea-a311109575ef_isrForCrosstalkSources_996974_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4506 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/eed28bb0-9cb7-4f7b-92ea-a311109575ef_isrForCrosstalkSources_996974_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:07 proc_lsst.multi:146 [INFO] found job 17075287 in provider checkpoint proc_lsst.multi INFO: found job 17075287 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4373 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2079e756df10c3ed34522b9a99bb2736 with result from task 4373 parsl.dataflow.dflow DEBUG: Task 4373 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/acbe27b0-298b-46bb-9258-bb0f09b23894_isrForCrosstalkSources_996972_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4373 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/acbe27b0-298b-46bb-9258-bb0f09b23894_isrForCrosstalkSources_996972_37.stderr parsl.dataflow.dflow INFO: Task 4623 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ad1808a9f959d6e8e50ecef18448bc3 with result from task 4623 parsl.dataflow.dflow DEBUG: Task 4623 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/38e6d9bb-784e-4829-b099-56ac448cb35f_isrForCrosstalkSources_997007_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4623 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/38e6d9bb-784e-4829-b099-56ac448cb35f_isrForCrosstalkSources_997007_60.stderr parsl.dataflow.dflow INFO: Task 4346 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2177f40e45a17a6205c953dd292cadc4 with result from task 4346 parsl.dataflow.dflow DEBUG: Task 4346 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/65b19e1e-9476-4439-91f0-6b91f037cc21_isrForCrosstalkSources_996993_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4346 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/65b19e1e-9476-4439-91f0-6b91f037cc21_isrForCrosstalkSources_996993_39.stderr parsl.dataflow.dflow INFO: Task 4481 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b31dce87dc8ac887a5d62ab0cf68148f with result from task 4481 parsl.dataflow.dflow DEBUG: Task 4481 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4d82dd88-527f-45c8-aaaa-ebcd7a4242fd_isrForCrosstalkSources_997005_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4481 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/4d82dd88-527f-45c8-aaaa-ebcd7a4242fd_isrForCrosstalkSources_997005_14.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:08 proc_lsst.multi:146 [INFO] found job 17075288 in provider checkpoint proc_lsst.multi INFO: found job 17075288 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4645 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f3fe06a1b5cc87001618325829ce4f4a with result from task 4645 parsl.dataflow.dflow DEBUG: Task 4645 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a42ae75a-4fbd-45c0-8f71-203b6a8e4720_isrForCrosstalkSources_996981_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4645 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/a42ae75a-4fbd-45c0-8f71-203b6a8e4720_isrForCrosstalkSources_996981_20.stderr parsl.dataflow.dflow INFO: Task 4414 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac695b621857e0ea4ae1099a9b2ffe82 with result from task 4414 parsl.dataflow.dflow DEBUG: Task 4414 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/350af285-d959-4e3a-8c27-54ca70da7dba_isrForCrosstalkSources_996970_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4414 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/350af285-d959-4e3a-8c27-54ca70da7dba_isrForCrosstalkSources_996970_12.stderr parsl.dataflow.dflow INFO: Task 4380 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb1b4173bfbae4bb85ae967bac99e767 with result from task 4380 parsl.dataflow.dflow DEBUG: Task 4380 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c8ac9bbb-3f37-456e-84dc-7a388c5ac31e_isrForCrosstalkSources_997014_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4380 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/c8ac9bbb-3f37-456e-84dc-7a388c5ac31e_isrForCrosstalkSources_997014_7.stderr parsl.dataflow.dflow INFO: Task 4311 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9e1aab78def83b25fac751e88dadb504 with result from task 4311 parsl.dataflow.dflow DEBUG: Task 4311 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/36b5bfd6-9406-45cc-9daa-455326fb3eac_isrForCrosstalkSources_996965_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4311 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/36b5bfd6-9406-45cc-9daa-455326fb3eac_isrForCrosstalkSources_996965_31.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:08 proc_lsst.multi:146 [INFO] found job 17075289 in provider checkpoint proc_lsst.multi INFO: found job 17075289 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4525 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d184055a3540b24b04207d311d52cd53 with result from task 4525 parsl.dataflow.dflow DEBUG: Task 4525 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/53ec19e0-6dcf-4e5c-83fb-5deee1157f71_isrForCrosstalkSources_996982_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4525 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/53ec19e0-6dcf-4e5c-83fb-5deee1157f71_isrForCrosstalkSources_996982_5.stderr parsl.dataflow.dflow INFO: Task 4670 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e3725eaac46ba29add61007e13c7db9 with result from task 4670 parsl.dataflow.dflow DEBUG: Task 4670 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a4352089-796d-4294-95c6-f5b23f81e101_isrForCrosstalkSources_996961_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4670 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/a4352089-796d-4294-95c6-f5b23f81e101_isrForCrosstalkSources_996961_27.stderr parsl.dataflow.dflow INFO: Task 4649 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8597a7ba589fee765f8319d0fbbc852 with result from task 4649 parsl.dataflow.dflow DEBUG: Task 4649 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/fc932746-fc33-4322-9a83-6bff7add5c07_isrForCrosstalkSources_997028_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4649 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/fc932746-fc33-4322-9a83-6bff7add5c07_isrForCrosstalkSources_997028_15.stderr parsl.dataflow.dflow INFO: Task 4422 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c9c9ac91ed7b339e7adc3cc579e3ed4 with result from task 4422 parsl.dataflow.dflow DEBUG: Task 4422 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b3978908-c77d-4544-9861-5ce3f5c24575_isrForCrosstalkSources_996986_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4422 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b3978908-c77d-4544-9861-5ce3f5c24575_isrForCrosstalkSources_996986_4.stderr parsl.dataflow.dflow INFO: Task 4394 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b58169658efbaad77c293a506cb6165 with result from task 4394 parsl.dataflow.dflow DEBUG: Task 4394 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a4d5ddf-e6d0-4312-858d-c696c6f9d66a_isrForCrosstalkSources_997004_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4394 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a4d5ddf-e6d0-4312-858d-c696c6f9d66a_isrForCrosstalkSources_997004_5.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:09 proc_lsst.multi:146 [INFO] found job 17075290 in provider checkpoint proc_lsst.multi INFO: found job 17075290 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4460 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b66b35b5cef51b1b82e2d36f453cc9b with result from task 4460 parsl.dataflow.dflow DEBUG: Task 4460 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/2aaf28e8-1624-4f06-b3b7-9fe02d81f89f_isrForCrosstalkSources_996996_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4460 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/2aaf28e8-1624-4f06-b3b7-9fe02d81f89f_isrForCrosstalkSources_996996_13.stderr parsl.dataflow.dflow INFO: Task 4495 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81e02ed6b76ad3949c07e7f7de49c31a with result from task 4495 parsl.dataflow.dflow DEBUG: Task 4495 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/eaeef10b-0b64-42e4-ba61-ca13764bda58_isrForCrosstalkSources_997012_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4495 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/eaeef10b-0b64-42e4-ba61-ca13764bda58_isrForCrosstalkSources_997012_33.stderr parsl.dataflow.dflow INFO: Task 4528 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4c882ffcd7db857e1700e7ffaea0bbe with result from task 4528 parsl.dataflow.dflow DEBUG: Task 4528 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/89ea3cf2-cf73-49bb-968d-cabab5862486_isrForCrosstalkSources_996979_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4528 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/89ea3cf2-cf73-49bb-968d-cabab5862486_isrForCrosstalkSources_996979_58.stderr parsl.dataflow.dflow INFO: Task 4637 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c18b5586160ae92b4ff88ea73e1046b3 with result from task 4637 parsl.dataflow.dflow DEBUG: Task 4637 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/711f9fd0-0b6c-4369-859e-1cfc6d8359be_isrForCrosstalkSources_997015_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4637 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/711f9fd0-0b6c-4369-859e-1cfc6d8359be_isrForCrosstalkSources_997015_9.stderr parsl.dataflow.dflow INFO: Task 4322 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 21e6bf4bc846faf9ccda327968b2ebdb with result from task 4322 parsl.dataflow.dflow DEBUG: Task 4322 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/586ef330-5159-4a5e-8b0a-37e245331a68_isrForCrosstalkSources_996966_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4322 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/586ef330-5159-4a5e-8b0a-37e245331a68_isrForCrosstalkSources_996966_51.stderr parsl.dataflow.dflow INFO: Task 4420 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 05f18963eade4cc1aa19b7225897fbac with result from task 4420 parsl.dataflow.dflow DEBUG: Task 4420 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/16bf48c2-9821-4c36-a82d-81d39db88e00_isrForCrosstalkSources_996974_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4420 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/16bf48c2-9821-4c36-a82d-81d39db88e00_isrForCrosstalkSources_996974_16.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:10 proc_lsst.multi:146 [INFO] found job 17075293 in provider checkpoint proc_lsst.multi INFO: found job 17075293 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4522 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1555a88add80a08e3cdde3bffdec66dd with result from task 4522 parsl.dataflow.dflow DEBUG: Task 4522 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/999bb3a6-49de-4cfe-acb9-638b56698a2b_isrForCrosstalkSources_996975_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4522 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/999bb3a6-49de-4cfe-acb9-638b56698a2b_isrForCrosstalkSources_996975_52.stderr parsl.dataflow.dflow INFO: Task 4277 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b197c2d0389a4372338c7d8b1f13adac with result from task 4277 parsl.dataflow.dflow DEBUG: Task 4277 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/bf85befa-bf41-4e7e-9475-3c0c66f1786a_isrForCrosstalkSources_996971_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4277 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/bf85befa-bf41-4e7e-9475-3c0c66f1786a_isrForCrosstalkSources_996971_29.stderr parsl.dataflow.dflow INFO: Task 4303 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bc7904b25a32215e5d52869019b9509a with result from task 4303 parsl.dataflow.dflow DEBUG: Task 4303 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/abf9385a-e125-46c2-bdaf-8bb2c01936bf_isrForCrosstalkSources_996972_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4303 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/abf9385a-e125-46c2-bdaf-8bb2c01936bf_isrForCrosstalkSources_996972_18.stderr parsl.dataflow.dflow INFO: Task 4400 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19bb641406c384399bbe418d19a5f1e2 with result from task 4400 parsl.dataflow.dflow DEBUG: Task 4400 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c8bc8f27-4d94-4245-a01a-0d20c6af5cc1_isrForCrosstalkSources_996980_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4400 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/c8bc8f27-4d94-4245-a01a-0d20c6af5cc1_isrForCrosstalkSources_996980_15.stderr parsl.dataflow.dflow INFO: Task 4433 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f7a47a9689b2c3bd510af9a060e207d3 with result from task 4433 parsl.dataflow.dflow DEBUG: Task 4433 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/94567f2a-0d67-47d9-bb30-55620136b102_isrForCrosstalkSources_997020_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4433 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/94567f2a-0d67-47d9-bb30-55620136b102_isrForCrosstalkSources_997020_24.stderr parsl.dataflow.dflow INFO: Task 4410 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00ea050434de9cf19fd10edc76e28ccb with result from task 4410 parsl.dataflow.dflow DEBUG: Task 4410 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/b694d2ad-1f6b-41e0-b6f1-855c352ee038_isrForCrosstalkSources_997018_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4410 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/b694d2ad-1f6b-41e0-b6f1-855c352ee038_isrForCrosstalkSources_997018_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:10 proc_lsst.multi:146 [INFO] found job 17075298 in provider checkpoint proc_lsst.multi INFO: found job 17075298 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4260 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 000c138b9679c125df1ca781eef8587c with result from task 4260 parsl.dataflow.dflow DEBUG: Task 4260 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/37132177-4bc8-49c1-a1e1-7c8f7fa31e26_isrForCrosstalkSources_996991_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4260 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/37132177-4bc8-49c1-a1e1-7c8f7fa31e26_isrForCrosstalkSources_996991_53.stderr parsl.dataflow.dflow INFO: Task 4429 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d18a40384922919e07c7cab5a05bfdf with result from task 4429 parsl.dataflow.dflow DEBUG: Task 4429 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/17331fa3-baeb-4a62-8cc5-da0a842c1ccb_isrForCrosstalkSources_997021_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4429 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/17331fa3-baeb-4a62-8cc5-da0a842c1ccb_isrForCrosstalkSources_997021_48.stderr parsl.dataflow.dflow INFO: Task 4550 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d044ebe77842088aa6aeb75a6997c08 with result from task 4550 parsl.dataflow.dflow DEBUG: Task 4550 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/83043a46-9526-4f11-9ddb-499374d37a58_isrForCrosstalkSources_997022_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4550 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/83043a46-9526-4f11-9ddb-499374d37a58_isrForCrosstalkSources_997022_25.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:11 proc_lsst.multi:146 [INFO] found job 17075299 in provider checkpoint proc_lsst.multi INFO: found job 17075299 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4655 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d92d42279599f3858857b6f7c9ffd638 with result from task 4655 parsl.dataflow.dflow DEBUG: Task 4655 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/94292aaf-47ac-45c0-94d0-b1ec436f475e_isrForCrosstalkSources_996979_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4655 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/94292aaf-47ac-45c0-94d0-b1ec436f475e_isrForCrosstalkSources_996979_56.stderr parsl.dataflow.dflow INFO: Task 4393 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5dc165743af693850e60586f2d7e8a5b with result from task 4393 parsl.dataflow.dflow DEBUG: Task 4393 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a28746a9-c48c-4f51-a7c7-d69458d7f44d_isrForCrosstalkSources_996991_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4393 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/a28746a9-c48c-4f51-a7c7-d69458d7f44d_isrForCrosstalkSources_996991_18.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:11 proc_lsst.multi:146 [INFO] found job 17075301 in provider checkpoint proc_lsst.multi INFO: found job 17075301 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4458 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c1380aca5311a9d94033b482affc0b5 with result from task 4458 parsl.dataflow.dflow DEBUG: Task 4458 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/f74a3fe6-fb76-44f2-996d-5b09dbf6c90b_isrForCrosstalkSources_996959_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4458 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/f74a3fe6-fb76-44f2-996d-5b09dbf6c90b_isrForCrosstalkSources_996959_1.stderr parsl.dataflow.dflow INFO: Task 4450 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7f467e55002c44111256f32ba2e605dd with result from task 4450 parsl.dataflow.dflow DEBUG: Task 4450 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6af55b86-f833-4927-98e2-2ab2951aa74a_isrForCrosstalkSources_996954_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4450 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/6af55b86-f833-4927-98e2-2ab2951aa74a_isrForCrosstalkSources_996954_51.stderr parsl.dataflow.dflow INFO: Task 4665 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 72a2e5e9a288e98e5fd7b2dc0c9e3718 with result from task 4665 parsl.dataflow.dflow DEBUG: Task 4665 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/23a139a2-ff08-455b-a0d3-944c8386caca_isrForCrosstalkSources_996959_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4665 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/23a139a2-ff08-455b-a0d3-944c8386caca_isrForCrosstalkSources_996959_61.stderr parsl.dataflow.dflow INFO: Task 4308 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8aed4a7fb5b0b74eeb9e7a387215ab82 with result from task 4308 parsl.dataflow.dflow DEBUG: Task 4308 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/16aac535-a7e9-43db-976b-6ec2bb72ce37_isrForCrosstalkSources_997019_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4308 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/16aac535-a7e9-43db-976b-6ec2bb72ce37_isrForCrosstalkSources_997019_52.stderr parsl.dataflow.dflow INFO: Task 4455 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 90559f099dba75799e353476885dac91 with result from task 4455 parsl.dataflow.dflow DEBUG: Task 4455 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0b90a53c-6013-46e8-a2d8-6a0d2c86b983_isrForCrosstalkSources_996997_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4455 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0b90a53c-6013-46e8-a2d8-6a0d2c86b983_isrForCrosstalkSources_996997_46.stderr parsl.dataflow.dflow INFO: Task 4638 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 96a75cec1e040a279bcbbb790a218622 with result from task 4638 parsl.dataflow.dflow DEBUG: Task 4638 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/214a3421-40e7-49b2-a094-fc369473278a_isrForCrosstalkSources_997001_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4638 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/214a3421-40e7-49b2-a094-fc369473278a_isrForCrosstalkSources_997001_1.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:12 proc_lsst.multi:146 [INFO] found job 17075306 in provider checkpoint proc_lsst.multi INFO: found job 17075306 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4502 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0418486987491d3255990d3b9dda9d45 with result from task 4502 parsl.dataflow.dflow DEBUG: Task 4502 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/8b957c19-5aba-451b-b65e-4c97110ac8ee_isrForCrosstalkSources_997012_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4502 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/8b957c19-5aba-451b-b65e-4c97110ac8ee_isrForCrosstalkSources_997012_51.stderr parsl.dataflow.dflow INFO: Task 4359 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 44f24b8bc03f1b4fbc02e8a3cdb4f78e with result from task 4359 parsl.dataflow.dflow DEBUG: Task 4359 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/59d0c293-3aae-4b87-a740-d9769c5711fd_isrForCrosstalkSources_996970_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4359 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/59d0c293-3aae-4b87-a740-d9769c5711fd_isrForCrosstalkSources_996970_18.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:12 proc_lsst.multi:146 [INFO] found job 17075307 in provider checkpoint proc_lsst.multi INFO: found job 17075307 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4430 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abaac825711c7ae8909a77b57e649d98 with result from task 4430 parsl.dataflow.dflow DEBUG: Task 4430 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/8913404d-4554-4a58-9802-be5cad119d00_isrForCrosstalkSources_997010_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4430 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/8913404d-4554-4a58-9802-be5cad119d00_isrForCrosstalkSources_997010_1.stderr parsl.dataflow.dflow INFO: Task 4402 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5911ca6a9296b055480a96a3b1e46538 with result from task 4402 parsl.dataflow.dflow DEBUG: Task 4402 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a81bc4b8-e845-40ff-8b72-c3d7af70e716_isrForCrosstalkSources_997008_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4402 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/a81bc4b8-e845-40ff-8b72-c3d7af70e716_isrForCrosstalkSources_997008_36.stderr parsl.dataflow.dflow INFO: Task 4297 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 321be3b1c9eadfd6aa254d637c4e4987 with result from task 4297 parsl.dataflow.dflow DEBUG: Task 4297 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/f896ce3e-79b4-45b3-a41e-25705e053f0c_isrForCrosstalkSources_996952_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4297 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/f896ce3e-79b4-45b3-a41e-25705e053f0c_isrForCrosstalkSources_996952_10.stderr parsl.dataflow.dflow INFO: Task 4673 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1be62125a5c3fa7a5d9ae4e9899ddf43 with result from task 4673 parsl.dataflow.dflow DEBUG: Task 4673 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4d94cd36-baef-4f1a-bb37-7a3be5b2fcf4_isrForCrosstalkSources_996998_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4673 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/4d94cd36-baef-4f1a-bb37-7a3be5b2fcf4_isrForCrosstalkSources_996998_46.stderr parsl.dataflow.dflow INFO: Task 4664 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dcf9832f74f0962c87e2647bfee1a540 with result from task 4664 parsl.dataflow.dflow DEBUG: Task 4664 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e5f0cddd-2e9f-483e-bcbd-1e1bd118b37c_isrForCrosstalkSources_997021_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4664 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/e5f0cddd-2e9f-483e-bcbd-1e1bd118b37c_isrForCrosstalkSources_997021_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:13 proc_lsst.multi:146 [INFO] found job 17075308 in provider checkpoint proc_lsst.multi INFO: found job 17075308 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4466 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01a26efec9394bda9027360a8d0934e8 with result from task 4466 parsl.dataflow.dflow DEBUG: Task 4466 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f68965bf-d6f9-41ce-b1b5-1c93385b6254_isrForCrosstalkSources_996977_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4466 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/f68965bf-d6f9-41ce-b1b5-1c93385b6254_isrForCrosstalkSources_996977_33.stderr parsl.dataflow.dflow INFO: Task 4444 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd34e7eaac5ed9e3ce3e0b787c6c7cdb with result from task 4444 parsl.dataflow.dflow DEBUG: Task 4444 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/16d08bc4-5592-48e2-a92e-d66d3a312943_isrForCrosstalkSources_996954_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4444 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/16d08bc4-5592-48e2-a92e-d66d3a312943_isrForCrosstalkSources_996954_38.stderr parsl.dataflow.dflow INFO: Task 4521 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ab4c92a051ec961c0a68226454eeffe with result from task 4521 parsl.dataflow.dflow DEBUG: Task 4521 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ea7a1f44-24f8-4bb4-848f-56f615d4945a_isrForCrosstalkSources_996969_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4521 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/ea7a1f44-24f8-4bb4-848f-56f615d4945a_isrForCrosstalkSources_996969_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:13 proc_lsst.multi:146 [INFO] found job 17075309 in provider checkpoint proc_lsst.multi INFO: found job 17075309 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4390 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8207dd8b27d73bccde86898511b6352c with result from task 4390 parsl.dataflow.dflow DEBUG: Task 4390 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/de3e5d4b-e478-47b1-bc93-21a732ff3134_isrForCrosstalkSources_997030_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4390 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/de3e5d4b-e478-47b1-bc93-21a732ff3134_isrForCrosstalkSources_997030_16.stderr parsl.dataflow.dflow INFO: Task 4573 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 792fb5fc72c882889a3354a97da7aa62 with result from task 4573 parsl.dataflow.dflow DEBUG: Task 4573 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0e0e6ba7-001a-4c1d-aa5f-646de60312c1_isrForCrosstalkSources_996983_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4573 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/0e0e6ba7-001a-4c1d-aa5f-646de60312c1_isrForCrosstalkSources_996983_21.stderr parsl.dataflow.dflow INFO: Task 4595 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1b9006296be7f60879d8340020634d17 with result from task 4595 parsl.dataflow.dflow DEBUG: Task 4595 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/6b3275f1-9788-463a-8772-24b90b3e4251_isrForCrosstalkSources_996999_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4595 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/6b3275f1-9788-463a-8772-24b90b3e4251_isrForCrosstalkSources_996999_14.stderr parsl.dataflow.dflow INFO: Task 4477 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9b40b499c932d8783a3c547072af1191 with result from task 4477 parsl.dataflow.dflow DEBUG: Task 4477 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e3810314-ba5b-4062-b1ab-7b5561d9edfe_isrForCrosstalkSources_996986_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4477 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/e3810314-ba5b-4062-b1ab-7b5561d9edfe_isrForCrosstalkSources_996986_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:14 proc_lsst.multi:146 [INFO] found job 17075310 in provider checkpoint proc_lsst.multi INFO: found job 17075310 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4411 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19b051a7439f9bfce07b62a9efd76c7e with result from task 4411 parsl.dataflow.dflow DEBUG: Task 4411 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8ed4aa16-7cec-4c1b-995b-3d00cb280e8a_isrForCrosstalkSources_996967_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4411 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/8ed4aa16-7cec-4c1b-995b-3d00cb280e8a_isrForCrosstalkSources_996967_31.stderr parsl.dataflow.dflow INFO: Task 4698 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be7ab23dca3b2f605106cb43f83b4fcf with result from task 4698 parsl.dataflow.dflow DEBUG: Task 4698 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c936105f-0dd5-4829-bf5e-3bb9767234ea_isrForCrosstalkSources_996966_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4698 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c936105f-0dd5-4829-bf5e-3bb9767234ea_isrForCrosstalkSources_996966_13.stderr parsl.dataflow.dflow INFO: Task 4680 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 829df41683217236b61900ebedf0b712 with result from task 4680 parsl.dataflow.dflow DEBUG: Task 4680 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/14e06baf-379a-4023-93e8-2b310d9f74cf_isrForCrosstalkSources_997014_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4680 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/14e06baf-379a-4023-93e8-2b310d9f74cf_isrForCrosstalkSources_997014_34.stderr parsl.dataflow.dflow INFO: Task 4483 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 84386c3252be7f118e79e37916d665a3 with result from task 4483 parsl.dataflow.dflow DEBUG: Task 4483 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/141f8555-e669-45b4-960d-bbbbe03ac3b7_isrForCrosstalkSources_996991_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4483 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/141f8555-e669-45b4-960d-bbbbe03ac3b7_isrForCrosstalkSources_996991_10.stderr parsl.dataflow.dflow INFO: Task 4682 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2bbe4164c2b01ec3ffc2367ab30fcc36 with result from task 4682 parsl.dataflow.dflow DEBUG: Task 4682 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/68605157-f841-4559-a470-83e5f67b4b1d_isrForCrosstalkSources_997010_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4682 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/68605157-f841-4559-a470-83e5f67b4b1d_isrForCrosstalkSources_997010_42.stderr parsl.dataflow.dflow INFO: Task 4672 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f8c88c77a557cce6503fd062fc7d7593 with result from task 4672 parsl.dataflow.dflow DEBUG: Task 4672 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a85c2bcb-d717-4de0-97dd-cffff7f306ab_isrForCrosstalkSources_997006_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4672 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/a85c2bcb-d717-4de0-97dd-cffff7f306ab_isrForCrosstalkSources_997006_47.stderr parsl.dataflow.dflow INFO: Task 4389 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 682446cfae344845b3b619edf1acdd30 with result from task 4389 parsl.dataflow.dflow DEBUG: Task 4389 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e6774f98-9b00-44f4-a86e-275d7777471b_isrForCrosstalkSources_996992_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4389 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/e6774f98-9b00-44f4-a86e-275d7777471b_isrForCrosstalkSources_996992_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4671 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry f31c5f798e73208972e2d366f254b587 with result from task 4671 parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4671 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9281d8b6-85ba-4f16-a44f-0cd60ea37cf5_isrForCrosstalkSources_996960_49.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4671 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9281d8b6-85ba-4f16-a44f-0cd60ea37cf5_isrForCrosstalkSources_996960_49.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:14 proc_lsst.multi:146 [INFO] found job 17075311 in provider checkpoint proc_lsst.multi INFO: found job 17075311 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4582 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0967b21cb2913408c6240e0a573834b8 with result from task 4582 parsl.dataflow.dflow DEBUG: Task 4582 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/920faa68-da1a-4017-af08-84018f26f304_isrForCrosstalkSources_996954_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4582 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/920faa68-da1a-4017-af08-84018f26f304_isrForCrosstalkSources_996954_13.stderr parsl.dataflow.dflow INFO: Task 4374 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 67299b687bc2e869c5f90f92e3541b10 with result from task 4374 parsl.dataflow.dflow DEBUG: Task 4374 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/857afb47-9db0-4eee-b5b2-3014a85af708_isrForCrosstalkSources_996954_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4374 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/857afb47-9db0-4eee-b5b2-3014a85af708_isrForCrosstalkSources_996954_28.stderr parsl.dataflow.dflow INFO: Task 4663 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abbf6574c0ad6fa44221f299a38b77bb with result from task 4663 parsl.dataflow.dflow DEBUG: Task 4663 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/137a3921-b6d2-44a7-a08b-d0a78bac5068_isrForCrosstalkSources_996954_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4663 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/137a3921-b6d2-44a7-a08b-d0a78bac5068_isrForCrosstalkSources_996954_50.stderr parsl.dataflow.dflow INFO: Task 4519 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c062c9c78257710326359116967d5d07 with result from task 4519 parsl.dataflow.dflow DEBUG: Task 4519 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ba5dffbe-330b-414c-8be0-8635aa91e7a4_isrForCrosstalkSources_996994_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4519 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/ba5dffbe-330b-414c-8be0-8635aa91e7a4_isrForCrosstalkSources_996994_56.stderr parsl.dataflow.dflow INFO: Task 4473 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3909b8a6c1270db72267e36754acdb1a with result from task 4473 parsl.dataflow.dflow DEBUG: Task 4473 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/d7f25149-05e6-45b2-aa45-335fd81dc371_isrForCrosstalkSources_996975_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4473 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/d7f25149-05e6-45b2-aa45-335fd81dc371_isrForCrosstalkSources_996975_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:15 proc_lsst.multi:146 [INFO] found job 17075312 in provider checkpoint proc_lsst.multi INFO: found job 17075312 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4259 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23322d401768747160e625518e0ae925 with result from task 4259 parsl.dataflow.dflow DEBUG: Task 4259 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/0537a6a6-9722-4f61-9968-c5589413f131_isrForCrosstalkSources_997005_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4259 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/0537a6a6-9722-4f61-9968-c5589413f131_isrForCrosstalkSources_997005_49.stderr parsl.dataflow.dflow INFO: Task 4491 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 36215315a905d78265296797f0e4bd8a with result from task 4491 parsl.dataflow.dflow DEBUG: Task 4491 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c031d22f-9520-4220-95ac-e504c18b6068_isrForCrosstalkSources_996969_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4491 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/c031d22f-9520-4220-95ac-e504c18b6068_isrForCrosstalkSources_996969_55.stderr parsl.dataflow.dflow INFO: Task 4685 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f642203801bd478698a0299aed570c23 with result from task 4685 parsl.dataflow.dflow DEBUG: Task 4685 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/813febc4-e29c-412b-a518-4ef5b5646cb7_isrForCrosstalkSources_996975_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4685 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/813febc4-e29c-412b-a518-4ef5b5646cb7_isrForCrosstalkSources_996975_12.stderr parsl.dataflow.dflow INFO: Task 4276 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 273d552bbb6832342886b081c53f2e2d with result from task 4276 parsl.dataflow.dflow DEBUG: Task 4276 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/52925fdb-99f0-4ea6-8ddf-821bbbac7741_isrForCrosstalkSources_996997_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4276 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/52925fdb-99f0-4ea6-8ddf-821bbbac7741_isrForCrosstalkSources_996997_30.stderr parsl.dataflow.dflow INFO: Task 4372 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4814afa1ff5f3319d136839b0f061f0f with result from task 4372 parsl.dataflow.dflow DEBUG: Task 4372 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/f1cc18f7-d3da-455d-92b9-2c2747c0436d_isrForCrosstalkSources_997029_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4372 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/f1cc18f7-d3da-455d-92b9-2c2747c0436d_isrForCrosstalkSources_997029_57.stderr parsl.dataflow.dflow INFO: Task 4690 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fbdc20d0719f74e83f83505ebdd1343d with result from task 4690 parsl.dataflow.dflow DEBUG: Task 4690 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a5ee799d-1e1a-4b24-a818-21b73e861cf9_isrForCrosstalkSources_997018_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4690 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/a5ee799d-1e1a-4b24-a818-21b73e861cf9_isrForCrosstalkSources_997018_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:15 proc_lsst.multi:146 [INFO] found job 17075315 in provider checkpoint proc_lsst.multi INFO: found job 17075315 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4421 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b07a1122858d4fffcbcca1b602a83c05 with result from task 4421 parsl.dataflow.dflow DEBUG: Task 4421 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/dd16daf2-5b2a-4d7c-be04-e398d359839f_isrForCrosstalkSources_997022_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4421 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/dd16daf2-5b2a-4d7c-be04-e398d359839f_isrForCrosstalkSources_997022_30.stderr parsl.dataflow.dflow INFO: Task 4489 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c775e2f51064f38051022ba8b4ef50e0 with result from task 4489 parsl.dataflow.dflow DEBUG: Task 4489 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b1e0a762-e13f-40fe-ad8f-f423309fe96d_isrForCrosstalkSources_996999_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4489 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/b1e0a762-e13f-40fe-ad8f-f423309fe96d_isrForCrosstalkSources_996999_46.stderr parsl.dataflow.dflow INFO: Task 4516 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c29c4e0590f1ae488f3e1a79b935972 with result from task 4516 parsl.dataflow.dflow DEBUG: Task 4516 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/3960529c-cbb9-4623-8988-ab44c17d4109_isrForCrosstalkSources_996966_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4516 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/3960529c-cbb9-4623-8988-ab44c17d4109_isrForCrosstalkSources_996966_15.stderr parsl.dataflow.dflow INFO: Task 4427 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb560916d1c46113609c99fb8aa49d20 with result from task 4427 parsl.dataflow.dflow DEBUG: Task 4427 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/35b49ebb-78d7-46d2-95b9-d8ccb297b939_isrForCrosstalkSources_996979_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4427 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/35b49ebb-78d7-46d2-95b9-d8ccb297b939_isrForCrosstalkSources_996979_18.stderr parsl.dataflow.dflow INFO: Task 4314 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29c2940ef0c6195ccfb3637d895d6c5a with result from task 4314 parsl.dataflow.dflow DEBUG: Task 4314 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c3be24fc-abb2-4d0d-9af6-893fb27b84a6_isrForCrosstalkSources_997009_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4314 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/c3be24fc-abb2-4d0d-9af6-893fb27b84a6_isrForCrosstalkSources_997009_61.stderr parsl.dataflow.dflow INFO: Task 4388 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e0c003d331a1f38b736316a63ce91d7a with result from task 4388 parsl.dataflow.dflow DEBUG: Task 4388 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9884be5d-7c5a-4d79-af31-88f2213582fd_isrForCrosstalkSources_996964_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4388 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/9884be5d-7c5a-4d79-af31-88f2213582fd_isrForCrosstalkSources_996964_2.stderr parsl.dataflow.dflow INFO: Task 4576 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0e879f73a06ce3e7d0234c175a1f9fa0 with result from task 4576 parsl.dataflow.dflow DEBUG: Task 4576 checkpointed parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5d642915-337b-4209-9cea-7e205c11e354_isrForCrosstalkSources_997018_14.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4576 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/5d642915-337b-4209-9cea-7e205c11e354_isrForCrosstalkSources_997018_14.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3989 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 69626b92ae2409f783b77a920f3ac920 with result from task 3989 parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3989 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c04671fa-05c1-4192-97c5-67c549abd8e9_isrForCrosstalkSources_997006_35.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c04671fa-05c1-4192-97c5-67c549abd8e9_isrForCrosstalkSources_997006_35.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3982 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 0c23ccc065d24e541d4874959f51b456 with result from task 3982 parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3982 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b7a4677c-784e-4ee4-99ec-acb5001bee37_isrForCrosstalkSources_996966_56.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/b7a4677c-784e-4ee4-99ec-acb5001bee37_isrForCrosstalkSources_996966_56.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3980 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 507cdcdfa5ad49565f00dedf30e9c925 with result from task 3980 parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 3980 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 3980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/b41c9c56-96d7-412d-b781-3a48b9e55be7_isrForCrosstalkSources_996952_2.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 3980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/b41c9c56-96d7-412d-b781-3a48b9e55be7_isrForCrosstalkSources_996952_2.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 3979 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:16 proc_lsst.multi:146 [INFO] found job 17075318 in provider checkpoint proc_lsst.multi INFO: found job 17075318 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.memoization DEBUG: Storing app cache entry cab786325e8dd91066f522a52b3f64d2 with result from task 3979 parsl.dataflow.dflow DEBUG: Task 3979 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/4716cb7c-62cc-4581-be97-5538dedb3961_isrForCrosstalkSources_996971_37.stdout parsl.dataflow.dflow INFO: Standard error for task 3979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/4716cb7c-62cc-4581-be97-5538dedb3961_isrForCrosstalkSources_996971_37.stderr parsl.dataflow.dflow INFO: Task 3976 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ee1420d5583c16e7f7779fb808e0400b with result from task 3976 parsl.dataflow.dflow DEBUG: Task 3976 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/e7554664-d4eb-4443-937e-0705743d4306_isrForCrosstalkSources_997012_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/e7554664-d4eb-4443-937e-0705743d4306_isrForCrosstalkSources_997012_17.stderr parsl.dataflow.dflow INFO: Task 3977 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c9c51c301f2cac5a3be7c74271bf774 with result from task 3977 parsl.dataflow.dflow DEBUG: Task 3977 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ce5fc6f5-f13a-44b8-9df3-97e43804bf71_isrForCrosstalkSources_996988_57.stdout parsl.dataflow.dflow INFO: Standard error for task 3977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ce5fc6f5-f13a-44b8-9df3-97e43804bf71_isrForCrosstalkSources_996988_57.stderr parsl.dataflow.dflow INFO: Task 3984 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8364365d14bf81fbfd729a072b55ba7 with result from task 3984 parsl.dataflow.dflow DEBUG: Task 3984 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/ee601cfd-b28d-4239-b4ea-258068245388_isrForCrosstalkSources_997008_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/ee601cfd-b28d-4239-b4ea-258068245388_isrForCrosstalkSources_997008_22.stderr parsl.dataflow.dflow INFO: Task 3988 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 64a7bcf3c0a68ae0713a820cae45adb7 with result from task 3988 parsl.dataflow.dflow DEBUG: Task 3988 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8f379c12-8114-4f2b-97e7-7b95f121ec2d_isrForCrosstalkSources_996960_8.stdout parsl.dataflow.dflow INFO: Standard error for task 3988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/8f379c12-8114-4f2b-97e7-7b95f121ec2d_isrForCrosstalkSources_996960_8.stderr parsl.dataflow.dflow INFO: Task 3985 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 07567b3d8781c4f60a58f3632dd4884f with result from task 3985 parsl.dataflow.dflow DEBUG: Task 3985 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b53ef759-8bd5-4dd7-8c33-6331e7f676ac_isrForCrosstalkSources_996998_28.stdout parsl.dataflow.dflow INFO: Standard error for task 3985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b53ef759-8bd5-4dd7-8c33-6331e7f676ac_isrForCrosstalkSources_996998_28.stderr parsl.dataflow.dflow INFO: Task 3990 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ade6e57ed0f1a8ab641d55743007408e with result from task 3990 parsl.dataflow.dflow DEBUG: Task 3990 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0cc70593-6513-491b-94df-840b5b801cd1_isrForCrosstalkSources_997013_15.stdout parsl.dataflow.dflow INFO: Standard error for task 3990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/0cc70593-6513-491b-94df-840b5b801cd1_isrForCrosstalkSources_997013_15.stderr parsl.dataflow.dflow INFO: Task 3991 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 055d10abdbe19dccf942a7e244e2bef8 with result from task 3991 parsl.dataflow.dflow DEBUG: Task 3991 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/6258eed0-8aa1-4d0b-bc4e-1a4d08f27d3c_isrForCrosstalkSources_996987_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/6258eed0-8aa1-4d0b-bc4e-1a4d08f27d3c_isrForCrosstalkSources_996987_7.stderr parsl.dataflow.dflow INFO: Task 3987 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f607b2492618fda27e54e01ada5a7bcf with result from task 3987 parsl.dataflow.dflow DEBUG: Task 3987 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9842d687-1e35-45dd-b5e6-11a9fdeb3687_isrForCrosstalkSources_996988_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/9842d687-1e35-45dd-b5e6-11a9fdeb3687_isrForCrosstalkSources_996988_41.stderr parsl.dataflow.dflow INFO: Task 3978 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 696974ca25ad81ce9ec0c47d85fd2dbe with result from task 3978 parsl.dataflow.dflow DEBUG: Task 3978 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/533aa9dc-aa2c-4779-bef3-5185a9b22988_isrForCrosstalkSources_997028_61.stdout parsl.dataflow.dflow INFO: Standard error for task 3978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/533aa9dc-aa2c-4779-bef3-5185a9b22988_isrForCrosstalkSources_997028_61.stderr parsl.dataflow.dflow INFO: Task 4247 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ed3363e609a62d0432750b1daf38b3b with result from task 4247 parsl.dataflow.dflow DEBUG: Task 4247 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/74d985e0-25d1-451a-9e9c-e097e69542e1_isrForCrosstalkSources_996990_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4247 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/74d985e0-25d1-451a-9e9c-e097e69542e1_isrForCrosstalkSources_996990_52.stderr parsl.dataflow.dflow INFO: Task 3986 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 292fd1ba0af6da61ddb0db3a573d6f99 with result from task 3986 parsl.dataflow.dflow DEBUG: Task 3986 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/c3547417-c095-48a3-8f89-a1c56cf5204b_isrForCrosstalkSources_996956_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/c3547417-c095-48a3-8f89-a1c56cf5204b_isrForCrosstalkSources_996956_48.stderr parsl.dataflow.dflow INFO: Task 4320 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02d3f17f71cd0b3d7896c163dd159cbc with result from task 4320 parsl.dataflow.dflow DEBUG: Task 4320 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/53120f4b-b538-452d-8191-997a1cdc43e9_isrForCrosstalkSources_996981_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4320 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/53120f4b-b538-452d-8191-997a1cdc43e9_isrForCrosstalkSources_996981_5.stderr parsl.dataflow.dflow INFO: Task 4313 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6a1370edf3dfaabc8f42e7c1bb7d529d with result from task 4313 parsl.dataflow.dflow DEBUG: Task 4313 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f9735b49-f07e-4fa2-9684-df0239ddefe0_isrForCrosstalkSources_997018_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4313 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/f9735b49-f07e-4fa2-9684-df0239ddefe0_isrForCrosstalkSources_997018_22.stderr parsl.dataflow.dflow INFO: Task 4251 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 549a42e1fd9edb8d3bf105d06a4fb33a with result from task 4251 parsl.dataflow.dflow DEBUG: Task 4251 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/2ebb57ee-4271-441e-be56-2f84ef944ae7_isrForCrosstalkSources_997009_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4251 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/2ebb57ee-4271-441e-be56-2f84ef944ae7_isrForCrosstalkSources_997009_11.stderr parsl.dataflow.dflow INFO: Task 4701 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f00dc3f61f3de3fa3144e49dad53211 with result from task 4701 parsl.dataflow.dflow DEBUG: Task 4701 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/098bf01d-a54d-428e-8ba0-cd7b8934e20a_isrForCrosstalkSources_996973_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4701 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/098bf01d-a54d-428e-8ba0-cd7b8934e20a_isrForCrosstalkSources_996973_9.stderr parsl.dataflow.dflow INFO: Task 3983 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bcaa556577bb32402b84cfb037bf04f3 with result from task 3983 parsl.dataflow.dflow DEBUG: Task 3983 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c556eb16-1681-44d6-a61e-7dc753193ea8_isrForCrosstalkSources_997019_18.stdout parsl.dataflow.dflow INFO: Standard error for task 3983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/c556eb16-1681-44d6-a61e-7dc753193ea8_isrForCrosstalkSources_997019_18.stderr parsl.dataflow.dflow INFO: Task 4321 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 404979965dc54a1d053f6d008646431f with result from task 4321 parsl.dataflow.dflow DEBUG: Task 4321 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/fec6715d-a714-4e1c-8782-3f7ee4d6fc4a_isrForCrosstalkSources_997021_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4321 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/fec6715d-a714-4e1c-8782-3f7ee4d6fc4a_isrForCrosstalkSources_997021_46.stderr parsl.dataflow.dflow INFO: Task 4391 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 026d8805cabb04bf61e326481b675a1c with result from task 4391 parsl.dataflow.dflow DEBUG: Task 4391 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/31fa0f08-84f7-4ccf-8de0-66b665318879_isrForCrosstalkSources_996973_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4391 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/31fa0f08-84f7-4ccf-8de0-66b665318879_isrForCrosstalkSources_996973_28.stderr parsl.dataflow.dflow INFO: Task 3981 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0178a4f8bff25b0c01cf9ed8feb6d068 with result from task 3981 parsl.dataflow.dflow DEBUG: Task 3981 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ede650df-bb3e-46cb-9b5c-4e003cea2eea_isrForCrosstalkSources_997006_20.stdout parsl.dataflow.dflow INFO: Standard error for task 3981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/ede650df-bb3e-46cb-9b5c-4e003cea2eea_isrForCrosstalkSources_997006_20.stderr parsl.dataflow.dflow INFO: Task 4230 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bfc5a9f9bda956afc0da553b9aa80082 with result from task 4230 parsl.dataflow.dflow DEBUG: Task 4230 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/512673a7-4ae8-4bae-a3d7-863bc70e8deb_isrForCrosstalkSources_997019_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4230 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/512673a7-4ae8-4bae-a3d7-863bc70e8deb_isrForCrosstalkSources_997019_15.stderr parsl.dataflow.dflow INFO: Task 4141 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 359d125692573a226cb07cccc638f125 with result from task 4141 parsl.dataflow.dflow DEBUG: Task 4141 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/73b9acd3-2dd9-4a98-8f6b-79dc68be948a_isrForCrosstalkSources_996985_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4141 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/73b9acd3-2dd9-4a98-8f6b-79dc68be948a_isrForCrosstalkSources_996985_56.stderr parsl.dataflow.dflow INFO: Task 4159 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c6801a8f65a41ed336edfb60e3434b3f with result from task 4159 parsl.dataflow.dflow DEBUG: Task 4159 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/1a3364ac-eb5d-4f27-9a84-2764a32ff099_isrForCrosstalkSources_996990_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4159 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/1a3364ac-eb5d-4f27-9a84-2764a32ff099_isrForCrosstalkSources_996990_5.stderr parsl.dataflow.dflow INFO: Task 4593 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04ada73a6cb85e4a354c71b14433acec with result from task 4593 parsl.dataflow.dflow DEBUG: Task 4593 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f9804382-df39-4d69-8705-e519efb288b5_isrForCrosstalkSources_997019_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4593 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/f9804382-df39-4d69-8705-e519efb288b5_isrForCrosstalkSources_997019_24.stderr parsl.dataflow.dflow INFO: Task 4345 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16299ec51a64605dc185aa84ec45bcc0 with result from task 4345 parsl.dataflow.dflow DEBUG: Task 4345 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/460c4e6a-ee33-4a34-9072-80fab7a21c4e_isrForCrosstalkSources_996991_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4345 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/460c4e6a-ee33-4a34-9072-80fab7a21c4e_isrForCrosstalkSources_996991_5.stderr parsl.dataflow.dflow INFO: Task 4431 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9cdf84130fc17e8fd8b92907b42a27e0 with result from task 4431 parsl.dataflow.dflow DEBUG: Task 4431 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/a5fb33c7-92ea-4c8b-85f2-71be42018b67_isrForCrosstalkSources_996979_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4431 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/a5fb33c7-92ea-4c8b-85f2-71be42018b67_isrForCrosstalkSources_996979_46.stderr parsl.dataflow.dflow INFO: Task 4287 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fe3918dbe4b973ecb3e59b2e3047c3b with result from task 4287 parsl.dataflow.dflow DEBUG: Task 4287 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ebd81182-cc9c-4ab6-a4a0-fb1265a116cd_isrForCrosstalkSources_997021_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4287 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/ebd81182-cc9c-4ab6-a4a0-fb1265a116cd_isrForCrosstalkSources_997021_42.stderr parsl.dataflow.dflow INFO: Task 4708 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e440777637d55d5e6929cfe64a03724f with result from task 4708 parsl.dataflow.dflow DEBUG: Task 4708 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7a85cd27-ea9f-4c9f-9986-7998bc8a917b_isrForCrosstalkSources_996984_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4708 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/7a85cd27-ea9f-4c9f-9986-7998bc8a917b_isrForCrosstalkSources_996984_49.stderr parsl.dataflow.dflow INFO: Task 4255 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5963d2470df43c7f6bd24858a3d15116 with result from task 4255 parsl.dataflow.dflow DEBUG: Task 4255 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/33506b5c-d824-4b39-99d4-7c623a9f124f_isrForCrosstalkSources_997016_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4255 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/33506b5c-d824-4b39-99d4-7c623a9f124f_isrForCrosstalkSources_997016_31.stderr parsl.dataflow.dflow INFO: Task 4706 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2a673f352e80d9766f6961e83d13c45e with result from task 4706 parsl.dataflow.dflow DEBUG: Task 4706 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/203dacff-24dd-4f9b-9dee-716f24e879a9_isrForCrosstalkSources_996999_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4706 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/203dacff-24dd-4f9b-9dee-716f24e879a9_isrForCrosstalkSources_996999_39.stderr parsl.dataflow.dflow INFO: Task 4343 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b43969451aad1568a6129383aeeee79a with result from task 4343 parsl.dataflow.dflow DEBUG: Task 4343 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/2d801789-67c2-48c1-a73f-3ea496aa27d5_isrForCrosstalkSources_997028_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4343 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/2d801789-67c2-48c1-a73f-3ea496aa27d5_isrForCrosstalkSources_997028_50.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:17 proc_lsst.multi:146 [INFO] found job 17075319 in provider checkpoint proc_lsst.multi INFO: found job 17075319 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4527 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a89c44e7a2974a6bb56583842471b94a with result from task 4527 parsl.dataflow.dflow DEBUG: Task 4527 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3a1ee839-4540-46e6-9729-e12b10d4bb3a_isrForCrosstalkSources_996995_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4527 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/3a1ee839-4540-46e6-9729-e12b10d4bb3a_isrForCrosstalkSources_996995_11.stderr parsl.dataflow.dflow INFO: Task 4384 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ac20fe232f7df0f77891e0551baa35f with result from task 4384 parsl.dataflow.dflow DEBUG: Task 4384 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3917908b-cb1f-4b7e-ad99-83f5472012da_isrForCrosstalkSources_997019_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4384 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/3917908b-cb1f-4b7e-ad99-83f5472012da_isrForCrosstalkSources_997019_5.stderr parsl.dataflow.dflow INFO: Task 4554 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be69dedcb457c71437611853d968923a with result from task 4554 parsl.dataflow.dflow DEBUG: Task 4554 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/f3fd096f-1b69-4265-99e8-1a6212065fb4_isrForCrosstalkSources_997006_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4554 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/f3fd096f-1b69-4265-99e8-1a6212065fb4_isrForCrosstalkSources_997006_50.stderr parsl.dataflow.dflow INFO: Task 4336 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8a9753418774c8ba5d8dfcf700f3983 with result from task 4336 parsl.dataflow.dflow DEBUG: Task 4336 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/c6f20bf7-683e-429f-b694-63e0f1a3d89c_isrForCrosstalkSources_997027_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4336 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/c6f20bf7-683e-429f-b694-63e0f1a3d89c_isrForCrosstalkSources_997027_58.stderr parsl.dataflow.dflow INFO: Task 4382 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab5730d370bc515db2dd74acaa6b59b6 with result from task 4382 parsl.dataflow.dflow DEBUG: Task 4382 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/6e285c0d-350c-49e2-81bb-41d5f6cb1fa3_isrForCrosstalkSources_996984_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4382 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/6e285c0d-350c-49e2-81bb-41d5f6cb1fa3_isrForCrosstalkSources_996984_54.stderr parsl.dataflow.dflow INFO: Task 4555 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d697ba71ac65211b6a38df06ff683e01 with result from task 4555 parsl.dataflow.dflow DEBUG: Task 4555 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/842ba0fa-1042-465b-8714-3be19d3ac021_isrForCrosstalkSources_996969_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4555 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/842ba0fa-1042-465b-8714-3be19d3ac021_isrForCrosstalkSources_996969_51.stderr parsl.dataflow.dflow INFO: Task 4286 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7fd37c5a2f59b779ccffa8ce96894e2 with result from task 4286 parsl.dataflow.dflow DEBUG: Task 4286 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4fbb04a1-598b-4134-b005-09996bcee303_isrForCrosstalkSources_997014_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4286 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4fbb04a1-598b-4134-b005-09996bcee303_isrForCrosstalkSources_997014_31.stderr parsl.dataflow.dflow INFO: Task 4307 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a1066f877c224c4ad06dbc5e472a380 with result from task 4307 parsl.dataflow.dflow DEBUG: Task 4307 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/bdc02af8-5d9c-4911-ba8d-db410328a452_isrForCrosstalkSources_997022_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4307 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/bdc02af8-5d9c-4911-ba8d-db410328a452_isrForCrosstalkSources_997022_41.stderr parsl.dataflow.dflow INFO: Task 4475 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23e38391d894260b14251499ec14fd83 with result from task 4475 parsl.dataflow.dflow DEBUG: Task 4475 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/57cd6856-5e82-471c-aa69-1cb653ee3845_isrForCrosstalkSources_997010_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4475 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/57cd6856-5e82-471c-aa69-1cb653ee3845_isrForCrosstalkSources_997010_18.stderr parsl.dataflow.dflow INFO: Task 4291 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ea0acd866ec8ad02e8d00abd6a990d83 with result from task 4291 parsl.dataflow.dflow DEBUG: Task 4291 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6f497fc6-ebaf-4f6a-9f0c-ec7c4e47f161_isrForCrosstalkSources_997015_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4291 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/6f497fc6-ebaf-4f6a-9f0c-ec7c4e47f161_isrForCrosstalkSources_997015_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:17 proc_lsst.multi:146 [INFO] found job 17075320 in provider checkpoint proc_lsst.multi INFO: found job 17075320 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4565 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4e3934c134307f04e818d6d55107504 with result from task 4565 parsl.dataflow.dflow DEBUG: Task 4565 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/bbc082a9-e128-4598-a6eb-b06c111b469b_isrForCrosstalkSources_996999_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4565 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/bbc082a9-e128-4598-a6eb-b06c111b469b_isrForCrosstalkSources_996999_11.stderr parsl.dataflow.dflow INFO: Task 4465 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e257e3bd0e6c3e7515d158fa64fbd6d9 with result from task 4465 parsl.dataflow.dflow DEBUG: Task 4465 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/54bcafc0-40b8-42b0-88b1-dd174d925b63_isrForCrosstalkSources_997017_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4465 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/54bcafc0-40b8-42b0-88b1-dd174d925b63_isrForCrosstalkSources_997017_1.stderr parsl.dataflow.dflow INFO: Task 4401 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 78d652759b0ae7ae0d8bb8bf367bbdc0 with result from task 4401 parsl.dataflow.dflow DEBUG: Task 4401 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/685279a8-2bd5-41ee-ab01-efe0fe2e5fa4_isrForCrosstalkSources_997020_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4401 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/685279a8-2bd5-41ee-ab01-efe0fe2e5fa4_isrForCrosstalkSources_997020_54.stderr parsl.dataflow.dflow INFO: Task 4723 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ccde9c1a347e4b765a683434b4591890 with result from task 4723 parsl.dataflow.dflow DEBUG: Task 4723 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/53d58de9-b54e-4504-8264-a451a0814ecc_isrForCrosstalkSources_997027_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4723 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/53d58de9-b54e-4504-8264-a451a0814ecc_isrForCrosstalkSources_997027_34.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:18 proc_lsst.multi:146 [INFO] found job 17075321 in provider checkpoint proc_lsst.multi INFO: found job 17075321 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4416 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb418a6bb58be2f743d214ecd8b0755f with result from task 4416 parsl.dataflow.dflow DEBUG: Task 4416 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/aa137d9b-d586-4d80-82ca-9c990f540385_isrForCrosstalkSources_996991_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4416 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/aa137d9b-d586-4d80-82ca-9c990f540385_isrForCrosstalkSources_996991_52.stderr parsl.dataflow.dflow INFO: Task 4709 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e18c697618fbfc27e052971d4991bb7 with result from task 4709 parsl.dataflow.dflow DEBUG: Task 4709 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b33d5895-a264-46c0-bb3b-9b7001254071_isrForCrosstalkSources_997031_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4709 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/b33d5895-a264-46c0-bb3b-9b7001254071_isrForCrosstalkSources_997031_42.stderr parsl.dataflow.dflow INFO: Task 4440 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 805703a669ba23c3f6c229301e63e9a8 with result from task 4440 parsl.dataflow.dflow DEBUG: Task 4440 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2a51d949-211d-4350-b3ea-0f65df25d599_isrForCrosstalkSources_996977_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4440 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/2a51d949-211d-4350-b3ea-0f65df25d599_isrForCrosstalkSources_996977_60.stderr parsl.dataflow.dflow INFO: Task 4332 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf60e9d12ce7a5d5fb0f3ef74eee12a5 with result from task 4332 parsl.dataflow.dflow DEBUG: Task 4332 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/fac44a26-6066-412e-b046-7b6db17e63fb_isrForCrosstalkSources_997031_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4332 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/fac44a26-6066-412e-b046-7b6db17e63fb_isrForCrosstalkSources_997031_58.stderr parsl.dataflow.dflow INFO: Task 4456 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9f3ae94fedfe5aad0e468fd95f3bbd78 with result from task 4456 parsl.dataflow.dflow DEBUG: Task 4456 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/1b7f0786-7e29-4fc4-9b5c-d66d7c2772a4_isrForCrosstalkSources_996995_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4456 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/1b7f0786-7e29-4fc4-9b5c-d66d7c2772a4_isrForCrosstalkSources_996995_61.stderr parsl.dataflow.dflow INFO: Task 4363 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e36e6fbad48ba96e8c75b73e0303868c with result from task 4363 parsl.dataflow.dflow DEBUG: Task 4363 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8aa5ea30-d15b-41b9-b6b5-116742ef6077_isrForCrosstalkSources_997013_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4363 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/8aa5ea30-d15b-41b9-b6b5-116742ef6077_isrForCrosstalkSources_997013_52.stderr parsl.dataflow.dflow INFO: Task 4474 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7de673d5a88f674452e52070dae940d with result from task 4474 parsl.dataflow.dflow DEBUG: Task 4474 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/eec2344d-9524-40ff-a139-7e344862e906_isrForCrosstalkSources_996994_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4474 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/eec2344d-9524-40ff-a139-7e344862e906_isrForCrosstalkSources_996994_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:18 proc_lsst.multi:146 [INFO] found job 17075326 in provider checkpoint proc_lsst.multi INFO: found job 17075326 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4462 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 82a9d71eddbec6cdc755d77ecbe2aa48 with result from task 4462 parsl.dataflow.dflow DEBUG: Task 4462 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/ebdee1b8-825c-44bf-99d1-633d50a8cab9_isrForCrosstalkSources_996996_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4462 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/ebdee1b8-825c-44bf-99d1-633d50a8cab9_isrForCrosstalkSources_996996_10.stderr parsl.dataflow.dflow INFO: Task 4561 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3645470017304a41950f17bd53ba6afe with result from task 4561 parsl.dataflow.dflow DEBUG: Task 4561 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ebf34ba6-ff4e-421b-b33c-65d98958622f_isrForCrosstalkSources_996959_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4561 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/ebf34ba6-ff4e-421b-b33c-65d98958622f_isrForCrosstalkSources_996959_62.stderr parsl.dataflow.dflow INFO: Task 4719 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 09dd852fd8e55c1b095a9b0c5ea44434 with result from task 4719 parsl.dataflow.dflow DEBUG: Task 4719 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2bc20b56-b2b0-4399-a74c-cfda6f1b5a14_isrForCrosstalkSources_996970_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4719 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/2bc20b56-b2b0-4399-a74c-cfda6f1b5a14_isrForCrosstalkSources_996970_55.stderr parsl.dataflow.dflow INFO: Task 4330 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f3700385e277b545859ef03c8b2639a2 with result from task 4330 parsl.dataflow.dflow DEBUG: Task 4330 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/12af1763-235e-41e6-96dc-b0ff15195e76_isrForCrosstalkSources_997032_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4330 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/12af1763-235e-41e6-96dc-b0ff15195e76_isrForCrosstalkSources_997032_48.stderr parsl.dataflow.dflow INFO: Task 4418 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 43442687367c887f49ee6d1ec4d9dec2 with result from task 4418 parsl.dataflow.dflow DEBUG: Task 4418 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/8020b4ad-1f46-4b32-ae51-a7216e339602_isrForCrosstalkSources_997025_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4418 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/8020b4ad-1f46-4b32-ae51-a7216e339602_isrForCrosstalkSources_997025_37.stderr parsl.dataflow.dflow INFO: Task 4490 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8b58976a3f81371c025b600e41b5dd9 with result from task 4490 parsl.dataflow.dflow DEBUG: Task 4490 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8b9332d2-f57d-4a40-86df-4558599c9f69_isrForCrosstalkSources_996984_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4490 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8b9332d2-f57d-4a40-86df-4558599c9f69_isrForCrosstalkSources_996984_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:19 proc_lsst.multi:146 [INFO] found job 17075329 in provider checkpoint proc_lsst.multi INFO: found job 17075329 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4591 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd0ab147be7080653f737470bc4c4f83 with result from task 4591 parsl.dataflow.dflow DEBUG: Task 4591 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/16727007-99ff-4806-9d3d-9a60e453b3f5_isrForCrosstalkSources_996981_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4591 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/16727007-99ff-4806-9d3d-9a60e453b3f5_isrForCrosstalkSources_996981_22.stderr parsl.dataflow.dflow INFO: Task 4496 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1bc8fd54ba9f991d3e5afbe72484fbf5 with result from task 4496 parsl.dataflow.dflow DEBUG: Task 4496 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c092cae6-6bfa-4c6a-91e8-4cd6090e8e3d_isrForCrosstalkSources_996973_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4496 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c092cae6-6bfa-4c6a-91e8-4cd6090e8e3d_isrForCrosstalkSources_996973_44.stderr parsl.dataflow.dflow INFO: Task 4729 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ef712a8944312461be000fbaf5d4432 with result from task 4729 parsl.dataflow.dflow DEBUG: Task 4729 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b053d659-3263-426d-87a8-cba77cd2e384_isrForCrosstalkSources_997006_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4729 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b053d659-3263-426d-87a8-cba77cd2e384_isrForCrosstalkSources_997006_36.stderr parsl.dataflow.dflow INFO: Task 4733 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d50d3702955d2c551d3f820cb7705d8 with result from task 4733 parsl.dataflow.dflow DEBUG: Task 4733 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/a440741f-fc45-4551-8d73-cf26622e4c84_isrForCrosstalkSources_997025_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4733 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/a440741f-fc45-4551-8d73-cf26622e4c84_isrForCrosstalkSources_997025_7.stderr parsl.dataflow.dflow INFO: Task 4571 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d602f641b35c99a7d566ea903bee8e56 with result from task 4571 parsl.dataflow.dflow DEBUG: Task 4571 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5296c4f8-8ca6-45a7-a50c-053bc38a4655_isrForCrosstalkSources_996979_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4571 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/5296c4f8-8ca6-45a7-a50c-053bc38a4655_isrForCrosstalkSources_996979_34.stderr parsl.dataflow.dflow INFO: Task 4590 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 838ce56f89136b3c0c7fec1a06d25070 with result from task 4590 parsl.dataflow.dflow DEBUG: Task 4590 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/36f293b7-d472-4f23-8955-1c963fac7afa_isrForCrosstalkSources_997023_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4590 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/36f293b7-d472-4f23-8955-1c963fac7afa_isrForCrosstalkSources_997023_38.stderr parsl.dataflow.dflow INFO: Task 4432 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ac9e559633e72240c0d6279b321e2a6 with result from task 4432 parsl.dataflow.dflow DEBUG: Task 4432 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b2523a06-99ee-422e-a172-10e144c3de4f_isrForCrosstalkSources_996986_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4432 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/b2523a06-99ee-422e-a172-10e144c3de4f_isrForCrosstalkSources_996986_12.stderr parsl.dataflow.dflow INFO: Task 4443 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c614ee477bbaa1b82732a14fb9883422 with result from task 4443 parsl.dataflow.dflow DEBUG: Task 4443 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/7e4c6609-8eb1-443a-b676-ac47c6b74e5b_isrForCrosstalkSources_997032_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4443 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/7e4c6609-8eb1-443a-b676-ac47c6b74e5b_isrForCrosstalkSources_997032_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:20 proc_lsst.multi:146 [INFO] found job 17075330 in provider checkpoint proc_lsst.multi INFO: found job 17075330 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4520 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8040e234c3438ca0089a7ea30e205a30 with result from task 4520 parsl.dataflow.dflow DEBUG: Task 4520 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/f7b1bc31-59d9-4def-855e-1fe49c85827c_isrForCrosstalkSources_997013_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4520 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/f7b1bc31-59d9-4def-855e-1fe49c85827c_isrForCrosstalkSources_997013_31.stderr parsl.dataflow.dflow INFO: Task 4726 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a724a16f87be414bd8cf4b89f6c99cb with result from task 4726 parsl.dataflow.dflow DEBUG: Task 4726 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/659077f5-d3ab-4e5a-81a3-bef50a820427_isrForCrosstalkSources_996985_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4726 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/659077f5-d3ab-4e5a-81a3-bef50a820427_isrForCrosstalkSources_996985_35.stderr parsl.dataflow.dflow INFO: Task 4704 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b484cccde7a54fc6d7ce49c94e39937 with result from task 4704 parsl.dataflow.dflow DEBUG: Task 4704 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/b73cf6ba-514b-4f79-9145-9257d7237a8f_isrForCrosstalkSources_996982_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4704 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/b73cf6ba-514b-4f79-9145-9257d7237a8f_isrForCrosstalkSources_996982_8.stderr parsl.dataflow.dflow INFO: Task 4588 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11c02431c644151c16886f18ba7f0129 with result from task 4588 parsl.dataflow.dflow DEBUG: Task 4588 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ad65a8d0-a20a-44fa-a8d3-451a65814c11_isrForCrosstalkSources_997028_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4588 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/ad65a8d0-a20a-44fa-a8d3-451a65814c11_isrForCrosstalkSources_997028_53.stderr parsl.dataflow.dflow INFO: Task 4438 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e3cad89840dc2368686917db70ded505 with result from task 4438 parsl.dataflow.dflow DEBUG: Task 4438 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8c9b0c96-8ec1-42e7-9b3c-4bb2aafb58af_isrForCrosstalkSources_996952_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4438 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/8c9b0c96-8ec1-42e7-9b3c-4bb2aafb58af_isrForCrosstalkSources_996952_43.stderr parsl.dataflow.dflow INFO: Task 4592 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df59456d0b7e936ddf04474f579127ac with result from task 4592 parsl.dataflow.dflow DEBUG: Task 4592 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/db7fa4dc-eada-4b0d-8de0-3d5960309b9b_isrForCrosstalkSources_996998_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4592 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/db7fa4dc-eada-4b0d-8de0-3d5960309b9b_isrForCrosstalkSources_996998_29.stderr parsl.dataflow.dflow INFO: Task 4747 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6fc05bd02b1f4af0b874019fa00aa633 with result from task 4747 parsl.dataflow.dflow DEBUG: Task 4747 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/288a71ae-9d00-4a5a-9ec6-273258a6a4b5_isrForCrosstalkSources_996974_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4747 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/288a71ae-9d00-4a5a-9ec6-273258a6a4b5_isrForCrosstalkSources_996974_20.stderr parsl.dataflow.dflow INFO: Task 4547 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53c266a5e05cd53cb6a50ed7e8ad6a42 with result from task 4547 parsl.dataflow.dflow DEBUG: Task 4547 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/bc03580f-8c79-49bc-82a4-daaeb08324cf_isrForCrosstalkSources_996954_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4547 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/bc03580f-8c79-49bc-82a4-daaeb08324cf_isrForCrosstalkSources_996954_8.stderr parsl.dataflow.dflow INFO: Task 4736 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4284935388f66b63374b822c8fca0c58 with result from task 4736 parsl.dataflow.dflow DEBUG: Task 4736 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/c3d3fefe-b0c4-4a25-99d0-04c8f71525f3_isrForCrosstalkSources_997011_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4736 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/c3d3fefe-b0c4-4a25-99d0-04c8f71525f3_isrForCrosstalkSources_997011_52.stderr parsl.dataflow.dflow INFO: Task 4464 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c5366895035ecf3ca64bdd8192877ce with result from task 4464 parsl.dataflow.dflow DEBUG: Task 4464 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/124b1898-2523-417f-a3f8-8956ed0ee4f6_isrForCrosstalkSources_997022_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4464 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/124b1898-2523-417f-a3f8-8956ed0ee4f6_isrForCrosstalkSources_997022_28.stderr parsl.dataflow.dflow INFO: Task 4492 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0cc5c15875da2c403c8976de45a86113 with result from task 4492 parsl.dataflow.dflow DEBUG: Task 4492 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/19fd3b51-5996-4108-9a31-04cac7d9695a_isrForCrosstalkSources_997000_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4492 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/19fd3b51-5996-4108-9a31-04cac7d9695a_isrForCrosstalkSources_997000_1.stderr parsl.dataflow.dflow INFO: Task 4741 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 259f5b3280a60c46947c0cabc1c192cf with result from task 4741 parsl.dataflow.dflow DEBUG: Task 4741 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4ea4d1c2-59d1-44fe-9636-e838e3bc4f67_isrForCrosstalkSources_996995_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4741 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/4ea4d1c2-59d1-44fe-9636-e838e3bc4f67_isrForCrosstalkSources_996995_18.stderr parsl.dataflow.dflow INFO: Task 4742 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f65319fbe378a96366706278ef526648 with result from task 4742 parsl.dataflow.dflow DEBUG: Task 4742 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f1e46bfc-28ab-4a64-9e63-2ddba3d72411_isrForCrosstalkSources_996955_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4742 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/f1e46bfc-28ab-4a64-9e63-2ddba3d72411_isrForCrosstalkSources_996955_30.stderr parsl.dataflow.dflow INFO: Task 4517 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4bc970c5ed00a658eda118800ebaaf2 with result from task 4517 parsl.dataflow.dflow DEBUG: Task 4517 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c2b5932e-e2c7-4890-a210-9be81bf0ad7f_isrForCrosstalkSources_997028_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4517 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/c2b5932e-e2c7-4890-a210-9be81bf0ad7f_isrForCrosstalkSources_997028_48.stderr parsl.dataflow.dflow INFO: Task 4578 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a353cd25d7a99359c353e40c981d5bc with result from task 4578 parsl.dataflow.dflow DEBUG: Task 4578 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/39915e4b-74bb-4e5e-97bf-d55c41c880d3_isrForCrosstalkSources_996978_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4578 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/39915e4b-74bb-4e5e-97bf-d55c41c880d3_isrForCrosstalkSources_996978_53.stderr parsl.dataflow.dflow INFO: Task 4539 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b4e0dfe3e385c80eb1d536282287164 with result from task 4539 parsl.dataflow.dflow DEBUG: Task 4539 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a85eca6d-2f55-4539-a9e9-20d928adc8b8_isrForCrosstalkSources_997022_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4539 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/a85eca6d-2f55-4539-a9e9-20d928adc8b8_isrForCrosstalkSources_997022_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:21 proc_lsst.multi:146 [INFO] found job 17075332 in provider checkpoint proc_lsst.multi INFO: found job 17075332 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4748 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry abba48c2df605b4f702d135811822fc4 with result from task 4748 parsl.dataflow.dflow DEBUG: Task 4748 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/5c8b2e0b-1674-4c91-8d53-7130deef92fe_isrForCrosstalkSources_997028_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4748 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/5c8b2e0b-1674-4c91-8d53-7130deef92fe_isrForCrosstalkSources_997028_44.stderr parsl.dataflow.dflow INFO: Task 4566 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 182f20eb507ce6b6b809349ee0bd5efd with result from task 4566 parsl.dataflow.dflow DEBUG: Task 4566 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3d23b953-c7e0-42fe-aae7-6c74be9b5b93_isrForCrosstalkSources_996968_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4566 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/3d23b953-c7e0-42fe-aae7-6c74be9b5b93_isrForCrosstalkSources_996968_61.stderr parsl.dataflow.dflow INFO: Task 4589 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 95718e27bb758984c6cdd2ed4a4f30f8 with result from task 4589 parsl.dataflow.dflow DEBUG: Task 4589 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/ac19a2a3-7465-4f7e-b6b0-0a1e1c8f77ff_isrForCrosstalkSources_996971_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4589 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/ac19a2a3-7465-4f7e-b6b0-0a1e1c8f77ff_isrForCrosstalkSources_996971_43.stderr parsl.dataflow.dflow INFO: Task 4546 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0dd6ae8fe5938a409163ae0f1678c2ac with result from task 4546 parsl.dataflow.dflow DEBUG: Task 4546 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/0988359b-2882-4c14-9b32-db46f43b1239_isrForCrosstalkSources_997022_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4546 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/0988359b-2882-4c14-9b32-db46f43b1239_isrForCrosstalkSources_997022_15.stderr parsl.dataflow.dflow INFO: Task 4711 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2627c6435f4435608140bbf40a75374 with result from task 4711 parsl.dataflow.dflow DEBUG: Task 4711 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/9a686d53-c1c8-4207-ae8d-75e07846e652_isrForCrosstalkSources_997027_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4711 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/9a686d53-c1c8-4207-ae8d-75e07846e652_isrForCrosstalkSources_997027_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:22 proc_lsst.multi:146 [INFO] found job 17075338 in provider checkpoint proc_lsst.multi INFO: found job 17075338 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4705 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f9cc949ce79193865dde8eee37452e5 with result from task 4705 parsl.dataflow.dflow DEBUG: Task 4705 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/74f4b4c7-0d05-4b13-852e-904a11f81dd3_isrForCrosstalkSources_996952_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4705 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/74f4b4c7-0d05-4b13-852e-904a11f81dd3_isrForCrosstalkSources_996952_20.stderr parsl.dataflow.dflow INFO: Task 4594 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d9ab6ce64b789f0c6818a81bb8e06b4 with result from task 4594 parsl.dataflow.dflow DEBUG: Task 4594 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e0f5570a-40a5-4d9d-8af2-a9565d0d871c_isrForCrosstalkSources_997026_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4594 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/e0f5570a-40a5-4d9d-8af2-a9565d0d871c_isrForCrosstalkSources_997026_43.stderr parsl.dataflow.dflow INFO: Task 4740 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca8d4e862330eae59e2fbaebe114dbf7 with result from task 4740 parsl.dataflow.dflow DEBUG: Task 4740 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fab0ebb2-d8e2-4cf6-bb8e-f973da118f70_isrForCrosstalkSources_996990_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4740 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/fab0ebb2-d8e2-4cf6-bb8e-f973da118f70_isrForCrosstalkSources_996990_16.stderr parsl.dataflow.dflow INFO: Task 4508 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca22a3fcc5b80a3ac807de3d2088c2a5 with result from task 4508 parsl.dataflow.dflow DEBUG: Task 4508 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b40ad668-05ae-443e-8526-3b4f961bcff2_isrForCrosstalkSources_996975_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4508 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/b40ad668-05ae-443e-8526-3b4f961bcff2_isrForCrosstalkSources_996975_22.stderr parsl.dataflow.dflow INFO: Task 4731 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 724e58f6bc470764c34bbd2c9e3c35d0 with result from task 4731 parsl.dataflow.dflow DEBUG: Task 4731 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/fc853780-10e0-434f-8bf6-86267b155c69_isrForCrosstalkSources_996959_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4731 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/fc853780-10e0-434f-8bf6-86267b155c69_isrForCrosstalkSources_996959_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:22 proc_lsst.multi:146 [INFO] found job 17075344 in provider checkpoint proc_lsst.multi INFO: found job 17075344 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4504 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8e400749ae08e094f7b420d2c262dd8 with result from task 4504 parsl.dataflow.dflow DEBUG: Task 4504 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7eb5eb31-87bb-41e7-84cd-3e54bd50629b_isrForCrosstalkSources_997005_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4504 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7eb5eb31-87bb-41e7-84cd-3e54bd50629b_isrForCrosstalkSources_997005_15.stderr parsl.dataflow.dflow INFO: Task 4700 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4a07fab6cde2409a0c3ab8047355480 with result from task 4700 parsl.dataflow.dflow DEBUG: Task 4700 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c060dc7d-f52c-4606-8d69-bfa1d48bc60f_isrForCrosstalkSources_996967_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4700 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c060dc7d-f52c-4606-8d69-bfa1d48bc60f_isrForCrosstalkSources_996967_25.stderr parsl.dataflow.dflow INFO: Task 4764 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5c5589d62bea03c9f17feb8f7db7dd82 with result from task 4764 parsl.dataflow.dflow DEBUG: Task 4764 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ca9e5713-805e-4107-8ea0-febb81096f73_isrForCrosstalkSources_996982_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4764 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/ca9e5713-805e-4107-8ea0-febb81096f73_isrForCrosstalkSources_996982_50.stderr parsl.dataflow.dflow INFO: Task 4710 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 244a14d4de53609499822248519cee76 with result from task 4710 parsl.dataflow.dflow DEBUG: Task 4710 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/4a398c7e-3161-4fd7-b69e-1d8a854877d8_isrForCrosstalkSources_996952_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4710 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/4a398c7e-3161-4fd7-b69e-1d8a854877d8_isrForCrosstalkSources_996952_39.stderr parsl.dataflow.dflow INFO: Task 4551 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 418c52606167698cc76d261decb3090e with result from task 4551 parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:23 proc_lsst.multi:146 [INFO] found job 17075345 in provider checkpoint proc_lsst.multi INFO: found job 17075345 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4551 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ade2383b-381f-4822-abb1-791115a6b1c7_isrForCrosstalkSources_997016_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4551 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ade2383b-381f-4822-abb1-791115a6b1c7_isrForCrosstalkSources_997016_53.stderr parsl.dataflow.dflow INFO: Task 4348 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 085c044df93105ac27e2c6fb2379b423 with result from task 4348 parsl.dataflow.dflow DEBUG: Task 4348 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/50c14ec3-4256-4506-9963-275eeef245cc_isrForCrosstalkSources_997014_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4348 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/50c14ec3-4256-4506-9963-275eeef245cc_isrForCrosstalkSources_997014_3.stderr parsl.dataflow.dflow INFO: Task 4541 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 63aeb6dd02e00c5b2bdeb03273a1a089 with result from task 4541 parsl.dataflow.dflow DEBUG: Task 4541 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/377ef795-dd2d-4afc-8b86-b6893f8d9abe_isrForCrosstalkSources_996990_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4541 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/377ef795-dd2d-4afc-8b86-b6893f8d9abe_isrForCrosstalkSources_996990_43.stderr parsl.dataflow.dflow INFO: Task 4514 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d604bc5e9070439aace045bacc42552d with result from task 4514 parsl.dataflow.dflow DEBUG: Task 4514 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/9fd0283c-93f4-444f-baf1-f473778f4f8b_isrForCrosstalkSources_996999_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4514 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/9fd0283c-93f4-444f-baf1-f473778f4f8b_isrForCrosstalkSources_996999_62.stderr parsl.dataflow.dflow INFO: Task 4643 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b30b697c958def3dd606fba28ce2b29 with result from task 4643 parsl.dataflow.dflow DEBUG: Task 4643 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e0eee41f-12a7-46e9-948b-d6e3a183bdb6_isrForCrosstalkSources_996997_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4643 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/e0eee41f-12a7-46e9-948b-d6e3a183bdb6_isrForCrosstalkSources_996997_20.stderr parsl.dataflow.dflow INFO: Task 4512 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f2991f4cdfb385547cf23fb848bd63bc with result from task 4512 parsl.dataflow.dflow DEBUG: Task 4512 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7ac1fa30-5940-4ab2-af44-135a7b909c6a_isrForCrosstalkSources_996990_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4512 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/7ac1fa30-5940-4ab2-af44-135a7b909c6a_isrForCrosstalkSources_996990_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:23 proc_lsst.multi:146 [INFO] found job 17075346 in provider checkpoint proc_lsst.multi INFO: found job 17075346 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4760 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eddf44658e738a653ab1b5c0cdc340d1 with result from task 4760 parsl.dataflow.dflow DEBUG: Task 4760 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9ba11b09-ec88-4221-b0c5-9f7b66aa669a_isrForCrosstalkSources_996971_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4760 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/9ba11b09-ec88-4221-b0c5-9f7b66aa669a_isrForCrosstalkSources_996971_21.stderr parsl.dataflow.dflow INFO: Task 4695 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a495c257f7c10106cef2445c97402339 with result from task 4695 parsl.dataflow.dflow DEBUG: Task 4695 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/a0efdfbe-016c-4a81-89c3-be9a47f0a224_isrForCrosstalkSources_996979_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4695 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/a0efdfbe-016c-4a81-89c3-be9a47f0a224_isrForCrosstalkSources_996979_51.stderr parsl.dataflow.dflow INFO: Task 4774 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b0c41eb45aafb5b82a06e163271e86ff with result from task 4774 parsl.dataflow.dflow DEBUG: Task 4774 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b033272c-1a03-4c26-a264-03cfb99596b3_isrForCrosstalkSources_997001_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4774 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/b033272c-1a03-4c26-a264-03cfb99596b3_isrForCrosstalkSources_997001_23.stderr parsl.dataflow.dflow INFO: Task 4652 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1a111a298722d23517fbc5aa0ee2a369 with result from task 4652 parsl.dataflow.dflow DEBUG: Task 4652 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c99baec9-14b9-4ff6-85a4-282fed867581_isrForCrosstalkSources_996970_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4652 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/c99baec9-14b9-4ff6-85a4-282fed867581_isrForCrosstalkSources_996970_34.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:24 proc_lsst.multi:146 [INFO] found job 17075347 in provider checkpoint proc_lsst.multi INFO: found job 17075347 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4641 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a357425da399b8f7288b1db27a3f9927 with result from task 4641 parsl.dataflow.dflow DEBUG: Task 4641 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b888e1db-ddc8-489c-9f01-c1526516b021_isrForCrosstalkSources_997006_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4641 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/b888e1db-ddc8-489c-9f01-c1526516b021_isrForCrosstalkSources_997006_6.stderr parsl.dataflow.dflow INFO: Task 4713 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ff41d92618e7dcdd344e93ea2ee9444 with result from task 4713 parsl.dataflow.dflow DEBUG: Task 4713 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/42c9ded1-5588-4471-8cd4-e79823cc5325_isrForCrosstalkSources_996988_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4713 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/42c9ded1-5588-4471-8cd4-e79823cc5325_isrForCrosstalkSources_996988_18.stderr parsl.dataflow.dflow INFO: Task 4662 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 198e005f2304918d6697be271ac57c8b with result from task 4662 parsl.dataflow.dflow DEBUG: Task 4662 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/dc2e2908-114f-43f3-b856-a970463455d2_isrForCrosstalkSources_996981_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4662 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/dc2e2908-114f-43f3-b856-a970463455d2_isrForCrosstalkSources_996981_26.stderr parsl.dataflow.dflow INFO: Task 4540 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa9431838b4e3be91bf788b94c934a90 with result from task 4540 parsl.dataflow.dflow DEBUG: Task 4540 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/174d47ea-18b5-4735-9032-0af0685c38fe_isrForCrosstalkSources_997006_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4540 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/174d47ea-18b5-4735-9032-0af0685c38fe_isrForCrosstalkSources_997006_62.stderr parsl.dataflow.dflow INFO: Task 4751 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5df9bef9f46cb2dba6b003d88b528529 with result from task 4751 parsl.dataflow.dflow DEBUG: Task 4751 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f9adef0b-fc29-4bbf-888d-78df9809996e_isrForCrosstalkSources_996993_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4751 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f9adef0b-fc29-4bbf-888d-78df9809996e_isrForCrosstalkSources_996993_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:24 proc_lsst.multi:146 [INFO] found job 17075348 in provider checkpoint proc_lsst.multi INFO: found job 17075348 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4417 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2441b83fd7c4a5fa414c64763245052 with result from task 4417 parsl.dataflow.dflow DEBUG: Task 4417 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/febcfde9-e100-417a-9b21-2c5ba47163a1_isrForCrosstalkSources_996996_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4417 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/febcfde9-e100-417a-9b21-2c5ba47163a1_isrForCrosstalkSources_996996_58.stderr parsl.dataflow.dflow INFO: Task 4783 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 899aa96deffcf5485a2ae5df1cebc448 with result from task 4783 parsl.dataflow.dflow DEBUG: Task 4783 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/4d991ba3-bb7b-4eed-a9a0-8be2c43700f9_isrForCrosstalkSources_997010_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4783 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/4d991ba3-bb7b-4eed-a9a0-8be2c43700f9_isrForCrosstalkSources_997010_36.stderr parsl.dataflow.dflow INFO: Task 4668 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ba5508c6b20a9cadaa55208301e3b659 with result from task 4668 parsl.dataflow.dflow DEBUG: Task 4668 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f11c1aa8-1045-4e13-b549-a2577ef404e7_isrForCrosstalkSources_996978_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4668 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f11c1aa8-1045-4e13-b549-a2577ef404e7_isrForCrosstalkSources_996978_20.stderr parsl.dataflow.dflow INFO: Task 4676 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ecff01e0d9bbbad9c43c1684c154495 with result from task 4676 parsl.dataflow.dflow DEBUG: Task 4676 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5c45ac06-d339-4531-b917-a10b1d194c45_isrForCrosstalkSources_997012_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4676 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/5c45ac06-d339-4531-b917-a10b1d194c45_isrForCrosstalkSources_997012_10.stderr parsl.dataflow.dflow INFO: Task 4639 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3155ae8379959fdebb8f904d941951e with result from task 4639 parsl.dataflow.dflow DEBUG: Task 4639 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/1f1d188a-4b95-413b-825d-49aa687deb49_isrForCrosstalkSources_996999_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4639 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/1f1d188a-4b95-413b-825d-49aa687deb49_isrForCrosstalkSources_996999_32.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075059 PD 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:25 proc_lsst.multi:146 [INFO] found job 17075351 in provider checkpoint proc_lsst.multi INFO: found job 17075351 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4786 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 198c9d789b1ca6b2dd17fba69d42e39c with result from task 4786 parsl.dataflow.dflow DEBUG: Task 4786 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/5367b1ba-1ef0-4ba4-bc92-9c96ae679b5f_isrForCrosstalkSources_996957_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4786 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/5367b1ba-1ef0-4ba4-bc92-9c96ae679b5f_isrForCrosstalkSources_996957_34.stderr parsl.dataflow.dflow INFO: Task 4773 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ebdb315d684d9eb9db193308c0620a9 with result from task 4773 parsl.dataflow.dflow DEBUG: Task 4773 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/1bb4588b-5f06-4486-ac82-32e65c0553d1_isrForCrosstalkSources_997002_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4773 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/1bb4588b-5f06-4486-ac82-32e65c0553d1_isrForCrosstalkSources_997002_30.stderr parsl.dataflow.dflow INFO: Task 4696 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65124a6e4edc376164443156fbb8bcc9 with result from task 4696 parsl.dataflow.dflow DEBUG: Task 4696 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/94d02795-3efd-4a37-bc59-0c553216a47d_isrForCrosstalkSources_997012_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4696 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/94d02795-3efd-4a37-bc59-0c553216a47d_isrForCrosstalkSources_997012_22.stderr parsl.dataflow.dflow INFO: Task 4678 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4d27b9d5fb70960feef48490515e83e5 with result from task 4678 parsl.dataflow.dflow DEBUG: Task 4678 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/06533278-32d8-40ce-a703-cbd8b889ef77_isrForCrosstalkSources_997014_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4678 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/06533278-32d8-40ce-a703-cbd8b889ef77_isrForCrosstalkSources_997014_28.stderr parsl.dataflow.dflow INFO: Task 4787 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27cc17dcfb18e9148c4d4940f0036c8b with result from task 4787 parsl.dataflow.dflow DEBUG: Task 4787 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/95ed680b-f5d7-4578-bff3-694b89b74caf_isrForCrosstalkSources_996990_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4787 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/95ed680b-f5d7-4578-bff3-694b89b74caf_isrForCrosstalkSources_996990_12.stderr parsl.dataflow.dflow INFO: Task 4661 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 696f352113ebb9d1ad041123182678c4 with result from task 4661 parsl.dataflow.dflow DEBUG: Task 4661 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a921f02b-30ad-4591-9e0f-f16caf24c8ed_isrForCrosstalkSources_997033_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4661 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a921f02b-30ad-4591-9e0f-f16caf24c8ed_isrForCrosstalkSources_997033_2.stderr parsl.dataflow.dflow INFO: Task 4604 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 040bb792ad88026e1ed2a9f207333dd6 with result from task 4604 parsl.dataflow.dflow DEBUG: Task 4604 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fdd00b2c-a9a6-4969-bc12-1b0c223e28e2_isrForCrosstalkSources_997001_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4604 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/fdd00b2c-a9a6-4969-bc12-1b0c223e28e2_isrForCrosstalkSources_997001_32.stderr parsl.dataflow.dflow INFO: Task 4658 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0a8bb686c2cd5433bea528bc916d7506 with result from task 4658 parsl.dataflow.dflow DEBUG: Task 4658 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bde1ab4d-a4d5-4aa2-9ed0-b4113dcc2054_isrForCrosstalkSources_996959_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4658 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/bde1ab4d-a4d5-4aa2-9ed0-b4113dcc2054_isrForCrosstalkSources_996959_56.stderr parsl.dataflow.dflow INFO: Task 4560 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5853ded2b62883b22da51ef59c3b1635 with result from task 4560 parsl.dataflow.dflow DEBUG: Task 4560 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3dea8121-7481-422b-a458-57f8755ae1d4_isrForCrosstalkSources_997014_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4560 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/3dea8121-7481-422b-a458-57f8755ae1d4_isrForCrosstalkSources_997014_5.stderr parsl.dataflow.dflow INFO: Task 4686 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 209e79b1deef3f3d2588974d75fbfcad with result from task 4686 parsl.dataflow.dflow DEBUG: Task 4686 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fabc437c-528f-42a6-afc4-8536a51760c1_isrForCrosstalkSources_996955_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4686 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/fabc437c-528f-42a6-afc4-8536a51760c1_isrForCrosstalkSources_996955_29.stderr parsl.dataflow.dflow INFO: Task 4669 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1348151b0356fe96ed0695fe501707cf with result from task 4669 parsl.dataflow.dflow DEBUG: Task 4669 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/6ca087a6-fc6c-4f4c-953f-11fd73c4fb4a_isrForCrosstalkSources_996972_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4669 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/6ca087a6-fc6c-4f4c-953f-11fd73c4fb4a_isrForCrosstalkSources_996972_40.stderr parsl.dataflow.dflow INFO: Task 4635 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ba9ec84571a824f3ea5ca86745407d7 with result from task 4635 parsl.dataflow.dflow DEBUG: Task 4635 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/603cd3d1-73e3-4b09-aea5-9b0f1e2c9cad_isrForCrosstalkSources_997000_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4635 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/603cd3d1-73e3-4b09-aea5-9b0f1e2c9cad_isrForCrosstalkSources_997000_41.stderr parsl.dataflow.dflow INFO: Task 4694 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1045db5127fcf8e426b0f329b19cb353 with result from task 4694 parsl.dataflow.dflow DEBUG: Task 4694 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0f61838a-6ef6-45a6-815f-d7096d594921_isrForCrosstalkSources_997012_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4694 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/0f61838a-6ef6-45a6-815f-d7096d594921_isrForCrosstalkSources_997012_28.stderr parsl.dataflow.dflow INFO: Task 4601 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 630640bcb25ae8baf81a1fe615c694d5 with result from task 4601 parsl.dataflow.dflow DEBUG: Task 4601 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/24a0ecae-6783-4f1c-a24c-7b1bee977b79_isrForCrosstalkSources_996998_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4601 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/24a0ecae-6783-4f1c-a24c-7b1bee977b79_isrForCrosstalkSources_996998_5.stderr parsl.dataflow.dflow INFO: Task 4767 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23b5817bbfe158d6e1f191a0215f5a9c with result from task 4767 parsl.dataflow.dflow DEBUG: Task 4767 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3d660650-f1cb-462b-9190-2b357fb14683_isrForCrosstalkSources_996984_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4767 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/3d660650-f1cb-462b-9190-2b357fb14683_isrForCrosstalkSources_996984_41.stderr parsl.dataflow.dflow INFO: Task 4660 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 653a565982151521536e1542fd0695d5 with result from task 4660 parsl.dataflow.dflow DEBUG: Task 4660 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/90a1370d-5aac-4252-968a-fe585e30e2eb_isrForCrosstalkSources_996959_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4660 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/90a1370d-5aac-4252-968a-fe585e30e2eb_isrForCrosstalkSources_996959_57.stderr parsl.dataflow.dflow INFO: Task 4797 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 60c2582222e057db6b605def0de6aaa4 with result from task 4797 parsl.dataflow.dflow DEBUG: Task 4797 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/cc1aa648-a89b-4111-bc55-6b780064702b_isrForCrosstalkSources_996961_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4797 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/cc1aa648-a89b-4111-bc55-6b780064702b_isrForCrosstalkSources_996961_12.stderr parsl.dataflow.dflow INFO: Task 4630 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6f3161c2e2a7fb187562c786cfb3788 with result from task 4630 parsl.dataflow.dflow DEBUG: Task 4630 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e8d3a893-8f6b-4b1f-9dc2-966796d59ba9_isrForCrosstalkSources_996978_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4630 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/e8d3a893-8f6b-4b1f-9dc2-966796d59ba9_isrForCrosstalkSources_996978_9.stderr parsl.dataflow.dflow INFO: Task 4600 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af21d2dbe97d8c1adaf82fcdde6e3013 with result from task 4600 parsl.dataflow.dflow DEBUG: Task 4600 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/39c872d3-7ef2-4014-8fce-dd2cea52f3b3_isrForCrosstalkSources_996979_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4600 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/39c872d3-7ef2-4014-8fce-dd2cea52f3b3_isrForCrosstalkSources_996979_7.stderr parsl.dataflow.dflow INFO: Task 4792 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03e3b4cd266fdc32231b276f2b2c863f with result from task 4792 parsl.dataflow.dflow DEBUG: Task 4792 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/a8b85d1a-2074-4869-9f17-b17235e33549_isrForCrosstalkSources_996987_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4792 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/a8b85d1a-2074-4869-9f17-b17235e33549_isrForCrosstalkSources_996987_32.stderr parsl.dataflow.dflow INFO: Task 4644 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9566da025c0e3755051eb88cb750081b with result from task 4644 parsl.dataflow.dflow DEBUG: Task 4644 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/5f5acaa2-7aff-4926-8d8d-40a3f91cabf3_isrForCrosstalkSources_997019_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4644 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997019/5f5acaa2-7aff-4926-8d8d-40a3f91cabf3_isrForCrosstalkSources_997019_62.stderr parsl.dataflow.dflow INFO: Task 4611 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51b17ae926f57cdbf47e05fe85465f7c with result from task 4611 parsl.dataflow.dflow DEBUG: Task 4611 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ff884701-8a7f-41df-bfa2-cd4650d9e556_isrForCrosstalkSources_996988_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4611 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/ff884701-8a7f-41df-bfa2-cd4650d9e556_isrForCrosstalkSources_996988_37.stderr parsl.dataflow.dflow INFO: Task 4800 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de9133980a07cc64abf294c2eebbb527 with result from task 4800 parsl.dataflow.dflow DEBUG: Task 4800 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a3d8b0d4-1d25-4358-9160-bc37f1f45e0f_isrForCrosstalkSources_996956_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4800 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a3d8b0d4-1d25-4358-9160-bc37f1f45e0f_isrForCrosstalkSources_996956_10.stderr parsl.dataflow.dflow INFO: Task 4608 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry edc86d86327bf793113d86888df2122b with result from task 4608 parsl.dataflow.dflow DEBUG: Task 4608 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/e1bc9a66-f6be-4a48-bd9b-b8250f8f2202_isrForCrosstalkSources_996971_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4608 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/e1bc9a66-f6be-4a48-bd9b-b8250f8f2202_isrForCrosstalkSources_996971_31.stderr parsl.dataflow.dflow INFO: Task 4697 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01008868bbe9861976d4678f7f73c3c9 with result from task 4697 parsl.dataflow.dflow DEBUG: Task 4697 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a44d2656-b814-443a-bfd4-4c0da18fd462_isrForCrosstalkSources_996952_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4697 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a44d2656-b814-443a-bfd4-4c0da18fd462_isrForCrosstalkSources_996952_7.stderr parsl.dataflow.dflow INFO: Task 4642 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 64159e12f681501c97d7749f00356746 with result from task 4642 parsl.dataflow.dflow DEBUG: Task 4642 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c97dec21-67fd-450c-a9bf-0f4f71604ca0_isrForCrosstalkSources_996994_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4642 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/c97dec21-67fd-450c-a9bf-0f4f71604ca0_isrForCrosstalkSources_996994_26.stderr parsl.dataflow.dflow INFO: Task 4633 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 37864f37520963b9b38d23aeec299d68 with result from task 4633 parsl.dataflow.dflow DEBUG: Task 4633 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/372a08c5-2767-45f7-97d7-f2aaaafd4539_isrForCrosstalkSources_996999_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4633 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/372a08c5-2767-45f7-97d7-f2aaaafd4539_isrForCrosstalkSources_996999_8.stderr parsl.dataflow.dflow INFO: Task 4683 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aa94b9e74c156ecbd072879489eec8e7 with result from task 4683 parsl.dataflow.dflow DEBUG: Task 4683 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a2f594ef-46b5-4a9a-8dab-6efe57a32117_isrForCrosstalkSources_996996_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4683 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/a2f594ef-46b5-4a9a-8dab-6efe57a32117_isrForCrosstalkSources_996996_33.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075136 PD 17075135 PD 17075134 PD 17075133 PD 17075132 PD 17075131 PD 17075129 PD 17075128 PD 17075127 PD 17075124 PD 17075122 PD 17075121 PD 17075118 PD 17075117 PD 17075116 PD 17075115 PD 17075114 PD 17075113 PD 17075112 PD 17075111 PD 17075110 PD 17075109 PD 17075103 PD 17075102 PD 17075101 PD 17075100 PD 17075099 PD 17075098 PD 17075097 PD 17075096 PD 17075095 PD 17075094 PD 17075093 PD 17075092 PD 17075091 PD 17075090 PD 17075088 PD 17075086 PD 17075085 PD 17075083 PD 17075082 PD 17075081 PD 17075080 PD 17075078 PD 17075076 PD 17075075 PD 17075074 PD 17075073 PD 17075072 PD 17075069 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:27 proc_lsst.multi:146 [INFO] found job 17075353 in provider checkpoint proc_lsst.multi INFO: found job 17075353 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4732 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c495ba56a3765facfe0e178b011bb1c with result from task 4732 parsl.dataflow.dflow DEBUG: Task 4732 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0134c818-d3ec-47ad-a7a3-c9266032b9c5_isrForCrosstalkSources_997025_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4732 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/0134c818-d3ec-47ad-a7a3-c9266032b9c5_isrForCrosstalkSources_997025_26.stderr parsl.dataflow.dflow INFO: Task 4789 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 38dfee8fc7ebfaf3d200b3f089be3a11 with result from task 4789 parsl.dataflow.dflow DEBUG: Task 4789 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d1a65c13-5878-4414-a268-9de23aba57d7_isrForCrosstalkSources_996980_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4789 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/d1a65c13-5878-4414-a268-9de23aba57d7_isrForCrosstalkSources_996980_60.stderr parsl.dataflow.dflow INFO: Task 4784 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4066e8fddcafe1c5bf4a9b02f69578e with result from task 4784 parsl.dataflow.dflow DEBUG: Task 4784 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e82a1d63-5d3c-4dc5-b8d5-44b8f9698a25_isrForCrosstalkSources_997002_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4784 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/e82a1d63-5d3c-4dc5-b8d5-44b8f9698a25_isrForCrosstalkSources_997002_59.stderr parsl.dataflow.dflow INFO: Task 4651 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry afac47251c3b0626acad3ce27589c44d with result from task 4651 parsl.dataflow.dflow DEBUG: Task 4651 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/789ba7a9-f5dd-47af-a688-f249548a37c2_isrForCrosstalkSources_996967_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4651 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/789ba7a9-f5dd-47af-a688-f249548a37c2_isrForCrosstalkSources_996967_48.stderr parsl.dataflow.dflow INFO: Task 4666 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 660f18cece6c03e5cace12f7ceefb82b with result from task 4666 parsl.dataflow.dflow DEBUG: Task 4666 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/b719fabc-74ab-4910-b8b7-b1d497762cfd_isrForCrosstalkSources_997017_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4666 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/b719fabc-74ab-4910-b8b7-b1d497762cfd_isrForCrosstalkSources_997017_36.stderr parsl.dataflow.dflow INFO: Task 4795 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23e0a511eef3b3519b8cdb4480bd9517 with result from task 4795 parsl.dataflow.dflow DEBUG: Task 4795 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/49fdecdd-f247-428b-b230-ad716093e476_isrForCrosstalkSources_997029_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4795 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/49fdecdd-f247-428b-b230-ad716093e476_isrForCrosstalkSources_997029_34.stderr parsl.dataflow.dflow INFO: Task 4581 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fb9e6014f622689e234a3b663c65367 with result from task 4581 parsl.dataflow.dflow DEBUG: Task 4581 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3255b58c-40ff-4b43-b764-d53f5f4519b9_isrForCrosstalkSources_996959_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4581 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/3255b58c-40ff-4b43-b764-d53f5f4519b9_isrForCrosstalkSources_996959_25.stderr parsl.dataflow.dflow INFO: Task 4679 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry da6175d18d8eaac4143e2c51f85526de with result from task 4679 parsl.dataflow.dflow DEBUG: Task 4679 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f85b2d43-2331-4bd6-a45c-41ee189396fa_isrForCrosstalkSources_996960_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4679 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f85b2d43-2331-4bd6-a45c-41ee189396fa_isrForCrosstalkSources_996960_25.stderr parsl.dataflow.dflow INFO: Task 4632 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d98fd5c527fc5a246ef8eac07da31339 with result from task 4632 parsl.dataflow.dflow DEBUG: Task 4632 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/5973f750-05eb-473d-9b29-eead1634baf4_isrForCrosstalkSources_996953_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4632 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/5973f750-05eb-473d-9b29-eead1634baf4_isrForCrosstalkSources_996953_24.stderr parsl.dataflow.dflow INFO: Task 4720 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bafcdfeed783205317a2ac9c276396bf with result from task 4720 parsl.dataflow.dflow DEBUG: Task 4720 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6b1bc84e-1a81-442f-9ce9-4f835c87d558_isrForCrosstalkSources_997010_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4720 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/6b1bc84e-1a81-442f-9ce9-4f835c87d558_isrForCrosstalkSources_997010_34.stderr parsl.dataflow.dflow INFO: Task 4557 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab4b18cfc6a6ac270780eabdc592909d with result from task 4557 parsl.dataflow.dflow DEBUG: Task 4557 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/3a0f1645-4446-4648-8f83-5c9b3508c0e0_isrForCrosstalkSources_996993_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4557 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/3a0f1645-4446-4648-8f83-5c9b3508c0e0_isrForCrosstalkSources_996993_32.stderr parsl.dataflow.dflow INFO: Task 4721 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1c78cea0f79370c6eec4a0ad7884e67d with result from task 4721 parsl.dataflow.dflow DEBUG: Task 4721 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9fa4c950-7f08-4901-9c86-b3df04082ebd_isrForCrosstalkSources_997023_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4721 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/9fa4c950-7f08-4901-9c86-b3df04082ebd_isrForCrosstalkSources_997023_32.stderr parsl.dataflow.dflow INFO: Task 4606 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f4e76ce0a8f7ce20c9b746fcb6e20b21 with result from task 4606 parsl.dataflow.dflow DEBUG: Task 4606 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a9632314-8900-465f-850c-499c4a53c152_isrForCrosstalkSources_996952_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4606 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/a9632314-8900-465f-850c-499c4a53c152_isrForCrosstalkSources_996952_19.stderr parsl.dataflow.dflow INFO: Task 4605 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4ed778cac6bf5afbd3db378327cd9866 with result from task 4605 parsl.dataflow.dflow DEBUG: Task 4605 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/da0fd872-8cd2-47b9-9c31-1b9bd9d7b66f_isrForCrosstalkSources_996965_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4605 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/da0fd872-8cd2-47b9-9c31-1b9bd9d7b66f_isrForCrosstalkSources_996965_26.stderr parsl.dataflow.dflow INFO: Task 4647 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bf7012010e5db97c12066084b7ebcc36 with result from task 4647 parsl.dataflow.dflow DEBUG: Task 4647 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/a487f763-84c0-4ce8-b87b-4676e49a8de5_isrForCrosstalkSources_997016_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4647 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/a487f763-84c0-4ce8-b87b-4676e49a8de5_isrForCrosstalkSources_997016_13.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:29 proc_lsst.multi:146 [INFO] found job 17075354 in provider checkpoint proc_lsst.multi INFO: found job 17075354 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4839 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b870536c1cd419045537b01f42fef87a with result from task 4839 parsl.dataflow.dflow DEBUG: Task 4839 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fb32cfbe-03c7-4cd5-a375-79eab16192bc_isrForCrosstalkSources_996961_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4839 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/fb32cfbe-03c7-4cd5-a375-79eab16192bc_isrForCrosstalkSources_996961_45.stderr parsl.dataflow.dflow INFO: Task 4807 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 642d15f004c9ea34be11406471dd160a with result from task 4807 parsl.dataflow.dflow DEBUG: Task 4807 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/236dcf1f-0e01-4358-9258-4e4dc1375e4c_isrForCrosstalkSources_996960_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4807 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/236dcf1f-0e01-4358-9258-4e4dc1375e4c_isrForCrosstalkSources_996960_20.stderr parsl.dataflow.dflow INFO: Task 4349 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d8e1d77d03bd2e84d2ed7db08811b34b with result from task 4349 parsl.dataflow.dflow DEBUG: Task 4349 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d1bf0ece-63f1-4ef7-9145-7ac5d3467628_isrForCrosstalkSources_997001_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4349 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/d1bf0ece-63f1-4ef7-9145-7ac5d3467628_isrForCrosstalkSources_997001_48.stderr parsl.dataflow.dflow INFO: Task 4840 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 392e3b5f89c96fa6f6bbbd2bd1f1696c with result from task 4840 parsl.dataflow.dflow DEBUG: Task 4840 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/d5e5a495-f4bf-44e9-9dff-5a9dc5f9d226_isrForCrosstalkSources_996956_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4840 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/d5e5a495-f4bf-44e9-9dff-5a9dc5f9d226_isrForCrosstalkSources_996956_24.stderr parsl.dataflow.dflow INFO: Task 4648 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c2f8bfe3e8e229e8dee7f0396e43aaa with result from task 4648 parsl.dataflow.dflow DEBUG: Task 4648 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/947fb6df-cd6a-47f4-abb3-fcfe00a5358c_isrForCrosstalkSources_997005_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4648 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/947fb6df-cd6a-47f4-abb3-fcfe00a5358c_isrForCrosstalkSources_997005_19.stderr parsl.dataflow.dflow INFO: Task 4599 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0bf19f52c921533ad85975ce240fca92 with result from task 4599 parsl.dataflow.dflow DEBUG: Task 4599 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/7f0c9631-85fd-48f5-aacb-9c7833a9ac95_isrForCrosstalkSources_996976_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4599 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/7f0c9631-85fd-48f5-aacb-9c7833a9ac95_isrForCrosstalkSources_996976_45.stderr parsl.dataflow.dflow INFO: Task 4826 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e31bb7b6e2321157c0d4948ec2a5c110 with result from task 4826 parsl.dataflow.dflow DEBUG: Task 4826 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/2baff398-d696-424e-a761-17a6807b66e6_isrForCrosstalkSources_996976_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4826 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/2baff398-d696-424e-a761-17a6807b66e6_isrForCrosstalkSources_996976_43.stderr parsl.dataflow.dflow INFO: Task 4838 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3bb9285dea6be423752d09356f5ca9a3 with result from task 4838 parsl.dataflow.dflow DEBUG: Task 4838 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/74d06804-736d-49fb-af8f-47aa089629a1_isrForCrosstalkSources_996978_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4838 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/74d06804-736d-49fb-af8f-47aa089629a1_isrForCrosstalkSources_996978_24.stderr parsl.dataflow.dflow INFO: Task 4756 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02af1547ee02c2a18f16e0997f93bb76 with result from task 4756 parsl.dataflow.dflow DEBUG: Task 4756 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/fceae346-14ce-46b2-aeea-d0f8d1dc41e4_isrForCrosstalkSources_996952_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4756 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/fceae346-14ce-46b2-aeea-d0f8d1dc41e4_isrForCrosstalkSources_996952_14.stderr parsl.dataflow.dflow INFO: Task 4617 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce420b5588214dbc6e54fcc62485b735 with result from task 4617 parsl.dataflow.dflow DEBUG: Task 4617 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/01ffc209-77ff-41b6-a36d-706d8792e6ee_isrForCrosstalkSources_997025_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4617 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/01ffc209-77ff-41b6-a36d-706d8792e6ee_isrForCrosstalkSources_997025_19.stderr parsl.dataflow.dflow INFO: Task 4618 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 861a71c7ba83453d07abb29eb497740f with result from task 4618 parsl.dataflow.dflow DEBUG: Task 4618 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2a2e5f04-d95f-4a00-98d8-bc75d383a498_isrForCrosstalkSources_996989_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4618 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/2a2e5f04-d95f-4a00-98d8-bc75d383a498_isrForCrosstalkSources_996989_1.stderr parsl.dataflow.dflow INFO: Task 4640 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9ba755da70735c298f716ed60466571 with result from task 4640 parsl.dataflow.dflow DEBUG: Task 4640 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3027b5ec-ca59-422b-926c-2ddbd8f8e3c8_isrForCrosstalkSources_996988_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4640 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/3027b5ec-ca59-422b-926c-2ddbd8f8e3c8_isrForCrosstalkSources_996988_10.stderr parsl.dataflow.dflow INFO: Task 4854 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ee1c6730314bb6a9b03194893dd3899a with result from task 4854 parsl.dataflow.dflow DEBUG: Task 4854 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/461210a5-4f4f-4217-b019-f873b4b7696b_isrForCrosstalkSources_997011_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4854 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/461210a5-4f4f-4217-b019-f873b4b7696b_isrForCrosstalkSources_997011_34.stderr parsl.dataflow.dflow INFO: Task 4636 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 02b7cba786f02c768e537ee3bb16b26d with result from task 4636 parsl.dataflow.dflow DEBUG: Task 4636 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/3b702984-18c3-46c7-a4d4-9f3815a9bed3_isrForCrosstalkSources_996998_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4636 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/3b702984-18c3-46c7-a4d4-9f3815a9bed3_isrForCrosstalkSources_996998_24.stderr parsl.dataflow.dflow INFO: Task 4865 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eaadffc22bfcf784a7c2fdc5500f41c9 with result from task 4865 parsl.dataflow.dflow DEBUG: Task 4865 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ec80df07-ff55-477e-b5e7-abbc3a87be50_isrForCrosstalkSources_996954_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4865 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/ec80df07-ff55-477e-b5e7-abbc3a87be50_isrForCrosstalkSources_996954_19.stderr parsl.dataflow.dflow INFO: Task 4563 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 144638e36e239a788b8288a9a0eba7b1 with result from task 4563 parsl.dataflow.dflow DEBUG: Task 4563 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c0ce3481-2983-4199-8802-4a8294496656_isrForCrosstalkSources_997004_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4563 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/c0ce3481-2983-4199-8802-4a8294496656_isrForCrosstalkSources_997004_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:30 proc_lsst.multi:146 [INFO] found job 17075355 in provider checkpoint proc_lsst.multi INFO: found job 17075355 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4728 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99ac3d388365832a6e1cb8eca930f9d0 with result from task 4728 parsl.dataflow.dflow DEBUG: Task 4728 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/49ab4feb-9f37-441f-8fe3-38531aa1560c_isrForCrosstalkSources_996985_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4728 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/49ab4feb-9f37-441f-8fe3-38531aa1560c_isrForCrosstalkSources_996985_40.stderr parsl.dataflow.dflow INFO: Task 4659 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a462967feafeeeab2a12534cbab87068 with result from task 4659 parsl.dataflow.dflow DEBUG: Task 4659 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/efb92a83-5bdc-4959-be37-906a430a2a99_isrForCrosstalkSources_996952_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4659 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/efb92a83-5bdc-4959-be37-906a430a2a99_isrForCrosstalkSources_996952_15.stderr parsl.dataflow.dflow INFO: Task 4572 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb7412ae894769bd83403bc242b2b17e with result from task 4572 parsl.dataflow.dflow DEBUG: Task 4572 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ab1118d9-3370-414c-9610-d947bc966921_isrForCrosstalkSources_997005_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4572 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ab1118d9-3370-414c-9610-d947bc966921_isrForCrosstalkSources_997005_17.stderr parsl.dataflow.dflow INFO: Task 4612 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 14f1a2533a70f41e436f24673765f6b7 with result from task 4612 parsl.dataflow.dflow DEBUG: Task 4612 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c3e36a4f-d697-4bd6-bf5c-8b63062015b0_isrForCrosstalkSources_996992_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4612 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/c3e36a4f-d697-4bd6-bf5c-8b63062015b0_isrForCrosstalkSources_996992_17.stderr parsl.dataflow.dflow INFO: Task 4674 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b44e158c9251d83a99cf3bb3a6f65ac4 with result from task 4674 parsl.dataflow.dflow DEBUG: Task 4674 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/38438d2e-c480-4f91-848e-d20fabb47943_isrForCrosstalkSources_996992_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4674 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/38438d2e-c480-4f91-848e-d20fabb47943_isrForCrosstalkSources_996992_56.stderr parsl.dataflow.dflow INFO: Task 4584 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8aa705bd23354db82416730553b77c91 with result from task 4584 parsl.dataflow.dflow DEBUG: Task 4584 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1d95aa2-5efe-491c-a963-b48f4ee12adb_isrForCrosstalkSources_996975_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4584 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/c1d95aa2-5efe-491c-a963-b48f4ee12adb_isrForCrosstalkSources_996975_59.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:31 proc_lsst.multi:146 [INFO] found job 17075357 in provider checkpoint proc_lsst.multi INFO: found job 17075357 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4616 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ca7ddd70d2a46ff334472b4d6f11319 with result from task 4616 parsl.dataflow.dflow DEBUG: Task 4616 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/aef47b4e-02cf-4812-959e-2e21ad4ab3e2_isrForCrosstalkSources_997023_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4616 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/aef47b4e-02cf-4812-959e-2e21ad4ab3e2_isrForCrosstalkSources_997023_12.stderr parsl.dataflow.dflow INFO: Task 4675 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53eeee45646091b8ad3bbb0b806c26ae with result from task 4675 parsl.dataflow.dflow DEBUG: Task 4675 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/2d68e52c-7df0-4ab6-82d6-7faf7879a021_isrForCrosstalkSources_997014_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4675 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/2d68e52c-7df0-4ab6-82d6-7faf7879a021_isrForCrosstalkSources_997014_36.stderr parsl.dataflow.dflow INFO: Task 4692 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6299f4d222da45671bacb9bc32fd0bec with result from task 4692 parsl.dataflow.dflow DEBUG: Task 4692 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bb70c79f-09e1-4e08-baf1-11e4ad2f05de_isrForCrosstalkSources_997030_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4692 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bb70c79f-09e1-4e08-baf1-11e4ad2f05de_isrForCrosstalkSources_997030_56.stderr parsl.dataflow.dflow INFO: Task 4893 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 52b8d269de5577720f05757a3a75dd78 with result from task 4893 parsl.dataflow.dflow DEBUG: Task 4893 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/2449369e-4d49-41fa-849b-008f3fbf6b77_isrForCrosstalkSources_997018_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4893 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/2449369e-4d49-41fa-849b-008f3fbf6b77_isrForCrosstalkSources_997018_26.stderr parsl.dataflow.dflow INFO: Task 4888 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 109716ce1c6a76ffc5a3f5d2d8674ce5 with result from task 4888 parsl.dataflow.dflow DEBUG: Task 4888 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/58aaa06e-87aa-4c63-a938-3e3a0b8617f1_isrForCrosstalkSources_997028_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4888 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/58aaa06e-87aa-4c63-a938-3e3a0b8617f1_isrForCrosstalkSources_997028_5.stderr parsl.dataflow.dflow INFO: Task 4716 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 31e99e4e3a6d91517ff16f20111cd352 with result from task 4716 parsl.dataflow.dflow DEBUG: Task 4716 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/c5fd04df-6de1-4939-81cb-684c8d6c2ce4_isrForCrosstalkSources_996979_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4716 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/c5fd04df-6de1-4939-81cb-684c8d6c2ce4_isrForCrosstalkSources_996979_20.stderr parsl.dataflow.dflow INFO: Task 4621 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25b01f1d029dcf730837f1d175c22c8a with result from task 4621 parsl.dataflow.dflow DEBUG: Task 4621 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f73343b2-fbd9-4271-9535-e8ee33f360e6_isrForCrosstalkSources_996999_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4621 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/f73343b2-fbd9-4271-9535-e8ee33f360e6_isrForCrosstalkSources_996999_33.stderr parsl.dataflow.dflow INFO: Task 4770 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4b3990b09d27a8d73ded54ecf467f7ec with result from task 4770 parsl.dataflow.dflow DEBUG: Task 4770 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e16d8268-0419-4ce7-962e-d8917dbb107b_isrForCrosstalkSources_997005_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4770 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/e16d8268-0419-4ce7-962e-d8917dbb107b_isrForCrosstalkSources_997005_45.stderr parsl.dataflow.dflow INFO: Task 4758 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd0e07a938e25a8c915b88907d082958 with result from task 4758 parsl.dataflow.dflow DEBUG: Task 4758 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/3a026ef4-f910-484e-99f5-b25ca316353c_isrForCrosstalkSources_997027_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4758 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/3a026ef4-f910-484e-99f5-b25ca316353c_isrForCrosstalkSources_997027_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:31 proc_lsst.multi:146 [INFO] found job 17075358 in provider checkpoint proc_lsst.multi INFO: found job 17075358 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4622 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 317336367868ba4b9d7e4c2a6c9c051e with result from task 4622 parsl.dataflow.dflow DEBUG: Task 4622 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/5ed8216e-d0af-4991-9d20-559356342810_isrForCrosstalkSources_997027_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4622 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/5ed8216e-d0af-4991-9d20-559356342810_isrForCrosstalkSources_997027_51.stderr parsl.dataflow.dflow INFO: Task 4871 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e00fdfecbd795c832baa0632576ef624 with result from task 4871 parsl.dataflow.dflow DEBUG: Task 4871 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/de339a2e-4a5c-4ade-bdd5-077a44e82373_isrForCrosstalkSources_996974_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4871 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/de339a2e-4a5c-4ade-bdd5-077a44e82373_isrForCrosstalkSources_996974_17.stderr parsl.dataflow.dflow INFO: Task 4702 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4137ac720d33ca2bb25c0616d1a32a85 with result from task 4702 parsl.dataflow.dflow DEBUG: Task 4702 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a2e10751-f7b4-4f3a-9d0a-8db3bd01fc84_isrForCrosstalkSources_996956_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4702 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a2e10751-f7b4-4f3a-9d0a-8db3bd01fc84_isrForCrosstalkSources_996956_56.stderr parsl.dataflow.dflow INFO: Task 4857 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01803a52899fabe292afa454574b35ea with result from task 4857 parsl.dataflow.dflow DEBUG: Task 4857 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/0af8e2db-4557-4350-a77f-68634768dfc1_isrForCrosstalkSources_996989_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4857 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/0af8e2db-4557-4350-a77f-68634768dfc1_isrForCrosstalkSources_996989_38.stderr parsl.dataflow.dflow INFO: Task 4562 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1d70dbe88092811040a3be2f659cdcf0 with result from task 4562 parsl.dataflow.dflow DEBUG: Task 4562 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/41b841f3-82f0-465f-a8c1-a5385f4f403d_isrForCrosstalkSources_997033_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4562 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/41b841f3-82f0-465f-a8c1-a5385f4f403d_isrForCrosstalkSources_997033_12.stderr parsl.dataflow.dflow INFO: Task 4620 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c34e680cbac7338b465960129c54ee3e with result from task 4620 parsl.dataflow.dflow DEBUG: Task 4620 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/ce65000b-1280-46bc-99e0-6bb147553e92_isrForCrosstalkSources_996968_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4620 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/ce65000b-1280-46bc-99e0-6bb147553e92_isrForCrosstalkSources_996968_52.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:32 proc_lsst.multi:146 [INFO] found job 17075359 in provider checkpoint proc_lsst.multi INFO: found job 17075359 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4855 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ac2a4d24d744aed87dbbe9fd4013dd76 with result from task 4855 parsl.dataflow.dflow DEBUG: Task 4855 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/3b2c707a-daec-4f68-a6ec-e0fadca47ee8_isrForCrosstalkSources_997023_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4855 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/3b2c707a-daec-4f68-a6ec-e0fadca47ee8_isrForCrosstalkSources_997023_51.stderr parsl.dataflow.dflow INFO: Task 4885 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42a63c9ef42b573929115d0644d89d1b with result from task 4885 parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:33 proc_lsst.multi:146 [INFO] found job 17075360 in provider checkpoint proc_lsst.multi INFO: found job 17075360 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4885 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2775c0f8-929c-4f95-b590-20a3cba77e4d_isrForCrosstalkSources_996962_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4885 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/2775c0f8-929c-4f95-b590-20a3cba77e4d_isrForCrosstalkSources_996962_51.stderr parsl.dataflow.dflow INFO: Task 4872 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a595b60e53a48624192f35576e349c69 with result from task 4872 parsl.dataflow.dflow DEBUG: Task 4872 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/89d9acef-6c9e-4ddf-a76a-686e13570b89_isrForCrosstalkSources_996987_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4872 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/89d9acef-6c9e-4ddf-a76a-686e13570b89_isrForCrosstalkSources_996987_45.stderr parsl.dataflow.dflow INFO: Task 4684 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b505a4e992306b3c036c6c1b4bcdc74d with result from task 4684 parsl.dataflow.dflow DEBUG: Task 4684 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8c217af1-d0b1-4c4c-af29-67d8527a1399_isrForCrosstalkSources_996984_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4684 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/8c217af1-d0b1-4c4c-af29-67d8527a1399_isrForCrosstalkSources_996984_15.stderr parsl.dataflow.dflow INFO: Task 4577 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56586b9571ff74e9521076831e01b944 with result from task 4577 parsl.dataflow.dflow DEBUG: Task 4577 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/473e6caa-524f-4a66-aa68-de539ca02081_isrForCrosstalkSources_996971_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4577 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/473e6caa-524f-4a66-aa68-de539ca02081_isrForCrosstalkSources_996971_15.stderr parsl.dataflow.dflow INFO: Task 4738 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 05d731541e54902a7f690b2be938c230 with result from task 4738 parsl.dataflow.dflow DEBUG: Task 4738 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/302de15a-8731-4a38-bd9b-bdf192c72588_isrForCrosstalkSources_996964_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4738 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/302de15a-8731-4a38-bd9b-bdf192c72588_isrForCrosstalkSources_996964_40.stderr parsl.dataflow.dflow INFO: Task 4790 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5fa4e748fc85ba96bac2fa38d11f13a with result from task 4790 parsl.dataflow.dflow DEBUG: Task 4790 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9424f062-afb4-4933-86b6-0d200eb82ef0_isrForCrosstalkSources_997018_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4790 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/9424f062-afb4-4933-86b6-0d200eb82ef0_isrForCrosstalkSources_997018_24.stderr parsl.dataflow.dflow INFO: Task 4703 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f53703ffdd6cd10baa613be6dd254eb4 with result from task 4703 parsl.dataflow.dflow DEBUG: Task 4703 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/47a4d2f6-b003-4c34-9a40-70a7789114d0_isrForCrosstalkSources_996982_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4703 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/47a4d2f6-b003-4c34-9a40-70a7789114d0_isrForCrosstalkSources_996982_18.stderr parsl.dataflow.dflow INFO: Task 4603 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dccb6e5f77f1c7b36d5cd0aab9bd7839 with result from task 4603 parsl.dataflow.dflow DEBUG: Task 4603 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/aa891d6c-653e-4c12-9e9f-dbf5341293aa_isrForCrosstalkSources_996991_22.stdout parsl.dataflow.dflow INFO: Standard error for task 4603 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/aa891d6c-653e-4c12-9e9f-dbf5341293aa_isrForCrosstalkSources_996991_22.stderr parsl.dataflow.dflow INFO: Task 4627 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4616b39a222947b59190518ddc56dd4 with result from task 4627 parsl.dataflow.dflow DEBUG: Task 4627 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/7d78e049-6c32-445b-aae3-3fe72846d1ce_isrForCrosstalkSources_996980_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4627 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/7d78e049-6c32-445b-aae3-3fe72846d1ce_isrForCrosstalkSources_996980_45.stderr parsl.dataflow.dflow INFO: Task 4878 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 47ab0504700bfac61efe98cf33a56097 with result from task 4878 parsl.dataflow.dflow DEBUG: Task 4878 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/75f1c6b9-1ab0-4d8f-9212-0524dcab0e3f_isrForCrosstalkSources_996976_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4878 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/75f1c6b9-1ab0-4d8f-9212-0524dcab0e3f_isrForCrosstalkSources_996976_4.stderr parsl.dataflow.dflow INFO: Task 4586 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry be86b80138558ccc8027a76a12953ec5 with result from task 4586 parsl.dataflow.dflow DEBUG: Task 4586 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/dac41d00-20ca-4ef7-8fa6-2cde21917c11_isrForCrosstalkSources_996993_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4586 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/dac41d00-20ca-4ef7-8fa6-2cde21917c11_isrForCrosstalkSources_996993_18.stderr parsl.dataflow.dflow INFO: Task 4861 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 860720b5a9642845d9c4a249562eca84 with result from task 4861 parsl.dataflow.dflow DEBUG: Task 4861 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/63c96795-3ff8-4be8-9915-c3d6ec489b1c_isrForCrosstalkSources_997018_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4861 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/63c96795-3ff8-4be8-9915-c3d6ec489b1c_isrForCrosstalkSources_997018_39.stderr parsl.dataflow.dflow INFO: Task 4851 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6f2b45c61298441b72f47ea2ae7a58cd with result from task 4851 parsl.dataflow.dflow DEBUG: Task 4851 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/901c8aa9-c6e8-4aa3-aabf-7f4d65cb24e0_isrForCrosstalkSources_997005_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4851 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/901c8aa9-c6e8-4aa3-aabf-7f4d65cb24e0_isrForCrosstalkSources_997005_56.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:34 proc_lsst.multi:146 [INFO] found job 17075364 in provider checkpoint proc_lsst.multi INFO: found job 17075364 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4619 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 036eff9a3216e416321843599010e45e with result from task 4619 parsl.dataflow.dflow DEBUG: Task 4619 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e4ad0eb6-ea9e-471c-966c-0772a81a3b8c_isrForCrosstalkSources_996952_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4619 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e4ad0eb6-ea9e-471c-966c-0772a81a3b8c_isrForCrosstalkSources_996952_38.stderr parsl.dataflow.dflow INFO: Task 4722 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5056a52c064dd8fdece555da46896d31 with result from task 4722 parsl.dataflow.dflow DEBUG: Task 4722 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/24e6d407-699e-4414-8314-aacc0f01d645_isrForCrosstalkSources_996959_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4722 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/24e6d407-699e-4414-8314-aacc0f01d645_isrForCrosstalkSources_996959_40.stderr parsl.dataflow.dflow INFO: Task 4646 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 09dd76efcfc060f80df4a040826ed703 with result from task 4646 parsl.dataflow.dflow DEBUG: Task 4646 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/9cfc607d-5239-4e44-be01-c7b53b880301_isrForCrosstalkSources_997028_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4646 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/9cfc607d-5239-4e44-be01-c7b53b880301_isrForCrosstalkSources_997028_56.stderr parsl.dataflow.dflow INFO: Task 4609 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5ab893faf024548e01da805a371649e9 with result from task 4609 parsl.dataflow.dflow DEBUG: Task 4609 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/05650d05-ed1f-4a92-911f-4339ca749ff4_isrForCrosstalkSources_996974_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4609 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/05650d05-ed1f-4a92-911f-4339ca749ff4_isrForCrosstalkSources_996974_62.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:34 proc_lsst.multi:146 [INFO] found job 17075368 in provider checkpoint proc_lsst.multi INFO: found job 17075368 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4657 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d05664e1a248dfafaeb7d7be23f88ab5 with result from task 4657 parsl.dataflow.dflow DEBUG: Task 4657 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/40aeb43a-8f41-4485-a7ec-ea4d409c514f_isrForCrosstalkSources_996982_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4657 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/40aeb43a-8f41-4485-a7ec-ea4d409c514f_isrForCrosstalkSources_996982_38.stderr parsl.dataflow.dflow INFO: Task 4889 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9b020f95c8d27c828e6228eb3ef48af with result from task 4889 parsl.dataflow.dflow DEBUG: Task 4889 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/fbfdab72-87bf-4c5d-a790-f1a1df1397fc_isrForCrosstalkSources_997021_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4889 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/fbfdab72-87bf-4c5d-a790-f1a1df1397fc_isrForCrosstalkSources_997021_36.stderr parsl.dataflow.dflow INFO: Task 4905 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 03a146b1fcefdd7e5c11d8be3543eed8 with result from task 4905 parsl.dataflow.dflow DEBUG: Task 4905 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/b5cd0d18-106a-4de1-a6d0-c95cf1eb0c8d_isrForCrosstalkSources_996988_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4905 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/b5cd0d18-106a-4de1-a6d0-c95cf1eb0c8d_isrForCrosstalkSources_996988_51.stderr parsl.dataflow.dflow INFO: Task 4614 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d16ec0aa755f7c176ab826258f1042ad with result from task 4614 parsl.dataflow.dflow DEBUG: Task 4614 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/615c1e2e-efa2-4b17-a634-9fc2d8849bc9_isrForCrosstalkSources_997027_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4614 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/615c1e2e-efa2-4b17-a634-9fc2d8849bc9_isrForCrosstalkSources_997027_55.stderr parsl.dataflow.dflow INFO: Task 4943 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 793970748d13257ae8805d73727cf1e5 with result from task 4943 parsl.dataflow.dflow DEBUG: Task 4943 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/c20f1c22-5bd6-46af-9adb-6098cf454720_isrForCrosstalkSources_996978_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4943 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/c20f1c22-5bd6-46af-9adb-6098cf454720_isrForCrosstalkSources_996978_10.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:35 proc_lsst.multi:146 [INFO] found job 17075372 in provider checkpoint proc_lsst.multi INFO: found job 17075372 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4656 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 045803940ab5e339e027772f32f94b92 with result from task 4656 parsl.dataflow.dflow DEBUG: Task 4656 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c72911f2-f69f-436e-b420-1479a0d959e3_isrForCrosstalkSources_997001_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4656 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/c72911f2-f69f-436e-b420-1479a0d959e3_isrForCrosstalkSources_997001_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4755 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry d742791b80133b8a651a06d8b66c3f75 with result from task 4755 parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:35 proc_lsst.multi:146 [INFO] found job 17075376 in provider checkpoint proc_lsst.multi INFO: found job 17075376 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4755 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c08bd6c5-e2b1-4fbd-934e-9aa9dc6c799b_isrForCrosstalkSources_996973_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4755 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/c08bd6c5-e2b1-4fbd-934e-9aa9dc6c799b_isrForCrosstalkSources_996973_2.stderr parsl.dataflow.dflow INFO: Task 4915 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e100f5c37ddff4dd808cee7782048d4b with result from task 4915 parsl.dataflow.dflow DEBUG: Task 4915 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8b073c14-6335-49bb-841b-bfaeabff2e00_isrForCrosstalkSources_996997_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4915 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/8b073c14-6335-49bb-841b-bfaeabff2e00_isrForCrosstalkSources_996997_3.stderr parsl.dataflow.dflow INFO: Task 4809 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ea245bb48004ba13c0cd986b8744ac4 with result from task 4809 parsl.dataflow.dflow DEBUG: Task 4809 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/53c6479b-38e9-47b3-8f67-7b664218bb6b_isrForCrosstalkSources_997018_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4809 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/53c6479b-38e9-47b3-8f67-7b664218bb6b_isrForCrosstalkSources_997018_32.stderr parsl.dataflow.dflow INFO: Task 4927 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2118b89ea9335d381594e0cb6a521604 with result from task 4927 parsl.dataflow.dflow DEBUG: Task 4927 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/42d0dc47-74c9-4232-a9d8-1ca7de92eac3_isrForCrosstalkSources_996971_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4927 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/42d0dc47-74c9-4232-a9d8-1ca7de92eac3_isrForCrosstalkSources_996971_27.stderr parsl.dataflow.dflow INFO: Task 4759 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65ed0eb24ef72ec846cecdd7fb8e3aa7 with result from task 4759 parsl.dataflow.dflow DEBUG: Task 4759 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/42c68201-dac8-495a-a559-3e7836002ce8_isrForCrosstalkSources_996978_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4759 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/42c68201-dac8-495a-a559-3e7836002ce8_isrForCrosstalkSources_996978_5.stderr parsl.dataflow.dflow INFO: Task 4727 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d369333b6113da67904393d6b71e64bf with result from task 4727 parsl.dataflow.dflow DEBUG: Task 4727 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b086b559-4c08-4ec6-97fa-504a063d80b8_isrForCrosstalkSources_997004_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4727 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/b086b559-4c08-4ec6-97fa-504a063d80b8_isrForCrosstalkSources_997004_24.stderr parsl.dataflow.dflow INFO: Task 4725 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8a3ee4f7d99d57e22b98e1d697ad1b66 with result from task 4725 parsl.dataflow.dflow DEBUG: Task 4725 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a07c454-7e09-4628-b064-7205d96445c4_isrForCrosstalkSources_997004_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4725 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1a07c454-7e09-4628-b064-7205d96445c4_isrForCrosstalkSources_997004_33.stderr parsl.dataflow.dflow INFO: Task 4707 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cd3763209e661d7651687aa7cd12cbbc with result from task 4707 parsl.dataflow.dflow DEBUG: Task 4707 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/c9d9bd19-337a-4613-8e18-e14410f9c5c0_isrForCrosstalkSources_997022_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4707 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/c9d9bd19-337a-4613-8e18-e14410f9c5c0_isrForCrosstalkSources_997022_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4873 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd26cc9b959d4392c6f53ecedd4c7451 with result from task 4873 parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:37 proc_lsst.multi:146 [INFO] found job 17075379 in provider checkpoint proc_lsst.multi INFO: found job 17075379 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4873 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/549f1d73-dc5a-4b28-a55c-70014509215b_isrForCrosstalkSources_996962_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4873 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/549f1d73-dc5a-4b28-a55c-70014509215b_isrForCrosstalkSources_996962_15.stderr parsl.dataflow.dflow INFO: Task 4575 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9bc355141feda8c9ba60c42d6fd180e with result from task 4575 parsl.dataflow.dflow DEBUG: Task 4575 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b64bf3cb-4a6f-43d2-b26e-965720dfd4f2_isrForCrosstalkSources_997015_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4575 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/b64bf3cb-4a6f-43d2-b26e-965720dfd4f2_isrForCrosstalkSources_997015_18.stderr parsl.dataflow.dflow INFO: Task 4718 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c5fe6e10e23e3550241f026462ed06ba with result from task 4718 parsl.dataflow.dflow DEBUG: Task 4718 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/3cc1e13e-ad79-4f51-bfab-b38f5688fc48_isrForCrosstalkSources_996991_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4718 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/3cc1e13e-ad79-4f51-bfab-b38f5688fc48_isrForCrosstalkSources_996991_9.stderr parsl.dataflow.dflow INFO: Task 4847 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 66cd989023a616fc3aaf11e9518c61ae with result from task 4847 parsl.dataflow.dflow DEBUG: Task 4847 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d72acd6a-a4e5-4a5d-be89-5e3cbcc826de_isrForCrosstalkSources_997030_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4847 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/d72acd6a-a4e5-4a5d-be89-5e3cbcc826de_isrForCrosstalkSources_997030_1.stderr parsl.dataflow.dflow INFO: Task 4844 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0daca9c05cf626beae29e3bc17a3c6b4 with result from task 4844 parsl.dataflow.dflow DEBUG: Task 4844 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/be930fe0-76f6-4a5d-93fd-a690d1aca6c5_isrForCrosstalkSources_996962_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4844 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/be930fe0-76f6-4a5d-93fd-a690d1aca6c5_isrForCrosstalkSources_996962_5.stderr parsl.dataflow.dflow INFO: Task 4689 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ad57beffa69aed805585663877803cae with result from task 4689 parsl.dataflow.dflow DEBUG: Task 4689 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/402cfe6b-045e-44df-9683-80752a164f39_isrForCrosstalkSources_997004_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4689 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/402cfe6b-045e-44df-9683-80752a164f39_isrForCrosstalkSources_997004_15.stderr parsl.dataflow.dflow INFO: Task 4691 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d6a65f7100cb1fe0b4b12c0788c164c1 with result from task 4691 parsl.dataflow.dflow DEBUG: Task 4691 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/f25ad7e6-fc7b-489d-821c-3b4526ec3122_isrForCrosstalkSources_996969_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4691 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/f25ad7e6-fc7b-489d-821c-3b4526ec3122_isrForCrosstalkSources_996969_9.stderr parsl.dataflow.dflow INFO: Task 4849 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2bdda7bb2c9587da5fa9aa4dd26cf14b with result from task 4849 parsl.dataflow.dflow DEBUG: Task 4849 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/913e98e9-f594-4fd4-9783-18b62b617b4d_isrForCrosstalkSources_996960_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4849 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/913e98e9-f594-4fd4-9783-18b62b617b4d_isrForCrosstalkSources_996960_16.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:37 proc_lsst.multi:146 [INFO] found job 17075380 in provider checkpoint proc_lsst.multi INFO: found job 17075380 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4860 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 133ffe5134e518420be119628919fd4b with result from task 4860 parsl.dataflow.dflow DEBUG: Task 4860 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/aca07a5c-1f40-48b4-84e1-dc6b297b349c_isrForCrosstalkSources_996962_32.stdout parsl.dataflow.dflow INFO: Standard error for task 4860 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/aca07a5c-1f40-48b4-84e1-dc6b297b349c_isrForCrosstalkSources_996962_32.stderr parsl.dataflow.dflow INFO: Task 4778 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b27f94dce951058353e04ef8226d6e4 with result from task 4778 parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:38 proc_lsst.multi:146 [INFO] found job 17075381 in provider checkpoint proc_lsst.multi INFO: found job 17075381 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4778 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7fa6c051-78e3-45ba-8ab8-80068e26ca35_isrForCrosstalkSources_996971_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4778 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/7fa6c051-78e3-45ba-8ab8-80068e26ca35_isrForCrosstalkSources_996971_20.stderr parsl.dataflow.dflow INFO: Task 4693 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3da33a4d6c6a1ab35114ed0e143fae72 with result from task 4693 parsl.dataflow.dflow DEBUG: Task 4693 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6f96a91f-3ff6-4cba-8da2-bbbd547c9067_isrForCrosstalkSources_996971_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4693 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/6f96a91f-3ff6-4cba-8da2-bbbd547c9067_isrForCrosstalkSources_996971_47.stderr parsl.dataflow.dflow INFO: Task 4650 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 22473bf1b07f637d486b71471f9757d7 with result from task 4650 parsl.dataflow.dflow DEBUG: Task 4650 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0f04d8ce-d2b0-43da-8711-c225705fd684_isrForCrosstalkSources_997007_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4650 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/0f04d8ce-d2b0-43da-8711-c225705fd684_isrForCrosstalkSources_997007_1.stderr parsl.dataflow.dflow INFO: Task 4913 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 53acd412f0a5b902e8163e2d46bf4522 with result from task 4913 parsl.dataflow.dflow DEBUG: Task 4913 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/281b2ad1-d141-4be2-8f34-2134a20c9cdd_isrForCrosstalkSources_997025_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4913 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/281b2ad1-d141-4be2-8f34-2134a20c9cdd_isrForCrosstalkSources_997025_18.stderr parsl.dataflow.dflow INFO: Task 4923 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f1f951f7d294a7cbc0df69b34d61a16b with result from task 4923 parsl.dataflow.dflow DEBUG: Task 4923 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/49bbb278-1d14-44b0-a954-cdfb4e8956b6_isrForCrosstalkSources_996973_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4923 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/49bbb278-1d14-44b0-a954-cdfb4e8956b6_isrForCrosstalkSources_996973_38.stderr parsl.dataflow.dflow INFO: Task 4765 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2b84d48f4ef9056cb6f46da4a14fa5f7 with result from task 4765 parsl.dataflow.dflow DEBUG: Task 4765 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/34654c7b-9da6-441e-ae37-8407383cd6e5_isrForCrosstalkSources_996999_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4765 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/34654c7b-9da6-441e-ae37-8407383cd6e5_isrForCrosstalkSources_996999_36.stderr parsl.dataflow.dflow INFO: Task 4835 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9580ca544f2e27335548c2c9331e3d8 with result from task 4835 parsl.dataflow.dflow DEBUG: Task 4835 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/1ed953c9-901e-44a7-9542-343bcd545442_isrForCrosstalkSources_996999_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4835 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/1ed953c9-901e-44a7-9542-343bcd545442_isrForCrosstalkSources_996999_4.stderr parsl.dataflow.dflow INFO: Task 4811 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c14906e1deade80ce86b7324a7c9829f with result from task 4811 parsl.dataflow.dflow DEBUG: Task 4811 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a82beef8-147e-445e-9d00-4a8f635e5606_isrForCrosstalkSources_996956_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4811 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/a82beef8-147e-445e-9d00-4a8f635e5606_isrForCrosstalkSources_996956_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:39 proc_lsst.multi:146 [INFO] found job 17075382 in provider checkpoint proc_lsst.multi INFO: found job 17075382 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4699 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 77b4e466268cf07ae5806aadb73cb0c7 with result from task 4699 parsl.dataflow.dflow DEBUG: Task 4699 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/bd023fdb-3319-49dc-a9dd-c6583542a75b_isrForCrosstalkSources_996976_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4699 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/bd023fdb-3319-49dc-a9dd-c6583542a75b_isrForCrosstalkSources_996976_6.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:39 proc_lsst.multi:146 [INFO] found job 17075384 in provider checkpoint proc_lsst.multi INFO: found job 17075384 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4852 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b331de0127706ef588196141fcdc71d8 with result from task 4852 parsl.dataflow.dflow DEBUG: Task 4852 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7267ffef-01af-429d-8fe5-f86cdefce52a_isrForCrosstalkSources_997026_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4852 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7267ffef-01af-429d-8fe5-f86cdefce52a_isrForCrosstalkSources_997026_42.stderr parsl.dataflow.dflow INFO: Task 4928 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d8ba6d5c17bfa4c599397d6e435cf269 with result from task 4928 parsl.dataflow.dflow DEBUG: Task 4928 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ad49c6f0-08a9-4c3a-a488-9872a2451a6f_isrForCrosstalkSources_997002_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4928 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/ad49c6f0-08a9-4c3a-a488-9872a2451a6f_isrForCrosstalkSources_997002_53.stderr parsl.dataflow.dflow INFO: Task 4931 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c020467f2c7305f3be9bedf43b2e76e with result from task 4931 parsl.dataflow.dflow DEBUG: Task 4931 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7ffb6109-7f17-457c-b3a5-0f7f085cd071_isrForCrosstalkSources_996974_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4931 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/7ffb6109-7f17-457c-b3a5-0f7f085cd071_isrForCrosstalkSources_996974_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:40 proc_lsst.multi:146 [INFO] found job 17075385 in provider checkpoint proc_lsst.multi INFO: found job 17075385 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4761 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ceb53fcbbdeed68ade046c6135130e4 with result from task 4761 parsl.dataflow.dflow DEBUG: Task 4761 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b450fee8-c754-4e65-b34d-5ccec9f1047a_isrForCrosstalkSources_996998_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4761 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b450fee8-c754-4e65-b34d-5ccec9f1047a_isrForCrosstalkSources_996998_23.stderr parsl.dataflow.dflow INFO: Task 4906 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bfc8086285f6d5b968d2d817776d7966 with result from task 4906 parsl.dataflow.dflow DEBUG: Task 4906 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/fc8557e3-2514-441e-a4ae-78bacbf75a94_isrForCrosstalkSources_996972_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4906 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/fc8557e3-2514-441e-a4ae-78bacbf75a94_isrForCrosstalkSources_996972_26.stderr parsl.dataflow.dflow INFO: Task 4782 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29c7fcc9f6b2200408317a24450bbb11 with result from task 4782 parsl.dataflow.dflow DEBUG: Task 4782 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/b8b429f9-b990-4d82-aeba-f036a6478c33_isrForCrosstalkSources_996968_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4782 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/b8b429f9-b990-4d82-aeba-f036a6478c33_isrForCrosstalkSources_996968_17.stderr parsl.dataflow.dflow INFO: Task 4829 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b215bfd3ef695e275fc109e913cec6fe with result from task 4829 parsl.dataflow.dflow DEBUG: Task 4829 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/de583e6e-e657-4a2e-924a-b842200f27c2_isrForCrosstalkSources_997032_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4829 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/de583e6e-e657-4a2e-924a-b842200f27c2_isrForCrosstalkSources_997032_16.stderr parsl.dataflow.dflow INFO: Task 4894 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f2c3ed8724565a39a123c5e5ba3deba with result from task 4894 parsl.dataflow.dflow DEBUG: Task 4894 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/4fd2ace2-b453-482a-8b97-40e4c77eb9ed_isrForCrosstalkSources_996990_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4894 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/4fd2ace2-b453-482a-8b97-40e4c77eb9ed_isrForCrosstalkSources_996990_30.stderr parsl.dataflow.dflow INFO: Task 4949 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b36087d4cbaf42e49c947e1dbfc0ac6b with result from task 4949 parsl.dataflow.dflow DEBUG: Task 4949 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a5fb8e34-080b-4ef1-a4be-000910358540_isrForCrosstalkSources_997007_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4949 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a5fb8e34-080b-4ef1-a4be-000910358540_isrForCrosstalkSources_997007_10.stderr parsl.dataflow.dflow INFO: Task 4892 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 00b6a931e71680dd307bf11cbf7d4a02 with result from task 4892 parsl.dataflow.dflow DEBUG: Task 4892 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/81a8a68b-d858-48d3-aa57-0b2de4feb496_isrForCrosstalkSources_997020_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4892 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/81a8a68b-d858-48d3-aa57-0b2de4feb496_isrForCrosstalkSources_997020_4.stderr parsl.dataflow.dflow INFO: Task 4891 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a0b0a635f9f56e7ea6468fdcd8daa41a with result from task 4891 parsl.dataflow.dflow DEBUG: Task 4891 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8df19e22-1f35-4085-bf65-2333e65076f8_isrForCrosstalkSources_997027_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4891 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8df19e22-1f35-4085-bf65-2333e65076f8_isrForCrosstalkSources_997027_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:40 proc_lsst.multi:146 [INFO] found job 17075386 in provider checkpoint proc_lsst.multi INFO: found job 17075386 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4791 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7ffa2c93431e4d858c19e8616955e6eb with result from task 4791 parsl.dataflow.dflow DEBUG: Task 4791 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2d7c59b2-956a-45be-849f-c1d7f932cac1_isrForCrosstalkSources_996956_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4791 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/2d7c59b2-956a-45be-849f-c1d7f932cac1_isrForCrosstalkSources_996956_33.stderr parsl.dataflow.dflow INFO: Task 4626 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9c101f01cd7ba221e24e028d46ab075 with result from task 4626 parsl.dataflow.dflow DEBUG: Task 4626 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/952a4e13-1af5-44a2-a797-e9530704cf86_isrForCrosstalkSources_997003_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4626 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/952a4e13-1af5-44a2-a797-e9530704cf86_isrForCrosstalkSources_997003_2.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:41 proc_lsst.multi:146 [INFO] found job 17075387 in provider checkpoint proc_lsst.multi INFO: found job 17075387 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4995 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5f1d9d8254e1b0642ea4dedf370dabab with result from task 4995 parsl.dataflow.dflow DEBUG: Task 4995 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8c20d3d5-82ac-4c3b-8748-21d033281bf3_isrForCrosstalkSources_997027_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4995 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/8c20d3d5-82ac-4c3b-8748-21d033281bf3_isrForCrosstalkSources_997027_20.stderr parsl.dataflow.dflow INFO: Task 4724 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29f69a84ca86471d57e112db4f1464e8 with result from task 4724 parsl.dataflow.dflow DEBUG: Task 4724 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/5a915cfd-6cd2-4234-8359-f4b3429ee550_isrForCrosstalkSources_997022_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4724 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/5a915cfd-6cd2-4234-8359-f4b3429ee550_isrForCrosstalkSources_997022_14.stderr parsl.dataflow.dflow INFO: Task 4941 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fcf7ba183e028e237458dc4ed3f1722b with result from task 4941 parsl.dataflow.dflow DEBUG: Task 4941 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b3037a65-bbba-4c69-bb41-6598b1c7194b_isrForCrosstalkSources_997008_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4941 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b3037a65-bbba-4c69-bb41-6598b1c7194b_isrForCrosstalkSources_997008_48.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:41 proc_lsst.multi:146 [INFO] found job 17075388 in provider checkpoint proc_lsst.multi INFO: found job 17075388 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4771 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry de7e1f6531087383ea9266709ccb67ea with result from task 4771 parsl.dataflow.dflow DEBUG: Task 4771 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/3ba58704-102b-431b-b7d2-2a6976685f03_isrForCrosstalkSources_997020_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4771 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/3ba58704-102b-431b-b7d2-2a6976685f03_isrForCrosstalkSources_997020_14.stderr parsl.dataflow.dflow INFO: Task 4956 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 577ed3183c589481459632c56ca6d343 with result from task 4956 parsl.dataflow.dflow DEBUG: Task 4956 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f22722b1-7cc0-4d5f-b48e-b9c11a0111db_isrForCrosstalkSources_996993_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4956 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/f22722b1-7cc0-4d5f-b48e-b9c11a0111db_isrForCrosstalkSources_996993_56.stderr parsl.dataflow.dflow INFO: Task 4816 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2bbed5e7c3b66905e9bf9ef7bf2c1a54 with result from task 4816 parsl.dataflow.dflow DEBUG: Task 4816 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/5c515fff-e439-4be7-9abe-e4a0013a99e1_isrForCrosstalkSources_996976_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4816 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/5c515fff-e439-4be7-9abe-e4a0013a99e1_isrForCrosstalkSources_996976_56.stderr parsl.dataflow.dflow INFO: Task 4543 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 72c89e7e0287855c76ef53b181602ef2 with result from task 4543 parsl.dataflow.dflow DEBUG: Task 4543 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/21034041-8abe-459b-a790-a0403090c5e5_isrForCrosstalkSources_996990_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4543 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/21034041-8abe-459b-a790-a0403090c5e5_isrForCrosstalkSources_996990_31.stderr parsl.dataflow.dflow INFO: Task 4900 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e27dd5de2150bde1cc22965ab4df8e94 with result from task 4900 parsl.dataflow.dflow DEBUG: Task 4900 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/390e9d8f-e982-42dc-88ab-a6570915e4ac_isrForCrosstalkSources_996970_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4900 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/390e9d8f-e982-42dc-88ab-a6570915e4ac_isrForCrosstalkSources_996970_39.stderr parsl.dataflow.dflow INFO: Task 4808 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1ac1f8c8ebfb8b802e6ebc9c6519465 with result from task 4808 parsl.dataflow.dflow DEBUG: Task 4808 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/6ab39b32-d9a9-4213-abf7-bf7c19256863_isrForCrosstalkSources_997020_8.stdout parsl.dataflow.dflow INFO: Standard error for task 4808 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/6ab39b32-d9a9-4213-abf7-bf7c19256863_isrForCrosstalkSources_997020_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:42 proc_lsst.multi:146 [INFO] found job 17075392 in provider checkpoint proc_lsst.multi INFO: found job 17075392 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4772 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c79d5122bb1ec3238dd98bc4a91749d9 with result from task 4772 parsl.dataflow.dflow DEBUG: Task 4772 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5243bd19-f8ea-4413-bbb4-efe14284d74d_isrForCrosstalkSources_996954_39.stdout parsl.dataflow.dflow INFO: Standard error for task 4772 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5243bd19-f8ea-4413-bbb4-efe14284d74d_isrForCrosstalkSources_996954_39.stderr parsl.dataflow.dflow INFO: Task 4545 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0eeedea1fb93c86d38f28f94c12f9b91 with result from task 4545 parsl.dataflow.dflow DEBUG: Task 4545 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ea690f34-da4f-4b46-84e8-b56e9cc9186d_isrForCrosstalkSources_996978_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4545 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/ea690f34-da4f-4b46-84e8-b56e9cc9186d_isrForCrosstalkSources_996978_6.stderr parsl.dataflow.dflow INFO: Task 4946 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16fe66dfa58434e25baee440b3c0846f with result from task 4946 parsl.dataflow.dflow DEBUG: Task 4946 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5f43eab8-126a-47d4-ae03-4e4ea0df3e26_isrForCrosstalkSources_996991_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4946 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5f43eab8-126a-47d4-ae03-4e4ea0df3e26_isrForCrosstalkSources_996991_58.stderr parsl.dataflow.dflow INFO: Task 4818 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9ae9d595ab207db7ec59580c5456b763 with result from task 4818 parsl.dataflow.dflow DEBUG: Task 4818 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/aff4f67a-2d14-44cd-b0e6-29af457bd2a7_isrForCrosstalkSources_997012_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4818 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/aff4f67a-2d14-44cd-b0e6-29af457bd2a7_isrForCrosstalkSources_997012_2.stderr parsl.dataflow.dflow INFO: Task 4977 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 11935ee2a94e4246a9b3235df8dd2ab1 with result from task 4977 parsl.dataflow.dflow DEBUG: Task 4977 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7b8bf87f-6d07-4469-a4b2-05b23af3da21_isrForCrosstalkSources_996993_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4977 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/7b8bf87f-6d07-4469-a4b2-05b23af3da21_isrForCrosstalkSources_996993_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:42 proc_lsst.multi:146 [INFO] found job 17075394 in provider checkpoint proc_lsst.multi INFO: found job 17075394 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4688 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 256e9e2e24b03180cd02906e7041d26a with result from task 4688 parsl.dataflow.dflow DEBUG: Task 4688 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7a9932b1-e86a-4963-b106-e947b3edab05_isrForCrosstalkSources_997027_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4688 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/7a9932b1-e86a-4963-b106-e947b3edab05_isrForCrosstalkSources_997027_48.stderr parsl.dataflow.dflow INFO: Task 4879 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb186ba41920a807cade99e0f8411e48 with result from task 4879 parsl.dataflow.dflow DEBUG: Task 4879 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ba35184e-5f6c-4e48-86ef-b87f6c5d0873_isrForCrosstalkSources_997015_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4879 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/ba35184e-5f6c-4e48-86ef-b87f6c5d0873_isrForCrosstalkSources_997015_15.stderr parsl.dataflow.dflow INFO: Task 4597 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9b8b395647d69c64b9e9145732e8fbd with result from task 4597 parsl.dataflow.dflow DEBUG: Task 4597 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/c5835ab0-786a-4949-9915-f57efd1fa9df_isrForCrosstalkSources_996956_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4597 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/c5835ab0-786a-4949-9915-f57efd1fa9df_isrForCrosstalkSources_996956_3.stderr parsl.dataflow.dflow INFO: Task 4805 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8f8472af76b8ba4f477e8d0bffbd37c with result from task 4805 parsl.dataflow.dflow DEBUG: Task 4805 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/5f809ef4-58f0-4d03-8095-c7f2b70d90da_isrForCrosstalkSources_997023_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4805 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/5f809ef4-58f0-4d03-8095-c7f2b70d90da_isrForCrosstalkSources_997023_6.stderr parsl.dataflow.dflow INFO: Task 4714 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0ea277a1c6d6bde40612be5be87dbcc0 with result from task 4714 parsl.dataflow.dflow DEBUG: Task 4714 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/741bdeb7-3a86-46a8-8b11-23ff1f36c169_isrForCrosstalkSources_996984_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4714 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/741bdeb7-3a86-46a8-8b11-23ff1f36c169_isrForCrosstalkSources_996984_9.stderr parsl.dataflow.dflow INFO: Task 4912 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5de1c80e18f0714f164b60f1885f0f9a with result from task 4912 parsl.dataflow.dflow DEBUG: Task 4912 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/619d40ce-6585-4e6b-945c-2f980bd50885_isrForCrosstalkSources_996963_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4912 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/619d40ce-6585-4e6b-945c-2f980bd50885_isrForCrosstalkSources_996963_13.stderr parsl.dataflow.dflow INFO: Task 4734 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 896f708dbd3077098c7c22eca31833a2 with result from task 4734 parsl.dataflow.dflow DEBUG: Task 4734 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/63144f9f-dd47-4aa4-b13a-b6f3695f220b_isrForCrosstalkSources_996982_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4734 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/63144f9f-dd47-4aa4-b13a-b6f3695f220b_isrForCrosstalkSources_996982_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:43 proc_lsst.multi:146 [INFO] found job 17075399 in provider checkpoint proc_lsst.multi INFO: found job 17075399 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4859 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10643daaf6fd0f7c0b15bea03b21debb with result from task 4859 parsl.dataflow.dflow DEBUG: Task 4859 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/515faa0d-ab74-447c-9112-f6c18a8ed663_isrForCrosstalkSources_996988_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4859 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/515faa0d-ab74-447c-9112-f6c18a8ed663_isrForCrosstalkSources_996988_7.stderr parsl.dataflow.dflow INFO: Task 4976 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6baf1a4e6c96dae70ad04a83812af9b8 with result from task 4976 parsl.dataflow.dflow DEBUG: Task 4976 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/0459e4d3-f5a2-422d-a22a-d0608db1ec1c_isrForCrosstalkSources_996952_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4976 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/0459e4d3-f5a2-422d-a22a-d0608db1ec1c_isrForCrosstalkSources_996952_58.stderr parsl.dataflow.dflow INFO: Task 4957 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3f8a1315e3146677cd2083ead118b827 with result from task 4957 parsl.dataflow.dflow DEBUG: Task 4957 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b57b4c00-2769-417e-8f30-292cc3c670fa_isrForCrosstalkSources_996998_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4957 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/b57b4c00-2769-417e-8f30-292cc3c670fa_isrForCrosstalkSources_996998_20.stderr parsl.dataflow.dflow INFO: Task 5002 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry aae1db6277d8297ea6f1de5bc6533a47 with result from task 5002 parsl.dataflow.dflow DEBUG: Task 5002 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/8818c06d-75ad-4cb6-a3fc-d87b1c011862_isrForCrosstalkSources_996990_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5002 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996990/8818c06d-75ad-4cb6-a3fc-d87b1c011862_isrForCrosstalkSources_996990_23.stderr parsl.dataflow.dflow INFO: Task 4667 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f03a5b322d0f5431f13ace04a07cd76 with result from task 4667 parsl.dataflow.dflow DEBUG: Task 4667 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/435342d8-e46b-491c-b2e1-0607913cacc1_isrForCrosstalkSources_997031_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4667 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/435342d8-e46b-491c-b2e1-0607913cacc1_isrForCrosstalkSources_997031_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:43 proc_lsst.multi:146 [INFO] found job 17075400 in provider checkpoint proc_lsst.multi INFO: found job 17075400 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 5008 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b09393502490784bc584729693002113 with result from task 5008 parsl.dataflow.dflow DEBUG: Task 5008 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/da48764b-584e-42e3-b15b-78309b37e700_isrForCrosstalkSources_996986_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5008 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/da48764b-584e-42e3-b15b-78309b37e700_isrForCrosstalkSources_996986_8.stderr parsl.dataflow.dflow INFO: Task 4984 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94c6bf294ccd6f0f4b11d986232ca090 with result from task 4984 parsl.dataflow.dflow DEBUG: Task 4984 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/123e4ecd-1cb5-4713-acef-2eb1c4adc511_isrForCrosstalkSources_996980_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4984 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/123e4ecd-1cb5-4713-acef-2eb1c4adc511_isrForCrosstalkSources_996980_9.stderr parsl.dataflow.dflow INFO: Task 4898 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.memoization DEBUG: Storing app cache entry d1379656b37634e2acb41872b84ca2ff with result from task 4898 parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4898 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/50553538-638d-42cf-85df-b3dd71a064f9_isrForCrosstalkSources_996974_15.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4898 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/50553538-638d-42cf-85df-b3dd71a064f9_isrForCrosstalkSources_996974_15.stderr parsl.dataflow.dflow INFO: Task 4846 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2e980df614a56311b6e0dcb69dfa1247 with result from task 4846 parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4846 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:44 proc_lsst.multi:146 [INFO] found job 17075401 in provider checkpoint proc_lsst.multi INFO: found job 17075401 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/d8ad6cf1-6e33-4b72-816a-99c2606d9e2c_isrForCrosstalkSources_996977_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4846 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/d8ad6cf1-6e33-4b72-816a-99c2606d9e2c_isrForCrosstalkSources_996977_14.stderr parsl.dataflow.dflow INFO: Task 5027 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a6b36995bf647bdf9fe188df44088185 with result from task 5027 parsl.dataflow.dflow DEBUG: Task 5027 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/00f587f6-89f8-44cc-8a1f-dc9e639cd83d_isrForCrosstalkSources_996974_22.stdout parsl.dataflow.dflow INFO: Standard error for task 5027 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/00f587f6-89f8-44cc-8a1f-dc9e639cd83d_isrForCrosstalkSources_996974_22.stderr parsl.dataflow.dflow INFO: Task 4909 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry efddc89d339400d258ab313ed273d6b4 with result from task 4909 parsl.dataflow.dflow DEBUG: Task 4909 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/52add8eb-5885-4d5b-872d-069df61190f2_isrForCrosstalkSources_997017_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4909 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/52add8eb-5885-4d5b-872d-069df61190f2_isrForCrosstalkSources_997017_9.stderr parsl.dataflow.dflow INFO: Task 4744 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c76777e553167b9d7891184fa289033a with result from task 4744 parsl.dataflow.dflow DEBUG: Task 4744 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/08bdd6a2-5af3-4e95-bf37-db3d35ca82c4_isrForCrosstalkSources_996994_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4744 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/08bdd6a2-5af3-4e95-bf37-db3d35ca82c4_isrForCrosstalkSources_996994_28.stderr parsl.dataflow.dflow INFO: Task 4883 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f3f71a0e99d02024f580669d44ddebf6 with result from task 4883 parsl.dataflow.dflow DEBUG: Task 4883 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/1c6abc8a-b89d-482b-969d-7d8a2336b993_isrForCrosstalkSources_996987_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4883 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/1c6abc8a-b89d-482b-969d-7d8a2336b993_isrForCrosstalkSources_996987_50.stderr parsl.dataflow.dflow INFO: Task 4822 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a9cac2346a11c909d0a6aa3a725db0c with result from task 4822 parsl.dataflow.dflow DEBUG: Task 4822 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/42b64a96-8541-4abd-ab33-9e2626a82352_isrForCrosstalkSources_997018_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4822 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/42b64a96-8541-4abd-ab33-9e2626a82352_isrForCrosstalkSources_997018_21.stderr parsl.dataflow.dflow INFO: Task 4841 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 994738a3476f2fa269a1bc1c4c3249bc with result from task 4841 parsl.dataflow.dflow DEBUG: Task 4841 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/714525c9-4894-47f3-8d8f-719756cf39fd_isrForCrosstalkSources_996968_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4841 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/714525c9-4894-47f3-8d8f-719756cf39fd_isrForCrosstalkSources_996968_41.stderr parsl.dataflow.dflow INFO: Task 4653 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 54da84fad3ce0cb65e5030a410d864a7 with result from task 4653 parsl.dataflow.dflow DEBUG: Task 4653 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/8829c8b4-ae19-402e-9309-28565aae73dc_isrForCrosstalkSources_997020_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4653 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/8829c8b4-ae19-402e-9309-28565aae73dc_isrForCrosstalkSources_997020_31.stderr parsl.dataflow.dflow INFO: Task 4779 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 04bba4e9f52d96b2cb9b2e0995509e31 with result from task 4779 parsl.dataflow.dflow DEBUG: Task 4779 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/ff0f1246-213b-494c-8a36-7d05955b2d9c_isrForCrosstalkSources_997011_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4779 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/ff0f1246-213b-494c-8a36-7d05955b2d9c_isrForCrosstalkSources_997011_55.stderr parsl.dataflow.dflow INFO: Task 4876 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8e86eebf533d255a765c313693fb1ac with result from task 4876 parsl.dataflow.dflow DEBUG: Task 4876 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/80290703-9328-4809-ace0-aab95801ba51_isrForCrosstalkSources_997018_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4876 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997018/80290703-9328-4809-ace0-aab95801ba51_isrForCrosstalkSources_997018_45.stderr parsl.dataflow.dflow INFO: Task 4781 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5842eda515ea4f86f6873ebe45212511 with result from task 4781 parsl.dataflow.dflow DEBUG: Task 4781 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/2547de06-3557-43b9-8981-c0fff41dda87_isrForCrosstalkSources_996954_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4781 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/2547de06-3557-43b9-8981-c0fff41dda87_isrForCrosstalkSources_996954_27.stderr parsl.dataflow.dflow INFO: Task 4996 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 51868f8bf2c36a55ef5f83c1404128ba with result from task 4996 parsl.dataflow.dflow DEBUG: Task 4996 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e7a7b29d-3d73-48b0-939f-253ffe5d3846_isrForCrosstalkSources_996956_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4996 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e7a7b29d-3d73-48b0-939f-253ffe5d3846_isrForCrosstalkSources_996956_50.stderr parsl.dataflow.dflow INFO: Task 5029 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 59f7f481c2152b33a2c89067ff432f63 with result from task 5029 parsl.dataflow.dflow DEBUG: Task 5029 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/006575f2-d065-4be9-9eff-7f320cf39f5b_isrForCrosstalkSources_996965_34.stdout parsl.dataflow.dflow INFO: Standard error for task 5029 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/006575f2-d065-4be9-9eff-7f320cf39f5b_isrForCrosstalkSources_996965_34.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4866 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 8d5de7a1e347e59fde25b6c5fd8b9c2f with result from task 4866 parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4866 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/ea60fdc9-fa45-4678-adb9-396177bdc689_isrForCrosstalkSources_996963_31.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4866 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/ea60fdc9-fa45-4678-adb9-396177bdc689_isrForCrosstalkSources_996963_31.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:45 proc_lsst.multi:146 [INFO] found job 17075405 in provider checkpoint proc_lsst.multi INFO: found job 17075405 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 5032 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bff525030e67215a0a13b07ce37993cd with result from task 5032 parsl.dataflow.dflow DEBUG: Task 5032 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/eef1d50d-edc3-4381-b352-2d63421ae26d_isrForCrosstalkSources_996982_34.stdout parsl.dataflow.dflow INFO: Standard error for task 5032 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/eef1d50d-edc3-4381-b352-2d63421ae26d_isrForCrosstalkSources_996982_34.stderr parsl.dataflow.dflow INFO: Task 4907 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bd38a5f74223c935944e156e480b80d2 with result from task 4907 parsl.dataflow.dflow DEBUG: Task 4907 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/97ec02d2-472b-48eb-bdc3-91ca30b0b44a_isrForCrosstalkSources_996964_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4907 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/97ec02d2-472b-48eb-bdc3-91ca30b0b44a_isrForCrosstalkSources_996964_23.stderr parsl.dataflow.dflow INFO: Task 5009 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b9aa8eaaf8714eef9f48ea992876a946 with result from task 5009 parsl.dataflow.dflow DEBUG: Task 5009 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b4996b9f-cc50-45aa-92ef-174ad92fd0d4_isrForCrosstalkSources_996983_16.stdout parsl.dataflow.dflow INFO: Standard error for task 5009 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/b4996b9f-cc50-45aa-92ef-174ad92fd0d4_isrForCrosstalkSources_996983_16.stderr parsl.dataflow.dflow INFO: Task 4856 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 26495f7afdd9fbb7b3b6a683a7f8c21b with result from task 4856 parsl.dataflow.dflow DEBUG: Task 4856 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/1c2f889a-c7df-4cdf-a747-c24158191340_isrForCrosstalkSources_997016_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4856 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/1c2f889a-c7df-4cdf-a747-c24158191340_isrForCrosstalkSources_997016_44.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:46 proc_lsst.multi:146 [INFO] found job 17075408 in provider checkpoint proc_lsst.multi INFO: found job 17075408 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4739 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d2691b8bb921358d6c72817dfa7033d with result from task 4739 parsl.dataflow.dflow DEBUG: Task 4739 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/4b4da93f-0dc4-4a4b-937e-a9ddd4ca24f0_isrForCrosstalkSources_997032_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4739 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/4b4da93f-0dc4-4a4b-937e-a9ddd4ca24f0_isrForCrosstalkSources_997032_7.stderr parsl.dataflow.dflow INFO: Task 4715 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5071c59f5cbd0470ba03ad1b371bb799 with result from task 4715 parsl.dataflow.dflow DEBUG: Task 4715 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/160126b2-460d-4d8c-a3d1-5eb7371f8a65_isrForCrosstalkSources_997029_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4715 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/160126b2-460d-4d8c-a3d1-5eb7371f8a65_isrForCrosstalkSources_997029_35.stderr parsl.dataflow.dflow INFO: Task 5026 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7b46f073457a6439b98895e92d950ce5 with result from task 5026 parsl.dataflow.dflow DEBUG: Task 5026 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/33a82eb2-5f15-4f40-8e4f-1010e754848f_isrForCrosstalkSources_996987_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5026 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/33a82eb2-5f15-4f40-8e4f-1010e754848f_isrForCrosstalkSources_996987_8.stderr parsl.dataflow.dflow INFO: Task 4921 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5ebbb07fe3893fc419a3926b7167c67 with result from task 4921 parsl.dataflow.dflow DEBUG: Task 4921 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0c6489a3-7aa7-4c30-afd9-b750d7409ab2_isrForCrosstalkSources_996997_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4921 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0c6489a3-7aa7-4c30-afd9-b750d7409ab2_isrForCrosstalkSources_996997_2.stderr parsl.dataflow.dflow INFO: Task 4687 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3eeba0846b19c3ad00b2ae49b6f7c12e with result from task 4687 parsl.dataflow.dflow DEBUG: Task 4687 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/079db310-2456-46ad-8f0b-598de1a9c0cb_isrForCrosstalkSources_996989_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4687 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/079db310-2456-46ad-8f0b-598de1a9c0cb_isrForCrosstalkSources_996989_50.stderr parsl.dataflow.dflow INFO: Task 4801 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 62d973069c72b6e23a2dd6e542c8a316 with result from task 4801 parsl.dataflow.dflow DEBUG: Task 4801 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/84fdc6e0-79fe-4c4e-9c93-ae1f324f1667_isrForCrosstalkSources_997033_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4801 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/84fdc6e0-79fe-4c4e-9c93-ae1f324f1667_isrForCrosstalkSources_997033_61.stderr parsl.dataflow.dflow INFO: Task 4902 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb15b4a5aa8a212573f171827122bc55 with result from task 4902 parsl.dataflow.dflow DEBUG: Task 4902 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/22c9b2f0-062b-4d62-85d5-d0724d541091_isrForCrosstalkSources_997030_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4902 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/22c9b2f0-062b-4d62-85d5-d0724d541091_isrForCrosstalkSources_997030_52.stderr parsl.dataflow.dflow INFO: Task 4583 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7df9a1c62d9e1041ee23abb852c5edc3 with result from task 4583 parsl.dataflow.dflow DEBUG: Task 4583 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ad1bc874-1884-4e22-bbb9-b7fecac502a6_isrForCrosstalkSources_996991_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4583 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/ad1bc874-1884-4e22-bbb9-b7fecac502a6_isrForCrosstalkSources_996991_30.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:46 proc_lsst.multi:146 [INFO] found job 17075417 in provider checkpoint proc_lsst.multi INFO: found job 17075417 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4631 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0f88db3f831ea9bb4848a1a854df7df9 with result from task 4631 parsl.dataflow.dflow DEBUG: Task 4631 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/849dd612-e3a3-47c8-940f-16eb08b1b67f_isrForCrosstalkSources_997013_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4631 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/849dd612-e3a3-47c8-940f-16eb08b1b67f_isrForCrosstalkSources_997013_49.stderr parsl.dataflow.dflow INFO: Task 4973 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e86d39df429eecf21bb9a4132c4b8c74 with result from task 4973 parsl.dataflow.dflow DEBUG: Task 4973 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e36a2d4d-9203-4899-b89e-9ec7c397b87d_isrForCrosstalkSources_997001_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4973 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/e36a2d4d-9203-4899-b89e-9ec7c397b87d_isrForCrosstalkSources_997001_37.stderr parsl.dataflow.dflow INFO: Task 4763 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5233a35f874f8be76a7f39f351bfb153 with result from task 4763 parsl.dataflow.dflow DEBUG: Task 4763 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/34a7fec2-35cf-49ce-9793-5bc705fea95a_isrForCrosstalkSources_996969_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4763 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/34a7fec2-35cf-49ce-9793-5bc705fea95a_isrForCrosstalkSources_996969_17.stderr parsl.dataflow.dflow INFO: Task 4544 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab10a4b6d2b719b593d039a70d72484f with result from task 4544 parsl.dataflow.dflow DEBUG: Task 4544 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/cc3d4e97-33cb-4ee7-9b79-36a8db617b64_isrForCrosstalkSources_996998_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4544 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/cc3d4e97-33cb-4ee7-9b79-36a8db617b64_isrForCrosstalkSources_996998_41.stderr parsl.dataflow.dflow INFO: Task 4803 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 47f19f944909bee835466676a151af4f with result from task 4803 parsl.dataflow.dflow DEBUG: Task 4803 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/11c35dfe-3b4e-40d2-8511-9187a615f271_isrForCrosstalkSources_996967_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4803 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/11c35dfe-3b4e-40d2-8511-9187a615f271_isrForCrosstalkSources_996967_19.stderr parsl.dataflow.dflow INFO: Task 4897 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fbeebe9b6f7572e7798155c00b33bb98 with result from task 4897 parsl.dataflow.dflow DEBUG: Task 4897 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/e142ef6d-7478-44cd-be8d-ed95884b54e0_isrForCrosstalkSources_996967_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4897 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/e142ef6d-7478-44cd-be8d-ed95884b54e0_isrForCrosstalkSources_996967_61.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:47 proc_lsst.multi:146 [INFO] found job 17075428 in provider checkpoint proc_lsst.multi INFO: found job 17075428 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4598 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d05846e90c27bab4a91ff2aff7b98a7e with result from task 4598 parsl.dataflow.dflow DEBUG: Task 4598 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/a5f4017b-055e-46bd-bbe7-13b578a6be49_isrForCrosstalkSources_997030_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4598 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/a5f4017b-055e-46bd-bbe7-13b578a6be49_isrForCrosstalkSources_997030_47.stderr parsl.dataflow.dflow INFO: Task 4896 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 883ba7fe1de4ac2c13ad8c38591eae0e with result from task 4896 parsl.dataflow.dflow DEBUG: Task 4896 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d4da0a8c-8768-4f49-891d-25de523000e5_isrForCrosstalkSources_996997_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4896 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/d4da0a8c-8768-4f49-891d-25de523000e5_isrForCrosstalkSources_996997_37.stderr parsl.dataflow.dflow INFO: Task 4863 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 439b0338dcf178f28dcc2b3456722d5a with result from task 4863 parsl.dataflow.dflow DEBUG: Task 4863 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/87e07dee-a22d-4a31-ae90-9250d514a8ce_isrForCrosstalkSources_996993_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4863 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/87e07dee-a22d-4a31-ae90-9250d514a8ce_isrForCrosstalkSources_996993_21.stderr parsl.dataflow.dflow INFO: Task 4625 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5db9849e568a4816bd972204f0ff7f5e with result from task 4625 parsl.dataflow.dflow DEBUG: Task 4625 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/db30f475-61fc-4296-89dd-baf77a1a5fd4_isrForCrosstalkSources_996998_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4625 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/db30f475-61fc-4296-89dd-baf77a1a5fd4_isrForCrosstalkSources_996998_49.stderr parsl.dataflow.dflow INFO: Task 4875 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6c3c5725db8afecef0b0ecf8bdcf0719 with result from task 4875 parsl.dataflow.dflow DEBUG: Task 4875 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c65837e6-2eee-4e20-a048-30def3cf0987_isrForCrosstalkSources_997025_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4875 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/c65837e6-2eee-4e20-a048-30def3cf0987_isrForCrosstalkSources_997025_20.stderr parsl.dataflow.dflow INFO: Task 4910 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3c276f0e992b2f1bd94e194d947125b with result from task 4910 parsl.dataflow.dflow DEBUG: Task 4910 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a1eb21eb-cec1-42e3-8f59-59bba775649e_isrForCrosstalkSources_997011_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4910 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/a1eb21eb-cec1-42e3-8f59-59bba775649e_isrForCrosstalkSources_997011_56.stderr parsl.dataflow.dflow INFO: Task 4749 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3b4fb41c65506eab1fe882e34c3c1ae8 with result from task 4749 parsl.dataflow.dflow DEBUG: Task 4749 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/05946fe8-13b0-4022-a21d-e1ced42d768c_isrForCrosstalkSources_997022_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4749 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/05946fe8-13b0-4022-a21d-e1ced42d768c_isrForCrosstalkSources_997022_24.stderr parsl.dataflow.dflow INFO: Task 4750 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7e7928cb0d96401788800d4f50feeb1e with result from task 4750 parsl.dataflow.dflow DEBUG: Task 4750 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/4b90119e-8e19-4d0f-ac37-e576b4968f11_isrForCrosstalkSources_996973_51.stdout parsl.dataflow.dflow INFO: Standard error for task 4750 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/4b90119e-8e19-4d0f-ac37-e576b4968f11_isrForCrosstalkSources_996973_51.stderr parsl.dataflow.dflow INFO: Task 4556 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b15884c794519c0141bd77b624394c7 with result from task 4556 parsl.dataflow.dflow DEBUG: Task 4556 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/48a0e81c-c4cd-4658-adaa-9c8999a7c059_isrForCrosstalkSources_997006_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4556 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/48a0e81c-c4cd-4658-adaa-9c8999a7c059_isrForCrosstalkSources_997006_48.stderr parsl.dataflow.dflow INFO: Task 4820 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c78ef33fa7a1f59a01d3a1b064e790e7 with result from task 4820 parsl.dataflow.dflow DEBUG: Task 4820 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/5ffeb5fc-ea2f-4bcf-b262-cecd3e8e9aad_isrForCrosstalkSources_996975_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4820 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/5ffeb5fc-ea2f-4bcf-b262-cecd3e8e9aad_isrForCrosstalkSources_996975_37.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:47 proc_lsst.multi:146 [INFO] found job 17075430 in provider checkpoint proc_lsst.multi INFO: found job 17075430 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4785 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry af6b12d5c01731094749fbb8d9b80de9 with result from task 4785 parsl.dataflow.dflow DEBUG: Task 4785 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ef1d40e0-dfe9-469e-bcd9-7ae1f3d54367_isrForCrosstalkSources_997016_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4785 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/ef1d40e0-dfe9-469e-bcd9-7ae1f3d54367_isrForCrosstalkSources_997016_12.stderr parsl.dataflow.dflow INFO: Task 4867 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c3f6d52143313236c4af26f98051a660 with result from task 4867 parsl.dataflow.dflow DEBUG: Task 4867 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/de90c2c5-2258-4868-b4d4-691786f1b715_isrForCrosstalkSources_997020_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4867 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/de90c2c5-2258-4868-b4d4-691786f1b715_isrForCrosstalkSources_997020_40.stderr parsl.dataflow.dflow INFO: Task 4848 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 29c0987946d36a4f909860003b6a522f with result from task 4848 parsl.dataflow.dflow DEBUG: Task 4848 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/ba7c868c-821a-40ed-a48a-1ff65c94f278_isrForCrosstalkSources_996970_56.stdout parsl.dataflow.dflow INFO: Standard error for task 4848 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/ba7c868c-821a-40ed-a48a-1ff65c94f278_isrForCrosstalkSources_996970_56.stderr parsl.dataflow.dflow INFO: Task 4794 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d277c14b7cacfb6f8537e62c211c5724 with result from task 4794 parsl.dataflow.dflow DEBUG: Task 4794 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/2eab5ced-e763-41f9-940a-e35fb3166909_isrForCrosstalkSources_996991_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4794 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/2eab5ced-e763-41f9-940a-e35fb3166909_isrForCrosstalkSources_996991_60.stderr parsl.dataflow.dflow INFO: Task 4837 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7dc7727083f7661c8e08d405914df670 with result from task 4837 parsl.dataflow.dflow DEBUG: Task 4837 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8cd51ecd-9426-4660-8c1d-0ad74d3d466f_isrForCrosstalkSources_996977_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4837 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8cd51ecd-9426-4660-8c1d-0ad74d3d466f_isrForCrosstalkSources_996977_10.stderr parsl.dataflow.dflow INFO: Task 4681 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fb4ba7f0bc4aa2e8e2909bc8ba17160 with result from task 4681 parsl.dataflow.dflow DEBUG: Task 4681 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c1e9b93a-00ec-4e26-ad94-6afc715a0499_isrForCrosstalkSources_996967_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4681 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/c1e9b93a-00ec-4e26-ad94-6afc715a0499_isrForCrosstalkSources_996967_50.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:48 proc_lsst.multi:146 [INFO] found job 17075431 in provider checkpoint proc_lsst.multi INFO: found job 17075431 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4901 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48c80bbe9deba580b42703055da46c9b with result from task 4901 parsl.dataflow.dflow DEBUG: Task 4901 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/4ef14c5a-77a0-4b51-977a-642567c32103_isrForCrosstalkSources_996988_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4901 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/4ef14c5a-77a0-4b51-977a-642567c32103_isrForCrosstalkSources_996988_5.stderr parsl.dataflow.dflow INFO: Task 5034 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fd1e4f2115265bb6f386f0cc62493ad4 with result from task 5034 parsl.dataflow.dflow DEBUG: Task 5034 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/20f7522c-57cd-455a-ae92-aeef462cd3e6_isrForCrosstalkSources_997004_52.stdout parsl.dataflow.dflow INFO: Standard error for task 5034 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/20f7522c-57cd-455a-ae92-aeef462cd3e6_isrForCrosstalkSources_997004_52.stderr parsl.dataflow.dflow INFO: Task 5004 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2a3978cb720a287c7da032b6175398e1 with result from task 5004 parsl.dataflow.dflow DEBUG: Task 5004 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/255f521b-3f95-459b-8219-5b7f4137d154_isrForCrosstalkSources_997016_49.stdout parsl.dataflow.dflow INFO: Standard error for task 5004 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/255f521b-3f95-459b-8219-5b7f4137d154_isrForCrosstalkSources_997016_49.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:48 proc_lsst.multi:146 [INFO] found job 17075432 in provider checkpoint proc_lsst.multi INFO: found job 17075432 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4799 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b8d0687ef6717079f665e176029d830 with result from task 4799 parsl.dataflow.dflow DEBUG: Task 4799 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/bf3261b9-c171-4052-ba90-7fe346ca7633_isrForCrosstalkSources_996953_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4799 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996953/bf3261b9-c171-4052-ba90-7fe346ca7633_isrForCrosstalkSources_996953_47.stderr parsl.dataflow.dflow INFO: Task 4629 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fcbfd9b3439b3b09640411f065e32e7 with result from task 4629 parsl.dataflow.dflow DEBUG: Task 4629 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/94053b65-63eb-4762-b0f9-4e25a70db99d_isrForCrosstalkSources_996977_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4629 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/94053b65-63eb-4762-b0f9-4e25a70db99d_isrForCrosstalkSources_996977_52.stderr parsl.dataflow.dflow INFO: Task 4579 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 154ba2ffb787549ad050321b340bd6f0 with result from task 4579 parsl.dataflow.dflow DEBUG: Task 4579 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5e64222b-ddda-4c08-bf33-2367fd6d137b_isrForCrosstalkSources_996972_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4579 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996972/5e64222b-ddda-4c08-bf33-2367fd6d137b_isrForCrosstalkSources_996972_60.stderr parsl.dataflow.dflow INFO: Task 4814 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 19760ccd1a318225fe73693cd33de327 with result from task 4814 parsl.dataflow.dflow DEBUG: Task 4814 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f7998868-6567-4482-bdf6-b071cf94d4de_isrForCrosstalkSources_996989_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4814 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f7998868-6567-4482-bdf6-b071cf94d4de_isrForCrosstalkSources_996989_16.stderr parsl.dataflow.dflow INFO: Task 5007 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0c3750d52c911f5bcddee50eb69d3197 with result from task 5007 parsl.dataflow.dflow DEBUG: Task 5007 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/10dcbfc3-28aa-4a9d-81eb-6ee05d27295d_isrForCrosstalkSources_997007_29.stdout parsl.dataflow.dflow INFO: Standard error for task 5007 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/10dcbfc3-28aa-4a9d-81eb-6ee05d27295d_isrForCrosstalkSources_997007_29.stderr parsl.dataflow.dflow INFO: Task 4884 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e5fabc108f0194e5dae9771eda0c66dd with result from task 4884 parsl.dataflow.dflow DEBUG: Task 4884 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/3433fd14-8aa2-41b2-b4ff-cdfe0e02ed5d_isrForCrosstalkSources_997007_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4884 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/3433fd14-8aa2-41b2-b4ff-cdfe0e02ed5d_isrForCrosstalkSources_997007_48.stderr parsl.dataflow.dflow INFO: Task 4812 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 849f29d4280801a3e75c8c0ee81682c1 with result from task 4812 parsl.dataflow.dflow DEBUG: Task 4812 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/da4318bd-da56-4775-bdf4-b06cacb73d45_isrForCrosstalkSources_997012_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4812 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/da4318bd-da56-4775-bdf4-b06cacb73d45_isrForCrosstalkSources_997012_52.stderr parsl.dataflow.dflow INFO: Task 4753 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry df455c77c77307042831100619972266 with result from task 4753 parsl.dataflow.dflow DEBUG: Task 4753 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/674ee716-6a33-4390-99b3-79913d14db0e_isrForCrosstalkSources_996958_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4753 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/674ee716-6a33-4390-99b3-79913d14db0e_isrForCrosstalkSources_996958_31.stderr parsl.dataflow.dflow INFO: Task 4819 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry dc5a77e08a0b91c613d786819161445c with result from task 4819 parsl.dataflow.dflow DEBUG: Task 4819 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c0c39e1d-f687-4c3f-a8fd-e0e965c79713_isrForCrosstalkSources_996962_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4819 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/c0c39e1d-f687-4c3f-a8fd-e0e965c79713_isrForCrosstalkSources_996962_37.stderr parsl.dataflow.dflow INFO: Task 4845 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 077bd7ad3e91f2cbe4bd5297832b2489 with result from task 4845 parsl.dataflow.dflow DEBUG: Task 4845 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/74101d64-436a-48f5-bca8-f63c2025b222_isrForCrosstalkSources_996986_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4845 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996986/74101d64-436a-48f5-bca8-f63c2025b222_isrForCrosstalkSources_996986_18.stderr parsl.dataflow.dflow INFO: Task 4864 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8fa797d294c6dd25cb86bcec5748da00 with result from task 4864 parsl.dataflow.dflow DEBUG: Task 4864 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7a136b8c-811e-4918-8834-0a096b56319b_isrForCrosstalkSources_997025_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4864 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/7a136b8c-811e-4918-8834-0a096b56319b_isrForCrosstalkSources_997025_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 5052 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry b809dbe883607b7c154a969f148df529 with result from task 5052 parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 5052 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 5052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e61f810c-5c05-491c-8717-31053cb035f3_isrForCrosstalkSources_996955_31.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 5052 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/e61f810c-5c05-491c-8717-31053cb035f3_isrForCrosstalkSources_996955_31.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:49 proc_lsst.multi:146 [INFO] found job 17075433 in provider checkpoint proc_lsst.multi INFO: found job 17075433 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 5017 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 636b39dfa1929c5c6eac61ff582dc4fc with result from task 5017 parsl.dataflow.dflow DEBUG: Task 5017 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/877330a6-be6e-4164-be95-19aa343bf32e_isrForCrosstalkSources_997022_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5017 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/877330a6-be6e-4164-be95-19aa343bf32e_isrForCrosstalkSources_997022_23.stderr parsl.dataflow.dflow INFO: Task 4752 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0bc5d9637665651a71876c8f8eb75938 with result from task 4752 parsl.dataflow.dflow DEBUG: Task 4752 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/4fa6cc0c-8573-4e31-9bd3-53e02188860b_isrForCrosstalkSources_996981_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4752 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/4fa6cc0c-8573-4e31-9bd3-53e02188860b_isrForCrosstalkSources_996981_47.stderr parsl.dataflow.dflow INFO: Task 4886 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 834cdf175bda2da175780a7b815e8e72 with result from task 4886 parsl.dataflow.dflow DEBUG: Task 4886 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/9d5e14f3-5d6e-4a11-b41e-20862bb8aa65_isrForCrosstalkSources_997021_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4886 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/9d5e14f3-5d6e-4a11-b41e-20862bb8aa65_isrForCrosstalkSources_997021_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:50 proc_lsst.multi:146 [INFO] found job 17075434 in provider checkpoint proc_lsst.multi INFO: found job 17075434 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4804 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56d2303ca0bcab58ee3e19c26fe96ebe with result from task 4804 parsl.dataflow.dflow DEBUG: Task 4804 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/ebde189f-f0f0-4dd4-af70-8865fd8a8a2c_isrForCrosstalkSources_997003_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4804 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/ebde189f-f0f0-4dd4-af70-8865fd8a8a2c_isrForCrosstalkSources_997003_24.stderr parsl.dataflow.dflow INFO: Task 4654 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2457819bd1a2f932b64fa1676c655112 with result from task 4654 parsl.dataflow.dflow DEBUG: Task 4654 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/aea3ef04-7f03-485f-85e1-b995f00e324f_isrForCrosstalkSources_996969_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4654 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/aea3ef04-7f03-485f-85e1-b995f00e324f_isrForCrosstalkSources_996969_47.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:50 proc_lsst.multi:146 [INFO] found job 17075435 in provider checkpoint proc_lsst.multi INFO: found job 17075435 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4788 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 771dda9e9895294b76f644ce9088a6ad with result from task 4788 parsl.dataflow.dflow DEBUG: Task 4788 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/59154914-99ec-4d55-b40e-171b6e920dd6_isrForCrosstalkSources_997012_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4788 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/59154914-99ec-4d55-b40e-171b6e920dd6_isrForCrosstalkSources_997012_62.stderr parsl.dataflow.dflow INFO: Task 4806 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb44d2e3b98852ee245437c0c3a9d3d1 with result from task 4806 parsl.dataflow.dflow DEBUG: Task 4806 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4e763486-5a91-4ee4-9def-89c179e681df_isrForCrosstalkSources_997014_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4806 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/4e763486-5a91-4ee4-9def-89c179e681df_isrForCrosstalkSources_997014_1.stderr parsl.dataflow.dflow INFO: Task 5058 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 837944f024b6bad9fd2a01553a77664f with result from task 5058 parsl.dataflow.dflow DEBUG: Task 5058 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/4c0d7fbb-1324-4e3a-bb01-b2d5375a8d65_isrForCrosstalkSources_997001_4.stdout parsl.dataflow.dflow INFO: Standard error for task 5058 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/4c0d7fbb-1324-4e3a-bb01-b2d5375a8d65_isrForCrosstalkSources_997001_4.stderr parsl.dataflow.dflow INFO: Task 5059 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88cc7315750564cf2fdb644a2c60a4e1 with result from task 5059 parsl.dataflow.dflow DEBUG: Task 5059 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f01af821-e907-4bb9-829c-89918178a153_isrForCrosstalkSources_996989_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5059 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/f01af821-e907-4bb9-829c-89918178a153_isrForCrosstalkSources_996989_8.stderr parsl.dataflow.dflow INFO: Task 4607 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 620ee8c3f245c553f471b54c5bb26553 with result from task 4607 parsl.dataflow.dflow DEBUG: Task 4607 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2b176645-17a9-41fe-a28b-61fdb9466480_isrForCrosstalkSources_997005_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4607 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/2b176645-17a9-41fe-a28b-61fdb9466480_isrForCrosstalkSources_997005_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4769 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry 66d647bf352c9177ee23d228abae1e76 with result from task 4769 parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 4769 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 4769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0d8d5805-f21c-4882-996b-928a6e6933d7_isrForCrosstalkSources_996966_11.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 4769 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/0d8d5805-f21c-4882-996b-928a6e6933d7_isrForCrosstalkSources_996966_11.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:51 proc_lsst.multi:146 [INFO] found job 17075436 in provider checkpoint proc_lsst.multi INFO: found job 17075436 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4610 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 52709b3a2b114d8e23e091c45c2dcf55 with result from task 4610 parsl.dataflow.dflow DEBUG: Task 4610 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/5249b3de-3000-41ba-9e76-ff57247ee6a4_isrForCrosstalkSources_996998_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4610 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996998/5249b3de-3000-41ba-9e76-ff57247ee6a4_isrForCrosstalkSources_996998_48.stderr parsl.dataflow.dflow INFO: Task 5070 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a30b8929bce66d0248ce03997eba6cc7 with result from task 5070 parsl.dataflow.dflow DEBUG: Task 5070 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/656bfdb3-4b62-4387-a966-aeb399efe34e_isrForCrosstalkSources_997003_4.stdout parsl.dataflow.dflow INFO: Standard error for task 5070 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/656bfdb3-4b62-4387-a966-aeb399efe34e_isrForCrosstalkSources_997003_4.stderr parsl.dataflow.dflow INFO: Task 5051 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 91876d8b78691a9a92b96bfc897a3ee8 with result from task 5051 parsl.dataflow.dflow DEBUG: Task 5051 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/ebf3a113-bd8d-4e18-ba87-92c96d1b0e0b_isrForCrosstalkSources_996983_50.stdout parsl.dataflow.dflow INFO: Standard error for task 5051 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/ebf3a113-bd8d-4e18-ba87-92c96d1b0e0b_isrForCrosstalkSources_996983_50.stderr parsl.dataflow.dflow INFO: Task 4677 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 37008b4bc1e67ae4355ff34068fd8999 with result from task 4677 parsl.dataflow.dflow DEBUG: Task 4677 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b75c58b7-6b6d-4ed2-a54f-75958dfe7f28_isrForCrosstalkSources_996997_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4677 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b75c58b7-6b6d-4ed2-a54f-75958dfe7f28_isrForCrosstalkSources_996997_28.stderr parsl.dataflow.dflow INFO: Task 4858 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9fbdf167f72c49a9c696080ae86d986e with result from task 4858 parsl.dataflow.dflow DEBUG: Task 4858 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/a29ea7ff-9b99-460f-ab21-a87ab8b0c9ac_isrForCrosstalkSources_996975_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4858 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/a29ea7ff-9b99-460f-ab21-a87ab8b0c9ac_isrForCrosstalkSources_996975_57.stderr parsl.dataflow.dflow INFO: Task 4802 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8ab96c54e9ca8c9db74843cf3feb7459 with result from task 4802 parsl.dataflow.dflow DEBUG: Task 4802 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/45e1a12b-214c-44f4-a673-5a997b91d152_isrForCrosstalkSources_996978_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4802 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/45e1a12b-214c-44f4-a673-5a997b91d152_isrForCrosstalkSources_996978_21.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 4890 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c8fefc769781f5ecb70e63ac87d541aa with result from task 4890 parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:51 proc_lsst.multi:146 [INFO] found job 17075438 in provider checkpoint proc_lsst.multi INFO: found job 17075438 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow DEBUG: Task 4890 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/df44e15d-e7e7-4f67-b2cb-31f8e712ffc6_isrForCrosstalkSources_997025_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4890 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/df44e15d-e7e7-4f67-b2cb-31f8e712ffc6_isrForCrosstalkSources_997025_2.stderr parsl.dataflow.dflow INFO: Task 4757 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc9884dcfd37c4230432a6b2c0766571 with result from task 4757 parsl.dataflow.dflow DEBUG: Task 4757 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0666bbc9-c8a1-4866-8673-af4fe6d63488_isrForCrosstalkSources_996997_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4757 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/0666bbc9-c8a1-4866-8673-af4fe6d63488_isrForCrosstalkSources_996997_59.stderr parsl.dataflow.dflow INFO: Task 4717 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9efa502eb89194433a57614f802c74c1 with result from task 4717 parsl.dataflow.dflow DEBUG: Task 4717 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/643b1c2f-d554-40f0-9550-a73b54e06f46_isrForCrosstalkSources_997010_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4717 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/643b1c2f-d554-40f0-9550-a73b54e06f46_isrForCrosstalkSources_997010_3.stderr parsl.dataflow.dflow INFO: Task 4762 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a24eaffa64b9d2ebb903bb19a279855a with result from task 4762 parsl.dataflow.dflow DEBUG: Task 4762 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/94a527de-3642-459e-8942-ded8a49a4abe_isrForCrosstalkSources_996960_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4762 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/94a527de-3642-459e-8942-ded8a49a4abe_isrForCrosstalkSources_996960_31.stderr parsl.dataflow.dflow INFO: Task 4903 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry efddda1ebe4020a4ec0b77913ae30553 with result from task 4903 parsl.dataflow.dflow DEBUG: Task 4903 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/461d13b3-7a28-49d3-8ba3-a7ec57162d01_isrForCrosstalkSources_997033_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4903 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/461d13b3-7a28-49d3-8ba3-a7ec57162d01_isrForCrosstalkSources_997033_43.stderr parsl.dataflow.dflow INFO: Task 4929 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 568ab67759add8ec5119cbeae85e871f with result from task 4929 parsl.dataflow.dflow DEBUG: Task 4929 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/82b1896a-a608-40b8-85b8-7c8db851f68b_isrForCrosstalkSources_997003_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4929 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/82b1896a-a608-40b8-85b8-7c8db851f68b_isrForCrosstalkSources_997003_26.stderr parsl.dataflow.dflow INFO: Task 4877 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9bef2a30331c92993d330278f305dd57 with result from task 4877 parsl.dataflow.dflow DEBUG: Task 4877 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/cc7c8e67-8e4d-4503-8be0-bb6c37bbd282_isrForCrosstalkSources_996991_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4877 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/cc7c8e67-8e4d-4503-8be0-bb6c37bbd282_isrForCrosstalkSources_996991_15.stderr parsl.dataflow.dflow INFO: Task 4920 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b7fdeb0c79d6699c9025f0e9f35bff7e with result from task 4920 parsl.dataflow.dflow DEBUG: Task 4920 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7dddedd8-b6a9-4ce2-ac0a-5834316294fe_isrForCrosstalkSources_997005_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4920 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7dddedd8-b6a9-4ce2-ac0a-5834316294fe_isrForCrosstalkSources_997005_16.stderr parsl.dataflow.dflow INFO: Task 4737 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8cdb3557b71bdb0cce4b5eb7e565f38f with result from task 4737 parsl.dataflow.dflow DEBUG: Task 4737 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/45b107aa-3986-40ed-bf33-6c8633a56fd3_isrForCrosstalkSources_996980_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4737 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996980/45b107aa-3986-40ed-bf33-6c8633a56fd3_isrForCrosstalkSources_996980_62.stderr parsl.dataflow.dflow INFO: Task 4798 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb9e2c27b6df9146c9a474d27f120d9b with result from task 4798 parsl.dataflow.dflow DEBUG: Task 4798 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4ccbb321-10f3-4cf9-a357-6be685c31110_isrForCrosstalkSources_996976_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4798 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/4ccbb321-10f3-4cf9-a357-6be685c31110_isrForCrosstalkSources_996976_59.stderr parsl.dataflow.dflow INFO: Task 4882 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a8858fadcb81433ec5d1753e50299d04 with result from task 4882 parsl.dataflow.dflow DEBUG: Task 4882 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/dc9d1b16-ef70-4d59-95db-7e57e55f0fed_isrForCrosstalkSources_997017_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4882 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/dc9d1b16-ef70-4d59-95db-7e57e55f0fed_isrForCrosstalkSources_997017_27.stderr parsl.dataflow.dflow INFO: Task 4850 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a6e8a76f1f7ad2687696cf13c6dbec5 with result from task 4850 parsl.dataflow.dflow DEBUG: Task 4850 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1723d15d-ca3b-46a7-815f-799fdc1fc4dd_isrForCrosstalkSources_997017_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4850 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/1723d15d-ca3b-46a7-815f-799fdc1fc4dd_isrForCrosstalkSources_997017_50.stderr parsl.dataflow.dflow INFO: Task 5065 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6be4a885ef22cc758c73bdfd015eac2f with result from task 5065 parsl.dataflow.dflow DEBUG: Task 5065 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c4429549-7334-43f2-8320-b8dbaf39dfc1_isrForCrosstalkSources_997033_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5065 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/c4429549-7334-43f2-8320-b8dbaf39dfc1_isrForCrosstalkSources_997033_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:52 proc_lsst.multi:146 [INFO] found job 17075440 in provider checkpoint proc_lsst.multi INFO: found job 17075440 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4853 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a0322e05acaa80e9c77bd4442d799bcc with result from task 4853 parsl.dataflow.dflow DEBUG: Task 4853 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/9637e321-7a20-43b0-9bfb-866bd7e7dc8c_isrForCrosstalkSources_997017_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4853 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/9637e321-7a20-43b0-9bfb-866bd7e7dc8c_isrForCrosstalkSources_997017_55.stderr parsl.dataflow.dflow INFO: Task 4766 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 01edb9e1f571b45994cdab2f8a57606e with result from task 4766 parsl.dataflow.dflow DEBUG: Task 4766 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d45a8a2d-e8a1-4988-a9a0-5093baf7ab6e_isrForCrosstalkSources_996981_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4766 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/d45a8a2d-e8a1-4988-a9a0-5093baf7ab6e_isrForCrosstalkSources_996981_28.stderr parsl.dataflow.dflow INFO: Task 4817 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 661bf8a78e08efe642a4953540b0648a with result from task 4817 parsl.dataflow.dflow DEBUG: Task 4817 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c59fc0ae-2080-4154-b33c-517714d06e5e_isrForCrosstalkSources_997016_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4817 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/c59fc0ae-2080-4154-b33c-517714d06e5e_isrForCrosstalkSources_997016_54.stderr parsl.dataflow.dflow INFO: Task 4887 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e7c3930a478c746e4575fbe2e9a8c7d1 with result from task 4887 parsl.dataflow.dflow DEBUG: Task 4887 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f1152981-5493-48fd-9a4f-d1444415b2f1_isrForCrosstalkSources_997005_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4887 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/f1152981-5493-48fd-9a4f-d1444415b2f1_isrForCrosstalkSources_997005_38.stderr parsl.dataflow.dflow INFO: Task 4730 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 99d5d84591abf4c49fdbbe12799af696 with result from task 4730 parsl.dataflow.dflow DEBUG: Task 4730 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7468517c-3dfe-4dca-9d4d-2dac660b08b0_isrForCrosstalkSources_997005_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4730 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/7468517c-3dfe-4dca-9d4d-2dac660b08b0_isrForCrosstalkSources_997005_12.stderr parsl.dataflow.dflow INFO: Task 4938 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d1380783cdf611524073c58201c8628 with result from task 4938 parsl.dataflow.dflow DEBUG: Task 4938 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/571664c2-752e-4824-9111-fe5f2ba3bb78_isrForCrosstalkSources_996963_25.stdout parsl.dataflow.dflow INFO: Standard error for task 4938 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/571664c2-752e-4824-9111-fe5f2ba3bb78_isrForCrosstalkSources_996963_25.stderr parsl.dataflow.dflow INFO: Task 4932 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ffb05414cf589d478b61edf3b232c16d with result from task 4932 parsl.dataflow.dflow DEBUG: Task 4932 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4fae4627-297b-4d65-8e39-bf6b008f1137_isrForCrosstalkSources_997015_26.stdout parsl.dataflow.dflow INFO: Standard error for task 4932 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/4fae4627-297b-4d65-8e39-bf6b008f1137_isrForCrosstalkSources_997015_26.stderr parsl.dataflow.dflow INFO: Task 4743 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry facddfa92f09cdfad62530b28db4cedc with result from task 4743 parsl.dataflow.dflow DEBUG: Task 4743 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/23b9b745-cb60-4898-9db1-02084d2032a5_isrForCrosstalkSources_996984_35.stdout parsl.dataflow.dflow INFO: Standard error for task 4743 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/23b9b745-cb60-4898-9db1-02084d2032a5_isrForCrosstalkSources_996984_35.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:52 proc_lsst.multi:146 [INFO] found job 17075441 in provider checkpoint proc_lsst.multi INFO: found job 17075441 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4881 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2d914d75733782eeee4f14b40942dc6b with result from task 4881 parsl.dataflow.dflow DEBUG: Task 4881 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7659ef5f-43ad-47de-a238-a744f64f3bdd_isrForCrosstalkSources_996975_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4881 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/7659ef5f-43ad-47de-a238-a744f64f3bdd_isrForCrosstalkSources_996975_18.stderr parsl.dataflow.dflow INFO: Task 4796 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9d44dbb4c4dd0df86b90859a04c3720b with result from task 4796 parsl.dataflow.dflow DEBUG: Task 4796 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b81aee93-1d5d-41c3-a3e3-3fd57b1bea9b_isrForCrosstalkSources_997003_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4796 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/b81aee93-1d5d-41c3-a3e3-3fd57b1bea9b_isrForCrosstalkSources_997003_16.stderr parsl.dataflow.dflow INFO: Task 4735 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50aa183d91fa22d24b0470bbaa8522a1 with result from task 4735 parsl.dataflow.dflow DEBUG: Task 4735 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/28a39f36-7aa9-4e14-8612-26280800869c_isrForCrosstalkSources_997012_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4735 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/28a39f36-7aa9-4e14-8612-26280800869c_isrForCrosstalkSources_997012_7.stderr parsl.dataflow.dflow INFO: Task 5083 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 693b0eb2f416939fef1435464e354130 with result from task 5083 parsl.dataflow.dflow DEBUG: Task 5083 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/90fc5198-fd88-42c1-a930-129399d7e9c7_isrForCrosstalkSources_997002_28.stdout parsl.dataflow.dflow INFO: Standard error for task 5083 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/90fc5198-fd88-42c1-a930-129399d7e9c7_isrForCrosstalkSources_997002_28.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Task 4776 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.memoization DEBUG: Storing app cache entry 36cb5328f267e8c5ab60c7609b3d916b with result from task 4776 parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow DEBUG: Task 4776 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard output for task 4776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/f214442b-4c04-4ed0-846f-9f089f2c0166_isrForCrosstalkSources_996952_47.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.dflow INFO: Standard error for task 4776 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/f214442b-4c04-4ed0-846f-9f089f2c0166_isrForCrosstalkSources_996952_47.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:53 proc_lsst.multi:146 [INFO] found job 17075443 in provider checkpoint proc_lsst.multi INFO: found job 17075443 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4930 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e095706399f1867133731eb3f2a241e6 with result from task 4930 parsl.dataflow.dflow DEBUG: Task 4930 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/d89395d8-0741-49f8-9234-413c33d62c77_isrForCrosstalkSources_997033_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4930 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/d89395d8-0741-49f8-9234-413c33d62c77_isrForCrosstalkSources_997033_27.stderr parsl.dataflow.dflow INFO: Task 4746 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7302a81070692849be91f12c4975a6d1 with result from task 4746 parsl.dataflow.dflow DEBUG: Task 4746 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/09f8800d-5088-4632-865e-b1cc90e8c5fb_isrForCrosstalkSources_997012_46.stdout parsl.dataflow.dflow INFO: Standard error for task 4746 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/09f8800d-5088-4632-865e-b1cc90e8c5fb_isrForCrosstalkSources_997012_46.stderr parsl.dataflow.dflow INFO: Task 4950 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c9b4d3261046661f50698d3e9c596d78 with result from task 4950 parsl.dataflow.dflow DEBUG: Task 4950 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3248d48a-4048-4cec-9921-4c65e2c9158b_isrForCrosstalkSources_997001_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4950 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/3248d48a-4048-4cec-9921-4c65e2c9158b_isrForCrosstalkSources_997001_14.stderr parsl.dataflow.dflow INFO: Task 4628 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cca38b947862f8a58bc790beecaa6cec with result from task 4628 parsl.dataflow.dflow DEBUG: Task 4628 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eab9c3d-7a08-4de2-95fb-875cf0af85bc_isrForCrosstalkSources_996985_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4628 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0eab9c3d-7a08-4de2-95fb-875cf0af85bc_isrForCrosstalkSources_996985_60.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:53 proc_lsst.multi:146 [INFO] found job 17075444 in provider checkpoint proc_lsst.multi INFO: found job 17075444 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4948 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8b48c5055178e8163952780502184bb2 with result from task 4948 parsl.dataflow.dflow DEBUG: Task 4948 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a921803e-82ed-47e7-b84a-52034c863d83_isrForCrosstalkSources_996994_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4948 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/a921803e-82ed-47e7-b84a-52034c863d83_isrForCrosstalkSources_996994_20.stderr parsl.dataflow.dflow INFO: Task 4534 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f46f5fe3942a7615b8ceb3e02b463fd1 with result from task 4534 parsl.dataflow.dflow DEBUG: Task 4534 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/0af44847-7ce3-4b2f-baa5-2c8b29b19928_isrForCrosstalkSources_997032_13.stdout parsl.dataflow.dflow INFO: Standard error for task 4534 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/0af44847-7ce3-4b2f-baa5-2c8b29b19928_isrForCrosstalkSources_997032_13.stderr parsl.dataflow.dflow INFO: Task 4538 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a1efc8256e519d41690104e328e89370 with result from task 4538 parsl.dataflow.dflow DEBUG: Task 4538 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/58c60c6e-5b94-418f-9a9d-71ae6d28e61b_isrForCrosstalkSources_996984_27.stdout parsl.dataflow.dflow INFO: Standard error for task 4538 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/58c60c6e-5b94-418f-9a9d-71ae6d28e61b_isrForCrosstalkSources_996984_27.stderr parsl.dataflow.dflow INFO: Task 4535 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5f38fadb9f12a26433a5f3f291b34b7 with result from task 4535 parsl.dataflow.dflow DEBUG: Task 4535 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/8a429e39-34d6-4b12-a5cc-19fc9bf399f0_isrForCrosstalkSources_997000_33.stdout parsl.dataflow.dflow INFO: Standard error for task 4535 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997000/8a429e39-34d6-4b12-a5cc-19fc9bf399f0_isrForCrosstalkSources_997000_33.stderr parsl.dataflow.dflow INFO: Task 4536 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 94a7ca7774635b2c4567a54112ee4b95 with result from task 4536 parsl.dataflow.dflow DEBUG: Task 4536 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a0516b1d-0640-409d-840b-ef6b2fa06d96_isrForCrosstalkSources_996960_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4536 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/a0516b1d-0640-409d-840b-ef6b2fa06d96_isrForCrosstalkSources_996960_11.stderr parsl.dataflow.dflow INFO: Task 4537 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a07bf4aec97ad9375a625df6ee5be1fb with result from task 4537 parsl.dataflow.dflow DEBUG: Task 4537 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fc736d57-a58b-4a0c-8136-f96c9c7ba207_isrForCrosstalkSources_997012_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4537 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/fc736d57-a58b-4a0c-8136-f96c9c7ba207_isrForCrosstalkSources_997012_12.stderr parsl.dataflow.dflow INFO: Task 4531 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 734804a58b60c8499ded043fb7361797 with result from task 4531 parsl.dataflow.dflow DEBUG: Task 4531 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9e5c3541-745a-4bc2-be70-30dbf300945c_isrForCrosstalkSources_996960_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4531 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9e5c3541-745a-4bc2-be70-30dbf300945c_isrForCrosstalkSources_996960_30.stderr parsl.dataflow.dflow INFO: Task 4939 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8717ceeec0a367fde9da02bda4e1eda with result from task 4939 parsl.dataflow.dflow DEBUG: Task 4939 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/aa2d3eff-2e98-4e57-8925-e8a4e88889cc_isrForCrosstalkSources_996983_17.stdout parsl.dataflow.dflow INFO: Standard error for task 4939 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/aa2d3eff-2e98-4e57-8925-e8a4e88889cc_isrForCrosstalkSources_996983_17.stderr parsl.dataflow.dflow INFO: Task 4918 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fcd4921571459fa886368d61cb2b97b5 with result from task 4918 parsl.dataflow.dflow DEBUG: Task 4918 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f66c4f76-efdc-4e66-9afd-6e0a99db88a1_isrForCrosstalkSources_996978_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4918 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/f66c4f76-efdc-4e66-9afd-6e0a99db88a1_isrForCrosstalkSources_996978_34.stderr parsl.dataflow.dflow INFO: Task 4815 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a7900aedb9303f46393afc094dd5c6fd with result from task 4815 parsl.dataflow.dflow DEBUG: Task 4815 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/7a0bb4bd-7257-4743-9602-5bef11f0bef7_isrForCrosstalkSources_996999_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4815 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/7a0bb4bd-7257-4743-9602-5bef11f0bef7_isrForCrosstalkSources_996999_50.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:54 proc_lsst.multi:146 [INFO] found job 17075446 in provider checkpoint proc_lsst.multi INFO: found job 17075446 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.dataflow.dflow INFO: Task 4745 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 222f6bd35b7a0be8f3b8c349d0ef5317 with result from task 4745 parsl.dataflow.dflow DEBUG: Task 4745 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e28e04e1-65a5-4133-afff-baada127eda3_isrForCrosstalkSources_996957_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4745 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/e28e04e1-65a5-4133-afff-baada127eda3_isrForCrosstalkSources_996957_42.stderr parsl.dataflow.dflow INFO: Task 4532 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bb1d8d6b3604cc6225d5a560b0534190 with result from task 4532 parsl.dataflow.dflow DEBUG: Task 4532 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/56dc4c2f-b61c-452f-87b7-5b31aa49c915_isrForCrosstalkSources_996966_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4532 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/56dc4c2f-b61c-452f-87b7-5b31aa49c915_isrForCrosstalkSources_996966_7.stderr parsl.dataflow.dflow INFO: Task 4810 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0dd064a447408ddb46e581c60bb6fa67 with result from task 4810 parsl.dataflow.dflow DEBUG: Task 4810 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1ef2c8b7-3cf8-42b3-9ce5-914f6c28e379_isrForCrosstalkSources_997004_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4810 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/1ef2c8b7-3cf8-42b3-9ce5-914f6c28e379_isrForCrosstalkSources_997004_62.stderr parsl.dataflow.dflow INFO: Task 4842 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9289a91b9a0614f13c7aeb29f73fb93 with result from task 4842 parsl.dataflow.dflow DEBUG: Task 4842 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/509c5d9b-0708-4437-922b-c421cc1ed7ea_isrForCrosstalkSources_996977_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4842 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/509c5d9b-0708-4437-922b-c421cc1ed7ea_isrForCrosstalkSources_996977_28.stderr parsl.dataflow.dflow INFO: Task 4827 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3322c3073374527622afa30dbefadad8 with result from task 4827 parsl.dataflow.dflow DEBUG: Task 4827 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/324906f8-e6f2-45e1-972d-c3241b304a4d_isrForCrosstalkSources_996982_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4827 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/324906f8-e6f2-45e1-972d-c3241b304a4d_isrForCrosstalkSources_996982_44.stderr parsl.dataflow.dflow INFO: Task 4940 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cfb164ac33df3d16c43ab2eeb5314151 with result from task 4940 parsl.dataflow.dflow DEBUG: Task 4940 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f80dc8de-d66f-479e-b5db-e987f7e53f8f_isrForCrosstalkSources_997031_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4940 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/f80dc8de-d66f-479e-b5db-e987f7e53f8f_isrForCrosstalkSources_997031_1.stderr parsl.dataflow.dflow INFO: Task 5072 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5558564b3748788d20f535d8abb5afe0 with result from task 5072 parsl.dataflow.dflow DEBUG: Task 5072 checkpointed parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 5072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/176f5245-88be-4d27-bc84-f545bfe21666_isrForCrosstalkSources_996968_23.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 5072 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/176f5245-88be-4d27-bc84-f545bfe21666_isrForCrosstalkSources_996968_23.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Task 5048 completed (launched -> exec_done) parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.memoization DEBUG: Storing app cache entry fac223ed75dec3189464b2bdae754244 with result from task 5048 parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow DEBUG: Task 5048 checkpointed parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard output for task 5048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d84e23bb-2686-411d-8007-92012cf1f5df_isrForCrosstalkSources_997027_33.stdout parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.dataflow.dflow INFO: Standard error for task 5048 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/d84e23bb-2686-411d-8007-92012cf1f5df_isrForCrosstalkSources_997027_33.stderr parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.strategy DEBUG: general strategy starting with strategy_type simple for 1 executors parsl.dataflow.strategy DEBUG: Strategizing for executor multi parsl.dataflow.strategy DEBUG: Slot ratio calculation: active_slots = 312, active_tasks = 162 parsl.dataflow.strategy DEBUG: Executor multi has 162 active tasks, 86/226 running/pending blocks, and 297 connected workers parsl.dataflow.strategy DEBUG: Strategy case 4b: more slots than tasks parsl.dataflow.strategy DEBUG: This strategy does not support scaling in parsl.process_loggers DEBUG: Normal ending for _general_strategy on thread JobStatusPoller-Timer-Thread-22709913200592 2024-03-15 14:00:54 proc_lsst.multi:146 [INFO] found job 91028 in provider local proc_lsst.multi INFO: found job 91028 in provider local parsl.dataflow.dflow INFO: Task 4904 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 45fb94fb525428c07cde2ea474c127fa with result from task 4904 parsl.dataflow.dflow DEBUG: Task 4904 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/973e6b60-f0b4-4515-8b02-cdf7b8ded2ec_isrForCrosstalkSources_997023_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4904 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/973e6b60-f0b4-4515-8b02-cdf7b8ded2ec_isrForCrosstalkSources_997023_40.stderr 2024-03-15 14:00:54 proc_lsst.multi:146 [INFO] found job 17074833 in provider astro proc_lsst.multi INFO: found job 17074833 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5081 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42d4ae2333690d7c242576465e46160f with result from task 5081 parsl.dataflow.dflow DEBUG: Task 5081 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ef2d23cf-04f4-4e22-a2d0-c7d1b1aa8e9b_isrForCrosstalkSources_997005_4.stdout parsl.dataflow.dflow INFO: Standard error for task 5081 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/ef2d23cf-04f4-4e22-a2d0-c7d1b1aa8e9b_isrForCrosstalkSources_997005_4.stderr parsl.dataflow.dflow INFO: Task 4868 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9cea4c989387dc2527e12537b053a908 with result from task 4868 parsl.dataflow.dflow DEBUG: Task 4868 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/7aa437d6-79e6-497f-a9ee-92cd9d272e77_isrForCrosstalkSources_997031_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4868 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/7aa437d6-79e6-497f-a9ee-92cd9d272e77_isrForCrosstalkSources_997031_36.stderr parsl.dataflow.dflow INFO: Task 5074 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e0d31d9d13f322c131eeeeb881362b24 with result from task 5074 parsl.dataflow.dflow DEBUG: Task 5074 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/6fefeacf-50f1-4653-9c3d-05506bbc4d01_isrForCrosstalkSources_997022_62.stdout parsl.dataflow.dflow INFO: Standard error for task 5074 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/6fefeacf-50f1-4653-9c3d-05506bbc4d01_isrForCrosstalkSources_997022_62.stderr parsl.dataflow.dflow INFO: Task 4862 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9a4a8975312bde53663b6d670c37260b with result from task 4862 parsl.dataflow.dflow DEBUG: Task 4862 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b455cfb8-1a4c-4c65-bf30-118c52a3171e_isrForCrosstalkSources_997016_57.stdout parsl.dataflow.dflow INFO: Standard error for task 4862 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/b455cfb8-1a4c-4c65-bf30-118c52a3171e_isrForCrosstalkSources_997016_57.stderr parsl.dataflow.dflow INFO: Task 4754 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a307c08731529bc0281cb1d3e82b1cf2 with result from task 4754 parsl.dataflow.dflow DEBUG: Task 4754 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/cfb9691c-64fe-42a8-84c5-4c83420d6235_isrForCrosstalkSources_996976_47.stdout parsl.dataflow.dflow INFO: Standard error for task 4754 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/cfb9691c-64fe-42a8-84c5-4c83420d6235_isrForCrosstalkSources_996976_47.stderr parsl.dataflow.dflow INFO: Task 4914 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 24745cae888d54cfbed58e0716888404 with result from task 4914 parsl.dataflow.dflow DEBUG: Task 4914 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/acb4e012-17fb-4154-b557-f3aae791712a_isrForCrosstalkSources_997002_12.stdout parsl.dataflow.dflow INFO: Standard error for task 4914 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997002/acb4e012-17fb-4154-b557-f3aae791712a_isrForCrosstalkSources_997002_12.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:55 proc_lsst.multi:146 [INFO] found job 17074834 in provider astro proc_lsst.multi INFO: found job 17074834 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4925 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 52d185486b69e0cab7c478a56abf01ad with result from task 4925 parsl.dataflow.dflow DEBUG: Task 4925 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/00b78541-0aa0-4183-8790-8fc7bbdb703d_isrForCrosstalkSources_997016_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4925 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/00b78541-0aa0-4183-8790-8fc7bbdb703d_isrForCrosstalkSources_997016_9.stderr parsl.dataflow.dflow INFO: Task 4942 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 0935a922fdac3eb6efe6ebed8ed05b07 with result from task 4942 parsl.dataflow.dflow DEBUG: Task 4942 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5dbb3d6f-c032-422d-bdba-db45b21f6b46_isrForCrosstalkSources_996991_6.stdout parsl.dataflow.dflow INFO: Standard error for task 4942 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/5dbb3d6f-c032-422d-bdba-db45b21f6b46_isrForCrosstalkSources_996991_6.stderr parsl.dataflow.dflow INFO: Task 4916 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b5c880db4f780950f5d411d6b4f21e81 with result from task 4916 parsl.dataflow.dflow DEBUG: Task 4916 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/e9f6a769-f8ba-448f-be27-6de44baed749_isrForCrosstalkSources_996991_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4916 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/e9f6a769-f8ba-448f-be27-6de44baed749_isrForCrosstalkSources_996991_50.stderr parsl.dataflow.dflow INFO: Task 4830 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9c568dcb5c3bb466428b2a3080c9deab with result from task 4830 parsl.dataflow.dflow DEBUG: Task 4830 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8a0fae67-b371-4f10-85f1-824df8eb00ca_isrForCrosstalkSources_996977_62.stdout parsl.dataflow.dflow INFO: Standard error for task 4830 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/8a0fae67-b371-4f10-85f1-824df8eb00ca_isrForCrosstalkSources_996977_62.stderr parsl.dataflow.dflow INFO: Task 4919 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 697266183593dab53c4f7a06e66fcde6 with result from task 4919 parsl.dataflow.dflow DEBUG: Task 4919 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7f2f35f4-8053-4c8b-86ef-3089293697a6_isrForCrosstalkSources_997011_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4919 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/7f2f35f4-8053-4c8b-86ef-3089293697a6_isrForCrosstalkSources_997011_15.stderr parsl.dataflow.dflow INFO: Task 4768 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4fb7ba7d11ca22959e9cf6a0d27fb5e2 with result from task 4768 parsl.dataflow.dflow DEBUG: Task 4768 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/90dcd1b1-1213-4329-9a3f-94c6e6b85e7d_isrForCrosstalkSources_996960_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4768 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/90dcd1b1-1213-4329-9a3f-94c6e6b85e7d_isrForCrosstalkSources_996960_28.stderr parsl.dataflow.dflow INFO: Task 4836 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2ae0475d6424683829691415fbbeb0dc with result from task 4836 parsl.dataflow.dflow DEBUG: Task 4836 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9da524f2-0dd6-4d5e-9781-bc7760560e73_isrForCrosstalkSources_996960_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4836 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/9da524f2-0dd6-4d5e-9781-bc7760560e73_isrForCrosstalkSources_996960_38.stderr parsl.dataflow.dflow INFO: Task 4936 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c976c9e38b6cba2a1f8b3a91719246e8 with result from task 4936 parsl.dataflow.dflow DEBUG: Task 4936 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/33fb6ffe-bd91-46d5-9382-86b360af45ea_isrForCrosstalkSources_996970_43.stdout parsl.dataflow.dflow INFO: Standard error for task 4936 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/33fb6ffe-bd91-46d5-9382-86b360af45ea_isrForCrosstalkSources_996970_43.stderr parsl.dataflow.dflow INFO: Task 4533 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 06ceb4682193bbcffec9bb84ac0f3c51 with result from task 4533 parsl.dataflow.dflow DEBUG: Task 4533 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/ac208d0b-3c0e-4b77-9e15-7b5798da96c0_isrForCrosstalkSources_996995_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4533 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/ac208d0b-3c0e-4b77-9e15-7b5798da96c0_isrForCrosstalkSources_996995_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:55 proc_lsst.multi:146 [INFO] found job 17074835 in provider astro proc_lsst.multi INFO: found job 17074835 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5062 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab19c11f5ef0a31159ac5e1eb5d55b7b with result from task 5062 parsl.dataflow.dflow DEBUG: Task 5062 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/42ad355e-7de5-49d9-a5aa-03fc78d0b091_isrForCrosstalkSources_997004_40.stdout parsl.dataflow.dflow INFO: Standard error for task 5062 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997004/42ad355e-7de5-49d9-a5aa-03fc78d0b091_isrForCrosstalkSources_997004_40.stderr parsl.dataflow.dflow INFO: Task 4870 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 35a2e59c7b9abc19558b2497ca0c87fb with result from task 4870 parsl.dataflow.dflow DEBUG: Task 4870 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/13b340fa-e993-4fd5-bd3f-a59389c07beb_isrForCrosstalkSources_996977_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4870 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/13b340fa-e993-4fd5-bd3f-a59389c07beb_isrForCrosstalkSources_996977_61.stderr parsl.dataflow.dflow INFO: Task 4793 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ab3f71e98fbe4675a524733afbf6b0a1 with result from task 4793 parsl.dataflow.dflow DEBUG: Task 4793 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a5512902-ec3a-4984-83c4-3e4155142c7f_isrForCrosstalkSources_996999_60.stdout parsl.dataflow.dflow INFO: Standard error for task 4793 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/a5512902-ec3a-4984-83c4-3e4155142c7f_isrForCrosstalkSources_996999_60.stderr parsl.dataflow.dflow INFO: Task 5031 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c72d7a3fa7eaee7d5371837d51b16a71 with result from task 5031 parsl.dataflow.dflow DEBUG: Task 5031 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f38f50b5-1f72-4739-95a5-4a3dd7aec4aa_isrForCrosstalkSources_996971_39.stdout parsl.dataflow.dflow INFO: Standard error for task 5031 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/f38f50b5-1f72-4739-95a5-4a3dd7aec4aa_isrForCrosstalkSources_996971_39.stderr parsl.dataflow.dflow INFO: Task 4775 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 996733964046ff480e1c506e2cdd9f2d with result from task 4775 parsl.dataflow.dflow DEBUG: Task 4775 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9277410-5a42-4bbe-a09c-981df26d19ef_isrForCrosstalkSources_997009_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4775 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/d9277410-5a42-4bbe-a09c-981df26d19ef_isrForCrosstalkSources_997009_48.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:56 proc_lsst.multi:146 [INFO] found job 17074836 in provider astro proc_lsst.multi INFO: found job 17074836 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4947 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 56306f1db520038813d492b12a0460b6 with result from task 4947 parsl.dataflow.dflow DEBUG: Task 4947 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/123405ee-ca5e-4b54-bc57-32528372223b_isrForCrosstalkSources_996969_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4947 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996969/123405ee-ca5e-4b54-bc57-32528372223b_isrForCrosstalkSources_996969_2.stderr parsl.dataflow.dflow INFO: Task 4899 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cf5bcea1e046d55ca27c098016ff1912 with result from task 4899 parsl.dataflow.dflow DEBUG: Task 4899 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c2f9b681-164e-4677-94c1-f40d04b1c7a2_isrForCrosstalkSources_996995_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4899 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/c2f9b681-164e-4677-94c1-f40d04b1c7a2_isrForCrosstalkSources_996995_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:56 proc_lsst.multi:146 [INFO] found job 17074837 in provider astro proc_lsst.multi INFO: found job 17074837 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4777 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b6547901947c574d4f212f49d0670c97 with result from task 4777 parsl.dataflow.dflow DEBUG: Task 4777 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/7b3514a2-ee69-43a9-b436-a088aa3f5abc_isrForCrosstalkSources_996982_1.stdout parsl.dataflow.dflow INFO: Standard error for task 4777 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/7b3514a2-ee69-43a9-b436-a088aa3f5abc_isrForCrosstalkSources_996982_1.stderr parsl.dataflow.dflow INFO: Task 5037 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 13325ca46048833baf81c6b6cc579028 with result from task 5037 parsl.dataflow.dflow DEBUG: Task 5037 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8d67e711-1a22-4f91-a2bc-1e376b11021c_isrForCrosstalkSources_997032_9.stdout parsl.dataflow.dflow INFO: Standard error for task 5037 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/8d67e711-1a22-4f91-a2bc-1e376b11021c_isrForCrosstalkSources_997032_9.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:57 proc_lsst.multi:146 [INFO] found job 17074838 in provider astro proc_lsst.multi INFO: found job 17074838 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4813 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e00469b0b254d8889f8f783785e686c8 with result from task 4813 parsl.dataflow.dflow DEBUG: Task 4813 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bdc5dac7-ac9f-4d2c-aa72-36a6bd268f9a_isrForCrosstalkSources_997030_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4813 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/bdc5dac7-ac9f-4d2c-aa72-36a6bd268f9a_isrForCrosstalkSources_997030_59.stderr parsl.dataflow.dflow INFO: Task 4951 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a224a773b7f73e02bb3a1338c8ead304 with result from task 4951 parsl.dataflow.dflow DEBUG: Task 4951 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/dd5df0c5-4f77-4e58-8e9c-654a227a0a05_isrForCrosstalkSources_997022_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4951 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/dd5df0c5-4f77-4e58-8e9c-654a227a0a05_isrForCrosstalkSources_997022_7.stderr parsl.dataflow.dflow INFO: Task 4933 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c784a644341b903b9878f86932fba6e with result from task 4933 parsl.dataflow.dflow DEBUG: Task 4933 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a48bfe39-e8a2-44dc-8302-434a8e2cd56f_isrForCrosstalkSources_997015_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4933 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/a48bfe39-e8a2-44dc-8302-434a8e2cd56f_isrForCrosstalkSources_997015_3.stderr parsl.dataflow.dflow INFO: Task 4917 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fa30116e0a4cefe3bb0bd056ee6e6862 with result from task 4917 parsl.dataflow.dflow DEBUG: Task 4917 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/dc5a0731-8e68-4d36-b0e8-b368a1799219_isrForCrosstalkSources_997027_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4917 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997027/dc5a0731-8e68-4d36-b0e8-b368a1799219_isrForCrosstalkSources_997027_42.stderr parsl.dataflow.dflow INFO: Task 3156 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8c918c8bc0f2959762072811f8264819 with result from task 3156 parsl.dataflow.dflow DEBUG: Task 3156 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dc1bfb09-a884-47f8-a8b1-e7606fe00f8a_isrForCrosstalkSources_997001_25.stdout parsl.dataflow.dflow INFO: Standard error for task 3156 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/dc1bfb09-a884-47f8-a8b1-e7606fe00f8a_isrForCrosstalkSources_997001_25.stderr parsl.dataflow.dflow INFO: Task 5063 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cca2fed0ac8aee0ca3bcf384d28e9a9a with result from task 5063 parsl.dataflow.dflow DEBUG: Task 5063 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/009cd290-21df-44f4-a032-a7a5597bd7cc_isrForCrosstalkSources_996967_60.stdout parsl.dataflow.dflow INFO: Standard error for task 5063 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/009cd290-21df-44f4-a032-a7a5597bd7cc_isrForCrosstalkSources_996967_60.stderr parsl.dataflow.dflow INFO: Task 4911 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4c8b45dd0336625cd5447cb7da7182a with result from task 4911 parsl.dataflow.dflow DEBUG: Task 4911 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/25c4553a-16a9-4f2a-803a-03b00daf0292_isrForCrosstalkSources_996967_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4911 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996967/25c4553a-16a9-4f2a-803a-03b00daf0292_isrForCrosstalkSources_996967_49.stderr parsl.dataflow.dflow INFO: Task 5060 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bcf8763008e01bdc916ad85bfbb2e977 with result from task 5060 parsl.dataflow.dflow DEBUG: Task 5060 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d27c231c-660d-4b51-acd6-8bd2b21524a1_isrForCrosstalkSources_996993_44.stdout parsl.dataflow.dflow INFO: Standard error for task 5060 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/d27c231c-660d-4b51-acd6-8bd2b21524a1_isrForCrosstalkSources_996993_44.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:00:58 proc_lsst.multi:146 [INFO] found job 17074839 in provider astro proc_lsst.multi INFO: found job 17074839 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5067 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e8682aef0136f9a351dd620e27a93671 with result from task 5067 parsl.dataflow.dflow DEBUG: Task 5067 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/82c25e13-7eb6-476f-a9c2-dc61e7dc43fb_isrForCrosstalkSources_997025_1.stdout parsl.dataflow.dflow INFO: Standard error for task 5067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/82c25e13-7eb6-476f-a9c2-dc61e7dc43fb_isrForCrosstalkSources_997025_1.stderr parsl.dataflow.dflow INFO: Task 5039 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4039834e930b77a951e672383530dabb with result from task 5039 parsl.dataflow.dflow DEBUG: Task 5039 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/06ea4c39-60af-4d77-8a78-a1cdd48e9319_isrForCrosstalkSources_996996_9.stdout parsl.dataflow.dflow INFO: Standard error for task 5039 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/06ea4c39-60af-4d77-8a78-a1cdd48e9319_isrForCrosstalkSources_996996_9.stderr parsl.dataflow.dflow INFO: Task 4934 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 054cc3f9af5450fe1a29b03569f322ed with result from task 4934 parsl.dataflow.dflow DEBUG: Task 4934 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/ce77481d-b2cb-4972-87ec-2a2672dfb0b5_isrForCrosstalkSources_997014_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4934 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/ce77481d-b2cb-4972-87ec-2a2672dfb0b5_isrForCrosstalkSources_997014_58.stderr parsl.dataflow.dflow INFO: Task 4869 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6e2051911066271705fb9c93b0f0c8f6 with result from task 4869 parsl.dataflow.dflow DEBUG: Task 4869 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d664f007-9ffd-4a75-ba87-362b456aa6b8_isrForCrosstalkSources_996982_31.stdout parsl.dataflow.dflow INFO: Standard error for task 4869 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d664f007-9ffd-4a75-ba87-362b456aa6b8_isrForCrosstalkSources_996982_31.stderr parsl.dataflow.dflow INFO: Task 5064 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 95a6c55a07810fdd9bd4293ed52b66e3 with result from task 5064 parsl.dataflow.dflow DEBUG: Task 5064 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e7b6d8fb-34c1-4836-925a-ec6c44f9bd88_isrForCrosstalkSources_996952_48.stdout parsl.dataflow.dflow INFO: Standard error for task 5064 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/e7b6d8fb-34c1-4836-925a-ec6c44f9bd88_isrForCrosstalkSources_996952_48.stderr parsl.dataflow.dflow INFO: Task 4937 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fa52042dc69d629401b324fe2e64c38 with result from task 4937 parsl.dataflow.dflow DEBUG: Task 4937 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/21b9699a-b1b4-4366-90d6-f79bcbcc6173_isrForCrosstalkSources_996991_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4937 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/21b9699a-b1b4-4366-90d6-f79bcbcc6173_isrForCrosstalkSources_996991_54.stderr parsl.dataflow.dflow INFO: Task 4780 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c7939b62057ea5a6d4b5fa08db2732f5 with result from task 4780 parsl.dataflow.dflow DEBUG: Task 4780 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d90d15f2-0f57-422f-b81a-b32ff486e691_isrForCrosstalkSources_996982_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4780 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/d90d15f2-0f57-422f-b81a-b32ff486e691_isrForCrosstalkSources_996982_41.stderr parsl.dataflow.dflow INFO: Task 4924 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3941e01787102e26ff8b6162426bd253 with result from task 4924 parsl.dataflow.dflow DEBUG: Task 4924 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e3226332-5718-4cb8-8e41-fbbf057e1659_isrForCrosstalkSources_996993_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4924 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/e3226332-5718-4cb8-8e41-fbbf057e1659_isrForCrosstalkSources_996993_53.stderr parsl.dataflow.dflow INFO: Task 5066 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e26325852d12919f00989d67ad44a549 with result from task 5066 parsl.dataflow.dflow DEBUG: Task 5066 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/cee20815-c9d1-4493-96e0-763f8100daba_isrForCrosstalkSources_997003_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5066 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997003/cee20815-c9d1-4493-96e0-763f8100daba_isrForCrosstalkSources_997003_8.stderr parsl.dataflow.dflow INFO: Task 4874 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e24e187bf15c8efe90e7e7b6a4e8c1b4 with result from task 4874 parsl.dataflow.dflow DEBUG: Task 4874 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b5b7eb2b-99f1-43b5-860f-0c5792abdce3_isrForCrosstalkSources_997030_5.stdout parsl.dataflow.dflow INFO: Standard error for task 4874 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/b5b7eb2b-99f1-43b5-860f-0c5792abdce3_isrForCrosstalkSources_997030_5.stderr parsl.dataflow.dflow INFO: Task 4834 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 88c5d51d25a406d6854f4b1eb3adf82e with result from task 4834 parsl.dataflow.dflow DEBUG: Task 4834 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/59ecab66-c62f-4928-95d6-7634dc8ab023_isrForCrosstalkSources_996968_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4834 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996968/59ecab66-c62f-4928-95d6-7634dc8ab023_isrForCrosstalkSources_996968_10.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:00 proc_lsst.multi:146 [INFO] found job 17074841 in provider astro proc_lsst.multi INFO: found job 17074841 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3087 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b147eccca8403b687e3e5c44c281d0c1 with result from task 3087 parsl.dataflow.dflow DEBUG: Task 3087 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68aa2644-aebf-4fe4-b2e7-b56d61cece5b_isrForCrosstalkSources_996985_22.stdout parsl.dataflow.dflow INFO: Standard error for task 3087 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/68aa2644-aebf-4fe4-b2e7-b56d61cece5b_isrForCrosstalkSources_996985_22.stderr parsl.dataflow.dflow INFO: Task 5042 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a74b2d7b822704b427c59fbd9e20911b with result from task 5042 parsl.dataflow.dflow DEBUG: Task 5042 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/91e5d330-8f8d-4328-915e-794a9427e577_isrForCrosstalkSources_996952_34.stdout parsl.dataflow.dflow INFO: Standard error for task 5042 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996952/91e5d330-8f8d-4328-915e-794a9427e577_isrForCrosstalkSources_996952_34.stderr parsl.dataflow.dflow INFO: Task 4895 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c0c4014339bb68fa523e3c4cb06de5d5 with result from task 4895 parsl.dataflow.dflow DEBUG: Task 4895 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/2516195f-0706-4da8-b180-fed2fafe0e13_isrForCrosstalkSources_997028_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4895 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/2516195f-0706-4da8-b180-fed2fafe0e13_isrForCrosstalkSources_997028_3.stderr parsl.dataflow.dflow INFO: Task 5077 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 527bd3095b5af62af2aab0109e71e514 with result from task 5077 parsl.dataflow.dflow DEBUG: Task 5077 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e052d490-560c-424e-8bc7-96f98b736d1b_isrForCrosstalkSources_997020_39.stdout parsl.dataflow.dflow INFO: Standard error for task 5077 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997020/e052d490-560c-424e-8bc7-96f98b736d1b_isrForCrosstalkSources_997020_39.stderr parsl.dataflow.dflow INFO: Task 4880 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8f1f1b269d9df4485f1fbd8fa0df932e with result from task 4880 parsl.dataflow.dflow DEBUG: Task 4880 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b960d7b7-7fc4-40d4-b77c-1148fe977419_isrForCrosstalkSources_997008_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4880 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/b960d7b7-7fc4-40d4-b77c-1148fe977419_isrForCrosstalkSources_997008_41.stderr parsl.dataflow.dflow INFO: Task 5061 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fb074a7687fb10e2542ea54eddaaa3a with result from task 5061 parsl.dataflow.dflow DEBUG: Task 5061 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/b31a8430-fc24-44a8-b8e2-54f6a53c59f2_isrForCrosstalkSources_996994_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5061 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/b31a8430-fc24-44a8-b8e2-54f6a53c59f2_isrForCrosstalkSources_996994_46.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:00 proc_lsst.multi:146 [INFO] found job 17074842 in provider astro proc_lsst.multi INFO: found job 17074842 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4983 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ddd47665c9de3155b95189d294c41b71 with result from task 4983 parsl.dataflow.dflow DEBUG: Task 4983 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/bd85914c-dca6-4123-925f-7c4f6bdbf45d_isrForCrosstalkSources_996976_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4983 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/bd85914c-dca6-4123-925f-7c4f6bdbf45d_isrForCrosstalkSources_996976_19.stderr parsl.dataflow.dflow INFO: Task 5035 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 36cb8e1ed7cf9aa1e2c689bfcf53586a with result from task 5035 parsl.dataflow.dflow DEBUG: Task 5035 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/b93be41d-406c-4dc1-b1a1-7739c325908b_isrForCrosstalkSources_996960_51.stdout parsl.dataflow.dflow INFO: Standard error for task 5035 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/b93be41d-406c-4dc1-b1a1-7739c325908b_isrForCrosstalkSources_996960_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:01 proc_lsst.multi:146 [INFO] found job 17074843 in provider astro proc_lsst.multi INFO: found job 17074843 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4981 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a26e57fb196675d75816e08a97f9fbdd with result from task 4981 parsl.dataflow.dflow DEBUG: Task 4981 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/41cfd45a-5820-49ce-8e60-9a1f0a8eaf0e_isrForCrosstalkSources_996999_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4981 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/41cfd45a-5820-49ce-8e60-9a1f0a8eaf0e_isrForCrosstalkSources_996999_20.stderr parsl.dataflow.dflow INFO: Task 4993 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 76ea9f3d3200aa656b338cdbcd84f513 with result from task 4993 parsl.dataflow.dflow DEBUG: Task 4993 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/edb88570-bac2-4613-a27b-df5ff13ccd14_isrForCrosstalkSources_996996_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4993 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/edb88570-bac2-4613-a27b-df5ff13ccd14_isrForCrosstalkSources_996996_4.stderr parsl.dataflow.dflow INFO: Task 4954 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 50b79d783762f33dfd264b89255d2565 with result from task 4954 parsl.dataflow.dflow DEBUG: Task 4954 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/39aacde7-cf5d-4fee-910c-11d6ea65fd8c_isrForCrosstalkSources_997008_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4954 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/39aacde7-cf5d-4fee-910c-11d6ea65fd8c_isrForCrosstalkSources_997008_19.stderr parsl.dataflow.dflow INFO: Task 4971 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ca488654bb44a86557f7ff62eec5aac5 with result from task 4971 parsl.dataflow.dflow DEBUG: Task 4971 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1e65a889-7ad6-448a-8496-eb134e42d924_isrForCrosstalkSources_996958_24.stdout parsl.dataflow.dflow INFO: Standard error for task 4971 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/1e65a889-7ad6-448a-8496-eb134e42d924_isrForCrosstalkSources_996958_24.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:02 proc_lsst.multi:146 [INFO] found job 17074850 in provider astro proc_lsst.multi INFO: found job 17074850 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4824 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d71dc7a051b44a0dda22a0c017e4004b with result from task 4824 parsl.dataflow.dflow DEBUG: Task 4824 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0e95d1d3-a30f-41e5-9cd4-3f48011aea2d_isrForCrosstalkSources_996985_37.stdout parsl.dataflow.dflow INFO: Standard error for task 4824 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/0e95d1d3-a30f-41e5-9cd4-3f48011aea2d_isrForCrosstalkSources_996985_37.stderr parsl.dataflow.dflow INFO: Task 5013 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f070520d4a7ff24686f43c965c0a8a01 with result from task 5013 parsl.dataflow.dflow DEBUG: Task 5013 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/862cb02c-212a-451f-bfc2-d927a435554a_isrForCrosstalkSources_996954_15.stdout parsl.dataflow.dflow INFO: Standard error for task 5013 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/862cb02c-212a-451f-bfc2-d927a435554a_isrForCrosstalkSources_996954_15.stderr parsl.dataflow.dflow INFO: Task 4935 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d9ab695bc995e5f3c618b557db1567d4 with result from task 4935 parsl.dataflow.dflow DEBUG: Task 4935 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/1a2f32bf-7479-4f00-990a-439ed3590484_isrForCrosstalkSources_996974_45.stdout parsl.dataflow.dflow INFO: Standard error for task 4935 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/1a2f32bf-7479-4f00-990a-439ed3590484_isrForCrosstalkSources_996974_45.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:02 proc_lsst.multi:146 [INFO] found job 17074851 in provider astro proc_lsst.multi INFO: found job 17074851 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4944 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 15eea248b04d591bff3248939b5ffbd5 with result from task 4944 parsl.dataflow.dflow DEBUG: Task 4944 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a57e7056-f2cd-4b73-95a5-85c37fd42110_isrForCrosstalkSources_997033_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4944 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997033/a57e7056-f2cd-4b73-95a5-85c37fd42110_isrForCrosstalkSources_997033_4.stderr parsl.dataflow.dflow INFO: Task 4926 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a4c39c96b4810ea4951549fde1927671 with result from task 4926 parsl.dataflow.dflow DEBUG: Task 4926 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4776aab2-3028-409c-ab09-eee1d744776b_isrForCrosstalkSources_996957_40.stdout parsl.dataflow.dflow INFO: Standard error for task 4926 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/4776aab2-3028-409c-ab09-eee1d744776b_isrForCrosstalkSources_996957_40.stderr parsl.dataflow.dflow INFO: Task 4823 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3fa24af1cdc063607ac52a995a47d34c with result from task 4823 parsl.dataflow.dflow DEBUG: Task 4823 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5ccf8a06-5a1b-4ffd-9008-a08a22947950_isrForCrosstalkSources_996985_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4823 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/5ccf8a06-5a1b-4ffd-9008-a08a22947950_isrForCrosstalkSources_996985_10.stderr parsl.dataflow.dflow INFO: Task 4992 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 16a490fa810627bfe760bb30cad1a344 with result from task 4992 parsl.dataflow.dflow DEBUG: Task 4992 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/7247f74e-93c2-40fc-9ec4-39508355f5e7_isrForCrosstalkSources_996958_9.stdout parsl.dataflow.dflow INFO: Standard error for task 4992 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/7247f74e-93c2-40fc-9ec4-39508355f5e7_isrForCrosstalkSources_996958_9.stderr parsl.dataflow.dflow INFO: Task 4987 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5d2faa1548cd23feefd3a06173c5296b with result from task 4987 parsl.dataflow.dflow DEBUG: Task 4987 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/891a8797-b31c-4165-a0af-db743f1bf27d_isrForCrosstalkSources_997009_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4987 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/891a8797-b31c-4165-a0af-db743f1bf27d_isrForCrosstalkSources_997009_38.stderr parsl.dataflow.dflow INFO: Task 4990 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e532e13a7a3c415a2e6ce9f70b7a9929 with result from task 4990 parsl.dataflow.dflow DEBUG: Task 4990 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/1d386aaa-d4d8-411c-9d43-b693eaa674f5_isrForCrosstalkSources_996991_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4990 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996991/1d386aaa-d4d8-411c-9d43-b693eaa674f5_isrForCrosstalkSources_996991_29.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:03 proc_lsst.multi:146 [INFO] found job 17074852 in provider astro proc_lsst.multi INFO: found job 17074852 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5006 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 71f4a2bf4b1876dcf65e389ab007dd49 with result from task 5006 parsl.dataflow.dflow DEBUG: Task 5006 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/a84513e9-1517-49a8-9b24-77ef03f3b0d5_isrForCrosstalkSources_996988_27.stdout parsl.dataflow.dflow INFO: Standard error for task 5006 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/a84513e9-1517-49a8-9b24-77ef03f3b0d5_isrForCrosstalkSources_996988_27.stderr parsl.dataflow.dflow INFO: Task 4952 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry eb365cef957e87572ed6b12be0c3468f with result from task 4952 parsl.dataflow.dflow DEBUG: Task 4952 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/103bbf15-3378-4d70-b967-a3f418ea1177_isrForCrosstalkSources_997021_49.stdout parsl.dataflow.dflow INFO: Standard error for task 4952 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997021/103bbf15-3378-4d70-b967-a3f418ea1177_isrForCrosstalkSources_997021_49.stderr parsl.dataflow.dflow INFO: Task 4825 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 25ded45107323d88c4a237e7c0301e16 with result from task 4825 parsl.dataflow.dflow DEBUG: Task 4825 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/2e9fae70-1d68-4086-81f4-a15d2f457520_isrForCrosstalkSources_997017_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4825 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/2e9fae70-1d68-4086-81f4-a15d2f457520_isrForCrosstalkSources_997017_15.stderr parsl.dataflow.dflow INFO: Task 4945 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 86b60cced4116311e7a060f43053c529 with result from task 4945 parsl.dataflow.dflow DEBUG: Task 4945 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a5e0dd4d-f43b-4efa-82b2-941fd94784de_isrForCrosstalkSources_997007_55.stdout parsl.dataflow.dflow INFO: Standard error for task 4945 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/a5e0dd4d-f43b-4efa-82b2-941fd94784de_isrForCrosstalkSources_997007_55.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:03 proc_lsst.multi:146 [INFO] found job 17074853 in provider astro proc_lsst.multi INFO: found job 17074853 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4985 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d4b346acb013bb939e193fb5d046058f with result from task 4985 parsl.dataflow.dflow DEBUG: Task 4985 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d4d2c8ba-4fb5-4917-8fc8-f949841b34ca_isrForCrosstalkSources_997023_19.stdout parsl.dataflow.dflow INFO: Standard error for task 4985 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/d4d2c8ba-4fb5-4917-8fc8-f949841b34ca_isrForCrosstalkSources_997023_19.stderr parsl.dataflow.dflow INFO: Task 4959 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8cd64ad3b4b8a52a1e4da4ecfbb98a9f with result from task 4959 parsl.dataflow.dflow DEBUG: Task 4959 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/04f2de67-0580-4d08-af10-8410240d94de_isrForCrosstalkSources_996964_36.stdout parsl.dataflow.dflow INFO: Standard error for task 4959 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/04f2de67-0580-4d08-af10-8410240d94de_isrForCrosstalkSources_996964_36.stderr parsl.dataflow.dflow INFO: Task 4712 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e9d31d98e90e0d21539219c59f2da363 with result from task 4712 parsl.dataflow.dflow DEBUG: Task 4712 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/efaa3a93-95b7-40c7-b2dd-cbc5e4d2f03e_isrForCrosstalkSources_997011_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4712 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/efaa3a93-95b7-40c7-b2dd-cbc5e4d2f03e_isrForCrosstalkSources_997011_42.stderr parsl.dataflow.dflow INFO: Task 4832 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4e437f9b6b8d068dd494316d7737fb5f with result from task 4832 parsl.dataflow.dflow DEBUG: Task 4832 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/fea1923d-b180-47ab-a25b-2891491cbbdd_isrForCrosstalkSources_997005_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4832 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997005/fea1923d-b180-47ab-a25b-2891491cbbdd_isrForCrosstalkSources_997005_11.stderr parsl.dataflow.dflow INFO: Task 4828 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2da06926975f941a9a1b160b48b85739 with result from task 4828 parsl.dataflow.dflow DEBUG: Task 4828 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/47cdf5dc-47da-4bc6-9745-dac574079b24_isrForCrosstalkSources_997007_30.stdout parsl.dataflow.dflow INFO: Standard error for task 4828 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997007/47cdf5dc-47da-4bc6-9745-dac574079b24_isrForCrosstalkSources_997007_30.stderr parsl.dataflow.dflow INFO: Task 4978 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2fc36cf7c45f3f227a2f87c9d0b6731b with result from task 4978 parsl.dataflow.dflow DEBUG: Task 4978 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c5517f74-2e4a-46c1-8e25-00163fa94750_isrForCrosstalkSources_996966_20.stdout parsl.dataflow.dflow INFO: Standard error for task 4978 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/c5517f74-2e4a-46c1-8e25-00163fa94750_isrForCrosstalkSources_996966_20.stderr parsl.dataflow.dflow INFO: Task 5046 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a172f6c8007ad7604cafd27d19cd7fcd with result from task 5046 parsl.dataflow.dflow DEBUG: Task 5046 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e44cfccb-b2f5-4e05-97c5-9f6229740e93_isrForCrosstalkSources_996970_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5046 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e44cfccb-b2f5-4e05-97c5-9f6229740e93_isrForCrosstalkSources_996970_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:04 proc_lsst.multi:146 [INFO] found job 17074854 in provider astro proc_lsst.multi INFO: found job 17074854 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4953 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a2bea23d83d18dd6cda509b336585022 with result from task 4953 parsl.dataflow.dflow DEBUG: Task 4953 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f50d2c37-d952-416f-ba61-8b794ed7f14f_isrForCrosstalkSources_996985_11.stdout parsl.dataflow.dflow INFO: Standard error for task 4953 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996985/f50d2c37-d952-416f-ba61-8b794ed7f14f_isrForCrosstalkSources_996985_11.stderr parsl.dataflow.dflow INFO: Task 5036 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a9e66dfb68055fb9cccd8a1db82ca6aa with result from task 5036 parsl.dataflow.dflow DEBUG: Task 5036 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/189c2320-d604-459a-880e-b830a9b3d1ef_isrForCrosstalkSources_997017_19.stdout parsl.dataflow.dflow INFO: Standard error for task 5036 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/189c2320-d604-459a-880e-b830a9b3d1ef_isrForCrosstalkSources_997017_19.stderr parsl.dataflow.dflow INFO: Task 4833 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d0760720adb6ce93bd3aee05217ed7c0 with result from task 4833 parsl.dataflow.dflow DEBUG: Task 4833 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/967532dc-78ee-41b2-b39f-ca253e47ebc3_isrForCrosstalkSources_997031_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4833 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997031/967532dc-78ee-41b2-b39f-ca253e47ebc3_isrForCrosstalkSources_997031_10.stderr parsl.dataflow.dflow INFO: Task 4958 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d21efcf2300773972b20a2aac4f6ab94 with result from task 4958 parsl.dataflow.dflow DEBUG: Task 4958 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e5c0deec-eecd-4305-a062-3863979564ba_isrForCrosstalkSources_997032_38.stdout parsl.dataflow.dflow INFO: Standard error for task 4958 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997032/e5c0deec-eecd-4305-a062-3863979564ba_isrForCrosstalkSources_997032_38.stderr parsl.dataflow.dflow INFO: Task 4965 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry fb1d61db9d062fad62b6bb07990fe2d1 with result from task 4965 parsl.dataflow.dflow DEBUG: Task 4965 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/bccc67bd-5350-4d56-975c-84acdffc926b_isrForCrosstalkSources_996994_34.stdout parsl.dataflow.dflow INFO: Standard error for task 4965 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996994/bccc67bd-5350-4d56-975c-84acdffc926b_isrForCrosstalkSources_996994_34.stderr parsl.dataflow.dflow INFO: Task 5056 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a081b3898dc5b3c132b0f095720a82f9 with result from task 5056 parsl.dataflow.dflow DEBUG: Task 5056 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/1e3bc188-4a07-4527-a07c-082edbf855fe_isrForCrosstalkSources_997013_20.stdout parsl.dataflow.dflow INFO: Standard error for task 5056 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/1e3bc188-4a07-4527-a07c-082edbf855fe_isrForCrosstalkSources_997013_20.stderr parsl.dataflow.dflow INFO: Task 5019 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6694b22e02b089972cd715a6b381e235 with result from task 5019 parsl.dataflow.dflow DEBUG: Task 5019 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/857fb17c-f8ef-48e8-8c64-fc382d28b71c_isrForCrosstalkSources_996977_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5019 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996977/857fb17c-f8ef-48e8-8c64-fc382d28b71c_isrForCrosstalkSources_996977_23.stderr parsl.dataflow.dflow INFO: Task 5053 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1118ed7dd8c21ce3c7417bb1914183a6 with result from task 5053 parsl.dataflow.dflow DEBUG: Task 5053 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/38f5956a-ac16-4274-a134-8d77ef6b6b61_isrForCrosstalkSources_996995_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5053 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996995/38f5956a-ac16-4274-a134-8d77ef6b6b61_isrForCrosstalkSources_996995_8.stderr parsl.dataflow.dflow INFO: Task 5000 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 72dba624ab99d23e000c815fc16adc4d with result from task 5000 parsl.dataflow.dflow DEBUG: Task 5000 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/91a77a0c-2bf7-4f25-b1fb-a3d3afe484d9_isrForCrosstalkSources_996992_8.stdout parsl.dataflow.dflow INFO: Standard error for task 5000 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/91a77a0c-2bf7-4f25-b1fb-a3d3afe484d9_isrForCrosstalkSources_996992_8.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:05 proc_lsst.multi:146 [INFO] found job 17074855 in provider astro proc_lsst.multi INFO: found job 17074855 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5057 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 10697f915532af533be23f5f2f91f3ff with result from task 5057 parsl.dataflow.dflow DEBUG: Task 5057 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cc9deb70-8f6c-4faf-a6c1-f5bc8c738192_isrForCrosstalkSources_996978_60.stdout parsl.dataflow.dflow INFO: Standard error for task 5057 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/cc9deb70-8f6c-4faf-a6c1-f5bc8c738192_isrForCrosstalkSources_996978_60.stderr parsl.dataflow.dflow INFO: Task 5043 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6ee8105319f2145cd41651e0f33ebca3 with result from task 5043 parsl.dataflow.dflow DEBUG: Task 5043 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/86215e29-39c7-40d2-8dca-f24c3786f835_isrForCrosstalkSources_997008_58.stdout parsl.dataflow.dflow INFO: Standard error for task 5043 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997008/86215e29-39c7-40d2-8dca-f24c3786f835_isrForCrosstalkSources_997008_58.stderr parsl.dataflow.dflow INFO: Task 5028 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7fb38ff3c49012f3aa183022c2cb69ee with result from task 5028 parsl.dataflow.dflow DEBUG: Task 5028 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/165a8d3e-cd83-45c3-9194-fc1235c3d084_isrForCrosstalkSources_996960_15.stdout parsl.dataflow.dflow INFO: Standard error for task 5028 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/165a8d3e-cd83-45c3-9194-fc1235c3d084_isrForCrosstalkSources_996960_15.stderr parsl.dataflow.dflow INFO: Task 4980 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c38e620b085679deab07bbc912c0e4d8 with result from task 4980 parsl.dataflow.dflow DEBUG: Task 4980 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/181b69fc-6f12-4241-9afd-16bf1e81eb7f_isrForCrosstalkSources_996981_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4980 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/181b69fc-6f12-4241-9afd-16bf1e81eb7f_isrForCrosstalkSources_996981_58.stderr parsl.dataflow.dflow INFO: Task 4831 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42353360cbc4abe011161e64965ce6e3 with result from task 4831 parsl.dataflow.dflow DEBUG: Task 4831 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/684534c7-9f23-405f-9f65-73f1beaaf65c_isrForCrosstalkSources_997010_48.stdout parsl.dataflow.dflow INFO: Standard error for task 4831 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997010/684534c7-9f23-405f-9f65-73f1beaaf65c_isrForCrosstalkSources_997010_48.stderr parsl.dataflow.dflow INFO: Task 4988 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 247a11367889facf79726314095b493b with result from task 4988 parsl.dataflow.dflow DEBUG: Task 4988 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e51bcad0-bc27-40b5-a1f7-7a86317b77f8_isrForCrosstalkSources_996970_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4988 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996970/e51bcad0-bc27-40b5-a1f7-7a86317b77f8_isrForCrosstalkSources_996970_29.stderr parsl.dataflow.dflow INFO: Task 4982 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4a7a3f8574f0ac9cc168a97d1b44a19f with result from task 4982 parsl.dataflow.dflow DEBUG: Task 4982 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b76cf203-be82-446a-9b83-72132c3dda26_isrForCrosstalkSources_996997_21.stdout parsl.dataflow.dflow INFO: Standard error for task 4982 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996997/b76cf203-be82-446a-9b83-72132c3dda26_isrForCrosstalkSources_996997_21.stderr parsl.dataflow.dflow INFO: Task 4821 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5bde44d965b641b6f7c22e34ddf549c3 with result from task 4821 parsl.dataflow.dflow DEBUG: Task 4821 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a70b1ec5-526e-4309-aa69-2e4166f1e0d8_isrForCrosstalkSources_996954_29.stdout parsl.dataflow.dflow INFO: Standard error for task 4821 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/a70b1ec5-526e-4309-aa69-2e4166f1e0d8_isrForCrosstalkSources_996954_29.stderr parsl.dataflow.dflow INFO: Task 5005 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ef7d68d891e8589d49b197d79fdd6e98 with result from task 5005 parsl.dataflow.dflow DEBUG: Task 5005 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/863ecf3e-6722-4679-b4fb-bbf338c2ca53_isrForCrosstalkSources_996988_58.stdout parsl.dataflow.dflow INFO: Standard error for task 5005 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/863ecf3e-6722-4679-b4fb-bbf338c2ca53_isrForCrosstalkSources_996988_58.stderr parsl.dataflow.dflow INFO: Task 4963 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 517b41888cbe7af0f78e9749977444b5 with result from task 4963 parsl.dataflow.dflow DEBUG: Task 4963 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d8aeca74-e26d-4ef8-bdc3-9a1130890544_isrForCrosstalkSources_997017_3.stdout parsl.dataflow.dflow INFO: Standard error for task 4963 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997017/d8aeca74-e26d-4ef8-bdc3-9a1130890544_isrForCrosstalkSources_997017_3.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:05 proc_lsst.multi:146 [INFO] found job 17074856 in provider astro proc_lsst.multi INFO: found job 17074856 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4955 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 650d0fcc5508f9a06fb3b84b09d16290 with result from task 4955 parsl.dataflow.dflow DEBUG: Task 4955 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ba88985c-70a5-43c7-9478-bb9fc96d5ea4_isrForCrosstalkSources_996976_18.stdout parsl.dataflow.dflow INFO: Standard error for task 4955 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/ba88985c-70a5-43c7-9478-bb9fc96d5ea4_isrForCrosstalkSources_996976_18.stderr parsl.dataflow.dflow INFO: Task 5001 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39e22972809d3851690a19f61520f930 with result from task 5001 parsl.dataflow.dflow DEBUG: Task 5001 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/efcbf166-1103-41ad-9f4a-e229e73c708a_isrForCrosstalkSources_996983_23.stdout parsl.dataflow.dflow INFO: Standard error for task 5001 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996983/efcbf166-1103-41ad-9f4a-e229e73c708a_isrForCrosstalkSources_996983_23.stderr parsl.dataflow.dflow INFO: Task 5014 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c27f522229672361c6551b89df175bd6 with result from task 5014 parsl.dataflow.dflow DEBUG: Task 5014 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/901d8d09-51f2-49e3-a277-b3263506af2d_isrForCrosstalkSources_996971_45.stdout parsl.dataflow.dflow INFO: Standard error for task 5014 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996971/901d8d09-51f2-49e3-a277-b3263506af2d_isrForCrosstalkSources_996971_45.stderr parsl.dataflow.dflow INFO: Task 4969 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 232538f470e23fceb78921f5d406191e with result from task 4969 parsl.dataflow.dflow DEBUG: Task 4969 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/52006b67-e71c-41c2-89be-fce49a01ef3c_isrForCrosstalkSources_996984_14.stdout parsl.dataflow.dflow INFO: Standard error for task 4969 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/52006b67-e71c-41c2-89be-fce49a01ef3c_isrForCrosstalkSources_996984_14.stderr parsl.dataflow.dflow INFO: Task 5024 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 927661e3a47757a28bc18a88e693a06a with result from task 5024 parsl.dataflow.dflow DEBUG: Task 5024 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/674cbf07-0ce2-4fe0-a07c-1ee07661ff78_isrForCrosstalkSources_996989_57.stdout parsl.dataflow.dflow INFO: Standard error for task 5024 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/674cbf07-0ce2-4fe0-a07c-1ee07661ff78_isrForCrosstalkSources_996989_57.stderr parsl.dataflow.dflow INFO: Task 5020 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 23ae9e43add366a2cfe3c06ff54345d6 with result from task 5020 parsl.dataflow.dflow DEBUG: Task 5020 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/4edc9036-a140-49a4-9aae-c7c0a7bdb14b_isrForCrosstalkSources_996993_24.stdout parsl.dataflow.dflow INFO: Standard error for task 5020 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996993/4edc9036-a140-49a4-9aae-c7c0a7bdb14b_isrForCrosstalkSources_996993_24.stderr parsl.dataflow.dflow INFO: Task 5038 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry d7aaa3bde7439add7835cf86e777a143 with result from task 5038 parsl.dataflow.dflow DEBUG: Task 5038 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2c70e87d-7c03-4baa-9788-cfa6551ea170_isrForCrosstalkSources_996974_12.stdout parsl.dataflow.dflow INFO: Standard error for task 5038 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/2c70e87d-7c03-4baa-9788-cfa6551ea170_isrForCrosstalkSources_996974_12.stderr parsl.dataflow.dflow INFO: Task 5080 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7aa0d63e8599384b43f1d4ecd788e22f with result from task 5080 parsl.dataflow.dflow DEBUG: Task 5080 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/aa4d7fe6-badf-46d9-8bf4-a76a97c8c8f0_isrForCrosstalkSources_997030_40.stdout parsl.dataflow.dflow INFO: Standard error for task 5080 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997030/aa4d7fe6-badf-46d9-8bf4-a76a97c8c8f0_isrForCrosstalkSources_997030_40.stderr parsl.dataflow.dflow INFO: Task 4960 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 847ea697062e2271fb4f9d2aa9d2084f with result from task 4960 parsl.dataflow.dflow DEBUG: Task 4960 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/86f97569-30a2-422d-94ba-b75a19d86b97_isrForCrosstalkSources_996976_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4960 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996976/86f97569-30a2-422d-94ba-b75a19d86b97_isrForCrosstalkSources_996976_58.stderr parsl.dataflow.dflow INFO: Task 5076 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5cebcfdc8e10e1806ac27213f8ad4f5c with result from task 5076 parsl.dataflow.dflow DEBUG: Task 5076 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f27a10a0-8d14-428d-9c62-30e2ab06eb1c_isrForCrosstalkSources_996957_11.stdout parsl.dataflow.dflow INFO: Standard error for task 5076 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/f27a10a0-8d14-428d-9c62-30e2ab06eb1c_isrForCrosstalkSources_996957_11.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:06 proc_lsst.multi:146 [INFO] found job 17074857 in provider astro proc_lsst.multi INFO: found job 17074857 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4843 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e2d5866a7d79c4c563b19458fb260ac3 with result from task 4843 parsl.dataflow.dflow DEBUG: Task 4843 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c33fc834-4714-47b5-9962-03a8d2113be5_isrForCrosstalkSources_997006_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4843 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997006/c33fc834-4714-47b5-9962-03a8d2113be5_isrForCrosstalkSources_997006_53.stderr parsl.dataflow.dflow INFO: Task 5012 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1304159ee2b17bba34195f153c94f092 with result from task 5012 parsl.dataflow.dflow DEBUG: Task 5012 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5c147f8d-b89d-49c2-b030-8c8cce0ab03b_isrForCrosstalkSources_996978_18.stdout parsl.dataflow.dflow INFO: Standard error for task 5012 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/5c147f8d-b89d-49c2-b030-8c8cce0ab03b_isrForCrosstalkSources_996978_18.stderr parsl.dataflow.dflow INFO: Task 4966 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry ce79982783d46e5aed65604b0d4afb49 with result from task 4966 parsl.dataflow.dflow DEBUG: Task 4966 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a5cde737-adc0-402a-9552-73187a8e7bd2_isrForCrosstalkSources_997001_7.stdout parsl.dataflow.dflow INFO: Standard error for task 4966 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/a5cde737-adc0-402a-9552-73187a8e7bd2_isrForCrosstalkSources_997001_7.stderr parsl.dataflow.dflow INFO: Task 4991 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3c04e7e49f699050a116379f0d793304 with result from task 4991 parsl.dataflow.dflow DEBUG: Task 4991 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/cf3acd91-d6e8-44c9-99b4-e908e3c8d765_isrForCrosstalkSources_996966_59.stdout parsl.dataflow.dflow INFO: Standard error for task 4991 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996966/cf3acd91-d6e8-44c9-99b4-e908e3c8d765_isrForCrosstalkSources_996966_59.stderr parsl.dataflow.dflow INFO: Task 4961 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 306d4db939a5b7eaaaa5cb000825abb4 with result from task 4961 parsl.dataflow.dflow DEBUG: Task 4961 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/51915ae6-f1e1-4ac0-a64a-d6d2b7a0270e_isrForCrosstalkSources_996975_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4961 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/51915ae6-f1e1-4ac0-a64a-d6d2b7a0270e_isrForCrosstalkSources_996975_23.stderr parsl.dataflow.dflow INFO: Task 5011 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3ede90da9f680b7eab56ae6d9476e391 with result from task 5011 parsl.dataflow.dflow DEBUG: Task 5011 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/fb01ce7e-da52-4db4-8076-5644df6b9079_isrForCrosstalkSources_996974_3.stdout parsl.dataflow.dflow INFO: Standard error for task 5011 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/fb01ce7e-da52-4db4-8076-5644df6b9079_isrForCrosstalkSources_996974_3.stderr parsl.dataflow.dflow INFO: Task 3220 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39294c674e8523ae40f3eb849e56e7f9 with result from task 3220 parsl.dataflow.dflow DEBUG: Task 3220 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/05260c5a-fefd-4162-9220-65ecb2a345e6_isrForCrosstalkSources_997001_49.stdout parsl.dataflow.dflow INFO: Standard error for task 3220 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997001/05260c5a-fefd-4162-9220-65ecb2a345e6_isrForCrosstalkSources_997001_49.stderr parsl.dataflow.dflow INFO: Task 5018 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a00a5c713bb5f8dae2b67663a36345a2 with result from task 5018 parsl.dataflow.dflow DEBUG: Task 5018 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e45fd657-67b6-4f75-8c96-a0b306278297_isrForCrosstalkSources_996989_11.stdout parsl.dataflow.dflow INFO: Standard error for task 5018 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/e45fd657-67b6-4f75-8c96-a0b306278297_isrForCrosstalkSources_996989_11.stderr parsl.dataflow.dflow INFO: Task 4999 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f463e182c526cbd056c2bf6b03f2b5e9 with result from task 4999 parsl.dataflow.dflow DEBUG: Task 4999 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0568dc0b-ad31-4d4c-a3a5-8412c2ce13b9_isrForCrosstalkSources_996975_4.stdout parsl.dataflow.dflow INFO: Standard error for task 4999 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996975/0568dc0b-ad31-4d4c-a3a5-8412c2ce13b9_isrForCrosstalkSources_996975_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:08 proc_lsst.multi:146 [INFO] found job 17074859 in provider astro proc_lsst.multi INFO: found job 17074859 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5015 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4c4217c089d6ac3a77cf0ec6b33d71fa with result from task 5015 parsl.dataflow.dflow DEBUG: Task 5015 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/35b16825-8e1c-4a8b-a0e7-ed59e1bbaa07_isrForCrosstalkSources_996963_12.stdout parsl.dataflow.dflow INFO: Standard error for task 5015 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996963/35b16825-8e1c-4a8b-a0e7-ed59e1bbaa07_isrForCrosstalkSources_996963_12.stderr parsl.dataflow.dflow INFO: Task 4922 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65088e00f82c1cca7bf23c95c2b92628 with result from task 4922 parsl.dataflow.dflow DEBUG: Task 4922 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/75c56022-3198-46ef-85a3-8ce460afc4b3_isrForCrosstalkSources_996958_10.stdout parsl.dataflow.dflow INFO: Standard error for task 4922 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/75c56022-3198-46ef-85a3-8ce460afc4b3_isrForCrosstalkSources_996958_10.stderr parsl.dataflow.dflow INFO: Task 4986 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e4d2fc266cace071725063eec21a12a4 with result from task 4986 parsl.dataflow.dflow DEBUG: Task 4986 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/4ee42cb6-dacf-417d-b138-ea9bf053f1c9_isrForCrosstalkSources_997025_44.stdout parsl.dataflow.dflow INFO: Standard error for task 4986 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997025/4ee42cb6-dacf-417d-b138-ea9bf053f1c9_isrForCrosstalkSources_997025_44.stderr parsl.dataflow.dflow INFO: Task 5016 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 4616a5ada10f9b73f18733e2a69e8217 with result from task 5016 parsl.dataflow.dflow DEBUG: Task 5016 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/78fd9e6a-c649-44d4-a8ae-4a917d33d7b1_isrForCrosstalkSources_997029_42.stdout parsl.dataflow.dflow INFO: Standard error for task 5016 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/78fd9e6a-c649-44d4-a8ae-4a917d33d7b1_isrForCrosstalkSources_997029_42.stderr parsl.dataflow.dflow INFO: Task 5021 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9464f7965702cc65dab104f58cebbd08 with result from task 5021 parsl.dataflow.dflow DEBUG: Task 5021 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bf4c7109-62ca-4578-8372-d0bb2c715291_isrForCrosstalkSources_997014_51.stdout parsl.dataflow.dflow INFO: Standard error for task 5021 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997014/bf4c7109-62ca-4578-8372-d0bb2c715291_isrForCrosstalkSources_997014_51.stderr parsl.dataflow.dflow INFO: Task 5030 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9424f05c8a5c8ab360c9dd71eb681d17 with result from task 5030 parsl.dataflow.dflow DEBUG: Task 5030 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/b060026e-266e-4105-8db9-f0c509347655_isrForCrosstalkSources_997028_17.stdout parsl.dataflow.dflow INFO: Standard error for task 5030 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997028/b060026e-266e-4105-8db9-f0c509347655_isrForCrosstalkSources_997028_17.stderr parsl.dataflow.dflow INFO: Task 3094 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fa8ff7f3a7600943b4041b7e3034b50 with result from task 3094 parsl.dataflow.dflow DEBUG: Task 3094 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5b03ed22-6a15-4441-aaca-4554315f2b98_isrForCrosstalkSources_996954_7.stdout parsl.dataflow.dflow INFO: Standard error for task 3094 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996954/5b03ed22-6a15-4441-aaca-4554315f2b98_isrForCrosstalkSources_996954_7.stderr parsl.dataflow.dflow INFO: Task 5041 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 6b16f24a766bd4ae6affa454efe31de8 with result from task 5041 parsl.dataflow.dflow DEBUG: Task 5041 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/302fd97d-869f-4ffa-9a39-b2c43a917efe_isrForCrosstalkSources_997029_48.stdout parsl.dataflow.dflow INFO: Standard error for task 5041 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/302fd97d-869f-4ffa-9a39-b2c43a917efe_isrForCrosstalkSources_997029_48.stderr parsl.dataflow.dflow INFO: Task 5022 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 48b31a8771ea57fc25a62b0d97594a26 with result from task 5022 parsl.dataflow.dflow DEBUG: Task 5022 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f2e2bd15-3ed6-46ca-b96c-4c6365c247f1_isrForCrosstalkSources_996960_62.stdout parsl.dataflow.dflow INFO: Standard error for task 5022 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996960/f2e2bd15-3ed6-46ca-b96c-4c6365c247f1_isrForCrosstalkSources_996960_62.stderr parsl.dataflow.dflow INFO: Task 4970 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 45b7d4abdbdd4268ed0a6272a3b9c874 with result from task 4970 parsl.dataflow.dflow DEBUG: Task 4970 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ea5e6aaf-a722-41db-bc18-9637d2286e50_isrForCrosstalkSources_996981_53.stdout parsl.dataflow.dflow INFO: Standard error for task 4970 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/ea5e6aaf-a722-41db-bc18-9637d2286e50_isrForCrosstalkSources_996981_53.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:09 proc_lsst.multi:146 [INFO] found job 17074861 in provider astro proc_lsst.multi INFO: found job 17074861 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 3167 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2639005c90196ace37b8d59c99c096f5 with result from task 3167 parsl.dataflow.dflow DEBUG: Task 3167 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/88366b76-4063-4fe2-9b3a-e6a45b5d50a3_isrForCrosstalkSources_996959_10.stdout parsl.dataflow.dflow INFO: Standard error for task 3167 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996959/88366b76-4063-4fe2-9b3a-e6a45b5d50a3_isrForCrosstalkSources_996959_10.stderr parsl.dataflow.dflow INFO: Task 4962 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8895cd78d040f4924441a3013d68a9a9 with result from task 4962 parsl.dataflow.dflow DEBUG: Task 4962 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e37aa698-d90c-44ee-9aa5-6498263e5592_isrForCrosstalkSources_997023_42.stdout parsl.dataflow.dflow INFO: Standard error for task 4962 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997023/e37aa698-d90c-44ee-9aa5-6498263e5592_isrForCrosstalkSources_997023_42.stderr parsl.dataflow.dflow INFO: Task 4989 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 599372c7184f012c0d2acb5fe7c3e49c with result from task 4989 parsl.dataflow.dflow DEBUG: Task 4989 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/3d4bbcdd-e477-4f02-a458-01fa1bdf05bd_isrForCrosstalkSources_997012_16.stdout parsl.dataflow.dflow INFO: Standard error for task 4989 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/3d4bbcdd-e477-4f02-a458-01fa1bdf05bd_isrForCrosstalkSources_997012_16.stderr parsl.dataflow.dflow INFO: Task 5049 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 39c381c67eae3ad8b459253e514083fa with result from task 5049 parsl.dataflow.dflow DEBUG: Task 5049 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/38b545a0-1918-44ef-bd54-f9d38761d778_isrForCrosstalkSources_996957_6.stdout parsl.dataflow.dflow INFO: Standard error for task 5049 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996957/38b545a0-1918-44ef-bd54-f9d38761d778_isrForCrosstalkSources_996957_6.stderr parsl.dataflow.dflow INFO: Task 4994 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 27fc6cb4f5bd1cfd30e3f2d233ec79e0 with result from task 4994 parsl.dataflow.dflow DEBUG: Task 4994 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ac2014e5-1011-4a5c-ad3b-34fc70dcaade_isrForCrosstalkSources_997013_2.stdout parsl.dataflow.dflow INFO: Standard error for task 4994 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997013/ac2014e5-1011-4a5c-ad3b-34fc70dcaade_isrForCrosstalkSources_997013_2.stderr parsl.dataflow.dflow INFO: Task 5040 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 459b5c16d3af49cb2247e4fef93d56e8 with result from task 5040 parsl.dataflow.dflow DEBUG: Task 5040 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/9ac9997d-80f1-4b50-b336-7f711dab242e_isrForCrosstalkSources_997011_57.stdout parsl.dataflow.dflow INFO: Standard error for task 5040 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997011/9ac9997d-80f1-4b50-b336-7f711dab242e_isrForCrosstalkSources_997011_57.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:10 proc_lsst.multi:146 [INFO] found job 17074863 in provider astro proc_lsst.multi INFO: found job 17074863 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4968 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bebbaf2ec1da4a468deaaa610792102c with result from task 4968 parsl.dataflow.dflow DEBUG: Task 4968 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4ba671ab-8fe2-4d2c-b6cf-2d25faf193e5_isrForCrosstalkSources_996999_28.stdout parsl.dataflow.dflow INFO: Standard error for task 4968 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996999/4ba671ab-8fe2-4d2c-b6cf-2d25faf193e5_isrForCrosstalkSources_996999_28.stderr parsl.dataflow.dflow INFO: Task 5010 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 58377774491fe98dbcd0cb5c1d1b73cf with result from task 5010 parsl.dataflow.dflow DEBUG: Task 5010 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d21474e5-81e0-4d08-9de3-d615ed5c15df_isrForCrosstalkSources_996992_57.stdout parsl.dataflow.dflow INFO: Standard error for task 5010 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996992/d21474e5-81e0-4d08-9de3-d615ed5c15df_isrForCrosstalkSources_996992_57.stderr parsl.dataflow.dflow INFO: Task 4975 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7c70f0641c012fd4f413b7a14f820c00 with result from task 4975 parsl.dataflow.dflow DEBUG: Task 4975 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d5c4cfba-dc0b-476f-81aa-f05b32ecc1d8_isrForCrosstalkSources_996973_41.stdout parsl.dataflow.dflow INFO: Standard error for task 4975 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996973/d5c4cfba-dc0b-476f-81aa-f05b32ecc1d8_isrForCrosstalkSources_996973_41.stderr parsl.dataflow.dflow INFO: Task 4967 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e6e920deee3c4800310858be4585fde8 with result from task 4967 parsl.dataflow.dflow DEBUG: Task 4967 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/ae8f94e1-4aee-45d8-a21b-8ceebf15ce6a_isrForCrosstalkSources_996996_52.stdout parsl.dataflow.dflow INFO: Standard error for task 4967 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/ae8f94e1-4aee-45d8-a21b-8ceebf15ce6a_isrForCrosstalkSources_996996_52.stderr parsl.dataflow.dflow INFO: Task 5069 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 2f1026cc7661d2e671a36be9dd20c968 with result from task 5069 parsl.dataflow.dflow DEBUG: Task 5069 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/cf5eb713-d074-45b7-a445-86cbd18cd1b2_isrForCrosstalkSources_996979_61.stdout parsl.dataflow.dflow INFO: Standard error for task 5069 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/cf5eb713-d074-45b7-a445-86cbd18cd1b2_isrForCrosstalkSources_996979_61.stderr parsl.dataflow.dflow INFO: Task 5044 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry afa25d607f3d564c643f2eaf496bce65 with result from task 5044 parsl.dataflow.dflow DEBUG: Task 5044 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/154377d1-4122-4b06-bb7d-5f58113ea6b2_isrForCrosstalkSources_997012_42.stdout parsl.dataflow.dflow INFO: Standard error for task 5044 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997012/154377d1-4122-4b06-bb7d-5f58113ea6b2_isrForCrosstalkSources_997012_42.stderr parsl.dataflow.dflow INFO: Task 4964 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 65fd4cb91296c61c0f07a7c7cd6360c0 with result from task 4964 parsl.dataflow.dflow DEBUG: Task 4964 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/31eb6657-1929-4afc-b8ce-ddc0652ea54f_isrForCrosstalkSources_996996_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4964 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996996/31eb6657-1929-4afc-b8ce-ddc0652ea54f_isrForCrosstalkSources_996996_50.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:10 proc_lsst.multi:146 [INFO] found job 17074865 in provider astro proc_lsst.multi INFO: found job 17074865 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5055 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fa2bdc36d3e15ab114e38a67e01014e with result from task 5055 parsl.dataflow.dflow DEBUG: Task 5055 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/bd1a4a4f-7865-42f3-9e56-4c0c2566e752_isrForCrosstalkSources_996978_33.stdout parsl.dataflow.dflow INFO: Standard error for task 5055 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996978/bd1a4a4f-7865-42f3-9e56-4c0c2566e752_isrForCrosstalkSources_996978_33.stderr parsl.dataflow.dflow INFO: Task 5082 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry f495816daf3da4b7b98ddd94a860af5d with result from task 5082 parsl.dataflow.dflow DEBUG: Task 5082 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b1cf1aa9-0a59-4e75-82e6-375f940d0c19_isrForCrosstalkSources_996962_34.stdout parsl.dataflow.dflow INFO: Standard error for task 5082 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996962/b1cf1aa9-0a59-4e75-82e6-375f940d0c19_isrForCrosstalkSources_996962_34.stderr parsl.dataflow.dflow INFO: Task 5003 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5fed8b3c74bbe9c5e612f2ff3cc4cdff with result from task 5003 parsl.dataflow.dflow DEBUG: Task 5003 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7d834461-981c-4490-801e-9613a8b6d997_isrForCrosstalkSources_997026_36.stdout parsl.dataflow.dflow INFO: Standard error for task 5003 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997026/7d834461-981c-4490-801e-9613a8b6d997_isrForCrosstalkSources_997026_36.stderr parsl.dataflow.dflow INFO: Task 4908 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 194d2f6fd643482fd6e02f30c6dc2aac with result from task 4908 parsl.dataflow.dflow DEBUG: Task 4908 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/90864c8b-24d3-4e0f-856a-9f2ef66e8b38_isrForCrosstalkSources_997015_23.stdout parsl.dataflow.dflow INFO: Standard error for task 4908 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997015/90864c8b-24d3-4e0f-856a-9f2ef66e8b38_isrForCrosstalkSources_997015_23.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:11 proc_lsst.multi:146 [INFO] found job 17074869 in provider astro proc_lsst.multi INFO: found job 17074869 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5054 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry c07e86cf1da485d03293d5d65378a606 with result from task 5054 parsl.dataflow.dflow DEBUG: Task 5054 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/89dc1ece-171b-4410-bc47-de10adae5e6d_isrForCrosstalkSources_997009_29.stdout parsl.dataflow.dflow INFO: Standard error for task 5054 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997009/89dc1ece-171b-4410-bc47-de10adae5e6d_isrForCrosstalkSources_997009_29.stderr parsl.dataflow.dflow INFO: Task 5025 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3838736245e04cc48a431314a65102b9 with result from task 5025 parsl.dataflow.dflow DEBUG: Task 5025 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/4d9d889c-c7e2-4f0b-9bc8-c62fb58f062d_isrForCrosstalkSources_997022_43.stdout parsl.dataflow.dflow INFO: Standard error for task 5025 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997022/4d9d889c-c7e2-4f0b-9bc8-c62fb58f062d_isrForCrosstalkSources_997022_43.stderr parsl.dataflow.dflow INFO: Task 5045 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry b8c247ae007478e8485fb33da9df5524 with result from task 5045 parsl.dataflow.dflow DEBUG: Task 5045 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/8fa379e6-2175-4c78-bcc3-e197b2069b0c_isrForCrosstalkSources_996982_36.stdout parsl.dataflow.dflow INFO: Standard error for task 5045 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996982/8fa379e6-2175-4c78-bcc3-e197b2069b0c_isrForCrosstalkSources_996982_36.stderr parsl.dataflow.dflow INFO: Task 5075 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 20aeb1f0956f8f3c4cc5250daca3506a with result from task 5075 parsl.dataflow.dflow DEBUG: Task 5075 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/583eac7d-13cd-4730-8b12-4bfa2eb5c620_isrForCrosstalkSources_997029_7.stdout parsl.dataflow.dflow INFO: Standard error for task 5075 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/583eac7d-13cd-4730-8b12-4bfa2eb5c620_isrForCrosstalkSources_997029_7.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:11 proc_lsst.multi:146 [INFO] found job 17074870 in provider astro proc_lsst.multi INFO: found job 17074870 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4974 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 5cc240efbc258b4e2ac90753ed8c7790 with result from task 4974 parsl.dataflow.dflow DEBUG: Task 4974 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/be5f8e99-8371-4ee5-9e7c-a4d64a98062b_isrForCrosstalkSources_996981_61.stdout parsl.dataflow.dflow INFO: Standard error for task 4974 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996981/be5f8e99-8371-4ee5-9e7c-a4d64a98062b_isrForCrosstalkSources_996981_61.stderr parsl.dataflow.dflow INFO: Task 5033 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 81b4590a204a6e073e69668f57bab6c4 with result from task 5033 parsl.dataflow.dflow DEBUG: Task 5033 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/f5f5f7d8-e523-4883-8ba4-f9f73172ba60_isrForCrosstalkSources_996964_28.stdout parsl.dataflow.dflow INFO: Standard error for task 5033 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996964/f5f5f7d8-e523-4883-8ba4-f9f73172ba60_isrForCrosstalkSources_996964_28.stderr parsl.dataflow.dflow INFO: Task 4979 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry bfedad5d6ef8d0a28dc2f6b02764b520 with result from task 4979 parsl.dataflow.dflow DEBUG: Task 4979 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/77947747-57b0-4f35-b7c7-feb0677c03c4_isrForCrosstalkSources_996958_54.stdout parsl.dataflow.dflow INFO: Standard error for task 4979 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996958/77947747-57b0-4f35-b7c7-feb0677c03c4_isrForCrosstalkSources_996958_54.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:12 proc_lsst.multi:146 [INFO] found job 17074871 in provider astro proc_lsst.multi INFO: found job 17074871 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5078 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 7742c0ed0de3cb5aa72cf732a37cb441 with result from task 5078 parsl.dataflow.dflow DEBUG: Task 5078 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/943c7e17-48e1-49fa-ac2d-f50859822fe0_isrForCrosstalkSources_996984_52.stdout parsl.dataflow.dflow INFO: Standard error for task 5078 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996984/943c7e17-48e1-49fa-ac2d-f50859822fe0_isrForCrosstalkSources_996984_52.stderr parsl.dataflow.dflow INFO: Task 3095 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 37fdd7bc349da7e5549d9d4591b6404b with result from task 3095 parsl.dataflow.dflow DEBUG: Task 3095 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/285b4a39-a344-4ae3-a649-6b93608cf698_isrForCrosstalkSources_996974_41.stdout parsl.dataflow.dflow INFO: Standard error for task 3095 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996974/285b4a39-a344-4ae3-a649-6b93608cf698_isrForCrosstalkSources_996974_41.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:12 proc_lsst.multi:146 [INFO] found job 17074872 in provider astro proc_lsst.multi INFO: found job 17074872 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 4972 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry feb31a8ed15fce53f62bb71da0465e2b with result from task 4972 parsl.dataflow.dflow DEBUG: Task 4972 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/423f2ddc-9184-4841-bfd3-6a4abfa2bb63_isrForCrosstalkSources_997029_58.stdout parsl.dataflow.dflow INFO: Standard error for task 4972 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997029/423f2ddc-9184-4841-bfd3-6a4abfa2bb63_isrForCrosstalkSources_997029_58.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:13 proc_lsst.multi:146 [INFO] found job 17074873 in provider astro proc_lsst.multi INFO: found job 17074873 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5050 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry e88010e841536c531ba70631bc0794f8 with result from task 5050 parsl.dataflow.dflow DEBUG: Task 5050 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/086bf235-659c-4413-b2e9-41e3c31e5e67_isrForCrosstalkSources_997024_51.stdout parsl.dataflow.dflow INFO: Standard error for task 5050 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997024/086bf235-659c-4413-b2e9-41e3c31e5e67_isrForCrosstalkSources_997024_51.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:13 proc_lsst.multi:146 [INFO] found job 17074876 in provider astro proc_lsst.multi INFO: found job 17074876 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5047 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cc9d4e069c74b3e1a163a22aa88dc766 with result from task 5047 parsl.dataflow.dflow DEBUG: Task 5047 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e0a51ac5-66c9-4740-a704-b7ce8560c4bd_isrForCrosstalkSources_996956_14.stdout parsl.dataflow.dflow INFO: Standard error for task 5047 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996956/e0a51ac5-66c9-4740-a704-b7ce8560c4bd_isrForCrosstalkSources_996956_14.stderr parsl.dataflow.dflow INFO: Task 5073 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 718be5989251fcd2f59befb7fffc562f with result from task 5073 parsl.dataflow.dflow DEBUG: Task 5073 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/380aaf61-dbcc-46c9-b747-6c175629c95d_isrForCrosstalkSources_996987_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5073 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/380aaf61-dbcc-46c9-b747-6c175629c95d_isrForCrosstalkSources_996987_46.stderr parsl.dataflow.dflow INFO: Task 5071 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 9db79731ad6df9636a15641f254ffffb with result from task 5071 parsl.dataflow.dflow DEBUG: Task 5071 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/77a9606f-7d0e-4df8-aa77-0c467d150fc3_isrForCrosstalkSources_996989_46.stdout parsl.dataflow.dflow INFO: Standard error for task 5071 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996989/77a9606f-7d0e-4df8-aa77-0c467d150fc3_isrForCrosstalkSources_996989_46.stderr parsl.dataflow.dflow INFO: Task 5079 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry a5711657fa75eacdf9ea4f1ed4fd2b6e with result from task 5079 parsl.dataflow.dflow DEBUG: Task 5079 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/9abf096f-04e3-4b0f-af70-59c95a22a73e_isrForCrosstalkSources_996961_4.stdout parsl.dataflow.dflow INFO: Standard error for task 5079 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996961/9abf096f-04e3-4b0f-af70-59c95a22a73e_isrForCrosstalkSources_996961_4.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:14 proc_lsst.multi:146 [INFO] found job 17074877 in provider astro proc_lsst.multi INFO: found job 17074877 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.dataflow.dflow INFO: Task 5068 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 3a13c82a22452448481c3cbad69219a0 with result from task 5068 parsl.dataflow.dflow DEBUG: Task 5068 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c953cf3b-3247-4099-8ba6-a944c9077aa0_isrForCrosstalkSources_996965_15.stdout parsl.dataflow.dflow INFO: Standard error for task 5068 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996965/c953cf3b-3247-4099-8ba6-a944c9077aa0_isrForCrosstalkSources_996965_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:14 proc_lsst.multi:146 [INFO] found job 17074879 in provider astro proc_lsst.multi INFO: found job 17074879 in provider astro parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074833,17074834,17074835,17074836,17074837,17074838,17074839,17074841,17074842,17074843,17074850,17074851,17074852,17074853,17074854,17074855,17074856,17074857,17074859,17074861,17074863,17074865,17074869,17074870,17074871,17074872,17074873,17074876,17074877,17074879' parsl.providers.slurm.slurm DEBUG: squeue returned 17074879 R 17074877 R 17074873 R 17074876 R 17074872 R 17074870 R 17074871 R 17074869 R 17074863 R 17074865 R 17074861 R 17074857 R 17074859 R 17074856 R 17074854 R 17074855 R 17074853 R 17074851 R 17074852 R 17074850 R 17074843 R 17074842 R 17074841 R 17074837 R 17074838 R 17074839 R 17074836 R 17074833 R 17074834 R 17074835 R parsl.providers.slurm.slurm DEBUG: Updating job 17074879 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074877 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074873 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074876 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074872 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074870 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074871 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074869 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074863 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074865 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074861 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074857 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074859 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074856 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074854 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074855 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074853 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074851 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074852 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074850 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074843 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074842 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074841 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074837 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074838 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074839 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074836 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074833 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074834 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074835 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:14 proc_lsst.multi:146 [INFO] found job 17074880 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074880 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:15 proc_lsst.multi:146 [INFO] found job 17074881 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074881 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 3067 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 003497dc6ecb95758657ef41cf23c216 with result from task 3067 parsl.dataflow.dflow DEBUG: Task 3067 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e3c2a0bd-c124-40e8-b6c6-536607cd459f_isrForCrosstalkSources_997016_17.stdout parsl.dataflow.dflow INFO: Standard error for task 3067 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/997016/e3c2a0bd-c124-40e8-b6c6-536607cd459f_isrForCrosstalkSources_997016_17.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:15 proc_lsst.multi:146 [INFO] found job 17074887 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074887 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 5023 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 42118a2180d4fd52ecc68d3d86467fee with result from task 5023 parsl.dataflow.dflow DEBUG: Task 5023 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 5023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/496a8a03-dc7e-47fe-841b-98f4dad63e8b_isrForCrosstalkSources_996988_56.stdout parsl.dataflow.dflow INFO: Standard error for task 5023 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996988/496a8a03-dc7e-47fe-841b-98f4dad63e8b_isrForCrosstalkSources_996988_56.stderr parsl.dataflow.dflow INFO: Task 4998 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 8d307bab97d591a5419465fcd36efc6e with result from task 4998 parsl.dataflow.dflow DEBUG: Task 4998 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f8d2771a-13dd-4b24-918d-655c115124ad_isrForCrosstalkSources_996987_15.stdout parsl.dataflow.dflow INFO: Standard error for task 4998 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996987/f8d2771a-13dd-4b24-918d-655c115124ad_isrForCrosstalkSources_996987_15.stderr parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:16 proc_lsst.multi:146 [INFO] found job 17074888 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074888 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.dataflow.dflow INFO: Task 4997 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry 1fc970260159f6af5eb255fd48a69641 with result from task 4997 parsl.dataflow.dflow DEBUG: Task 4997 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 4997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/dfc93ba7-58cc-493e-8139-8ffbe08133de_isrForCrosstalkSources_996955_50.stdout parsl.dataflow.dflow INFO: Standard error for task 4997 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996955/dfc93ba7-58cc-493e-8139-8ffbe08133de_isrForCrosstalkSources_996955_50.stderr parsl.dataflow.dflow INFO: Task 3092 completed (launched -> exec_done) parsl.dataflow.memoization DEBUG: Storing app cache entry cb1a2069d81441632688400a9851e131 with result from task 3092 parsl.dataflow.dflow INFO: DFK cleanup initiated parsl.dataflow.dflow INFO: Summary of tasks in DFK: parsl.dataflow.dflow INFO: Tasks in state States.unsched: 0 parsl.dataflow.dflow INFO: Tasks in state States.pending: 0 parsl.dataflow.dflow INFO: Tasks in state States.running: 0 parsl.dataflow.dflow INFO: Tasks in state States.exec_done: 5084 parsl.dataflow.dflow INFO: Tasks in state States.failed: 0 parsl.dataflow.dflow INFO: Tasks in state States.dep_fail: 0 parsl.dataflow.dflow INFO: Tasks in state States.launched: 0 parsl.dataflow.dflow INFO: Tasks in state States.fail_retryable: 0 parsl.dataflow.dflow INFO: Tasks in state States.memo_done: 0 parsl.dataflow.dflow INFO: Tasks in state States.joining: 0 parsl.dataflow.dflow INFO: Tasks in state States.running_ended: 0 parsl.dataflow.dflow INFO: End of summary parsl.dataflow.dflow DEBUG: Task 3092 checkpointed parsl.dataflow.dflow INFO: Done checkpointing 1 tasks parsl.dataflow.dflow INFO: Standard output for task 3092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1ccfd1b0-f028-482b-b5f1-2e24563e8c51_isrForCrosstalkSources_996979_48.stdout parsl.dataflow.dflow INFO: Standard error for task 3092 available at /mmfs1/home/stevengs/dirac/DEEP/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/submit/DEEP/20210513/A1c/science#step0/20240315T134820Z/logs/isrForCrosstalkSources/996979/1ccfd1b0-f028-482b-b5f1-2e24563e8c51_isrForCrosstalkSources_996979_48.stderr parsl.dataflow.dflow DEBUG: No tasks checkpointed in this pass. parsl.dataflow.dflow INFO: Closing job status poller parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:16 proc_lsst.multi:146 [INFO] found job 17074889 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074889 in provider escience_rtx6k parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074880,17074881,17074887,17074888,17074889' parsl.providers.slurm.slurm DEBUG: squeue returned 17074889 R 17074888 R 17074887 R 17074880 R 17074881 R parsl.providers.slurm.slurm DEBUG: Updating job 17074889 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074888 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074887 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074880 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074881 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:17 proc_lsst.multi:146 [INFO] found job 17074894 in provider escience_a40 proc_lsst.multi INFO: found job 17074894 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:17 proc_lsst.multi:146 [INFO] found job 17074895 in provider escience_a40 proc_lsst.multi INFO: found job 17074895 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:18 proc_lsst.multi:146 [INFO] found job 17074897 in provider escience_a40 proc_lsst.multi INFO: found job 17074897 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:18 proc_lsst.multi:146 [INFO] found job 17074898 in provider escience_a40 proc_lsst.multi INFO: found job 17074898 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:19 proc_lsst.multi:146 [INFO] found job 17074932 in provider escience_a40 proc_lsst.multi INFO: found job 17074932 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:19 proc_lsst.multi:146 [INFO] found job 17074933 in provider escience_a40 proc_lsst.multi INFO: found job 17074933 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:20 proc_lsst.multi:146 [INFO] found job 17074934 in provider escience_a40 proc_lsst.multi INFO: found job 17074934 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:20 proc_lsst.multi:146 [INFO] found job 17074935 in provider escience_a40 proc_lsst.multi INFO: found job 17074935 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:21 proc_lsst.multi:146 [INFO] found job 17074936 in provider escience_a40 proc_lsst.multi INFO: found job 17074936 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:21 proc_lsst.multi:146 [INFO] found job 17074937 in provider escience_a40 proc_lsst.multi INFO: found job 17074937 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:21 proc_lsst.multi:146 [INFO] found job 17074938 in provider escience_a40 proc_lsst.multi INFO: found job 17074938 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:22 proc_lsst.multi:146 [INFO] found job 17074939 in provider escience_a40 proc_lsst.multi INFO: found job 17074939 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:23 proc_lsst.multi:146 [INFO] found job 17074940 in provider escience_a40 proc_lsst.multi INFO: found job 17074940 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:23 proc_lsst.multi:146 [INFO] found job 17074942 in provider escience_a40 proc_lsst.multi INFO: found job 17074942 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:23 proc_lsst.multi:146 [INFO] found job 17074974 in provider escience_a40 proc_lsst.multi INFO: found job 17074974 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:24 proc_lsst.multi:146 [INFO] found job 17075003 in provider escience_a40 proc_lsst.multi INFO: found job 17075003 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:25 proc_lsst.multi:146 [INFO] found job 17075018 in provider escience_a40 proc_lsst.multi INFO: found job 17075018 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:25 proc_lsst.multi:146 [INFO] found job 17075019 in provider escience_a40 proc_lsst.multi INFO: found job 17075019 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:25 proc_lsst.multi:146 [INFO] found job 17075020 in provider escience_a40 proc_lsst.multi INFO: found job 17075020 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:26 proc_lsst.multi:146 [INFO] found job 17075021 in provider escience_a40 proc_lsst.multi INFO: found job 17075021 in provider escience_a40 parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17074894,17074895,17074897,17074898,17074932,17074933,17074934,17074935,17074936,17074937,17074938,17074939,17074940,17074942,17074974,17075003,17075018,17075019,17075020,17075021' parsl.providers.slurm.slurm DEBUG: squeue returned 17075018 R 17075019 R 17075020 R 17075021 R 17074974 R 17075003 R 17074942 R 17074939 R 17074940 R 17074938 R 17074934 R 17074935 R 17074936 R 17074937 R 17074932 R 17074933 R 17074898 R 17074895 R 17074897 R 17074894 R parsl.providers.slurm.slurm DEBUG: Updating job 17075018 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075019 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075020 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075021 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074974 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075003 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074942 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074939 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074940 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074938 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074934 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074935 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074936 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074937 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074932 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074933 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074898 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074895 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074897 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17074894 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:26 proc_lsst.multi:146 [INFO] found job 17075022 in provider checkpoint proc_lsst.multi INFO: found job 17075022 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:27 proc_lsst.multi:146 [INFO] found job 17075030 in provider checkpoint proc_lsst.multi INFO: found job 17075030 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:27 proc_lsst.multi:146 [INFO] found job 17075031 in provider checkpoint proc_lsst.multi INFO: found job 17075031 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:28 proc_lsst.multi:146 [INFO] found job 17075032 in provider checkpoint proc_lsst.multi INFO: found job 17075032 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:28 proc_lsst.multi:146 [INFO] found job 17075033 in provider checkpoint proc_lsst.multi INFO: found job 17075033 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075069 PD 17075072 PD 17075073 PD 17075074 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:29 proc_lsst.multi:146 [INFO] found job 17075034 in provider checkpoint proc_lsst.multi INFO: found job 17075034 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:31 proc_lsst.multi:146 [INFO] found job 17075035 in provider checkpoint proc_lsst.multi INFO: found job 17075035 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:31 proc_lsst.multi:146 [INFO] found job 17075038 in provider checkpoint proc_lsst.multi INFO: found job 17075038 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:32 proc_lsst.multi:146 [INFO] found job 17075041 in provider checkpoint proc_lsst.multi INFO: found job 17075041 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:32 proc_lsst.multi:146 [INFO] found job 17075042 in provider checkpoint proc_lsst.multi INFO: found job 17075042 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:33 proc_lsst.multi:146 [INFO] found job 17075046 in provider checkpoint proc_lsst.multi INFO: found job 17075046 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:33 proc_lsst.multi:146 [INFO] found job 17075047 in provider checkpoint proc_lsst.multi INFO: found job 17075047 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:34 proc_lsst.multi:146 [INFO] found job 17075048 in provider checkpoint proc_lsst.multi INFO: found job 17075048 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075060 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:34 proc_lsst.multi:146 [INFO] found job 17075049 in provider checkpoint proc_lsst.multi INFO: found job 17075049 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:34 proc_lsst.multi:146 [INFO] found job 17075050 in provider checkpoint proc_lsst.multi INFO: found job 17075050 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:35 proc_lsst.multi:146 [INFO] found job 17075052 in provider checkpoint proc_lsst.multi INFO: found job 17075052 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:35 proc_lsst.multi:146 [INFO] found job 17075053 in provider checkpoint proc_lsst.multi INFO: found job 17075053 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:36 proc_lsst.multi:146 [INFO] found job 17075054 in provider checkpoint proc_lsst.multi INFO: found job 17075054 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:36 proc_lsst.multi:146 [INFO] found job 17075055 in provider checkpoint proc_lsst.multi INFO: found job 17075055 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:37 proc_lsst.multi:146 [INFO] found job 17075056 in provider checkpoint proc_lsst.multi INFO: found job 17075056 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:37 proc_lsst.multi:146 [INFO] found job 17075057 in provider checkpoint proc_lsst.multi INFO: found job 17075057 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:38 proc_lsst.multi:146 [INFO] found job 17075058 in provider checkpoint proc_lsst.multi INFO: found job 17075058 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:38 proc_lsst.multi:146 [INFO] found job 17075059 in provider checkpoint proc_lsst.multi INFO: found job 17075059 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:39 proc_lsst.multi:146 [INFO] found job 17075060 in provider checkpoint proc_lsst.multi INFO: found job 17075060 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:39 proc_lsst.multi:146 [INFO] found job 17075061 in provider checkpoint proc_lsst.multi INFO: found job 17075061 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:40 proc_lsst.multi:146 [INFO] found job 17075062 in provider checkpoint proc_lsst.multi INFO: found job 17075062 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:40 proc_lsst.multi:146 [INFO] found job 17075063 in provider checkpoint proc_lsst.multi INFO: found job 17075063 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:41 proc_lsst.multi:146 [INFO] found job 17075064 in provider checkpoint proc_lsst.multi INFO: found job 17075064 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:41 proc_lsst.multi:146 [INFO] found job 17075065 in provider checkpoint proc_lsst.multi INFO: found job 17075065 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:42 proc_lsst.multi:146 [INFO] found job 17075066 in provider checkpoint proc_lsst.multi INFO: found job 17075066 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:42 proc_lsst.multi:146 [INFO] found job 17075068 in provider checkpoint proc_lsst.multi INFO: found job 17075068 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:43 proc_lsst.multi:146 [INFO] found job 17075069 in provider checkpoint proc_lsst.multi INFO: found job 17075069 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:43 proc_lsst.multi:146 [INFO] found job 17075072 in provider checkpoint proc_lsst.multi INFO: found job 17075072 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:44 proc_lsst.multi:146 [INFO] found job 17075073 in provider checkpoint proc_lsst.multi INFO: found job 17075073 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:44 proc_lsst.multi:146 [INFO] found job 17075074 in provider checkpoint proc_lsst.multi INFO: found job 17075074 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:45 proc_lsst.multi:146 [INFO] found job 17075075 in provider checkpoint proc_lsst.multi INFO: found job 17075075 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:45 proc_lsst.multi:146 [INFO] found job 17075076 in provider checkpoint proc_lsst.multi INFO: found job 17075076 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:46 proc_lsst.multi:146 [INFO] found job 17075078 in provider checkpoint proc_lsst.multi INFO: found job 17075078 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:47 proc_lsst.multi:146 [INFO] found job 17075080 in provider checkpoint proc_lsst.multi INFO: found job 17075080 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:47 proc_lsst.multi:146 [INFO] found job 17075081 in provider checkpoint proc_lsst.multi INFO: found job 17075081 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:48 proc_lsst.multi:146 [INFO] found job 17075082 in provider checkpoint proc_lsst.multi INFO: found job 17075082 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:48 proc_lsst.multi:146 [INFO] found job 17075083 in provider checkpoint proc_lsst.multi INFO: found job 17075083 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:49 proc_lsst.multi:146 [INFO] found job 17075085 in provider checkpoint proc_lsst.multi INFO: found job 17075085 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:49 proc_lsst.multi:146 [INFO] found job 17075086 in provider checkpoint proc_lsst.multi INFO: found job 17075086 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:50 proc_lsst.multi:146 [INFO] found job 17075088 in provider checkpoint proc_lsst.multi INFO: found job 17075088 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:50 proc_lsst.multi:146 [INFO] found job 17075090 in provider checkpoint proc_lsst.multi INFO: found job 17075090 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:51 proc_lsst.multi:146 [INFO] found job 17075091 in provider checkpoint proc_lsst.multi INFO: found job 17075091 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:51 proc_lsst.multi:146 [INFO] found job 17075092 in provider checkpoint proc_lsst.multi INFO: found job 17075092 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:52 proc_lsst.multi:146 [INFO] found job 17075093 in provider checkpoint proc_lsst.multi INFO: found job 17075093 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:52 proc_lsst.multi:146 [INFO] found job 17075094 in provider checkpoint proc_lsst.multi INFO: found job 17075094 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:53 proc_lsst.multi:146 [INFO] found job 17075095 in provider checkpoint proc_lsst.multi INFO: found job 17075095 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:53 proc_lsst.multi:146 [INFO] found job 17075096 in provider checkpoint proc_lsst.multi INFO: found job 17075096 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:54 proc_lsst.multi:146 [INFO] found job 17075097 in provider checkpoint proc_lsst.multi INFO: found job 17075097 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:54 proc_lsst.multi:146 [INFO] found job 17075098 in provider checkpoint proc_lsst.multi INFO: found job 17075098 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:55 proc_lsst.multi:146 [INFO] found job 17075099 in provider checkpoint proc_lsst.multi INFO: found job 17075099 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:55 proc_lsst.multi:146 [INFO] found job 17075100 in provider checkpoint proc_lsst.multi INFO: found job 17075100 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:56 proc_lsst.multi:146 [INFO] found job 17075101 in provider checkpoint proc_lsst.multi INFO: found job 17075101 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:56 proc_lsst.multi:146 [INFO] found job 17075102 in provider checkpoint proc_lsst.multi INFO: found job 17075102 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:57 proc_lsst.multi:146 [INFO] found job 17075103 in provider checkpoint proc_lsst.multi INFO: found job 17075103 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:57 proc_lsst.multi:146 [INFO] found job 17075109 in provider checkpoint proc_lsst.multi INFO: found job 17075109 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:58 proc_lsst.multi:146 [INFO] found job 17075110 in provider checkpoint proc_lsst.multi INFO: found job 17075110 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:58 proc_lsst.multi:146 [INFO] found job 17075111 in provider checkpoint proc_lsst.multi INFO: found job 17075111 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:59 proc_lsst.multi:146 [INFO] found job 17075112 in provider checkpoint proc_lsst.multi INFO: found job 17075112 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:01:59 proc_lsst.multi:146 [INFO] found job 17075113 in provider checkpoint proc_lsst.multi INFO: found job 17075113 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:00 proc_lsst.multi:146 [INFO] found job 17075114 in provider checkpoint proc_lsst.multi INFO: found job 17075114 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075140 PD 17075139 PD 17075138 PD 17075137 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:00 proc_lsst.multi:146 [INFO] found job 17075115 in provider checkpoint proc_lsst.multi INFO: found job 17075115 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:02 proc_lsst.multi:146 [INFO] found job 17075116 in provider checkpoint proc_lsst.multi INFO: found job 17075116 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:02 proc_lsst.multi:146 [INFO] found job 17075117 in provider checkpoint proc_lsst.multi INFO: found job 17075117 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:03 proc_lsst.multi:146 [INFO] found job 17075118 in provider checkpoint proc_lsst.multi INFO: found job 17075118 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:03 proc_lsst.multi:146 [INFO] found job 17075121 in provider checkpoint proc_lsst.multi INFO: found job 17075121 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:04 proc_lsst.multi:146 [INFO] found job 17075122 in provider checkpoint proc_lsst.multi INFO: found job 17075122 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:04 proc_lsst.multi:146 [INFO] found job 17075124 in provider checkpoint proc_lsst.multi INFO: found job 17075124 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:05 proc_lsst.multi:146 [INFO] found job 17075127 in provider checkpoint proc_lsst.multi INFO: found job 17075127 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:05 proc_lsst.multi:146 [INFO] found job 17075128 in provider checkpoint proc_lsst.multi INFO: found job 17075128 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:06 proc_lsst.multi:146 [INFO] found job 17075129 in provider checkpoint proc_lsst.multi INFO: found job 17075129 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:06 proc_lsst.multi:146 [INFO] found job 17075131 in provider checkpoint proc_lsst.multi INFO: found job 17075131 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:07 proc_lsst.multi:146 [INFO] found job 17075132 in provider checkpoint proc_lsst.multi INFO: found job 17075132 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:07 proc_lsst.multi:146 [INFO] found job 17075133 in provider checkpoint proc_lsst.multi INFO: found job 17075133 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:08 proc_lsst.multi:146 [INFO] found job 17075134 in provider checkpoint proc_lsst.multi INFO: found job 17075134 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:08 proc_lsst.multi:146 [INFO] found job 17075135 in provider checkpoint proc_lsst.multi INFO: found job 17075135 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:09 proc_lsst.multi:146 [INFO] found job 17075136 in provider checkpoint proc_lsst.multi INFO: found job 17075136 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:09 proc_lsst.multi:146 [INFO] found job 17075137 in provider checkpoint proc_lsst.multi INFO: found job 17075137 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:10 proc_lsst.multi:146 [INFO] found job 17075138 in provider checkpoint proc_lsst.multi INFO: found job 17075138 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:10 proc_lsst.multi:146 [INFO] found job 17075139 in provider checkpoint proc_lsst.multi INFO: found job 17075139 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:11 proc_lsst.multi:146 [INFO] found job 17075140 in provider checkpoint proc_lsst.multi INFO: found job 17075140 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:11 proc_lsst.multi:146 [INFO] found job 17075141 in provider checkpoint proc_lsst.multi INFO: found job 17075141 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:12 proc_lsst.multi:146 [INFO] found job 17075142 in provider checkpoint proc_lsst.multi INFO: found job 17075142 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:12 proc_lsst.multi:146 [INFO] found job 17075143 in provider checkpoint proc_lsst.multi INFO: found job 17075143 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:13 proc_lsst.multi:146 [INFO] found job 17075144 in provider checkpoint proc_lsst.multi INFO: found job 17075144 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:13 proc_lsst.multi:146 [INFO] found job 17075145 in provider checkpoint proc_lsst.multi INFO: found job 17075145 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:14 proc_lsst.multi:146 [INFO] found job 17075146 in provider checkpoint proc_lsst.multi INFO: found job 17075146 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:14 proc_lsst.multi:146 [INFO] found job 17075147 in provider checkpoint proc_lsst.multi INFO: found job 17075147 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:15 proc_lsst.multi:146 [INFO] found job 17075148 in provider checkpoint proc_lsst.multi INFO: found job 17075148 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:15 proc_lsst.multi:146 [INFO] found job 17075149 in provider checkpoint proc_lsst.multi INFO: found job 17075149 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:16 proc_lsst.multi:146 [INFO] found job 17075150 in provider checkpoint proc_lsst.multi INFO: found job 17075150 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:16 proc_lsst.multi:146 [INFO] found job 17075151 in provider checkpoint proc_lsst.multi INFO: found job 17075151 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:17 proc_lsst.multi:146 [INFO] found job 17075152 in provider checkpoint proc_lsst.multi INFO: found job 17075152 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:17 proc_lsst.multi:146 [INFO] found job 17075153 in provider checkpoint proc_lsst.multi INFO: found job 17075153 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:18 proc_lsst.multi:146 [INFO] found job 17075154 in provider checkpoint proc_lsst.multi INFO: found job 17075154 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:18 proc_lsst.multi:146 [INFO] found job 17075155 in provider checkpoint proc_lsst.multi INFO: found job 17075155 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:19 proc_lsst.multi:146 [INFO] found job 17075157 in provider checkpoint proc_lsst.multi INFO: found job 17075157 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:19 proc_lsst.multi:146 [INFO] found job 17075159 in provider checkpoint proc_lsst.multi INFO: found job 17075159 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:20 proc_lsst.multi:146 [INFO] found job 17075160 in provider checkpoint proc_lsst.multi INFO: found job 17075160 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:20 proc_lsst.multi:146 [INFO] found job 17075161 in provider checkpoint proc_lsst.multi INFO: found job 17075161 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:21 proc_lsst.multi:146 [INFO] found job 17075162 in provider checkpoint proc_lsst.multi INFO: found job 17075162 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:21 proc_lsst.multi:146 [INFO] found job 17075163 in provider checkpoint proc_lsst.multi INFO: found job 17075163 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:22 proc_lsst.multi:146 [INFO] found job 17075166 in provider checkpoint proc_lsst.multi INFO: found job 17075166 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:22 proc_lsst.multi:146 [INFO] found job 17075167 in provider checkpoint proc_lsst.multi INFO: found job 17075167 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:23 proc_lsst.multi:146 [INFO] found job 17075168 in provider checkpoint proc_lsst.multi INFO: found job 17075168 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:23 proc_lsst.multi:146 [INFO] found job 17075171 in provider checkpoint proc_lsst.multi INFO: found job 17075171 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:24 proc_lsst.multi:146 [INFO] found job 17075172 in provider checkpoint proc_lsst.multi INFO: found job 17075172 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:24 proc_lsst.multi:146 [INFO] found job 17075173 in provider checkpoint proc_lsst.multi INFO: found job 17075173 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:25 proc_lsst.multi:146 [INFO] found job 17075179 in provider checkpoint proc_lsst.multi INFO: found job 17075179 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:25 proc_lsst.multi:146 [INFO] found job 17075180 in provider checkpoint proc_lsst.multi INFO: found job 17075180 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:26 proc_lsst.multi:146 [INFO] found job 17075181 in provider checkpoint proc_lsst.multi INFO: found job 17075181 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:26 proc_lsst.multi:146 [INFO] found job 17075182 in provider checkpoint proc_lsst.multi INFO: found job 17075182 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:27 proc_lsst.multi:146 [INFO] found job 17075183 in provider checkpoint proc_lsst.multi INFO: found job 17075183 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:27 proc_lsst.multi:146 [INFO] found job 17075185 in provider checkpoint proc_lsst.multi INFO: found job 17075185 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:28 proc_lsst.multi:146 [INFO] found job 17075187 in provider checkpoint proc_lsst.multi INFO: found job 17075187 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:28 proc_lsst.multi:146 [INFO] found job 17075188 in provider checkpoint proc_lsst.multi INFO: found job 17075188 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:29 proc_lsst.multi:146 [INFO] found job 17075190 in provider checkpoint proc_lsst.multi INFO: found job 17075190 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:30 proc_lsst.multi:146 [INFO] found job 17075191 in provider checkpoint proc_lsst.multi INFO: found job 17075191 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:30 proc_lsst.multi:146 [INFO] found job 17075197 in provider checkpoint proc_lsst.multi INFO: found job 17075197 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:31 proc_lsst.multi:146 [INFO] found job 17075200 in provider checkpoint proc_lsst.multi INFO: found job 17075200 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:31 proc_lsst.multi:146 [INFO] found job 17075201 in provider checkpoint proc_lsst.multi INFO: found job 17075201 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:32 proc_lsst.multi:146 [INFO] found job 17075202 in provider checkpoint proc_lsst.multi INFO: found job 17075202 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:32 proc_lsst.multi:146 [INFO] found job 17075203 in provider checkpoint proc_lsst.multi INFO: found job 17075203 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:33 proc_lsst.multi:146 [INFO] found job 17075204 in provider checkpoint proc_lsst.multi INFO: found job 17075204 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:33 proc_lsst.multi:146 [INFO] found job 17075205 in provider checkpoint proc_lsst.multi INFO: found job 17075205 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:34 proc_lsst.multi:146 [INFO] found job 17075206 in provider checkpoint proc_lsst.multi INFO: found job 17075206 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:34 proc_lsst.multi:146 [INFO] found job 17075207 in provider checkpoint proc_lsst.multi INFO: found job 17075207 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:35 proc_lsst.multi:146 [INFO] found job 17075208 in provider checkpoint proc_lsst.multi INFO: found job 17075208 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:35 proc_lsst.multi:146 [INFO] found job 17075209 in provider checkpoint proc_lsst.multi INFO: found job 17075209 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:36 proc_lsst.multi:146 [INFO] found job 17075210 in provider checkpoint proc_lsst.multi INFO: found job 17075210 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:36 proc_lsst.multi:146 [INFO] found job 17075211 in provider checkpoint proc_lsst.multi INFO: found job 17075211 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:37 proc_lsst.multi:146 [INFO] found job 17075212 in provider checkpoint proc_lsst.multi INFO: found job 17075212 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:37 proc_lsst.multi:146 [INFO] found job 17075213 in provider checkpoint proc_lsst.multi INFO: found job 17075213 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:38 proc_lsst.multi:146 [INFO] found job 17075214 in provider checkpoint proc_lsst.multi INFO: found job 17075214 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:38 proc_lsst.multi:146 [INFO] found job 17075216 in provider checkpoint proc_lsst.multi INFO: found job 17075216 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:39 proc_lsst.multi:146 [INFO] found job 17075217 in provider checkpoint proc_lsst.multi INFO: found job 17075217 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:39 proc_lsst.multi:146 [INFO] found job 17075218 in provider checkpoint proc_lsst.multi INFO: found job 17075218 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 CG 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status CG to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:40 proc_lsst.multi:146 [INFO] found job 17075219 in provider checkpoint proc_lsst.multi INFO: found job 17075219 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:40 proc_lsst.multi:146 [INFO] found job 17075220 in provider checkpoint proc_lsst.multi INFO: found job 17075220 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:41 proc_lsst.multi:146 [INFO] found job 17075221 in provider checkpoint proc_lsst.multi INFO: found job 17075221 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:41 proc_lsst.multi:146 [INFO] found job 17075222 in provider checkpoint proc_lsst.multi INFO: found job 17075222 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:42 proc_lsst.multi:146 [INFO] found job 17075223 in provider checkpoint proc_lsst.multi INFO: found job 17075223 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:42 proc_lsst.multi:146 [INFO] found job 17075224 in provider checkpoint proc_lsst.multi INFO: found job 17075224 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:43 proc_lsst.multi:146 [INFO] found job 17075225 in provider checkpoint proc_lsst.multi INFO: found job 17075225 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:43 proc_lsst.multi:146 [INFO] found job 17075226 in provider checkpoint proc_lsst.multi INFO: found job 17075226 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:44 proc_lsst.multi:146 [INFO] found job 17075227 in provider checkpoint proc_lsst.multi INFO: found job 17075227 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:44 proc_lsst.multi:146 [INFO] found job 17075228 in provider checkpoint proc_lsst.multi INFO: found job 17075228 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:45 proc_lsst.multi:146 [INFO] found job 17075229 in provider checkpoint proc_lsst.multi INFO: found job 17075229 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:45 proc_lsst.multi:146 [INFO] found job 17075230 in provider checkpoint proc_lsst.multi INFO: found job 17075230 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:46 proc_lsst.multi:146 [INFO] found job 17075231 in provider checkpoint proc_lsst.multi INFO: found job 17075231 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:46 proc_lsst.multi:146 [INFO] found job 17075232 in provider checkpoint proc_lsst.multi INFO: found job 17075232 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:47 proc_lsst.multi:146 [INFO] found job 17075233 in provider checkpoint proc_lsst.multi INFO: found job 17075233 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:47 proc_lsst.multi:146 [INFO] found job 17075234 in provider checkpoint proc_lsst.multi INFO: found job 17075234 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:48 proc_lsst.multi:146 [INFO] found job 17075238 in provider checkpoint proc_lsst.multi INFO: found job 17075238 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:48 proc_lsst.multi:146 [INFO] found job 17075242 in provider checkpoint proc_lsst.multi INFO: found job 17075242 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:49 proc_lsst.multi:146 [INFO] found job 17075245 in provider checkpoint proc_lsst.multi INFO: found job 17075245 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:49 proc_lsst.multi:146 [INFO] found job 17075246 in provider checkpoint proc_lsst.multi INFO: found job 17075246 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:50 proc_lsst.multi:146 [INFO] found job 17075247 in provider checkpoint proc_lsst.multi INFO: found job 17075247 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:50 proc_lsst.multi:146 [INFO] found job 17075250 in provider checkpoint proc_lsst.multi INFO: found job 17075250 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:51 proc_lsst.multi:146 [INFO] found job 17075254 in provider checkpoint proc_lsst.multi INFO: found job 17075254 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:51 proc_lsst.multi:146 [INFO] found job 17075255 in provider checkpoint proc_lsst.multi INFO: found job 17075255 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:52 proc_lsst.multi:146 [INFO] found job 17075256 in provider checkpoint proc_lsst.multi INFO: found job 17075256 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:52 proc_lsst.multi:146 [INFO] found job 17075257 in provider checkpoint proc_lsst.multi INFO: found job 17075257 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:53 proc_lsst.multi:146 [INFO] found job 17075258 in provider checkpoint proc_lsst.multi INFO: found job 17075258 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:53 proc_lsst.multi:146 [INFO] found job 17075264 in provider checkpoint proc_lsst.multi INFO: found job 17075264 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:54 proc_lsst.multi:146 [INFO] found job 17075265 in provider checkpoint proc_lsst.multi INFO: found job 17075265 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:54 proc_lsst.multi:146 [INFO] found job 17075273 in provider checkpoint proc_lsst.multi INFO: found job 17075273 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:55 proc_lsst.multi:146 [INFO] found job 17075274 in provider checkpoint proc_lsst.multi INFO: found job 17075274 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:55 proc_lsst.multi:146 [INFO] found job 17075275 in provider checkpoint proc_lsst.multi INFO: found job 17075275 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:56 proc_lsst.multi:146 [INFO] found job 17075276 in provider checkpoint proc_lsst.multi INFO: found job 17075276 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:56 proc_lsst.multi:146 [INFO] found job 17075278 in provider checkpoint proc_lsst.multi INFO: found job 17075278 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:57 proc_lsst.multi:146 [INFO] found job 17075279 in provider checkpoint proc_lsst.multi INFO: found job 17075279 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:57 proc_lsst.multi:146 [INFO] found job 17075280 in provider checkpoint proc_lsst.multi INFO: found job 17075280 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:58 proc_lsst.multi:146 [INFO] found job 17075281 in provider checkpoint proc_lsst.multi INFO: found job 17075281 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:59 proc_lsst.multi:146 [INFO] found job 17075282 in provider checkpoint proc_lsst.multi INFO: found job 17075282 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:02:59 proc_lsst.multi:146 [INFO] found job 17075283 in provider checkpoint proc_lsst.multi INFO: found job 17075283 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:00 proc_lsst.multi:146 [INFO] found job 17075284 in provider checkpoint proc_lsst.multi INFO: found job 17075284 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:00 proc_lsst.multi:146 [INFO] found job 17075285 in provider checkpoint proc_lsst.multi INFO: found job 17075285 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:01 proc_lsst.multi:146 [INFO] found job 17075286 in provider checkpoint proc_lsst.multi INFO: found job 17075286 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:01 proc_lsst.multi:146 [INFO] found job 17075287 in provider checkpoint proc_lsst.multi INFO: found job 17075287 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:02 proc_lsst.multi:146 [INFO] found job 17075288 in provider checkpoint proc_lsst.multi INFO: found job 17075288 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:03 proc_lsst.multi:146 [INFO] found job 17075289 in provider checkpoint proc_lsst.multi INFO: found job 17075289 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:03 proc_lsst.multi:146 [INFO] found job 17075290 in provider checkpoint proc_lsst.multi INFO: found job 17075290 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:04 proc_lsst.multi:146 [INFO] found job 17075293 in provider checkpoint proc_lsst.multi INFO: found job 17075293 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:04 proc_lsst.multi:146 [INFO] found job 17075298 in provider checkpoint proc_lsst.multi INFO: found job 17075298 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:05 proc_lsst.multi:146 [INFO] found job 17075299 in provider checkpoint proc_lsst.multi INFO: found job 17075299 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:05 proc_lsst.multi:146 [INFO] found job 17075301 in provider checkpoint proc_lsst.multi INFO: found job 17075301 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:06 proc_lsst.multi:146 [INFO] found job 17075306 in provider checkpoint proc_lsst.multi INFO: found job 17075306 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:06 proc_lsst.multi:146 [INFO] found job 17075307 in provider checkpoint proc_lsst.multi INFO: found job 17075307 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:07 proc_lsst.multi:146 [INFO] found job 17075308 in provider checkpoint proc_lsst.multi INFO: found job 17075308 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:07 proc_lsst.multi:146 [INFO] found job 17075309 in provider checkpoint proc_lsst.multi INFO: found job 17075309 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:08 proc_lsst.multi:146 [INFO] found job 17075310 in provider checkpoint proc_lsst.multi INFO: found job 17075310 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:08 proc_lsst.multi:146 [INFO] found job 17075311 in provider checkpoint proc_lsst.multi INFO: found job 17075311 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:09 proc_lsst.multi:146 [INFO] found job 17075312 in provider checkpoint proc_lsst.multi INFO: found job 17075312 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:09 proc_lsst.multi:146 [INFO] found job 17075315 in provider checkpoint proc_lsst.multi INFO: found job 17075315 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:10 proc_lsst.multi:146 [INFO] found job 17075318 in provider checkpoint proc_lsst.multi INFO: found job 17075318 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:10 proc_lsst.multi:146 [INFO] found job 17075319 in provider checkpoint proc_lsst.multi INFO: found job 17075319 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:11 proc_lsst.multi:146 [INFO] found job 17075320 in provider checkpoint proc_lsst.multi INFO: found job 17075320 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:11 proc_lsst.multi:146 [INFO] found job 17075321 in provider checkpoint proc_lsst.multi INFO: found job 17075321 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:12 proc_lsst.multi:146 [INFO] found job 17075326 in provider checkpoint proc_lsst.multi INFO: found job 17075326 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:12 proc_lsst.multi:146 [INFO] found job 17075329 in provider checkpoint proc_lsst.multi INFO: found job 17075329 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:13 proc_lsst.multi:146 [INFO] found job 17075330 in provider checkpoint proc_lsst.multi INFO: found job 17075330 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:13 proc_lsst.multi:146 [INFO] found job 17075332 in provider checkpoint proc_lsst.multi INFO: found job 17075332 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:14 proc_lsst.multi:146 [INFO] found job 17075338 in provider checkpoint proc_lsst.multi INFO: found job 17075338 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:14 proc_lsst.multi:146 [INFO] found job 17075344 in provider checkpoint proc_lsst.multi INFO: found job 17075344 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:15 proc_lsst.multi:146 [INFO] found job 17075345 in provider checkpoint proc_lsst.multi INFO: found job 17075345 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:15 proc_lsst.multi:146 [INFO] found job 17075346 in provider checkpoint proc_lsst.multi INFO: found job 17075346 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:16 proc_lsst.multi:146 [INFO] found job 17075347 in provider checkpoint proc_lsst.multi INFO: found job 17075347 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:16 proc_lsst.multi:146 [INFO] found job 17075348 in provider checkpoint proc_lsst.multi INFO: found job 17075348 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:17 proc_lsst.multi:146 [INFO] found job 17075351 in provider checkpoint proc_lsst.multi INFO: found job 17075351 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:17 proc_lsst.multi:146 [INFO] found job 17075353 in provider checkpoint proc_lsst.multi INFO: found job 17075353 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:18 proc_lsst.multi:146 [INFO] found job 17075354 in provider checkpoint proc_lsst.multi INFO: found job 17075354 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:18 proc_lsst.multi:146 [INFO] found job 17075355 in provider checkpoint proc_lsst.multi INFO: found job 17075355 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:19 proc_lsst.multi:146 [INFO] found job 17075357 in provider checkpoint proc_lsst.multi INFO: found job 17075357 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:19 proc_lsst.multi:146 [INFO] found job 17075358 in provider checkpoint proc_lsst.multi INFO: found job 17075358 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:20 proc_lsst.multi:146 [INFO] found job 17075359 in provider checkpoint proc_lsst.multi INFO: found job 17075359 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:20 proc_lsst.multi:146 [INFO] found job 17075360 in provider checkpoint proc_lsst.multi INFO: found job 17075360 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:21 proc_lsst.multi:146 [INFO] found job 17075364 in provider checkpoint proc_lsst.multi INFO: found job 17075364 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:21 proc_lsst.multi:146 [INFO] found job 17075368 in provider checkpoint proc_lsst.multi INFO: found job 17075368 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:22 proc_lsst.multi:146 [INFO] found job 17075372 in provider checkpoint proc_lsst.multi INFO: found job 17075372 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:22 proc_lsst.multi:146 [INFO] found job 17075376 in provider checkpoint proc_lsst.multi INFO: found job 17075376 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:23 proc_lsst.multi:146 [INFO] found job 17075379 in provider checkpoint proc_lsst.multi INFO: found job 17075379 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:23 proc_lsst.multi:146 [INFO] found job 17075380 in provider checkpoint proc_lsst.multi INFO: found job 17075380 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:24 proc_lsst.multi:146 [INFO] found job 17075381 in provider checkpoint proc_lsst.multi INFO: found job 17075381 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:24 proc_lsst.multi:146 [INFO] found job 17075382 in provider checkpoint proc_lsst.multi INFO: found job 17075382 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:25 proc_lsst.multi:146 [INFO] found job 17075384 in provider checkpoint proc_lsst.multi INFO: found job 17075384 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:25 proc_lsst.multi:146 [INFO] found job 17075385 in provider checkpoint proc_lsst.multi INFO: found job 17075385 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:26 proc_lsst.multi:146 [INFO] found job 17075386 in provider checkpoint proc_lsst.multi INFO: found job 17075386 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:26 proc_lsst.multi:146 [INFO] found job 17075387 in provider checkpoint proc_lsst.multi INFO: found job 17075387 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:27 proc_lsst.multi:146 [INFO] found job 17075388 in provider checkpoint proc_lsst.multi INFO: found job 17075388 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:27 proc_lsst.multi:146 [INFO] found job 17075392 in provider checkpoint proc_lsst.multi INFO: found job 17075392 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:28 proc_lsst.multi:146 [INFO] found job 17075394 in provider checkpoint proc_lsst.multi INFO: found job 17075394 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:28 proc_lsst.multi:146 [INFO] found job 17075399 in provider checkpoint proc_lsst.multi INFO: found job 17075399 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:29 proc_lsst.multi:146 [INFO] found job 17075400 in provider checkpoint proc_lsst.multi INFO: found job 17075400 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:29 proc_lsst.multi:146 [INFO] found job 17075401 in provider checkpoint proc_lsst.multi INFO: found job 17075401 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:30 proc_lsst.multi:146 [INFO] found job 17075405 in provider checkpoint proc_lsst.multi INFO: found job 17075405 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:30 proc_lsst.multi:146 [INFO] found job 17075408 in provider checkpoint proc_lsst.multi INFO: found job 17075408 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:31 proc_lsst.multi:146 [INFO] found job 17075417 in provider checkpoint proc_lsst.multi INFO: found job 17075417 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:31 proc_lsst.multi:146 [INFO] found job 17075428 in provider checkpoint proc_lsst.multi INFO: found job 17075428 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:32 proc_lsst.multi:146 [INFO] found job 17075430 in provider checkpoint proc_lsst.multi INFO: found job 17075430 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:33 proc_lsst.multi:146 [INFO] found job 17075431 in provider checkpoint proc_lsst.multi INFO: found job 17075431 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:33 proc_lsst.multi:146 [INFO] found job 17075432 in provider checkpoint proc_lsst.multi INFO: found job 17075432 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:34 proc_lsst.multi:146 [INFO] found job 17075433 in provider checkpoint proc_lsst.multi INFO: found job 17075433 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:34 proc_lsst.multi:146 [INFO] found job 17075434 in provider checkpoint proc_lsst.multi INFO: found job 17075434 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:34 proc_lsst.multi:146 [INFO] found job 17075435 in provider checkpoint proc_lsst.multi INFO: found job 17075435 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:35 proc_lsst.multi:146 [INFO] found job 17075436 in provider checkpoint proc_lsst.multi INFO: found job 17075436 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:35 proc_lsst.multi:146 [INFO] found job 17075438 in provider checkpoint proc_lsst.multi INFO: found job 17075438 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:36 proc_lsst.multi:146 [INFO] found job 17075440 in provider checkpoint proc_lsst.multi INFO: found job 17075440 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:36 proc_lsst.multi:146 [INFO] found job 17075441 in provider checkpoint proc_lsst.multi INFO: found job 17075441 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:37 proc_lsst.multi:146 [INFO] found job 17075443 in provider checkpoint proc_lsst.multi INFO: found job 17075443 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:37 proc_lsst.multi:146 [INFO] found job 17075444 in provider checkpoint proc_lsst.multi INFO: found job 17075444 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING 2024-03-15 14:03:38 proc_lsst.multi:146 [INFO] found job 17075446 in provider checkpoint proc_lsst.multi INFO: found job 17075446 in provider checkpoint parsl.providers.slurm.slurm DEBUG: Executing squeue --noheader --format='%i %t' --job '17075022,17075030,17075031,17075032,17075033,17075034,17075035,17075038,17075041,17075042,17075046,17075047,17075048,17075049,17075050,17075052,17075053,17075054,17075055,17075056,17075057,17075058,17075059,17075060,17075061,17075062,17075063,17075064,17075065,17075066,17075068,17075069,17075072,17075073,17075074,17075075,17075076,17075078,17075080,17075081,17075082,17075083,17075085,17075086,17075088,17075090,17075091,17075092,17075093,17075094,17075095,17075096,17075097,17075098,17075099,17075100,17075101,17075102,17075103,17075109,17075110,17075111,17075112,17075113,17075114,17075115,17075116,17075117,17075118,17075121,17075122,17075124,17075127,17075128,17075129,17075131,17075132,17075133,17075134,17075135,17075136,17075137,17075138,17075139,17075140,17075141,17075142,17075143,17075144,17075145,17075146,17075147,17075148,17075149,17075150,17075151,17075152,17075153,17075154,17075155,17075157,17075159,17075160,17075161,17075162,17075163,17075166,17075167,17075168,17075171,17075172,17075173,17075179,17075180,17075181,17075182,17075183,17075185,17075187,17075188,17075190,17075191,17075197,17075200,17075201,17075202,17075203,17075204,17075205,17075206,17075207,17075208,17075209,17075210,17075211,17075212,17075213,17075214,17075216,17075217,17075218,17075219,17075220,17075221,17075222,17075223,17075224,17075225,17075226,17075227,17075228,17075229,17075230,17075231,17075232,17075233,17075234,17075238,17075242,17075245,17075246,17075247,17075250,17075254,17075255,17075256,17075257,17075258,17075264,17075265,17075273,17075274,17075275,17075276,17075278,17075279,17075280,17075281,17075282,17075283,17075284,17075285,17075286,17075287,17075288,17075289,17075290,17075293,17075298,17075299,17075301,17075306,17075307,17075308,17075309,17075310,17075311,17075312,17075315,17075318,17075319,17075320,17075321,17075326,17075329,17075330,17075332,17075338,17075344,17075345,17075346,17075347,17075348,17075351,17075353,17075354,17075355,17075357,17075358,17075359,17075360,17075364,17075368,17075372,17075376,17075379,17075380,17075381,17075382,17075384,17075385,17075386,17075387,17075388,17075392,17075394,17075399,17075400,17075401,17075405,17075408,17075417,17075428,17075430,17075431,17075432,17075433,17075434,17075435,17075436,17075438,17075440,17075441,17075443,17075444,17075446' parsl.providers.slurm.slurm DEBUG: squeue returned 17075060 PD 17075075 PD 17075076 PD 17075078 PD 17075080 PD 17075081 PD 17075082 PD 17075083 PD 17075085 PD 17075086 PD 17075088 PD 17075090 PD 17075091 PD 17075092 PD 17075093 PD 17075094 PD 17075095 PD 17075096 PD 17075097 PD 17075098 PD 17075099 PD 17075100 PD 17075101 PD 17075102 PD 17075103 PD 17075109 PD 17075110 PD 17075111 PD 17075112 PD 17075113 PD 17075114 PD 17075115 PD 17075116 PD 17075117 PD 17075118 PD 17075121 PD 17075122 PD 17075124 PD 17075127 PD 17075128 PD 17075129 PD 17075131 PD 17075132 PD 17075133 PD 17075134 PD 17075135 PD 17075136 PD 17075137 PD 17075138 PD 17075139 PD 17075140 PD 17075446 PD 17075444 PD 17075443 PD 17075441 PD 17075440 PD 17075438 PD 17075436 PD 17075435 PD 17075434 PD 17075433 PD 17075432 PD 17075431 PD 17075430 PD 17075428 PD 17075417 PD 17075408 PD 17075405 PD 17075401 PD 17075400 PD 17075399 PD 17075394 PD 17075392 PD 17075388 PD 17075387 PD 17075386 PD 17075385 PD 17075384 PD 17075382 PD 17075381 PD 17075380 PD 17075379 PD 17075376 PD 17075372 PD 17075368 PD 17075364 PD 17075360 PD 17075359 PD 17075358 PD 17075357 PD 17075355 PD 17075354 PD 17075353 PD 17075351 PD 17075348 PD 17075347 PD 17075346 PD 17075345 PD 17075344 PD 17075338 PD 17075332 PD 17075330 PD 17075329 PD 17075326 PD 17075321 PD 17075320 PD 17075319 PD 17075318 PD 17075315 PD 17075312 PD 17075311 PD 17075310 PD 17075309 PD 17075308 PD 17075307 PD 17075306 PD 17075301 PD 17075299 PD 17075298 PD 17075293 PD 17075290 PD 17075289 PD 17075288 PD 17075287 PD 17075286 PD 17075285 PD 17075284 PD 17075283 PD 17075282 PD 17075281 PD 17075280 PD 17075279 PD 17075278 PD 17075276 PD 17075275 PD 17075274 PD 17075273 PD 17075265 PD 17075264 PD 17075258 PD 17075257 PD 17075256 PD 17075255 PD 17075254 PD 17075250 PD 17075247 PD 17075246 PD 17075245 PD 17075242 PD 17075238 PD 17075234 PD 17075233 PD 17075232 PD 17075231 PD 17075230 PD 17075229 PD 17075228 PD 17075227 PD 17075226 PD 17075225 PD 17075224 PD 17075223 PD 17075222 PD 17075221 PD 17075220 PD 17075219 PD 17075218 PD 17075217 PD 17075216 PD 17075214 PD 17075213 PD 17075212 PD 17075211 PD 17075210 PD 17075209 PD 17075208 PD 17075207 PD 17075206 PD 17075205 PD 17075204 PD 17075203 PD 17075202 PD 17075201 PD 17075200 PD 17075197 PD 17075191 PD 17075190 PD 17075188 PD 17075187 PD 17075185 PD 17075183 PD 17075182 PD 17075181 PD 17075180 PD 17075179 PD 17075173 PD 17075172 PD 17075171 PD 17075168 PD 17075167 PD 17075166 PD 17075163 PD 17075162 PD 17075161 PD 17075160 PD 17075159 PD 17075157 PD 17075155 PD 17075154 PD 17075153 PD 17075152 PD 17075151 PD 17075150 PD 17075149 PD 17075148 PD 17075147 PD 17075146 PD 17075145 PD 17075144 PD 17075143 PD 17075142 PD 17075141 PD 17075059 PD 17075069 R 17075072 R 17075073 R 17075074 R 17075068 R 17075065 R 17075066 R 17075064 R 17075055 R 17075056 R 17075057 R 17075058 R 17075061 R 17075062 R 17075063 R 17075053 R 17075054 R 17075052 R 17075022 R 17075030 R 17075031 R 17075032 R 17075033 R 17075034 R 17075035 R 17075038 R 17075041 R 17075042 R 17075046 R 17075047 R 17075048 R 17075049 R 17075050 R parsl.providers.slurm.slurm DEBUG: Updating job 17075060 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075075 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075076 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075078 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075080 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075081 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075082 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075083 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075085 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075086 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075088 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075090 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075091 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075092 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075093 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075094 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075095 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075096 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075097 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075098 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075099 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075100 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075101 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075102 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075103 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075109 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075110 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075111 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075112 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075113 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075114 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075115 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075116 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075117 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075118 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075121 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075122 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075124 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075127 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075128 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075129 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075131 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075132 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075133 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075134 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075135 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075136 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075137 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075138 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075139 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075140 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075446 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075444 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075443 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075441 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075440 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075438 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075436 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075435 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075434 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075433 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075432 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075431 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075430 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075428 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075417 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075408 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075405 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075401 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075400 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075399 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075394 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075392 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075388 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075387 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075386 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075385 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075384 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075382 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075381 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075380 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075379 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075376 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075372 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075368 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075364 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075360 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075359 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075358 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075357 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075355 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075354 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075353 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075351 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075348 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075347 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075346 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075345 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075344 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075338 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075332 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075330 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075329 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075326 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075321 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075320 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075319 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075318 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075315 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075312 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075311 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075310 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075309 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075308 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075307 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075306 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075301 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075299 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075298 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075293 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075290 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075289 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075288 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075287 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075286 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075285 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075284 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075283 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075282 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075281 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075280 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075279 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075278 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075276 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075275 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075274 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075273 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075265 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075264 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075258 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075257 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075256 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075255 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075254 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075250 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075247 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075246 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075245 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075242 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075238 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075234 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075233 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075232 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075231 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075230 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075229 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075228 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075227 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075226 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075225 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075224 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075223 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075222 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075221 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075220 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075219 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075218 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075217 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075216 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075214 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075213 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075212 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075211 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075210 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075209 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075208 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075207 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075206 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075205 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075204 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075203 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075202 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075201 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075200 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075197 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075191 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075190 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075188 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075187 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075185 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075183 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075182 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075181 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075180 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075179 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075173 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075172 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075171 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075168 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075167 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075166 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075163 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075162 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075161 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075160 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075159 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075157 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075155 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075154 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075153 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075152 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075151 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075150 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075149 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075148 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075147 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075146 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075145 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075144 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075143 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075142 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075141 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075059 with slurm status PD to parsl state JobState.PENDING parsl.providers.slurm.slurm DEBUG: Updating job 17075069 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075072 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075073 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075074 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075068 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075065 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075066 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075064 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075055 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075056 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075057 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075058 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075061 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075062 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075063 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075053 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075054 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075052 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075022 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075030 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075031 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075032 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075033 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075034 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075035 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075038 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075041 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075042 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075046 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075047 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075048 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075049 with slurm status R to parsl state JobState.RUNNING parsl.providers.slurm.slurm DEBUG: Updating job 17075050 with slurm status R to parsl state JobState.RUNNING parsl.dataflow.strategy DEBUG: general strategy starting with strategy_type simple for 1 executors parsl.dataflow.strategy DEBUG: Strategizing for executor multi parsl.dataflow.strategy DEBUG: Slot ratio calculation: active_slots = 312, active_tasks = 0 parsl.dataflow.strategy DEBUG: Executor multi has 0 active tasks, 90/222 running/pending blocks, and 321 connected workers parsl.dataflow.strategy DEBUG: Strategy case 1: Executor has no active tasks parsl.dataflow.strategy DEBUG: Strategy case 1b: Executor has no active tasks, and more (312) than minimum blocks (0) parsl.dataflow.strategy DEBUG: Starting idle timer for executor. If idle time exceeds 120.0s, blocks will be scaled in parsl.dataflow.strategy DEBUG: Idle time 9.5367431640625e-07s is less than max_idletime 120.0s for executor multi; not scaling in parsl.process_loggers DEBUG: Normal ending for _general_strategy on thread JobStatusPoller-Timer-Thread-22709913200592 parsl.dataflow.dflow INFO: Terminated job status poller parsl.dataflow.dflow INFO: Scaling in and shutting down executors parsl.dataflow.dflow INFO: Scaling in executor multi parsl.executors.high_throughput.executor DEBUG: Scale in called, blocks=312, block_ids=[] parsl.executors.high_throughput.executor DEBUG: Scale in selecting from 89 blocks parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 1b972926d4e0 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 1b972926d4e0 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 6c7f4aa77664 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 6c7f4aa77664 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: c552f76b2b1f parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: c552f76b2b1f parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 63d737d3569c parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 63d737d3569c parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: d7a7ff57dae0 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: d7a7ff57dae0 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: d2bffa4d84cd parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: d2bffa4d84cd parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: faf4015f0659 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: faf4015f0659 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: fccb8b92c445 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: fccb8b92c445 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 20e427254306 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 20e427254306 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 229e7dba5d1d parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 229e7dba5d1d parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 3b197f05c301 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 3b197f05c301 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 4273dac37356 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 4273dac37356 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: b9cf7f54fe3c parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: b9cf7f54fe3c parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: c026fd74d765 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: c026fd74d765 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 332e87b01aa3 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 332e87b01aa3 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: cb6e2dfd1bfc parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: cb6e2dfd1bfc parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 1a1062a229f6 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 1a1062a229f6 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 59d20f0ba128 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 59d20f0ba128 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: d53c27ed065e parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: d53c27ed065e parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 271a26100405 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 271a26100405 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 70578ac8b1a3 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 70578ac8b1a3 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 063740c510c0 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 063740c510c0 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 62772b0f63da parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 62772b0f63da parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: bf242b625fde parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: bf242b625fde parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: cc9e316ba9a6 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: cc9e316ba9a6 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 4c7bf37437f8 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 4c7bf37437f8 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: b3ba858c43c0 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: b3ba858c43c0 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 5cdb226c7c57 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 5cdb226c7c57 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: e6ee98e70182 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: e6ee98e70182 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: ef8448770cdc parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: ef8448770cdc parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: be0d32fb747c parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: be0d32fb747c parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 2c636f3f7047 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 2c636f3f7047 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 521a3cecf9a5 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 521a3cecf9a5 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 49e462f3bfd9 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 49e462f3bfd9 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 0d7801fc2e9d parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 0d7801fc2e9d parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 4c228b14aaf7 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 4c228b14aaf7 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 286f865c90f9 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 286f865c90f9 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 4d4f093cb4eb parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 4d4f093cb4eb parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 471a35b25d28 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 471a35b25d28 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 09b112b7e98c parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 09b112b7e98c parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: c4b2dc40c841 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: c4b2dc40c841 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: df5d1934c659 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: df5d1934c659 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: e08da7e36ecd parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: e08da7e36ecd parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 39bbf6c5dbc2 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 39bbf6c5dbc2 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 6f054a4a42cd parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 6f054a4a42cd parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 58d35a3ad03e parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 58d35a3ad03e parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: e1f8559db38c parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: e1f8559db38c parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 416fce2e6b84 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 416fce2e6b84 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: c36953c6476d parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: c36953c6476d parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 2106f45c3a2e parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 2106f45c3a2e parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 262ff9f76989 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 262ff9f76989 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: f93f627fc39d parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: f93f627fc39d parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 7d743924ce31 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 7d743924ce31 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 5a4d0ab30908 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 5a4d0ab30908 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 72b83fb670ea parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 72b83fb670ea parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: b83f5bbe933f parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: b83f5bbe933f parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: e92212d4c043 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: e92212d4c043 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: c7a61f3d760e parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: c7a61f3d760e parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 9d8a088d2040 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 9d8a088d2040 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: a488d914a899 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: a488d914a899 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 9770ca0fce8e parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 9770ca0fce8e parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: dadcaafa122c parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: dadcaafa122c parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: b5a1ec0cd7b5 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: b5a1ec0cd7b5 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 2764911a3623 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 2764911a3623 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 5fb09901b16f parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 5fb09901b16f parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 984b2132e47a parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 984b2132e47a parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 92935490ea8a parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 92935490ea8a parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: ebb5c657518c parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: ebb5c657518c parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 76c71c7a013b parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 76c71c7a013b parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 271395b57e1e parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 271395b57e1e parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 8feb0fba92ca parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 8feb0fba92ca parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 0dc6627c4b54 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 0dc6627c4b54 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 296f9b6b2cab parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 296f9b6b2cab parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: cb6db5b5fc07 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: cb6db5b5fc07 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 48966b2562fb parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 48966b2562fb parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: c52b649aa128 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: c52b649aa128 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 75daba6d216e parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 75daba6d216e parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: a9a8573c5d53 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: a9a8573c5d53 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 5cfd8ad0fc40 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 5cfd8ad0fc40 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: bbc2193506a7 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: bbc2193506a7 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: bb79eebfd5cf parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: bb79eebfd5cf parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: bf6fc54c1333 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: bf6fc54c1333 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 464a7fe55d70 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 464a7fe55d70 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: a3cc7af86217 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: a3cc7af86217 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: fc608b158f4d parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: fc608b158f4d parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: d895a16ee6fd parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: d895a16ee6fd parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 69e847481885 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 69e847481885 parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: b3fd9c73e95a parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: b3fd9c73e95a parsl.executors.high_throughput.executor DEBUG: Sending hold to manager: 80621833f833 parsl.executors.high_throughput.executor DEBUG: Sent hold request to manager: 80621833f833 2024-03-15 14:03:38 proc_lsst.multi:146 [INFO] found job 17075073 in provider checkpoint proc_lsst.multi INFO: found job 17075073 in provider checkpoint 2024-03-15 14:03:38 proc_lsst.multi:201 [INFO] cancelling 17075073 on provider checkpoint proc_lsst.multi INFO: cancelling 17075073 on provider checkpoint 2024-03-15 14:03:39 proc_lsst.multi:146 [INFO] found job 17075072 in provider checkpoint proc_lsst.multi INFO: found job 17075072 in provider checkpoint 2024-03-15 14:03:39 proc_lsst.multi:201 [INFO] cancelling 17075072 on provider checkpoint proc_lsst.multi INFO: cancelling 17075072 on provider checkpoint 2024-03-15 14:03:39 proc_lsst.multi:146 [INFO] found job 17075069 in provider checkpoint proc_lsst.multi INFO: found job 17075069 in provider checkpoint 2024-03-15 14:03:39 proc_lsst.multi:201 [INFO] cancelling 17075069 on provider checkpoint proc_lsst.multi INFO: cancelling 17075069 on provider checkpoint 2024-03-15 14:03:40 proc_lsst.multi:146 [INFO] found job 17075074 in provider checkpoint proc_lsst.multi INFO: found job 17075074 in provider checkpoint 2024-03-15 14:03:40 proc_lsst.multi:201 [INFO] cancelling 17075074 on provider checkpoint proc_lsst.multi INFO: cancelling 17075074 on provider checkpoint 2024-03-15 14:03:40 proc_lsst.multi:146 [INFO] found job 17075047 in provider checkpoint proc_lsst.multi INFO: found job 17075047 in provider checkpoint 2024-03-15 14:03:40 proc_lsst.multi:201 [INFO] cancelling 17075047 on provider checkpoint proc_lsst.multi INFO: cancelling 17075047 on provider checkpoint 2024-03-15 14:03:40 proc_lsst.multi:146 [INFO] found job 17075035 in provider checkpoint proc_lsst.multi INFO: found job 17075035 in provider checkpoint 2024-03-15 14:03:40 proc_lsst.multi:201 [INFO] cancelling 17075035 on provider checkpoint proc_lsst.multi INFO: cancelling 17075035 on provider checkpoint 2024-03-15 14:03:41 proc_lsst.multi:146 [INFO] found job 17075038 in provider checkpoint proc_lsst.multi INFO: found job 17075038 in provider checkpoint 2024-03-15 14:03:41 proc_lsst.multi:201 [INFO] cancelling 17075038 on provider checkpoint proc_lsst.multi INFO: cancelling 17075038 on provider checkpoint 2024-03-15 14:03:41 proc_lsst.multi:146 [INFO] found job 17075048 in provider checkpoint proc_lsst.multi INFO: found job 17075048 in provider checkpoint 2024-03-15 14:03:41 proc_lsst.multi:201 [INFO] cancelling 17075048 on provider checkpoint proc_lsst.multi INFO: cancelling 17075048 on provider checkpoint 2024-03-15 14:03:42 proc_lsst.multi:146 [INFO] found job 17075065 in provider checkpoint proc_lsst.multi INFO: found job 17075065 in provider checkpoint 2024-03-15 14:03:42 proc_lsst.multi:201 [INFO] cancelling 17075065 on provider checkpoint proc_lsst.multi INFO: cancelling 17075065 on provider checkpoint 2024-03-15 14:03:42 proc_lsst.multi:146 [INFO] found job 17075062 in provider checkpoint proc_lsst.multi INFO: found job 17075062 in provider checkpoint 2024-03-15 14:03:42 proc_lsst.multi:201 [INFO] cancelling 17075062 on provider checkpoint proc_lsst.multi INFO: cancelling 17075062 on provider checkpoint 2024-03-15 14:03:43 proc_lsst.multi:146 [INFO] found job 17075066 in provider checkpoint proc_lsst.multi INFO: found job 17075066 in provider checkpoint 2024-03-15 14:03:43 proc_lsst.multi:201 [INFO] cancelling 17075066 on provider checkpoint proc_lsst.multi INFO: cancelling 17075066 on provider checkpoint 2024-03-15 14:03:43 proc_lsst.multi:146 [INFO] found job 91028 in provider local proc_lsst.multi INFO: found job 91028 in provider local 2024-03-15 14:03:43 proc_lsst.multi:201 [INFO] cancelling 91028 on provider local proc_lsst.multi INFO: cancelling 91028 on provider local parsl.providers.local.local DEBUG: Terminating job/proc_id: 91028 2024-03-15 14:03:43 proc_lsst.multi:146 [INFO] found job 17074838 in provider astro proc_lsst.multi INFO: found job 17074838 in provider astro 2024-03-15 14:03:43 proc_lsst.multi:201 [INFO] cancelling 17074838 on provider astro proc_lsst.multi INFO: cancelling 17074838 on provider astro 2024-03-15 14:03:43 proc_lsst.multi:146 [INFO] found job 17074833 in provider astro proc_lsst.multi INFO: found job 17074833 in provider astro 2024-03-15 14:03:43 proc_lsst.multi:201 [INFO] cancelling 17074833 on provider astro proc_lsst.multi INFO: cancelling 17074833 on provider astro 2024-03-15 14:03:44 proc_lsst.multi:146 [INFO] found job 17074939 in provider escience_a40 proc_lsst.multi INFO: found job 17074939 in provider escience_a40 2024-03-15 14:03:44 proc_lsst.multi:201 [INFO] cancelling 17074939 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074939 on provider escience_a40 2024-03-15 14:03:44 proc_lsst.multi:146 [INFO] found job 17074898 in provider escience_a40 proc_lsst.multi INFO: found job 17074898 in provider escience_a40 2024-03-15 14:03:44 proc_lsst.multi:201 [INFO] cancelling 17074898 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074898 on provider escience_a40 2024-03-15 14:03:45 proc_lsst.multi:146 [INFO] found job 17074932 in provider escience_a40 proc_lsst.multi INFO: found job 17074932 in provider escience_a40 2024-03-15 14:03:45 proc_lsst.multi:201 [INFO] cancelling 17074932 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074932 on provider escience_a40 2024-03-15 14:03:45 proc_lsst.multi:146 [INFO] found job 17075032 in provider checkpoint proc_lsst.multi INFO: found job 17075032 in provider checkpoint 2024-03-15 14:03:45 proc_lsst.multi:201 [INFO] cancelling 17075032 on provider checkpoint proc_lsst.multi INFO: cancelling 17075032 on provider checkpoint 2024-03-15 14:03:46 proc_lsst.multi:146 [INFO] found job 17074974 in provider escience_a40 proc_lsst.multi INFO: found job 17074974 in provider escience_a40 2024-03-15 14:03:46 proc_lsst.multi:201 [INFO] cancelling 17074974 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074974 on provider escience_a40 2024-03-15 14:03:46 proc_lsst.multi:146 [INFO] found job 17074836 in provider astro proc_lsst.multi INFO: found job 17074836 in provider astro 2024-03-15 14:03:46 proc_lsst.multi:201 [INFO] cancelling 17074836 on provider astro proc_lsst.multi INFO: cancelling 17074836 on provider astro 2024-03-15 14:03:46 proc_lsst.multi:146 [INFO] found job 17074834 in provider astro proc_lsst.multi INFO: found job 17074834 in provider astro 2024-03-15 14:03:46 proc_lsst.multi:201 [INFO] cancelling 17074834 on provider astro proc_lsst.multi INFO: cancelling 17074834 on provider astro 2024-03-15 14:03:47 proc_lsst.multi:146 [INFO] found job 17074839 in provider astro proc_lsst.multi INFO: found job 17074839 in provider astro 2024-03-15 14:03:47 proc_lsst.multi:201 [INFO] cancelling 17074839 on provider astro proc_lsst.multi INFO: cancelling 17074839 on provider astro 2024-03-15 14:03:47 proc_lsst.multi:146 [INFO] found job 17075031 in provider checkpoint proc_lsst.multi INFO: found job 17075031 in provider checkpoint 2024-03-15 14:03:47 proc_lsst.multi:201 [INFO] cancelling 17075031 on provider checkpoint proc_lsst.multi INFO: cancelling 17075031 on provider checkpoint 2024-03-15 14:03:48 proc_lsst.multi:146 [INFO] found job 17075030 in provider checkpoint proc_lsst.multi INFO: found job 17075030 in provider checkpoint 2024-03-15 14:03:48 proc_lsst.multi:201 [INFO] cancelling 17075030 on provider checkpoint proc_lsst.multi INFO: cancelling 17075030 on provider checkpoint 2024-03-15 14:03:48 proc_lsst.multi:146 [INFO] found job 17075003 in provider escience_a40 proc_lsst.multi INFO: found job 17075003 in provider escience_a40 2024-03-15 14:03:48 proc_lsst.multi:201 [INFO] cancelling 17075003 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075003 on provider escience_a40 2024-03-15 14:03:49 proc_lsst.multi:146 [INFO] found job 17074835 in provider astro proc_lsst.multi INFO: found job 17074835 in provider astro 2024-03-15 14:03:49 proc_lsst.multi:201 [INFO] cancelling 17074835 on provider astro proc_lsst.multi INFO: cancelling 17074835 on provider astro 2024-03-15 14:03:49 proc_lsst.multi:146 [INFO] found job 17074841 in provider astro proc_lsst.multi INFO: found job 17074841 in provider astro 2024-03-15 14:03:49 proc_lsst.multi:201 [INFO] cancelling 17074841 on provider astro proc_lsst.multi INFO: cancelling 17074841 on provider astro 2024-03-15 14:03:49 proc_lsst.multi:146 [INFO] found job 17074934 in provider escience_a40 proc_lsst.multi INFO: found job 17074934 in provider escience_a40 2024-03-15 14:03:49 proc_lsst.multi:201 [INFO] cancelling 17074934 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074934 on provider escience_a40 2024-03-15 14:03:50 proc_lsst.multi:146 [INFO] found job 17075022 in provider checkpoint proc_lsst.multi INFO: found job 17075022 in provider checkpoint 2024-03-15 14:03:50 proc_lsst.multi:201 [INFO] cancelling 17075022 on provider checkpoint proc_lsst.multi INFO: cancelling 17075022 on provider checkpoint 2024-03-15 14:03:50 proc_lsst.multi:146 [INFO] found job 17075046 in provider checkpoint proc_lsst.multi INFO: found job 17075046 in provider checkpoint 2024-03-15 14:03:50 proc_lsst.multi:201 [INFO] cancelling 17075046 on provider checkpoint proc_lsst.multi INFO: cancelling 17075046 on provider checkpoint 2024-03-15 14:03:51 proc_lsst.multi:146 [INFO] found job 17075052 in provider checkpoint proc_lsst.multi INFO: found job 17075052 in provider checkpoint 2024-03-15 14:03:51 proc_lsst.multi:201 [INFO] cancelling 17075052 on provider checkpoint proc_lsst.multi INFO: cancelling 17075052 on provider checkpoint 2024-03-15 14:03:51 proc_lsst.multi:146 [INFO] found job 17074895 in provider escience_a40 proc_lsst.multi INFO: found job 17074895 in provider escience_a40 2024-03-15 14:03:51 proc_lsst.multi:201 [INFO] cancelling 17074895 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074895 on provider escience_a40 2024-03-15 14:03:51 proc_lsst.multi:146 [INFO] found job 17075042 in provider checkpoint proc_lsst.multi INFO: found job 17075042 in provider checkpoint 2024-03-15 14:03:51 proc_lsst.multi:201 [INFO] cancelling 17075042 on provider checkpoint proc_lsst.multi INFO: cancelling 17075042 on provider checkpoint 2024-03-15 14:03:52 proc_lsst.multi:146 [INFO] found job 17074842 in provider astro proc_lsst.multi INFO: found job 17074842 in provider astro 2024-03-15 14:03:52 proc_lsst.multi:201 [INFO] cancelling 17074842 on provider astro proc_lsst.multi INFO: cancelling 17074842 on provider astro 2024-03-15 14:03:52 proc_lsst.multi:146 [INFO] found job 17074837 in provider astro proc_lsst.multi INFO: found job 17074837 in provider astro 2024-03-15 14:03:52 proc_lsst.multi:201 [INFO] cancelling 17074837 on provider astro proc_lsst.multi INFO: cancelling 17074837 on provider astro 2024-03-15 14:03:53 proc_lsst.multi:146 [INFO] found job 17075057 in provider checkpoint proc_lsst.multi INFO: found job 17075057 in provider checkpoint 2024-03-15 14:03:53 proc_lsst.multi:201 [INFO] cancelling 17075057 on provider checkpoint proc_lsst.multi INFO: cancelling 17075057 on provider checkpoint 2024-03-15 14:03:53 proc_lsst.multi:146 [INFO] found job 17074937 in provider escience_a40 proc_lsst.multi INFO: found job 17074937 in provider escience_a40 2024-03-15 14:03:53 proc_lsst.multi:201 [INFO] cancelling 17074937 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074937 on provider escience_a40 2024-03-15 14:03:53 proc_lsst.multi:146 [INFO] found job 17074933 in provider escience_a40 proc_lsst.multi INFO: found job 17074933 in provider escience_a40 2024-03-15 14:03:53 proc_lsst.multi:201 [INFO] cancelling 17074933 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074933 on provider escience_a40 2024-03-15 14:03:54 proc_lsst.multi:146 [INFO] found job 17074894 in provider escience_a40 proc_lsst.multi INFO: found job 17074894 in provider escience_a40 2024-03-15 14:03:54 proc_lsst.multi:201 [INFO] cancelling 17074894 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074894 on provider escience_a40 2024-03-15 14:03:54 proc_lsst.multi:146 [INFO] found job 17074935 in provider escience_a40 proc_lsst.multi INFO: found job 17074935 in provider escience_a40 2024-03-15 14:03:54 proc_lsst.multi:201 [INFO] cancelling 17074935 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074935 on provider escience_a40 2024-03-15 14:03:55 proc_lsst.multi:146 [INFO] found job 17075041 in provider checkpoint proc_lsst.multi INFO: found job 17075041 in provider checkpoint 2024-03-15 14:03:55 proc_lsst.multi:201 [INFO] cancelling 17075041 on provider checkpoint proc_lsst.multi INFO: cancelling 17075041 on provider checkpoint 2024-03-15 14:03:55 proc_lsst.multi:146 [INFO] found job 17075061 in provider checkpoint proc_lsst.multi INFO: found job 17075061 in provider checkpoint 2024-03-15 14:03:55 proc_lsst.multi:201 [INFO] cancelling 17075061 on provider checkpoint proc_lsst.multi INFO: cancelling 17075061 on provider checkpoint 2024-03-15 14:03:55 proc_lsst.multi:146 [INFO] found job 17075058 in provider checkpoint proc_lsst.multi INFO: found job 17075058 in provider checkpoint 2024-03-15 14:03:55 proc_lsst.multi:201 [INFO] cancelling 17075058 on provider checkpoint proc_lsst.multi INFO: cancelling 17075058 on provider checkpoint 2024-03-15 14:03:56 proc_lsst.multi:146 [INFO] found job 17075056 in provider checkpoint proc_lsst.multi INFO: found job 17075056 in provider checkpoint 2024-03-15 14:03:56 proc_lsst.multi:201 [INFO] cancelling 17075056 on provider checkpoint proc_lsst.multi INFO: cancelling 17075056 on provider checkpoint 2024-03-15 14:03:56 proc_lsst.multi:146 [INFO] found job 17075049 in provider checkpoint proc_lsst.multi INFO: found job 17075049 in provider checkpoint 2024-03-15 14:03:56 proc_lsst.multi:201 [INFO] cancelling 17075049 on provider checkpoint proc_lsst.multi INFO: cancelling 17075049 on provider checkpoint 2024-03-15 14:03:57 proc_lsst.multi:146 [INFO] found job 17074856 in provider astro proc_lsst.multi INFO: found job 17074856 in provider astro 2024-03-15 14:03:57 proc_lsst.multi:201 [INFO] cancelling 17074856 on provider astro proc_lsst.multi INFO: cancelling 17074856 on provider astro 2024-03-15 14:03:57 proc_lsst.multi:146 [INFO] found job 17074855 in provider astro proc_lsst.multi INFO: found job 17074855 in provider astro 2024-03-15 14:03:57 proc_lsst.multi:201 [INFO] cancelling 17074855 on provider astro proc_lsst.multi INFO: cancelling 17074855 on provider astro 2024-03-15 14:03:58 proc_lsst.multi:146 [INFO] found job 17074854 in provider astro proc_lsst.multi INFO: found job 17074854 in provider astro 2024-03-15 14:03:58 proc_lsst.multi:201 [INFO] cancelling 17074854 on provider astro proc_lsst.multi INFO: cancelling 17074854 on provider astro 2024-03-15 14:03:58 proc_lsst.multi:146 [INFO] found job 17075055 in provider checkpoint proc_lsst.multi INFO: found job 17075055 in provider checkpoint 2024-03-15 14:03:58 proc_lsst.multi:201 [INFO] cancelling 17075055 on provider checkpoint proc_lsst.multi INFO: cancelling 17075055 on provider checkpoint 2024-03-15 14:03:58 proc_lsst.multi:146 [INFO] found job 17075063 in provider checkpoint proc_lsst.multi INFO: found job 17075063 in provider checkpoint 2024-03-15 14:03:58 proc_lsst.multi:201 [INFO] cancelling 17075063 on provider checkpoint proc_lsst.multi INFO: cancelling 17075063 on provider checkpoint 2024-03-15 14:03:59 proc_lsst.multi:146 [INFO] found job 17074861 in provider astro proc_lsst.multi INFO: found job 17074861 in provider astro 2024-03-15 14:03:59 proc_lsst.multi:201 [INFO] cancelling 17074861 on provider astro proc_lsst.multi INFO: cancelling 17074861 on provider astro 2024-03-15 14:03:59 proc_lsst.multi:146 [INFO] found job 17074877 in provider astro proc_lsst.multi INFO: found job 17074877 in provider astro 2024-03-15 14:03:59 proc_lsst.multi:201 [INFO] cancelling 17074877 on provider astro proc_lsst.multi INFO: cancelling 17074877 on provider astro 2024-03-15 14:04:00 proc_lsst.multi:146 [INFO] found job 17074876 in provider astro proc_lsst.multi INFO: found job 17074876 in provider astro 2024-03-15 14:04:00 proc_lsst.multi:201 [INFO] cancelling 17074876 on provider astro proc_lsst.multi INFO: cancelling 17074876 on provider astro 2024-03-15 14:04:00 proc_lsst.multi:146 [INFO] found job 17074871 in provider astro proc_lsst.multi INFO: found job 17074871 in provider astro 2024-03-15 14:04:00 proc_lsst.multi:201 [INFO] cancelling 17074871 on provider astro proc_lsst.multi INFO: cancelling 17074871 on provider astro 2024-03-15 14:04:00 proc_lsst.multi:146 [INFO] found job 17075068 in provider checkpoint proc_lsst.multi INFO: found job 17075068 in provider checkpoint 2024-03-15 14:04:00 proc_lsst.multi:201 [INFO] cancelling 17075068 on provider checkpoint proc_lsst.multi INFO: cancelling 17075068 on provider checkpoint 2024-03-15 14:04:01 proc_lsst.multi:146 [INFO] found job 17075064 in provider checkpoint proc_lsst.multi INFO: found job 17075064 in provider checkpoint 2024-03-15 14:04:01 proc_lsst.multi:201 [INFO] cancelling 17075064 on provider checkpoint proc_lsst.multi INFO: cancelling 17075064 on provider checkpoint 2024-03-15 14:04:01 proc_lsst.multi:146 [INFO] found job 17074863 in provider astro proc_lsst.multi INFO: found job 17074863 in provider astro 2024-03-15 14:04:01 proc_lsst.multi:201 [INFO] cancelling 17074863 on provider astro proc_lsst.multi INFO: cancelling 17074863 on provider astro 2024-03-15 14:04:02 proc_lsst.multi:146 [INFO] found job 17074873 in provider astro proc_lsst.multi INFO: found job 17074873 in provider astro 2024-03-15 14:04:02 proc_lsst.multi:201 [INFO] cancelling 17074873 on provider astro proc_lsst.multi INFO: cancelling 17074873 on provider astro 2024-03-15 14:04:02 proc_lsst.multi:146 [INFO] found job 17075020 in provider escience_a40 proc_lsst.multi INFO: found job 17075020 in provider escience_a40 2024-03-15 14:04:02 proc_lsst.multi:201 [INFO] cancelling 17075020 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075020 on provider escience_a40 2024-03-15 14:04:03 proc_lsst.multi:146 [INFO] found job 17075033 in provider checkpoint proc_lsst.multi INFO: found job 17075033 in provider checkpoint 2024-03-15 14:04:03 proc_lsst.multi:201 [INFO] cancelling 17075033 on provider checkpoint proc_lsst.multi INFO: cancelling 17075033 on provider checkpoint 2024-03-15 14:04:03 proc_lsst.multi:146 [INFO] found job 17074853 in provider astro proc_lsst.multi INFO: found job 17074853 in provider astro 2024-03-15 14:04:03 proc_lsst.multi:201 [INFO] cancelling 17074853 on provider astro proc_lsst.multi INFO: cancelling 17074853 on provider astro 2024-03-15 14:04:03 proc_lsst.multi:146 [INFO] found job 17074870 in provider astro proc_lsst.multi INFO: found job 17074870 in provider astro 2024-03-15 14:04:03 proc_lsst.multi:201 [INFO] cancelling 17074870 on provider astro proc_lsst.multi INFO: cancelling 17074870 on provider astro 2024-03-15 14:04:04 proc_lsst.multi:146 [INFO] found job 17074850 in provider astro proc_lsst.multi INFO: found job 17074850 in provider astro 2024-03-15 14:04:04 proc_lsst.multi:201 [INFO] cancelling 17074850 on provider astro proc_lsst.multi INFO: cancelling 17074850 on provider astro 2024-03-15 14:04:04 proc_lsst.multi:146 [INFO] found job 17074889 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074889 in provider escience_rtx6k 2024-03-15 14:04:04 proc_lsst.multi:201 [INFO] cancelling 17074889 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074889 on provider escience_rtx6k 2024-03-15 14:04:05 proc_lsst.multi:146 [INFO] found job 17075034 in provider checkpoint proc_lsst.multi INFO: found job 17075034 in provider checkpoint 2024-03-15 14:04:05 proc_lsst.multi:201 [INFO] cancelling 17075034 on provider checkpoint proc_lsst.multi INFO: cancelling 17075034 on provider checkpoint 2024-03-15 14:04:05 proc_lsst.multi:146 [INFO] found job 17075050 in provider checkpoint proc_lsst.multi INFO: found job 17075050 in provider checkpoint 2024-03-15 14:04:05 proc_lsst.multi:201 [INFO] cancelling 17075050 on provider checkpoint proc_lsst.multi INFO: cancelling 17075050 on provider checkpoint 2024-03-15 14:04:05 proc_lsst.multi:146 [INFO] found job 17074940 in provider escience_a40 proc_lsst.multi INFO: found job 17074940 in provider escience_a40 2024-03-15 14:04:05 proc_lsst.multi:201 [INFO] cancelling 17074940 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074940 on provider escience_a40 2024-03-15 14:04:06 proc_lsst.multi:146 [INFO] found job 17074869 in provider astro proc_lsst.multi INFO: found job 17074869 in provider astro 2024-03-15 14:04:06 proc_lsst.multi:201 [INFO] cancelling 17074869 on provider astro proc_lsst.multi INFO: cancelling 17074869 on provider astro 2024-03-15 14:04:06 proc_lsst.multi:146 [INFO] found job 17075053 in provider checkpoint proc_lsst.multi INFO: found job 17075053 in provider checkpoint 2024-03-15 14:04:06 proc_lsst.multi:201 [INFO] cancelling 17075053 on provider checkpoint proc_lsst.multi INFO: cancelling 17075053 on provider checkpoint 2024-03-15 14:04:07 proc_lsst.multi:146 [INFO] found job 17074872 in provider astro proc_lsst.multi INFO: found job 17074872 in provider astro 2024-03-15 14:04:07 proc_lsst.multi:201 [INFO] cancelling 17074872 on provider astro proc_lsst.multi INFO: cancelling 17074872 on provider astro 2024-03-15 14:04:07 proc_lsst.multi:146 [INFO] found job 17075054 in provider checkpoint proc_lsst.multi INFO: found job 17075054 in provider checkpoint 2024-03-15 14:04:07 proc_lsst.multi:201 [INFO] cancelling 17075054 on provider checkpoint proc_lsst.multi INFO: cancelling 17075054 on provider checkpoint 2024-03-15 14:04:08 proc_lsst.multi:146 [INFO] found job 17074865 in provider astro proc_lsst.multi INFO: found job 17074865 in provider astro 2024-03-15 14:04:08 proc_lsst.multi:201 [INFO] cancelling 17074865 on provider astro proc_lsst.multi INFO: cancelling 17074865 on provider astro 2024-03-15 14:04:08 proc_lsst.multi:146 [INFO] found job 17074888 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074888 in provider escience_rtx6k 2024-03-15 14:04:08 proc_lsst.multi:201 [INFO] cancelling 17074888 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074888 on provider escience_rtx6k 2024-03-15 14:04:08 proc_lsst.multi:146 [INFO] found job 17074881 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074881 in provider escience_rtx6k 2024-03-15 14:04:08 proc_lsst.multi:201 [INFO] cancelling 17074881 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074881 on provider escience_rtx6k 2024-03-15 14:04:09 proc_lsst.multi:146 [INFO] found job 17075019 in provider escience_a40 proc_lsst.multi INFO: found job 17075019 in provider escience_a40 2024-03-15 14:04:09 proc_lsst.multi:201 [INFO] cancelling 17075019 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075019 on provider escience_a40 2024-03-15 14:04:09 proc_lsst.multi:146 [INFO] found job 17075021 in provider escience_a40 proc_lsst.multi INFO: found job 17075021 in provider escience_a40 2024-03-15 14:04:09 proc_lsst.multi:201 [INFO] cancelling 17075021 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075021 on provider escience_a40 2024-03-15 14:04:10 proc_lsst.multi:146 [INFO] found job 17074857 in provider astro proc_lsst.multi INFO: found job 17074857 in provider astro 2024-03-15 14:04:10 proc_lsst.multi:201 [INFO] cancelling 17074857 on provider astro proc_lsst.multi INFO: cancelling 17074857 on provider astro 2024-03-15 14:04:10 proc_lsst.multi:146 [INFO] found job 17074851 in provider astro proc_lsst.multi INFO: found job 17074851 in provider astro 2024-03-15 14:04:10 proc_lsst.multi:201 [INFO] cancelling 17074851 on provider astro proc_lsst.multi INFO: cancelling 17074851 on provider astro 2024-03-15 14:04:10 proc_lsst.multi:146 [INFO] found job 17074880 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074880 in provider escience_rtx6k 2024-03-15 14:04:10 proc_lsst.multi:201 [INFO] cancelling 17074880 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074880 on provider escience_rtx6k 2024-03-15 14:04:11 proc_lsst.multi:146 [INFO] found job 17074897 in provider escience_a40 proc_lsst.multi INFO: found job 17074897 in provider escience_a40 2024-03-15 14:04:11 proc_lsst.multi:201 [INFO] cancelling 17074897 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074897 on provider escience_a40 2024-03-15 14:04:11 proc_lsst.multi:146 [INFO] found job 17075018 in provider escience_a40 proc_lsst.multi INFO: found job 17075018 in provider escience_a40 2024-03-15 14:04:11 proc_lsst.multi:201 [INFO] cancelling 17075018 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075018 on provider escience_a40 2024-03-15 14:04:12 proc_lsst.multi:146 [INFO] found job 17074887 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074887 in provider escience_rtx6k 2024-03-15 14:04:12 proc_lsst.multi:201 [INFO] cancelling 17074887 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074887 on provider escience_rtx6k 2024-03-15 14:04:12 proc_lsst.multi:146 [INFO] found job 17074859 in provider astro proc_lsst.multi INFO: found job 17074859 in provider astro 2024-03-15 14:04:12 proc_lsst.multi:201 [INFO] cancelling 17074859 on provider astro proc_lsst.multi INFO: cancelling 17074859 on provider astro 2024-03-15 14:04:12 proc_lsst.multi:146 [INFO] found job 17074879 in provider astro proc_lsst.multi INFO: found job 17074879 in provider astro 2024-03-15 14:04:12 proc_lsst.multi:201 [INFO] cancelling 17074879 on provider astro proc_lsst.multi INFO: cancelling 17074879 on provider astro 2024-03-15 14:04:13 proc_lsst.multi:146 [INFO] found job 17074936 in provider escience_a40 proc_lsst.multi INFO: found job 17074936 in provider escience_a40 2024-03-15 14:04:13 proc_lsst.multi:201 [INFO] cancelling 17074936 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074936 on provider escience_a40 2024-03-15 14:04:13 proc_lsst.multi:146 [INFO] found job 17074843 in provider astro proc_lsst.multi INFO: found job 17074843 in provider astro 2024-03-15 14:04:13 proc_lsst.multi:201 [INFO] cancelling 17074843 on provider astro proc_lsst.multi INFO: cancelling 17074843 on provider astro 2024-03-15 14:04:14 proc_lsst.multi:146 [INFO] found job 17074942 in provider escience_a40 proc_lsst.multi INFO: found job 17074942 in provider escience_a40 2024-03-15 14:04:14 proc_lsst.multi:201 [INFO] cancelling 17074942 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074942 on provider escience_a40 2024-03-15 14:04:14 proc_lsst.multi:146 [INFO] found job 17074938 in provider escience_a40 proc_lsst.multi INFO: found job 17074938 in provider escience_a40 2024-03-15 14:04:14 proc_lsst.multi:201 [INFO] cancelling 17074938 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074938 on provider escience_a40 2024-03-15 14:04:15 proc_lsst.multi:146 [INFO] found job 17074852 in provider astro proc_lsst.multi INFO: found job 17074852 in provider astro 2024-03-15 14:04:15 proc_lsst.multi:201 [INFO] cancelling 17074852 on provider astro proc_lsst.multi INFO: cancelling 17074852 on provider astro parsl.dataflow.dflow INFO: Shutting down executor multi 2024-03-15 14:04:15 proc_lsst.multi:40 [INFO] Cancelling all provider resources proc_lsst.multi INFO: Cancelling all provider resources 2024-03-15 14:04:15 proc_lsst.multi:47 [INFO] new jobs since last cancel ['17075022', '17075030', '17075031', '17075032', '17075033', '17075034', '17075035', '17075038', '17075041', '17075042', '17075046', '17075047', '17075048', '17075049', '17075050', '17075052', '17075053', '17075054', '17075055', '17075056', '17075057', '17075058', '17075059', '17075060', '17075061', '17075062', '17075063', '17075064', '17075065', '17075066', '17075068', '17075069', '17075072', '17075073', '17075074', '17075075', '17075076', '17075078', '17075080', '17075081', '17075082', '17075083', '17075085', '17075086', '17075088', '17075090', '17075091', '17075092', '17075093', '17075094', '17075095', '17075096', '17075097', '17075098', '17075099', '17075100', '17075101', '17075102', '17075103', '17075109', '17075110', '17075111', '17075112', '17075113', '17075114', '17075115', '17075116', '17075117', '17075118', '17075121', '17075122', '17075124', '17075127', '17075128', '17075129', '17075131', '17075132', '17075133', '17075134', '17075135', '17075136', '17075137', '17075138', '17075139', '17075140', '17075141', '17075142', '17075143', '17075144', '17075145', '17075146', '17075147', '17075148', '17075149', '17075150', '17075151', '17075152', '17075153', '17075154', '17075155', '17075157', '17075159', '17075160', '17075161', '17075162', '17075163', '17075166', '17075167', '17075168', '17075171', '17075172', '17075173', '17075179', '17075180', '17075181', '17075182', '17075183', '17075185', '17075187', '17075188', '17075190', '17075191', '17075197', '17075200', '17075201', '17075202', '17075203', '17075204', '17075205', '17075206', '17075207', '17075208', '17075209', '17075210', '17075211', '17075212', '17075213', '17075214', '17075216', '17075217', '17075218', '17075219', '17075220', '17075221', '17075222', '17075223', '17075224', '17075225', '17075226', '17075227', '17075228', '17075229', '17075230', '17075231', '17075232', '17075233', '17075234', '17075238', '17075242', '17075245', '17075246', '17075247', '17075250', '17075254', '17075255', '17075256', '17075257', '17075258', '17075264', '17075265', '17075273', '17075274', '17075275', '17075276', '17075278', '17075279', '17075280', '17075281', '17075282', '17075283', '17075284', '17075285', '17075286', '17075287', '17075288', '17075289', '17075290', '17075293', '17075298', '17075299', '17075301', '17075306', '17075307', '17075308', '17075309', '17075310', '17075311', '17075312', '17075315', '17075318', '17075319', '17075320', '17075321', '17075326', '17075329', '17075330', '17075332', '17075338', '17075344', '17075345', '17075346', '17075347', '17075348', '17075351', '17075353', '17075354', '17075355', '17075357', '17075358', '17075359', '17075360', '17075364', '17075368', '17075372', '17075376', '17075379', '17075380', '17075381', '17075382', '17075384', '17075385', '17075386', '17075387', '17075388', '17075392', '17075394', '17075399', '17075400', '17075401', '17075405', '17075408', '17075417', '17075428', '17075430', '17075431', '17075432', '17075433', '17075434', '17075435', '17075436', '17075438', '17075440', '17075441', '17075443', '17075444', '17075446', '17074833', '17074834', '17074835', '17074836', '17074837', '17074838', '17074839', '17074841', '17074842', '17074843', '17074850', '17074851', '17074852', '17074853', '17074854', '17074855', '17074856', '17074857', '17074859', '17074861', '17074863', '17074865', '17074869', '17074870', '17074871', '17074872', '17074873', '17074876', '17074877', '17074879', '17074894', '17074895', '17074897', '17074898', '17074932', '17074933', '17074934', '17074935', '17074936', '17074937', '17074938', '17074939', '17074940', '17074942', '17074974', '17075003', '17075018', '17075019', '17075020', '17075021', '17074880', '17074881', '17074887', '17074888', '17074889', '91028'] proc_lsst.multi INFO: new jobs since last cancel ['17075022', '17075030', '17075031', '17075032', '17075033', '17075034', '17075035', '17075038', '17075041', '17075042', '17075046', '17075047', '17075048', '17075049', '17075050', '17075052', '17075053', '17075054', '17075055', '17075056', '17075057', '17075058', '17075059', '17075060', '17075061', '17075062', '17075063', '17075064', '17075065', '17075066', '17075068', '17075069', '17075072', '17075073', '17075074', '17075075', '17075076', '17075078', '17075080', '17075081', '17075082', '17075083', '17075085', '17075086', '17075088', '17075090', '17075091', '17075092', '17075093', '17075094', '17075095', '17075096', '17075097', '17075098', '17075099', '17075100', '17075101', '17075102', '17075103', '17075109', '17075110', '17075111', '17075112', '17075113', '17075114', '17075115', '17075116', '17075117', '17075118', '17075121', '17075122', '17075124', '17075127', '17075128', '17075129', '17075131', '17075132', '17075133', '17075134', '17075135', '17075136', '17075137', '17075138', '17075139', '17075140', '17075141', '17075142', '17075143', '17075144', '17075145', '17075146', '17075147', '17075148', '17075149', '17075150', '17075151', '17075152', '17075153', '17075154', '17075155', '17075157', '17075159', '17075160', '17075161', '17075162', '17075163', '17075166', '17075167', '17075168', '17075171', '17075172', '17075173', '17075179', '17075180', '17075181', '17075182', '17075183', '17075185', '17075187', '17075188', '17075190', '17075191', '17075197', '17075200', '17075201', '17075202', '17075203', '17075204', '17075205', '17075206', '17075207', '17075208', '17075209', '17075210', '17075211', '17075212', '17075213', '17075214', '17075216', '17075217', '17075218', '17075219', '17075220', '17075221', '17075222', '17075223', '17075224', '17075225', '17075226', '17075227', '17075228', '17075229', '17075230', '17075231', '17075232', '17075233', '17075234', '17075238', '17075242', '17075245', '17075246', '17075247', '17075250', '17075254', '17075255', '17075256', '17075257', '17075258', '17075264', '17075265', '17075273', '17075274', '17075275', '17075276', '17075278', '17075279', '17075280', '17075281', '17075282', '17075283', '17075284', '17075285', '17075286', '17075287', '17075288', '17075289', '17075290', '17075293', '17075298', '17075299', '17075301', '17075306', '17075307', '17075308', '17075309', '17075310', '17075311', '17075312', '17075315', '17075318', '17075319', '17075320', '17075321', '17075326', '17075329', '17075330', '17075332', '17075338', '17075344', '17075345', '17075346', '17075347', '17075348', '17075351', '17075353', '17075354', '17075355', '17075357', '17075358', '17075359', '17075360', '17075364', '17075368', '17075372', '17075376', '17075379', '17075380', '17075381', '17075382', '17075384', '17075385', '17075386', '17075387', '17075388', '17075392', '17075394', '17075399', '17075400', '17075401', '17075405', '17075408', '17075417', '17075428', '17075430', '17075431', '17075432', '17075433', '17075434', '17075435', '17075436', '17075438', '17075440', '17075441', '17075443', '17075444', '17075446', '17074833', '17074834', '17074835', '17074836', '17074837', '17074838', '17074839', '17074841', '17074842', '17074843', '17074850', '17074851', '17074852', '17074853', '17074854', '17074855', '17074856', '17074857', '17074859', '17074861', '17074863', '17074865', '17074869', '17074870', '17074871', '17074872', '17074873', '17074876', '17074877', '17074879', '17074894', '17074895', '17074897', '17074898', '17074932', '17074933', '17074934', '17074935', '17074936', '17074937', '17074938', '17074939', '17074940', '17074942', '17074974', '17075003', '17075018', '17075019', '17075020', '17075021', '17074880', '17074881', '17074887', '17074888', '17074889', '91028'] 2024-03-15 14:04:15 proc_lsst.multi:146 [INFO] found job 17075022 in provider checkpoint proc_lsst.multi INFO: found job 17075022 in provider checkpoint 2024-03-15 14:04:15 proc_lsst.multi:201 [INFO] cancelling 17075022 on provider checkpoint proc_lsst.multi INFO: cancelling 17075022 on provider checkpoint 2024-03-15 14:04:16 proc_lsst.multi:146 [INFO] found job 17075030 in provider checkpoint proc_lsst.multi INFO: found job 17075030 in provider checkpoint 2024-03-15 14:04:16 proc_lsst.multi:201 [INFO] cancelling 17075030 on provider checkpoint proc_lsst.multi INFO: cancelling 17075030 on provider checkpoint 2024-03-15 14:04:16 proc_lsst.multi:146 [INFO] found job 17075031 in provider checkpoint proc_lsst.multi INFO: found job 17075031 in provider checkpoint 2024-03-15 14:04:16 proc_lsst.multi:201 [INFO] cancelling 17075031 on provider checkpoint proc_lsst.multi INFO: cancelling 17075031 on provider checkpoint 2024-03-15 14:04:16 proc_lsst.multi:146 [INFO] found job 17075032 in provider checkpoint proc_lsst.multi INFO: found job 17075032 in provider checkpoint 2024-03-15 14:04:16 proc_lsst.multi:201 [INFO] cancelling 17075032 on provider checkpoint proc_lsst.multi INFO: cancelling 17075032 on provider checkpoint 2024-03-15 14:04:17 proc_lsst.multi:146 [INFO] found job 17075033 in provider checkpoint proc_lsst.multi INFO: found job 17075033 in provider checkpoint 2024-03-15 14:04:17 proc_lsst.multi:201 [INFO] cancelling 17075033 on provider checkpoint proc_lsst.multi INFO: cancelling 17075033 on provider checkpoint 2024-03-15 14:04:17 proc_lsst.multi:146 [INFO] found job 17075034 in provider checkpoint proc_lsst.multi INFO: found job 17075034 in provider checkpoint 2024-03-15 14:04:17 proc_lsst.multi:201 [INFO] cancelling 17075034 on provider checkpoint proc_lsst.multi INFO: cancelling 17075034 on provider checkpoint 2024-03-15 14:04:18 proc_lsst.multi:146 [INFO] found job 17075035 in provider checkpoint proc_lsst.multi INFO: found job 17075035 in provider checkpoint 2024-03-15 14:04:18 proc_lsst.multi:201 [INFO] cancelling 17075035 on provider checkpoint proc_lsst.multi INFO: cancelling 17075035 on provider checkpoint 2024-03-15 14:04:18 proc_lsst.multi:146 [INFO] found job 17075038 in provider checkpoint proc_lsst.multi INFO: found job 17075038 in provider checkpoint 2024-03-15 14:04:18 proc_lsst.multi:201 [INFO] cancelling 17075038 on provider checkpoint proc_lsst.multi INFO: cancelling 17075038 on provider checkpoint 2024-03-15 14:04:19 proc_lsst.multi:146 [INFO] found job 17075041 in provider checkpoint proc_lsst.multi INFO: found job 17075041 in provider checkpoint 2024-03-15 14:04:19 proc_lsst.multi:201 [INFO] cancelling 17075041 on provider checkpoint proc_lsst.multi INFO: cancelling 17075041 on provider checkpoint 2024-03-15 14:04:19 proc_lsst.multi:146 [INFO] found job 17075042 in provider checkpoint proc_lsst.multi INFO: found job 17075042 in provider checkpoint 2024-03-15 14:04:19 proc_lsst.multi:201 [INFO] cancelling 17075042 on provider checkpoint proc_lsst.multi INFO: cancelling 17075042 on provider checkpoint 2024-03-15 14:04:19 proc_lsst.multi:146 [INFO] found job 17075046 in provider checkpoint proc_lsst.multi INFO: found job 17075046 in provider checkpoint 2024-03-15 14:04:19 proc_lsst.multi:201 [INFO] cancelling 17075046 on provider checkpoint proc_lsst.multi INFO: cancelling 17075046 on provider checkpoint 2024-03-15 14:04:20 proc_lsst.multi:146 [INFO] found job 17075047 in provider checkpoint proc_lsst.multi INFO: found job 17075047 in provider checkpoint 2024-03-15 14:04:20 proc_lsst.multi:201 [INFO] cancelling 17075047 on provider checkpoint proc_lsst.multi INFO: cancelling 17075047 on provider checkpoint 2024-03-15 14:04:20 proc_lsst.multi:146 [INFO] found job 17075048 in provider checkpoint proc_lsst.multi INFO: found job 17075048 in provider checkpoint 2024-03-15 14:04:20 proc_lsst.multi:201 [INFO] cancelling 17075048 on provider checkpoint proc_lsst.multi INFO: cancelling 17075048 on provider checkpoint 2024-03-15 14:04:21 proc_lsst.multi:146 [INFO] found job 17075049 in provider checkpoint proc_lsst.multi INFO: found job 17075049 in provider checkpoint 2024-03-15 14:04:21 proc_lsst.multi:201 [INFO] cancelling 17075049 on provider checkpoint proc_lsst.multi INFO: cancelling 17075049 on provider checkpoint 2024-03-15 14:04:21 proc_lsst.multi:146 [INFO] found job 17075050 in provider checkpoint proc_lsst.multi INFO: found job 17075050 in provider checkpoint 2024-03-15 14:04:21 proc_lsst.multi:201 [INFO] cancelling 17075050 on provider checkpoint proc_lsst.multi INFO: cancelling 17075050 on provider checkpoint 2024-03-15 14:04:21 proc_lsst.multi:146 [INFO] found job 17075052 in provider checkpoint proc_lsst.multi INFO: found job 17075052 in provider checkpoint 2024-03-15 14:04:21 proc_lsst.multi:201 [INFO] cancelling 17075052 on provider checkpoint proc_lsst.multi INFO: cancelling 17075052 on provider checkpoint 2024-03-15 14:04:22 proc_lsst.multi:146 [INFO] found job 17075053 in provider checkpoint proc_lsst.multi INFO: found job 17075053 in provider checkpoint 2024-03-15 14:04:22 proc_lsst.multi:201 [INFO] cancelling 17075053 on provider checkpoint proc_lsst.multi INFO: cancelling 17075053 on provider checkpoint 2024-03-15 14:04:22 proc_lsst.multi:146 [INFO] found job 17075054 in provider checkpoint proc_lsst.multi INFO: found job 17075054 in provider checkpoint 2024-03-15 14:04:22 proc_lsst.multi:201 [INFO] cancelling 17075054 on provider checkpoint proc_lsst.multi INFO: cancelling 17075054 on provider checkpoint 2024-03-15 14:04:23 proc_lsst.multi:146 [INFO] found job 17075055 in provider checkpoint proc_lsst.multi INFO: found job 17075055 in provider checkpoint 2024-03-15 14:04:23 proc_lsst.multi:201 [INFO] cancelling 17075055 on provider checkpoint proc_lsst.multi INFO: cancelling 17075055 on provider checkpoint 2024-03-15 14:04:23 proc_lsst.multi:146 [INFO] found job 17075056 in provider checkpoint proc_lsst.multi INFO: found job 17075056 in provider checkpoint 2024-03-15 14:04:23 proc_lsst.multi:201 [INFO] cancelling 17075056 on provider checkpoint proc_lsst.multi INFO: cancelling 17075056 on provider checkpoint 2024-03-15 14:04:23 proc_lsst.multi:146 [INFO] found job 17075057 in provider checkpoint proc_lsst.multi INFO: found job 17075057 in provider checkpoint 2024-03-15 14:04:23 proc_lsst.multi:201 [INFO] cancelling 17075057 on provider checkpoint proc_lsst.multi INFO: cancelling 17075057 on provider checkpoint 2024-03-15 14:04:24 proc_lsst.multi:146 [INFO] found job 17075058 in provider checkpoint proc_lsst.multi INFO: found job 17075058 in provider checkpoint 2024-03-15 14:04:24 proc_lsst.multi:201 [INFO] cancelling 17075058 on provider checkpoint proc_lsst.multi INFO: cancelling 17075058 on provider checkpoint 2024-03-15 14:04:24 proc_lsst.multi:146 [INFO] found job 17075059 in provider checkpoint proc_lsst.multi INFO: found job 17075059 in provider checkpoint 2024-03-15 14:04:24 proc_lsst.multi:201 [INFO] cancelling 17075059 on provider checkpoint proc_lsst.multi INFO: cancelling 17075059 on provider checkpoint 2024-03-15 14:04:25 proc_lsst.multi:146 [INFO] found job 17075060 in provider checkpoint proc_lsst.multi INFO: found job 17075060 in provider checkpoint 2024-03-15 14:04:25 proc_lsst.multi:201 [INFO] cancelling 17075060 on provider checkpoint proc_lsst.multi INFO: cancelling 17075060 on provider checkpoint 2024-03-15 14:04:25 proc_lsst.multi:146 [INFO] found job 17075061 in provider checkpoint proc_lsst.multi INFO: found job 17075061 in provider checkpoint 2024-03-15 14:04:25 proc_lsst.multi:201 [INFO] cancelling 17075061 on provider checkpoint proc_lsst.multi INFO: cancelling 17075061 on provider checkpoint 2024-03-15 14:04:25 proc_lsst.multi:146 [INFO] found job 17075062 in provider checkpoint proc_lsst.multi INFO: found job 17075062 in provider checkpoint 2024-03-15 14:04:25 proc_lsst.multi:201 [INFO] cancelling 17075062 on provider checkpoint proc_lsst.multi INFO: cancelling 17075062 on provider checkpoint 2024-03-15 14:04:26 proc_lsst.multi:146 [INFO] found job 17075063 in provider checkpoint proc_lsst.multi INFO: found job 17075063 in provider checkpoint 2024-03-15 14:04:26 proc_lsst.multi:201 [INFO] cancelling 17075063 on provider checkpoint proc_lsst.multi INFO: cancelling 17075063 on provider checkpoint 2024-03-15 14:04:26 proc_lsst.multi:146 [INFO] found job 17075064 in provider checkpoint proc_lsst.multi INFO: found job 17075064 in provider checkpoint 2024-03-15 14:04:26 proc_lsst.multi:201 [INFO] cancelling 17075064 on provider checkpoint proc_lsst.multi INFO: cancelling 17075064 on provider checkpoint 2024-03-15 14:04:27 proc_lsst.multi:146 [INFO] found job 17075065 in provider checkpoint proc_lsst.multi INFO: found job 17075065 in provider checkpoint 2024-03-15 14:04:27 proc_lsst.multi:201 [INFO] cancelling 17075065 on provider checkpoint proc_lsst.multi INFO: cancelling 17075065 on provider checkpoint 2024-03-15 14:04:27 proc_lsst.multi:146 [INFO] found job 17075066 in provider checkpoint proc_lsst.multi INFO: found job 17075066 in provider checkpoint 2024-03-15 14:04:27 proc_lsst.multi:201 [INFO] cancelling 17075066 on provider checkpoint proc_lsst.multi INFO: cancelling 17075066 on provider checkpoint 2024-03-15 14:04:28 proc_lsst.multi:146 [INFO] found job 17075068 in provider checkpoint proc_lsst.multi INFO: found job 17075068 in provider checkpoint 2024-03-15 14:04:28 proc_lsst.multi:201 [INFO] cancelling 17075068 on provider checkpoint proc_lsst.multi INFO: cancelling 17075068 on provider checkpoint 2024-03-15 14:04:28 proc_lsst.multi:146 [INFO] found job 17075069 in provider checkpoint proc_lsst.multi INFO: found job 17075069 in provider checkpoint 2024-03-15 14:04:28 proc_lsst.multi:201 [INFO] cancelling 17075069 on provider checkpoint proc_lsst.multi INFO: cancelling 17075069 on provider checkpoint 2024-03-15 14:04:28 proc_lsst.multi:146 [INFO] found job 17075072 in provider checkpoint proc_lsst.multi INFO: found job 17075072 in provider checkpoint 2024-03-15 14:04:28 proc_lsst.multi:201 [INFO] cancelling 17075072 on provider checkpoint proc_lsst.multi INFO: cancelling 17075072 on provider checkpoint 2024-03-15 14:04:29 proc_lsst.multi:146 [INFO] found job 17075073 in provider checkpoint proc_lsst.multi INFO: found job 17075073 in provider checkpoint 2024-03-15 14:04:29 proc_lsst.multi:201 [INFO] cancelling 17075073 on provider checkpoint proc_lsst.multi INFO: cancelling 17075073 on provider checkpoint 2024-03-15 14:04:29 proc_lsst.multi:146 [INFO] found job 17075074 in provider checkpoint proc_lsst.multi INFO: found job 17075074 in provider checkpoint 2024-03-15 14:04:29 proc_lsst.multi:201 [INFO] cancelling 17075074 on provider checkpoint proc_lsst.multi INFO: cancelling 17075074 on provider checkpoint 2024-03-15 14:04:30 proc_lsst.multi:146 [INFO] found job 17075075 in provider checkpoint proc_lsst.multi INFO: found job 17075075 in provider checkpoint 2024-03-15 14:04:30 proc_lsst.multi:201 [INFO] cancelling 17075075 on provider checkpoint proc_lsst.multi INFO: cancelling 17075075 on provider checkpoint 2024-03-15 14:04:30 proc_lsst.multi:146 [INFO] found job 17075076 in provider checkpoint proc_lsst.multi INFO: found job 17075076 in provider checkpoint 2024-03-15 14:04:30 proc_lsst.multi:201 [INFO] cancelling 17075076 on provider checkpoint proc_lsst.multi INFO: cancelling 17075076 on provider checkpoint 2024-03-15 14:04:30 proc_lsst.multi:146 [INFO] found job 17075078 in provider checkpoint proc_lsst.multi INFO: found job 17075078 in provider checkpoint 2024-03-15 14:04:30 proc_lsst.multi:201 [INFO] cancelling 17075078 on provider checkpoint proc_lsst.multi INFO: cancelling 17075078 on provider checkpoint 2024-03-15 14:04:31 proc_lsst.multi:146 [INFO] found job 17075080 in provider checkpoint proc_lsst.multi INFO: found job 17075080 in provider checkpoint 2024-03-15 14:04:31 proc_lsst.multi:201 [INFO] cancelling 17075080 on provider checkpoint proc_lsst.multi INFO: cancelling 17075080 on provider checkpoint 2024-03-15 14:04:31 proc_lsst.multi:146 [INFO] found job 17075081 in provider checkpoint proc_lsst.multi INFO: found job 17075081 in provider checkpoint 2024-03-15 14:04:31 proc_lsst.multi:201 [INFO] cancelling 17075081 on provider checkpoint proc_lsst.multi INFO: cancelling 17075081 on provider checkpoint 2024-03-15 14:04:32 proc_lsst.multi:146 [INFO] found job 17075082 in provider checkpoint proc_lsst.multi INFO: found job 17075082 in provider checkpoint 2024-03-15 14:04:32 proc_lsst.multi:201 [INFO] cancelling 17075082 on provider checkpoint proc_lsst.multi INFO: cancelling 17075082 on provider checkpoint 2024-03-15 14:04:32 proc_lsst.multi:146 [INFO] found job 17075083 in provider checkpoint proc_lsst.multi INFO: found job 17075083 in provider checkpoint 2024-03-15 14:04:32 proc_lsst.multi:201 [INFO] cancelling 17075083 on provider checkpoint proc_lsst.multi INFO: cancelling 17075083 on provider checkpoint 2024-03-15 14:04:32 proc_lsst.multi:146 [INFO] found job 17075085 in provider checkpoint proc_lsst.multi INFO: found job 17075085 in provider checkpoint 2024-03-15 14:04:32 proc_lsst.multi:201 [INFO] cancelling 17075085 on provider checkpoint proc_lsst.multi INFO: cancelling 17075085 on provider checkpoint 2024-03-15 14:04:33 proc_lsst.multi:146 [INFO] found job 17075086 in provider checkpoint proc_lsst.multi INFO: found job 17075086 in provider checkpoint 2024-03-15 14:04:33 proc_lsst.multi:201 [INFO] cancelling 17075086 on provider checkpoint proc_lsst.multi INFO: cancelling 17075086 on provider checkpoint 2024-03-15 14:04:33 proc_lsst.multi:146 [INFO] found job 17075088 in provider checkpoint proc_lsst.multi INFO: found job 17075088 in provider checkpoint 2024-03-15 14:04:33 proc_lsst.multi:201 [INFO] cancelling 17075088 on provider checkpoint proc_lsst.multi INFO: cancelling 17075088 on provider checkpoint 2024-03-15 14:04:34 proc_lsst.multi:146 [INFO] found job 17075090 in provider checkpoint proc_lsst.multi INFO: found job 17075090 in provider checkpoint 2024-03-15 14:04:34 proc_lsst.multi:201 [INFO] cancelling 17075090 on provider checkpoint proc_lsst.multi INFO: cancelling 17075090 on provider checkpoint 2024-03-15 14:04:34 proc_lsst.multi:146 [INFO] found job 17075091 in provider checkpoint proc_lsst.multi INFO: found job 17075091 in provider checkpoint 2024-03-15 14:04:34 proc_lsst.multi:201 [INFO] cancelling 17075091 on provider checkpoint proc_lsst.multi INFO: cancelling 17075091 on provider checkpoint 2024-03-15 14:04:34 proc_lsst.multi:146 [INFO] found job 17075092 in provider checkpoint proc_lsst.multi INFO: found job 17075092 in provider checkpoint 2024-03-15 14:04:34 proc_lsst.multi:201 [INFO] cancelling 17075092 on provider checkpoint proc_lsst.multi INFO: cancelling 17075092 on provider checkpoint 2024-03-15 14:04:35 proc_lsst.multi:146 [INFO] found job 17075093 in provider checkpoint proc_lsst.multi INFO: found job 17075093 in provider checkpoint 2024-03-15 14:04:35 proc_lsst.multi:201 [INFO] cancelling 17075093 on provider checkpoint proc_lsst.multi INFO: cancelling 17075093 on provider checkpoint 2024-03-15 14:04:35 proc_lsst.multi:146 [INFO] found job 17075094 in provider checkpoint proc_lsst.multi INFO: found job 17075094 in provider checkpoint 2024-03-15 14:04:35 proc_lsst.multi:201 [INFO] cancelling 17075094 on provider checkpoint proc_lsst.multi INFO: cancelling 17075094 on provider checkpoint 2024-03-15 14:04:35 proc_lsst.multi:146 [INFO] found job 17075095 in provider checkpoint proc_lsst.multi INFO: found job 17075095 in provider checkpoint 2024-03-15 14:04:35 proc_lsst.multi:201 [INFO] cancelling 17075095 on provider checkpoint proc_lsst.multi INFO: cancelling 17075095 on provider checkpoint 2024-03-15 14:04:36 proc_lsst.multi:146 [INFO] found job 17075096 in provider checkpoint proc_lsst.multi INFO: found job 17075096 in provider checkpoint 2024-03-15 14:04:36 proc_lsst.multi:201 [INFO] cancelling 17075096 on provider checkpoint proc_lsst.multi INFO: cancelling 17075096 on provider checkpoint 2024-03-15 14:04:36 proc_lsst.multi:146 [INFO] found job 17075097 in provider checkpoint proc_lsst.multi INFO: found job 17075097 in provider checkpoint 2024-03-15 14:04:36 proc_lsst.multi:201 [INFO] cancelling 17075097 on provider checkpoint proc_lsst.multi INFO: cancelling 17075097 on provider checkpoint 2024-03-15 14:04:36 proc_lsst.multi:146 [INFO] found job 17075098 in provider checkpoint proc_lsst.multi INFO: found job 17075098 in provider checkpoint 2024-03-15 14:04:36 proc_lsst.multi:201 [INFO] cancelling 17075098 on provider checkpoint proc_lsst.multi INFO: cancelling 17075098 on provider checkpoint 2024-03-15 14:04:37 proc_lsst.multi:146 [INFO] found job 17075099 in provider checkpoint proc_lsst.multi INFO: found job 17075099 in provider checkpoint 2024-03-15 14:04:37 proc_lsst.multi:201 [INFO] cancelling 17075099 on provider checkpoint proc_lsst.multi INFO: cancelling 17075099 on provider checkpoint 2024-03-15 14:04:37 proc_lsst.multi:146 [INFO] found job 17075100 in provider checkpoint proc_lsst.multi INFO: found job 17075100 in provider checkpoint 2024-03-15 14:04:37 proc_lsst.multi:201 [INFO] cancelling 17075100 on provider checkpoint proc_lsst.multi INFO: cancelling 17075100 on provider checkpoint 2024-03-15 14:04:38 proc_lsst.multi:146 [INFO] found job 17075101 in provider checkpoint proc_lsst.multi INFO: found job 17075101 in provider checkpoint 2024-03-15 14:04:38 proc_lsst.multi:201 [INFO] cancelling 17075101 on provider checkpoint proc_lsst.multi INFO: cancelling 17075101 on provider checkpoint 2024-03-15 14:04:38 proc_lsst.multi:146 [INFO] found job 17075102 in provider checkpoint proc_lsst.multi INFO: found job 17075102 in provider checkpoint 2024-03-15 14:04:38 proc_lsst.multi:201 [INFO] cancelling 17075102 on provider checkpoint proc_lsst.multi INFO: cancelling 17075102 on provider checkpoint 2024-03-15 14:04:38 proc_lsst.multi:146 [INFO] found job 17075103 in provider checkpoint proc_lsst.multi INFO: found job 17075103 in provider checkpoint 2024-03-15 14:04:38 proc_lsst.multi:201 [INFO] cancelling 17075103 on provider checkpoint proc_lsst.multi INFO: cancelling 17075103 on provider checkpoint 2024-03-15 14:04:39 proc_lsst.multi:146 [INFO] found job 17075109 in provider checkpoint proc_lsst.multi INFO: found job 17075109 in provider checkpoint 2024-03-15 14:04:39 proc_lsst.multi:201 [INFO] cancelling 17075109 on provider checkpoint proc_lsst.multi INFO: cancelling 17075109 on provider checkpoint 2024-03-15 14:04:39 proc_lsst.multi:146 [INFO] found job 17075110 in provider checkpoint proc_lsst.multi INFO: found job 17075110 in provider checkpoint 2024-03-15 14:04:39 proc_lsst.multi:201 [INFO] cancelling 17075110 on provider checkpoint proc_lsst.multi INFO: cancelling 17075110 on provider checkpoint 2024-03-15 14:04:40 proc_lsst.multi:146 [INFO] found job 17075111 in provider checkpoint proc_lsst.multi INFO: found job 17075111 in provider checkpoint 2024-03-15 14:04:40 proc_lsst.multi:201 [INFO] cancelling 17075111 on provider checkpoint proc_lsst.multi INFO: cancelling 17075111 on provider checkpoint 2024-03-15 14:04:40 proc_lsst.multi:146 [INFO] found job 17075112 in provider checkpoint proc_lsst.multi INFO: found job 17075112 in provider checkpoint 2024-03-15 14:04:40 proc_lsst.multi:201 [INFO] cancelling 17075112 on provider checkpoint proc_lsst.multi INFO: cancelling 17075112 on provider checkpoint 2024-03-15 14:04:40 proc_lsst.multi:146 [INFO] found job 17075113 in provider checkpoint proc_lsst.multi INFO: found job 17075113 in provider checkpoint 2024-03-15 14:04:40 proc_lsst.multi:201 [INFO] cancelling 17075113 on provider checkpoint proc_lsst.multi INFO: cancelling 17075113 on provider checkpoint 2024-03-15 14:04:41 proc_lsst.multi:146 [INFO] found job 17075114 in provider checkpoint proc_lsst.multi INFO: found job 17075114 in provider checkpoint 2024-03-15 14:04:41 proc_lsst.multi:201 [INFO] cancelling 17075114 on provider checkpoint proc_lsst.multi INFO: cancelling 17075114 on provider checkpoint 2024-03-15 14:04:41 proc_lsst.multi:146 [INFO] found job 17075115 in provider checkpoint proc_lsst.multi INFO: found job 17075115 in provider checkpoint 2024-03-15 14:04:41 proc_lsst.multi:201 [INFO] cancelling 17075115 on provider checkpoint proc_lsst.multi INFO: cancelling 17075115 on provider checkpoint 2024-03-15 14:04:42 proc_lsst.multi:146 [INFO] found job 17075116 in provider checkpoint proc_lsst.multi INFO: found job 17075116 in provider checkpoint 2024-03-15 14:04:42 proc_lsst.multi:201 [INFO] cancelling 17075116 on provider checkpoint proc_lsst.multi INFO: cancelling 17075116 on provider checkpoint 2024-03-15 14:04:42 proc_lsst.multi:146 [INFO] found job 17075117 in provider checkpoint proc_lsst.multi INFO: found job 17075117 in provider checkpoint 2024-03-15 14:04:42 proc_lsst.multi:201 [INFO] cancelling 17075117 on provider checkpoint proc_lsst.multi INFO: cancelling 17075117 on provider checkpoint 2024-03-15 14:04:42 proc_lsst.multi:146 [INFO] found job 17075118 in provider checkpoint proc_lsst.multi INFO: found job 17075118 in provider checkpoint 2024-03-15 14:04:42 proc_lsst.multi:201 [INFO] cancelling 17075118 on provider checkpoint proc_lsst.multi INFO: cancelling 17075118 on provider checkpoint 2024-03-15 14:04:43 proc_lsst.multi:146 [INFO] found job 17075121 in provider checkpoint proc_lsst.multi INFO: found job 17075121 in provider checkpoint 2024-03-15 14:04:43 proc_lsst.multi:201 [INFO] cancelling 17075121 on provider checkpoint proc_lsst.multi INFO: cancelling 17075121 on provider checkpoint 2024-03-15 14:04:43 proc_lsst.multi:146 [INFO] found job 17075122 in provider checkpoint proc_lsst.multi INFO: found job 17075122 in provider checkpoint 2024-03-15 14:04:43 proc_lsst.multi:201 [INFO] cancelling 17075122 on provider checkpoint proc_lsst.multi INFO: cancelling 17075122 on provider checkpoint 2024-03-15 14:04:44 proc_lsst.multi:146 [INFO] found job 17075124 in provider checkpoint proc_lsst.multi INFO: found job 17075124 in provider checkpoint 2024-03-15 14:04:44 proc_lsst.multi:201 [INFO] cancelling 17075124 on provider checkpoint proc_lsst.multi INFO: cancelling 17075124 on provider checkpoint 2024-03-15 14:04:44 proc_lsst.multi:146 [INFO] found job 17075127 in provider checkpoint proc_lsst.multi INFO: found job 17075127 in provider checkpoint 2024-03-15 14:04:44 proc_lsst.multi:201 [INFO] cancelling 17075127 on provider checkpoint proc_lsst.multi INFO: cancelling 17075127 on provider checkpoint 2024-03-15 14:04:44 proc_lsst.multi:146 [INFO] found job 17075128 in provider checkpoint proc_lsst.multi INFO: found job 17075128 in provider checkpoint 2024-03-15 14:04:44 proc_lsst.multi:201 [INFO] cancelling 17075128 on provider checkpoint proc_lsst.multi INFO: cancelling 17075128 on provider checkpoint 2024-03-15 14:04:45 proc_lsst.multi:146 [INFO] found job 17075129 in provider checkpoint proc_lsst.multi INFO: found job 17075129 in provider checkpoint 2024-03-15 14:04:45 proc_lsst.multi:201 [INFO] cancelling 17075129 on provider checkpoint proc_lsst.multi INFO: cancelling 17075129 on provider checkpoint 2024-03-15 14:04:45 proc_lsst.multi:146 [INFO] found job 17075131 in provider checkpoint proc_lsst.multi INFO: found job 17075131 in provider checkpoint 2024-03-15 14:04:45 proc_lsst.multi:201 [INFO] cancelling 17075131 on provider checkpoint proc_lsst.multi INFO: cancelling 17075131 on provider checkpoint 2024-03-15 14:04:46 proc_lsst.multi:146 [INFO] found job 17075132 in provider checkpoint proc_lsst.multi INFO: found job 17075132 in provider checkpoint 2024-03-15 14:04:46 proc_lsst.multi:201 [INFO] cancelling 17075132 on provider checkpoint proc_lsst.multi INFO: cancelling 17075132 on provider checkpoint 2024-03-15 14:04:46 proc_lsst.multi:146 [INFO] found job 17075133 in provider checkpoint proc_lsst.multi INFO: found job 17075133 in provider checkpoint 2024-03-15 14:04:46 proc_lsst.multi:201 [INFO] cancelling 17075133 on provider checkpoint proc_lsst.multi INFO: cancelling 17075133 on provider checkpoint 2024-03-15 14:04:46 proc_lsst.multi:146 [INFO] found job 17075134 in provider checkpoint proc_lsst.multi INFO: found job 17075134 in provider checkpoint 2024-03-15 14:04:46 proc_lsst.multi:201 [INFO] cancelling 17075134 on provider checkpoint proc_lsst.multi INFO: cancelling 17075134 on provider checkpoint 2024-03-15 14:04:47 proc_lsst.multi:146 [INFO] found job 17075135 in provider checkpoint proc_lsst.multi INFO: found job 17075135 in provider checkpoint 2024-03-15 14:04:47 proc_lsst.multi:201 [INFO] cancelling 17075135 on provider checkpoint proc_lsst.multi INFO: cancelling 17075135 on provider checkpoint 2024-03-15 14:04:47 proc_lsst.multi:146 [INFO] found job 17075136 in provider checkpoint proc_lsst.multi INFO: found job 17075136 in provider checkpoint 2024-03-15 14:04:47 proc_lsst.multi:201 [INFO] cancelling 17075136 on provider checkpoint proc_lsst.multi INFO: cancelling 17075136 on provider checkpoint 2024-03-15 14:04:48 proc_lsst.multi:146 [INFO] found job 17075137 in provider checkpoint proc_lsst.multi INFO: found job 17075137 in provider checkpoint 2024-03-15 14:04:48 proc_lsst.multi:201 [INFO] cancelling 17075137 on provider checkpoint proc_lsst.multi INFO: cancelling 17075137 on provider checkpoint 2024-03-15 14:04:48 proc_lsst.multi:146 [INFO] found job 17075138 in provider checkpoint proc_lsst.multi INFO: found job 17075138 in provider checkpoint 2024-03-15 14:04:48 proc_lsst.multi:201 [INFO] cancelling 17075138 on provider checkpoint proc_lsst.multi INFO: cancelling 17075138 on provider checkpoint 2024-03-15 14:04:48 proc_lsst.multi:146 [INFO] found job 17075139 in provider checkpoint proc_lsst.multi INFO: found job 17075139 in provider checkpoint 2024-03-15 14:04:48 proc_lsst.multi:201 [INFO] cancelling 17075139 on provider checkpoint proc_lsst.multi INFO: cancelling 17075139 on provider checkpoint 2024-03-15 14:04:49 proc_lsst.multi:146 [INFO] found job 17075140 in provider checkpoint proc_lsst.multi INFO: found job 17075140 in provider checkpoint 2024-03-15 14:04:49 proc_lsst.multi:201 [INFO] cancelling 17075140 on provider checkpoint proc_lsst.multi INFO: cancelling 17075140 on provider checkpoint 2024-03-15 14:04:49 proc_lsst.multi:146 [INFO] found job 17075141 in provider checkpoint proc_lsst.multi INFO: found job 17075141 in provider checkpoint 2024-03-15 14:04:49 proc_lsst.multi:201 [INFO] cancelling 17075141 on provider checkpoint proc_lsst.multi INFO: cancelling 17075141 on provider checkpoint 2024-03-15 14:04:50 proc_lsst.multi:146 [INFO] found job 17075142 in provider checkpoint proc_lsst.multi INFO: found job 17075142 in provider checkpoint 2024-03-15 14:04:50 proc_lsst.multi:201 [INFO] cancelling 17075142 on provider checkpoint proc_lsst.multi INFO: cancelling 17075142 on provider checkpoint 2024-03-15 14:04:50 proc_lsst.multi:146 [INFO] found job 17075143 in provider checkpoint proc_lsst.multi INFO: found job 17075143 in provider checkpoint 2024-03-15 14:04:50 proc_lsst.multi:201 [INFO] cancelling 17075143 on provider checkpoint proc_lsst.multi INFO: cancelling 17075143 on provider checkpoint 2024-03-15 14:04:50 proc_lsst.multi:146 [INFO] found job 17075144 in provider checkpoint proc_lsst.multi INFO: found job 17075144 in provider checkpoint 2024-03-15 14:04:50 proc_lsst.multi:201 [INFO] cancelling 17075144 on provider checkpoint proc_lsst.multi INFO: cancelling 17075144 on provider checkpoint 2024-03-15 14:04:51 proc_lsst.multi:146 [INFO] found job 17075145 in provider checkpoint proc_lsst.multi INFO: found job 17075145 in provider checkpoint 2024-03-15 14:04:51 proc_lsst.multi:201 [INFO] cancelling 17075145 on provider checkpoint proc_lsst.multi INFO: cancelling 17075145 on provider checkpoint 2024-03-15 14:04:51 proc_lsst.multi:146 [INFO] found job 17075146 in provider checkpoint proc_lsst.multi INFO: found job 17075146 in provider checkpoint 2024-03-15 14:04:51 proc_lsst.multi:201 [INFO] cancelling 17075146 on provider checkpoint proc_lsst.multi INFO: cancelling 17075146 on provider checkpoint 2024-03-15 14:04:52 proc_lsst.multi:146 [INFO] found job 17075147 in provider checkpoint proc_lsst.multi INFO: found job 17075147 in provider checkpoint 2024-03-15 14:04:52 proc_lsst.multi:201 [INFO] cancelling 17075147 on provider checkpoint proc_lsst.multi INFO: cancelling 17075147 on provider checkpoint 2024-03-15 14:04:52 proc_lsst.multi:146 [INFO] found job 17075148 in provider checkpoint proc_lsst.multi INFO: found job 17075148 in provider checkpoint 2024-03-15 14:04:52 proc_lsst.multi:201 [INFO] cancelling 17075148 on provider checkpoint proc_lsst.multi INFO: cancelling 17075148 on provider checkpoint 2024-03-15 14:04:52 proc_lsst.multi:146 [INFO] found job 17075149 in provider checkpoint proc_lsst.multi INFO: found job 17075149 in provider checkpoint 2024-03-15 14:04:52 proc_lsst.multi:201 [INFO] cancelling 17075149 on provider checkpoint proc_lsst.multi INFO: cancelling 17075149 on provider checkpoint 2024-03-15 14:04:53 proc_lsst.multi:146 [INFO] found job 17075150 in provider checkpoint proc_lsst.multi INFO: found job 17075150 in provider checkpoint 2024-03-15 14:04:53 proc_lsst.multi:201 [INFO] cancelling 17075150 on provider checkpoint proc_lsst.multi INFO: cancelling 17075150 on provider checkpoint 2024-03-15 14:04:53 proc_lsst.multi:146 [INFO] found job 17075151 in provider checkpoint proc_lsst.multi INFO: found job 17075151 in provider checkpoint 2024-03-15 14:04:53 proc_lsst.multi:201 [INFO] cancelling 17075151 on provider checkpoint proc_lsst.multi INFO: cancelling 17075151 on provider checkpoint 2024-03-15 14:04:54 proc_lsst.multi:146 [INFO] found job 17075152 in provider checkpoint proc_lsst.multi INFO: found job 17075152 in provider checkpoint 2024-03-15 14:04:54 proc_lsst.multi:201 [INFO] cancelling 17075152 on provider checkpoint proc_lsst.multi INFO: cancelling 17075152 on provider checkpoint 2024-03-15 14:04:54 proc_lsst.multi:146 [INFO] found job 17075153 in provider checkpoint proc_lsst.multi INFO: found job 17075153 in provider checkpoint 2024-03-15 14:04:54 proc_lsst.multi:201 [INFO] cancelling 17075153 on provider checkpoint proc_lsst.multi INFO: cancelling 17075153 on provider checkpoint 2024-03-15 14:04:54 proc_lsst.multi:146 [INFO] found job 17075154 in provider checkpoint proc_lsst.multi INFO: found job 17075154 in provider checkpoint 2024-03-15 14:04:54 proc_lsst.multi:201 [INFO] cancelling 17075154 on provider checkpoint proc_lsst.multi INFO: cancelling 17075154 on provider checkpoint 2024-03-15 14:04:55 proc_lsst.multi:146 [INFO] found job 17075155 in provider checkpoint proc_lsst.multi INFO: found job 17075155 in provider checkpoint 2024-03-15 14:04:55 proc_lsst.multi:201 [INFO] cancelling 17075155 on provider checkpoint proc_lsst.multi INFO: cancelling 17075155 on provider checkpoint 2024-03-15 14:04:55 proc_lsst.multi:146 [INFO] found job 17075157 in provider checkpoint proc_lsst.multi INFO: found job 17075157 in provider checkpoint 2024-03-15 14:04:55 proc_lsst.multi:201 [INFO] cancelling 17075157 on provider checkpoint proc_lsst.multi INFO: cancelling 17075157 on provider checkpoint 2024-03-15 14:04:56 proc_lsst.multi:146 [INFO] found job 17075159 in provider checkpoint proc_lsst.multi INFO: found job 17075159 in provider checkpoint 2024-03-15 14:04:56 proc_lsst.multi:201 [INFO] cancelling 17075159 on provider checkpoint proc_lsst.multi INFO: cancelling 17075159 on provider checkpoint 2024-03-15 14:04:56 proc_lsst.multi:146 [INFO] found job 17075160 in provider checkpoint proc_lsst.multi INFO: found job 17075160 in provider checkpoint 2024-03-15 14:04:56 proc_lsst.multi:201 [INFO] cancelling 17075160 on provider checkpoint proc_lsst.multi INFO: cancelling 17075160 on provider checkpoint 2024-03-15 14:04:56 proc_lsst.multi:146 [INFO] found job 17075161 in provider checkpoint proc_lsst.multi INFO: found job 17075161 in provider checkpoint 2024-03-15 14:04:56 proc_lsst.multi:201 [INFO] cancelling 17075161 on provider checkpoint proc_lsst.multi INFO: cancelling 17075161 on provider checkpoint 2024-03-15 14:04:57 proc_lsst.multi:146 [INFO] found job 17075162 in provider checkpoint proc_lsst.multi INFO: found job 17075162 in provider checkpoint 2024-03-15 14:04:57 proc_lsst.multi:201 [INFO] cancelling 17075162 on provider checkpoint proc_lsst.multi INFO: cancelling 17075162 on provider checkpoint 2024-03-15 14:04:57 proc_lsst.multi:146 [INFO] found job 17075163 in provider checkpoint proc_lsst.multi INFO: found job 17075163 in provider checkpoint 2024-03-15 14:04:57 proc_lsst.multi:201 [INFO] cancelling 17075163 on provider checkpoint proc_lsst.multi INFO: cancelling 17075163 on provider checkpoint 2024-03-15 14:04:58 proc_lsst.multi:146 [INFO] found job 17075166 in provider checkpoint proc_lsst.multi INFO: found job 17075166 in provider checkpoint 2024-03-15 14:04:58 proc_lsst.multi:201 [INFO] cancelling 17075166 on provider checkpoint proc_lsst.multi INFO: cancelling 17075166 on provider checkpoint 2024-03-15 14:04:58 proc_lsst.multi:146 [INFO] found job 17075167 in provider checkpoint proc_lsst.multi INFO: found job 17075167 in provider checkpoint 2024-03-15 14:04:58 proc_lsst.multi:201 [INFO] cancelling 17075167 on provider checkpoint proc_lsst.multi INFO: cancelling 17075167 on provider checkpoint 2024-03-15 14:04:58 proc_lsst.multi:146 [INFO] found job 17075168 in provider checkpoint proc_lsst.multi INFO: found job 17075168 in provider checkpoint 2024-03-15 14:04:58 proc_lsst.multi:201 [INFO] cancelling 17075168 on provider checkpoint proc_lsst.multi INFO: cancelling 17075168 on provider checkpoint 2024-03-15 14:04:59 proc_lsst.multi:146 [INFO] found job 17075171 in provider checkpoint proc_lsst.multi INFO: found job 17075171 in provider checkpoint 2024-03-15 14:04:59 proc_lsst.multi:201 [INFO] cancelling 17075171 on provider checkpoint proc_lsst.multi INFO: cancelling 17075171 on provider checkpoint 2024-03-15 14:04:59 proc_lsst.multi:146 [INFO] found job 17075172 in provider checkpoint proc_lsst.multi INFO: found job 17075172 in provider checkpoint 2024-03-15 14:04:59 proc_lsst.multi:201 [INFO] cancelling 17075172 on provider checkpoint proc_lsst.multi INFO: cancelling 17075172 on provider checkpoint 2024-03-15 14:05:00 proc_lsst.multi:146 [INFO] found job 17075173 in provider checkpoint proc_lsst.multi INFO: found job 17075173 in provider checkpoint 2024-03-15 14:05:00 proc_lsst.multi:201 [INFO] cancelling 17075173 on provider checkpoint proc_lsst.multi INFO: cancelling 17075173 on provider checkpoint 2024-03-15 14:05:00 proc_lsst.multi:146 [INFO] found job 17075179 in provider checkpoint proc_lsst.multi INFO: found job 17075179 in provider checkpoint 2024-03-15 14:05:00 proc_lsst.multi:201 [INFO] cancelling 17075179 on provider checkpoint proc_lsst.multi INFO: cancelling 17075179 on provider checkpoint 2024-03-15 14:05:00 proc_lsst.multi:146 [INFO] found job 17075180 in provider checkpoint proc_lsst.multi INFO: found job 17075180 in provider checkpoint 2024-03-15 14:05:00 proc_lsst.multi:201 [INFO] cancelling 17075180 on provider checkpoint proc_lsst.multi INFO: cancelling 17075180 on provider checkpoint 2024-03-15 14:05:01 proc_lsst.multi:146 [INFO] found job 17075181 in provider checkpoint proc_lsst.multi INFO: found job 17075181 in provider checkpoint 2024-03-15 14:05:01 proc_lsst.multi:201 [INFO] cancelling 17075181 on provider checkpoint proc_lsst.multi INFO: cancelling 17075181 on provider checkpoint 2024-03-15 14:05:01 proc_lsst.multi:146 [INFO] found job 17075182 in provider checkpoint proc_lsst.multi INFO: found job 17075182 in provider checkpoint 2024-03-15 14:05:01 proc_lsst.multi:201 [INFO] cancelling 17075182 on provider checkpoint proc_lsst.multi INFO: cancelling 17075182 on provider checkpoint 2024-03-15 14:05:02 proc_lsst.multi:146 [INFO] found job 17075183 in provider checkpoint proc_lsst.multi INFO: found job 17075183 in provider checkpoint 2024-03-15 14:05:02 proc_lsst.multi:201 [INFO] cancelling 17075183 on provider checkpoint proc_lsst.multi INFO: cancelling 17075183 on provider checkpoint 2024-03-15 14:05:02 proc_lsst.multi:146 [INFO] found job 17075185 in provider checkpoint proc_lsst.multi INFO: found job 17075185 in provider checkpoint 2024-03-15 14:05:02 proc_lsst.multi:201 [INFO] cancelling 17075185 on provider checkpoint proc_lsst.multi INFO: cancelling 17075185 on provider checkpoint 2024-03-15 14:05:03 proc_lsst.multi:146 [INFO] found job 17075187 in provider checkpoint proc_lsst.multi INFO: found job 17075187 in provider checkpoint 2024-03-15 14:05:03 proc_lsst.multi:201 [INFO] cancelling 17075187 on provider checkpoint proc_lsst.multi INFO: cancelling 17075187 on provider checkpoint 2024-03-15 14:05:03 proc_lsst.multi:146 [INFO] found job 17075188 in provider checkpoint proc_lsst.multi INFO: found job 17075188 in provider checkpoint 2024-03-15 14:05:03 proc_lsst.multi:201 [INFO] cancelling 17075188 on provider checkpoint proc_lsst.multi INFO: cancelling 17075188 on provider checkpoint 2024-03-15 14:05:03 proc_lsst.multi:146 [INFO] found job 17075190 in provider checkpoint proc_lsst.multi INFO: found job 17075190 in provider checkpoint 2024-03-15 14:05:03 proc_lsst.multi:201 [INFO] cancelling 17075190 on provider checkpoint proc_lsst.multi INFO: cancelling 17075190 on provider checkpoint 2024-03-15 14:05:04 proc_lsst.multi:146 [INFO] found job 17075191 in provider checkpoint proc_lsst.multi INFO: found job 17075191 in provider checkpoint 2024-03-15 14:05:04 proc_lsst.multi:201 [INFO] cancelling 17075191 on provider checkpoint proc_lsst.multi INFO: cancelling 17075191 on provider checkpoint 2024-03-15 14:05:04 proc_lsst.multi:146 [INFO] found job 17075197 in provider checkpoint proc_lsst.multi INFO: found job 17075197 in provider checkpoint 2024-03-15 14:05:04 proc_lsst.multi:201 [INFO] cancelling 17075197 on provider checkpoint proc_lsst.multi INFO: cancelling 17075197 on provider checkpoint 2024-03-15 14:05:05 proc_lsst.multi:146 [INFO] found job 17075200 in provider checkpoint proc_lsst.multi INFO: found job 17075200 in provider checkpoint 2024-03-15 14:05:05 proc_lsst.multi:201 [INFO] cancelling 17075200 on provider checkpoint proc_lsst.multi INFO: cancelling 17075200 on provider checkpoint 2024-03-15 14:05:05 proc_lsst.multi:146 [INFO] found job 17075201 in provider checkpoint proc_lsst.multi INFO: found job 17075201 in provider checkpoint 2024-03-15 14:05:05 proc_lsst.multi:201 [INFO] cancelling 17075201 on provider checkpoint proc_lsst.multi INFO: cancelling 17075201 on provider checkpoint 2024-03-15 14:05:05 proc_lsst.multi:146 [INFO] found job 17075202 in provider checkpoint proc_lsst.multi INFO: found job 17075202 in provider checkpoint 2024-03-15 14:05:05 proc_lsst.multi:201 [INFO] cancelling 17075202 on provider checkpoint proc_lsst.multi INFO: cancelling 17075202 on provider checkpoint 2024-03-15 14:05:06 proc_lsst.multi:146 [INFO] found job 17075203 in provider checkpoint proc_lsst.multi INFO: found job 17075203 in provider checkpoint 2024-03-15 14:05:06 proc_lsst.multi:201 [INFO] cancelling 17075203 on provider checkpoint proc_lsst.multi INFO: cancelling 17075203 on provider checkpoint 2024-03-15 14:05:06 proc_lsst.multi:146 [INFO] found job 17075204 in provider checkpoint proc_lsst.multi INFO: found job 17075204 in provider checkpoint 2024-03-15 14:05:06 proc_lsst.multi:201 [INFO] cancelling 17075204 on provider checkpoint proc_lsst.multi INFO: cancelling 17075204 on provider checkpoint 2024-03-15 14:05:07 proc_lsst.multi:146 [INFO] found job 17075205 in provider checkpoint proc_lsst.multi INFO: found job 17075205 in provider checkpoint 2024-03-15 14:05:07 proc_lsst.multi:201 [INFO] cancelling 17075205 on provider checkpoint proc_lsst.multi INFO: cancelling 17075205 on provider checkpoint 2024-03-15 14:05:07 proc_lsst.multi:146 [INFO] found job 17075206 in provider checkpoint proc_lsst.multi INFO: found job 17075206 in provider checkpoint 2024-03-15 14:05:07 proc_lsst.multi:201 [INFO] cancelling 17075206 on provider checkpoint proc_lsst.multi INFO: cancelling 17075206 on provider checkpoint 2024-03-15 14:05:08 proc_lsst.multi:146 [INFO] found job 17075207 in provider checkpoint proc_lsst.multi INFO: found job 17075207 in provider checkpoint 2024-03-15 14:05:08 proc_lsst.multi:201 [INFO] cancelling 17075207 on provider checkpoint proc_lsst.multi INFO: cancelling 17075207 on provider checkpoint 2024-03-15 14:05:08 proc_lsst.multi:146 [INFO] found job 17075208 in provider checkpoint proc_lsst.multi INFO: found job 17075208 in provider checkpoint 2024-03-15 14:05:08 proc_lsst.multi:201 [INFO] cancelling 17075208 on provider checkpoint proc_lsst.multi INFO: cancelling 17075208 on provider checkpoint 2024-03-15 14:05:08 proc_lsst.multi:146 [INFO] found job 17075209 in provider checkpoint proc_lsst.multi INFO: found job 17075209 in provider checkpoint 2024-03-15 14:05:08 proc_lsst.multi:201 [INFO] cancelling 17075209 on provider checkpoint proc_lsst.multi INFO: cancelling 17075209 on provider checkpoint 2024-03-15 14:05:09 proc_lsst.multi:146 [INFO] found job 17075210 in provider checkpoint proc_lsst.multi INFO: found job 17075210 in provider checkpoint 2024-03-15 14:05:09 proc_lsst.multi:201 [INFO] cancelling 17075210 on provider checkpoint proc_lsst.multi INFO: cancelling 17075210 on provider checkpoint 2024-03-15 14:05:09 proc_lsst.multi:146 [INFO] found job 17075211 in provider checkpoint proc_lsst.multi INFO: found job 17075211 in provider checkpoint 2024-03-15 14:05:09 proc_lsst.multi:201 [INFO] cancelling 17075211 on provider checkpoint proc_lsst.multi INFO: cancelling 17075211 on provider checkpoint 2024-03-15 14:05:10 proc_lsst.multi:146 [INFO] found job 17075212 in provider checkpoint proc_lsst.multi INFO: found job 17075212 in provider checkpoint 2024-03-15 14:05:10 proc_lsst.multi:201 [INFO] cancelling 17075212 on provider checkpoint proc_lsst.multi INFO: cancelling 17075212 on provider checkpoint 2024-03-15 14:05:10 proc_lsst.multi:146 [INFO] found job 17075213 in provider checkpoint proc_lsst.multi INFO: found job 17075213 in provider checkpoint 2024-03-15 14:05:10 proc_lsst.multi:201 [INFO] cancelling 17075213 on provider checkpoint proc_lsst.multi INFO: cancelling 17075213 on provider checkpoint 2024-03-15 14:05:10 proc_lsst.multi:146 [INFO] found job 17075214 in provider checkpoint proc_lsst.multi INFO: found job 17075214 in provider checkpoint 2024-03-15 14:05:10 proc_lsst.multi:201 [INFO] cancelling 17075214 on provider checkpoint proc_lsst.multi INFO: cancelling 17075214 on provider checkpoint 2024-03-15 14:05:11 proc_lsst.multi:146 [INFO] found job 17075216 in provider checkpoint proc_lsst.multi INFO: found job 17075216 in provider checkpoint 2024-03-15 14:05:11 proc_lsst.multi:201 [INFO] cancelling 17075216 on provider checkpoint proc_lsst.multi INFO: cancelling 17075216 on provider checkpoint 2024-03-15 14:05:11 proc_lsst.multi:146 [INFO] found job 17075217 in provider checkpoint proc_lsst.multi INFO: found job 17075217 in provider checkpoint 2024-03-15 14:05:11 proc_lsst.multi:201 [INFO] cancelling 17075217 on provider checkpoint proc_lsst.multi INFO: cancelling 17075217 on provider checkpoint 2024-03-15 14:05:12 proc_lsst.multi:146 [INFO] found job 17075218 in provider checkpoint proc_lsst.multi INFO: found job 17075218 in provider checkpoint 2024-03-15 14:05:12 proc_lsst.multi:201 [INFO] cancelling 17075218 on provider checkpoint proc_lsst.multi INFO: cancelling 17075218 on provider checkpoint 2024-03-15 14:05:12 proc_lsst.multi:146 [INFO] found job 17075219 in provider checkpoint proc_lsst.multi INFO: found job 17075219 in provider checkpoint 2024-03-15 14:05:12 proc_lsst.multi:201 [INFO] cancelling 17075219 on provider checkpoint proc_lsst.multi INFO: cancelling 17075219 on provider checkpoint 2024-03-15 14:05:12 proc_lsst.multi:146 [INFO] found job 17075220 in provider checkpoint proc_lsst.multi INFO: found job 17075220 in provider checkpoint 2024-03-15 14:05:12 proc_lsst.multi:201 [INFO] cancelling 17075220 on provider checkpoint proc_lsst.multi INFO: cancelling 17075220 on provider checkpoint 2024-03-15 14:05:13 proc_lsst.multi:146 [INFO] found job 17075221 in provider checkpoint proc_lsst.multi INFO: found job 17075221 in provider checkpoint 2024-03-15 14:05:13 proc_lsst.multi:201 [INFO] cancelling 17075221 on provider checkpoint proc_lsst.multi INFO: cancelling 17075221 on provider checkpoint 2024-03-15 14:05:13 proc_lsst.multi:146 [INFO] found job 17075222 in provider checkpoint proc_lsst.multi INFO: found job 17075222 in provider checkpoint 2024-03-15 14:05:13 proc_lsst.multi:201 [INFO] cancelling 17075222 on provider checkpoint proc_lsst.multi INFO: cancelling 17075222 on provider checkpoint 2024-03-15 14:05:14 proc_lsst.multi:146 [INFO] found job 17075223 in provider checkpoint proc_lsst.multi INFO: found job 17075223 in provider checkpoint 2024-03-15 14:05:14 proc_lsst.multi:201 [INFO] cancelling 17075223 on provider checkpoint proc_lsst.multi INFO: cancelling 17075223 on provider checkpoint 2024-03-15 14:05:14 proc_lsst.multi:146 [INFO] found job 17075224 in provider checkpoint proc_lsst.multi INFO: found job 17075224 in provider checkpoint 2024-03-15 14:05:14 proc_lsst.multi:201 [INFO] cancelling 17075224 on provider checkpoint proc_lsst.multi INFO: cancelling 17075224 on provider checkpoint 2024-03-15 14:05:14 proc_lsst.multi:146 [INFO] found job 17075225 in provider checkpoint proc_lsst.multi INFO: found job 17075225 in provider checkpoint 2024-03-15 14:05:14 proc_lsst.multi:201 [INFO] cancelling 17075225 on provider checkpoint proc_lsst.multi INFO: cancelling 17075225 on provider checkpoint 2024-03-15 14:05:15 proc_lsst.multi:146 [INFO] found job 17075226 in provider checkpoint proc_lsst.multi INFO: found job 17075226 in provider checkpoint 2024-03-15 14:05:15 proc_lsst.multi:201 [INFO] cancelling 17075226 on provider checkpoint proc_lsst.multi INFO: cancelling 17075226 on provider checkpoint 2024-03-15 14:05:15 proc_lsst.multi:146 [INFO] found job 17075227 in provider checkpoint proc_lsst.multi INFO: found job 17075227 in provider checkpoint 2024-03-15 14:05:15 proc_lsst.multi:201 [INFO] cancelling 17075227 on provider checkpoint proc_lsst.multi INFO: cancelling 17075227 on provider checkpoint 2024-03-15 14:05:16 proc_lsst.multi:146 [INFO] found job 17075228 in provider checkpoint proc_lsst.multi INFO: found job 17075228 in provider checkpoint 2024-03-15 14:05:16 proc_lsst.multi:201 [INFO] cancelling 17075228 on provider checkpoint proc_lsst.multi INFO: cancelling 17075228 on provider checkpoint 2024-03-15 14:05:16 proc_lsst.multi:146 [INFO] found job 17075229 in provider checkpoint proc_lsst.multi INFO: found job 17075229 in provider checkpoint 2024-03-15 14:05:16 proc_lsst.multi:201 [INFO] cancelling 17075229 on provider checkpoint proc_lsst.multi INFO: cancelling 17075229 on provider checkpoint 2024-03-15 14:05:16 proc_lsst.multi:146 [INFO] found job 17075230 in provider checkpoint proc_lsst.multi INFO: found job 17075230 in provider checkpoint 2024-03-15 14:05:16 proc_lsst.multi:201 [INFO] cancelling 17075230 on provider checkpoint proc_lsst.multi INFO: cancelling 17075230 on provider checkpoint 2024-03-15 14:05:17 proc_lsst.multi:146 [INFO] found job 17075231 in provider checkpoint proc_lsst.multi INFO: found job 17075231 in provider checkpoint 2024-03-15 14:05:17 proc_lsst.multi:201 [INFO] cancelling 17075231 on provider checkpoint proc_lsst.multi INFO: cancelling 17075231 on provider checkpoint 2024-03-15 14:05:17 proc_lsst.multi:146 [INFO] found job 17075232 in provider checkpoint proc_lsst.multi INFO: found job 17075232 in provider checkpoint 2024-03-15 14:05:17 proc_lsst.multi:201 [INFO] cancelling 17075232 on provider checkpoint proc_lsst.multi INFO: cancelling 17075232 on provider checkpoint 2024-03-15 14:05:18 proc_lsst.multi:146 [INFO] found job 17075233 in provider checkpoint proc_lsst.multi INFO: found job 17075233 in provider checkpoint 2024-03-15 14:05:18 proc_lsst.multi:201 [INFO] cancelling 17075233 on provider checkpoint proc_lsst.multi INFO: cancelling 17075233 on provider checkpoint 2024-03-15 14:05:18 proc_lsst.multi:146 [INFO] found job 17075234 in provider checkpoint proc_lsst.multi INFO: found job 17075234 in provider checkpoint 2024-03-15 14:05:18 proc_lsst.multi:201 [INFO] cancelling 17075234 on provider checkpoint proc_lsst.multi INFO: cancelling 17075234 on provider checkpoint 2024-03-15 14:05:18 proc_lsst.multi:146 [INFO] found job 17075238 in provider checkpoint proc_lsst.multi INFO: found job 17075238 in provider checkpoint 2024-03-15 14:05:18 proc_lsst.multi:201 [INFO] cancelling 17075238 on provider checkpoint proc_lsst.multi INFO: cancelling 17075238 on provider checkpoint 2024-03-15 14:05:19 proc_lsst.multi:146 [INFO] found job 17075242 in provider checkpoint proc_lsst.multi INFO: found job 17075242 in provider checkpoint 2024-03-15 14:05:19 proc_lsst.multi:201 [INFO] cancelling 17075242 on provider checkpoint proc_lsst.multi INFO: cancelling 17075242 on provider checkpoint 2024-03-15 14:05:19 proc_lsst.multi:146 [INFO] found job 17075245 in provider checkpoint proc_lsst.multi INFO: found job 17075245 in provider checkpoint 2024-03-15 14:05:19 proc_lsst.multi:201 [INFO] cancelling 17075245 on provider checkpoint proc_lsst.multi INFO: cancelling 17075245 on provider checkpoint 2024-03-15 14:05:20 proc_lsst.multi:146 [INFO] found job 17075246 in provider checkpoint proc_lsst.multi INFO: found job 17075246 in provider checkpoint 2024-03-15 14:05:20 proc_lsst.multi:201 [INFO] cancelling 17075246 on provider checkpoint proc_lsst.multi INFO: cancelling 17075246 on provider checkpoint 2024-03-15 14:05:20 proc_lsst.multi:146 [INFO] found job 17075247 in provider checkpoint proc_lsst.multi INFO: found job 17075247 in provider checkpoint 2024-03-15 14:05:20 proc_lsst.multi:201 [INFO] cancelling 17075247 on provider checkpoint proc_lsst.multi INFO: cancelling 17075247 on provider checkpoint 2024-03-15 14:05:20 proc_lsst.multi:146 [INFO] found job 17075250 in provider checkpoint proc_lsst.multi INFO: found job 17075250 in provider checkpoint 2024-03-15 14:05:20 proc_lsst.multi:201 [INFO] cancelling 17075250 on provider checkpoint proc_lsst.multi INFO: cancelling 17075250 on provider checkpoint 2024-03-15 14:05:21 proc_lsst.multi:146 [INFO] found job 17075254 in provider checkpoint proc_lsst.multi INFO: found job 17075254 in provider checkpoint 2024-03-15 14:05:21 proc_lsst.multi:201 [INFO] cancelling 17075254 on provider checkpoint proc_lsst.multi INFO: cancelling 17075254 on provider checkpoint 2024-03-15 14:05:21 proc_lsst.multi:146 [INFO] found job 17075255 in provider checkpoint proc_lsst.multi INFO: found job 17075255 in provider checkpoint 2024-03-15 14:05:21 proc_lsst.multi:201 [INFO] cancelling 17075255 on provider checkpoint proc_lsst.multi INFO: cancelling 17075255 on provider checkpoint 2024-03-15 14:05:22 proc_lsst.multi:146 [INFO] found job 17075256 in provider checkpoint proc_lsst.multi INFO: found job 17075256 in provider checkpoint 2024-03-15 14:05:22 proc_lsst.multi:201 [INFO] cancelling 17075256 on provider checkpoint proc_lsst.multi INFO: cancelling 17075256 on provider checkpoint 2024-03-15 14:05:22 proc_lsst.multi:146 [INFO] found job 17075257 in provider checkpoint proc_lsst.multi INFO: found job 17075257 in provider checkpoint 2024-03-15 14:05:22 proc_lsst.multi:201 [INFO] cancelling 17075257 on provider checkpoint proc_lsst.multi INFO: cancelling 17075257 on provider checkpoint 2024-03-15 14:05:23 proc_lsst.multi:146 [INFO] found job 17075258 in provider checkpoint proc_lsst.multi INFO: found job 17075258 in provider checkpoint 2024-03-15 14:05:23 proc_lsst.multi:201 [INFO] cancelling 17075258 on provider checkpoint proc_lsst.multi INFO: cancelling 17075258 on provider checkpoint 2024-03-15 14:05:23 proc_lsst.multi:146 [INFO] found job 17075264 in provider checkpoint proc_lsst.multi INFO: found job 17075264 in provider checkpoint 2024-03-15 14:05:23 proc_lsst.multi:201 [INFO] cancelling 17075264 on provider checkpoint proc_lsst.multi INFO: cancelling 17075264 on provider checkpoint 2024-03-15 14:05:23 proc_lsst.multi:146 [INFO] found job 17075265 in provider checkpoint proc_lsst.multi INFO: found job 17075265 in provider checkpoint 2024-03-15 14:05:23 proc_lsst.multi:201 [INFO] cancelling 17075265 on provider checkpoint proc_lsst.multi INFO: cancelling 17075265 on provider checkpoint 2024-03-15 14:05:24 proc_lsst.multi:146 [INFO] found job 17075273 in provider checkpoint proc_lsst.multi INFO: found job 17075273 in provider checkpoint 2024-03-15 14:05:24 proc_lsst.multi:201 [INFO] cancelling 17075273 on provider checkpoint proc_lsst.multi INFO: cancelling 17075273 on provider checkpoint 2024-03-15 14:05:24 proc_lsst.multi:146 [INFO] found job 17075274 in provider checkpoint proc_lsst.multi INFO: found job 17075274 in provider checkpoint 2024-03-15 14:05:24 proc_lsst.multi:201 [INFO] cancelling 17075274 on provider checkpoint proc_lsst.multi INFO: cancelling 17075274 on provider checkpoint 2024-03-15 14:05:25 proc_lsst.multi:146 [INFO] found job 17075275 in provider checkpoint proc_lsst.multi INFO: found job 17075275 in provider checkpoint 2024-03-15 14:05:25 proc_lsst.multi:201 [INFO] cancelling 17075275 on provider checkpoint proc_lsst.multi INFO: cancelling 17075275 on provider checkpoint 2024-03-15 14:05:25 proc_lsst.multi:146 [INFO] found job 17075276 in provider checkpoint proc_lsst.multi INFO: found job 17075276 in provider checkpoint 2024-03-15 14:05:25 proc_lsst.multi:201 [INFO] cancelling 17075276 on provider checkpoint proc_lsst.multi INFO: cancelling 17075276 on provider checkpoint 2024-03-15 14:05:25 proc_lsst.multi:146 [INFO] found job 17075278 in provider checkpoint proc_lsst.multi INFO: found job 17075278 in provider checkpoint 2024-03-15 14:05:25 proc_lsst.multi:201 [INFO] cancelling 17075278 on provider checkpoint proc_lsst.multi INFO: cancelling 17075278 on provider checkpoint 2024-03-15 14:05:26 proc_lsst.multi:146 [INFO] found job 17075279 in provider checkpoint proc_lsst.multi INFO: found job 17075279 in provider checkpoint 2024-03-15 14:05:26 proc_lsst.multi:201 [INFO] cancelling 17075279 on provider checkpoint proc_lsst.multi INFO: cancelling 17075279 on provider checkpoint 2024-03-15 14:05:26 proc_lsst.multi:146 [INFO] found job 17075280 in provider checkpoint proc_lsst.multi INFO: found job 17075280 in provider checkpoint 2024-03-15 14:05:26 proc_lsst.multi:201 [INFO] cancelling 17075280 on provider checkpoint proc_lsst.multi INFO: cancelling 17075280 on provider checkpoint 2024-03-15 14:05:27 proc_lsst.multi:146 [INFO] found job 17075281 in provider checkpoint proc_lsst.multi INFO: found job 17075281 in provider checkpoint 2024-03-15 14:05:27 proc_lsst.multi:201 [INFO] cancelling 17075281 on provider checkpoint proc_lsst.multi INFO: cancelling 17075281 on provider checkpoint 2024-03-15 14:05:27 proc_lsst.multi:146 [INFO] found job 17075282 in provider checkpoint proc_lsst.multi INFO: found job 17075282 in provider checkpoint 2024-03-15 14:05:27 proc_lsst.multi:201 [INFO] cancelling 17075282 on provider checkpoint proc_lsst.multi INFO: cancelling 17075282 on provider checkpoint 2024-03-15 14:05:27 proc_lsst.multi:146 [INFO] found job 17075283 in provider checkpoint proc_lsst.multi INFO: found job 17075283 in provider checkpoint 2024-03-15 14:05:27 proc_lsst.multi:201 [INFO] cancelling 17075283 on provider checkpoint proc_lsst.multi INFO: cancelling 17075283 on provider checkpoint 2024-03-15 14:05:28 proc_lsst.multi:146 [INFO] found job 17075284 in provider checkpoint proc_lsst.multi INFO: found job 17075284 in provider checkpoint 2024-03-15 14:05:28 proc_lsst.multi:201 [INFO] cancelling 17075284 on provider checkpoint proc_lsst.multi INFO: cancelling 17075284 on provider checkpoint 2024-03-15 14:05:28 proc_lsst.multi:146 [INFO] found job 17075285 in provider checkpoint proc_lsst.multi INFO: found job 17075285 in provider checkpoint 2024-03-15 14:05:28 proc_lsst.multi:201 [INFO] cancelling 17075285 on provider checkpoint proc_lsst.multi INFO: cancelling 17075285 on provider checkpoint 2024-03-15 14:05:29 proc_lsst.multi:146 [INFO] found job 17075286 in provider checkpoint proc_lsst.multi INFO: found job 17075286 in provider checkpoint 2024-03-15 14:05:29 proc_lsst.multi:201 [INFO] cancelling 17075286 on provider checkpoint proc_lsst.multi INFO: cancelling 17075286 on provider checkpoint 2024-03-15 14:05:29 proc_lsst.multi:146 [INFO] found job 17075287 in provider checkpoint proc_lsst.multi INFO: found job 17075287 in provider checkpoint 2024-03-15 14:05:29 proc_lsst.multi:201 [INFO] cancelling 17075287 on provider checkpoint proc_lsst.multi INFO: cancelling 17075287 on provider checkpoint 2024-03-15 14:05:29 proc_lsst.multi:146 [INFO] found job 17075288 in provider checkpoint proc_lsst.multi INFO: found job 17075288 in provider checkpoint 2024-03-15 14:05:29 proc_lsst.multi:201 [INFO] cancelling 17075288 on provider checkpoint proc_lsst.multi INFO: cancelling 17075288 on provider checkpoint 2024-03-15 14:05:30 proc_lsst.multi:146 [INFO] found job 17075289 in provider checkpoint proc_lsst.multi INFO: found job 17075289 in provider checkpoint 2024-03-15 14:05:30 proc_lsst.multi:201 [INFO] cancelling 17075289 on provider checkpoint proc_lsst.multi INFO: cancelling 17075289 on provider checkpoint 2024-03-15 14:05:30 proc_lsst.multi:146 [INFO] found job 17075290 in provider checkpoint proc_lsst.multi INFO: found job 17075290 in provider checkpoint 2024-03-15 14:05:30 proc_lsst.multi:201 [INFO] cancelling 17075290 on provider checkpoint proc_lsst.multi INFO: cancelling 17075290 on provider checkpoint 2024-03-15 14:05:31 proc_lsst.multi:146 [INFO] found job 17075293 in provider checkpoint proc_lsst.multi INFO: found job 17075293 in provider checkpoint 2024-03-15 14:05:31 proc_lsst.multi:201 [INFO] cancelling 17075293 on provider checkpoint proc_lsst.multi INFO: cancelling 17075293 on provider checkpoint 2024-03-15 14:05:31 proc_lsst.multi:146 [INFO] found job 17075298 in provider checkpoint proc_lsst.multi INFO: found job 17075298 in provider checkpoint 2024-03-15 14:05:31 proc_lsst.multi:201 [INFO] cancelling 17075298 on provider checkpoint proc_lsst.multi INFO: cancelling 17075298 on provider checkpoint 2024-03-15 14:05:31 proc_lsst.multi:146 [INFO] found job 17075299 in provider checkpoint proc_lsst.multi INFO: found job 17075299 in provider checkpoint 2024-03-15 14:05:31 proc_lsst.multi:201 [INFO] cancelling 17075299 on provider checkpoint proc_lsst.multi INFO: cancelling 17075299 on provider checkpoint 2024-03-15 14:05:32 proc_lsst.multi:146 [INFO] found job 17075301 in provider checkpoint proc_lsst.multi INFO: found job 17075301 in provider checkpoint 2024-03-15 14:05:32 proc_lsst.multi:201 [INFO] cancelling 17075301 on provider checkpoint proc_lsst.multi INFO: cancelling 17075301 on provider checkpoint 2024-03-15 14:05:32 proc_lsst.multi:146 [INFO] found job 17075306 in provider checkpoint proc_lsst.multi INFO: found job 17075306 in provider checkpoint 2024-03-15 14:05:32 proc_lsst.multi:201 [INFO] cancelling 17075306 on provider checkpoint proc_lsst.multi INFO: cancelling 17075306 on provider checkpoint 2024-03-15 14:05:33 proc_lsst.multi:146 [INFO] found job 17075307 in provider checkpoint proc_lsst.multi INFO: found job 17075307 in provider checkpoint 2024-03-15 14:05:33 proc_lsst.multi:201 [INFO] cancelling 17075307 on provider checkpoint proc_lsst.multi INFO: cancelling 17075307 on provider checkpoint 2024-03-15 14:05:33 proc_lsst.multi:146 [INFO] found job 17075308 in provider checkpoint proc_lsst.multi INFO: found job 17075308 in provider checkpoint 2024-03-15 14:05:33 proc_lsst.multi:201 [INFO] cancelling 17075308 on provider checkpoint proc_lsst.multi INFO: cancelling 17075308 on provider checkpoint 2024-03-15 14:05:33 proc_lsst.multi:146 [INFO] found job 17075309 in provider checkpoint proc_lsst.multi INFO: found job 17075309 in provider checkpoint 2024-03-15 14:05:33 proc_lsst.multi:201 [INFO] cancelling 17075309 on provider checkpoint proc_lsst.multi INFO: cancelling 17075309 on provider checkpoint 2024-03-15 14:05:34 proc_lsst.multi:146 [INFO] found job 17075310 in provider checkpoint proc_lsst.multi INFO: found job 17075310 in provider checkpoint 2024-03-15 14:05:34 proc_lsst.multi:201 [INFO] cancelling 17075310 on provider checkpoint proc_lsst.multi INFO: cancelling 17075310 on provider checkpoint 2024-03-15 14:05:34 proc_lsst.multi:146 [INFO] found job 17075311 in provider checkpoint proc_lsst.multi INFO: found job 17075311 in provider checkpoint 2024-03-15 14:05:34 proc_lsst.multi:201 [INFO] cancelling 17075311 on provider checkpoint proc_lsst.multi INFO: cancelling 17075311 on provider checkpoint 2024-03-15 14:05:34 proc_lsst.multi:146 [INFO] found job 17075312 in provider checkpoint proc_lsst.multi INFO: found job 17075312 in provider checkpoint 2024-03-15 14:05:34 proc_lsst.multi:201 [INFO] cancelling 17075312 on provider checkpoint proc_lsst.multi INFO: cancelling 17075312 on provider checkpoint 2024-03-15 14:05:35 proc_lsst.multi:146 [INFO] found job 17075315 in provider checkpoint proc_lsst.multi INFO: found job 17075315 in provider checkpoint 2024-03-15 14:05:35 proc_lsst.multi:201 [INFO] cancelling 17075315 on provider checkpoint proc_lsst.multi INFO: cancelling 17075315 on provider checkpoint 2024-03-15 14:05:35 proc_lsst.multi:146 [INFO] found job 17075318 in provider checkpoint proc_lsst.multi INFO: found job 17075318 in provider checkpoint 2024-03-15 14:05:35 proc_lsst.multi:201 [INFO] cancelling 17075318 on provider checkpoint proc_lsst.multi INFO: cancelling 17075318 on provider checkpoint 2024-03-15 14:05:35 proc_lsst.multi:146 [INFO] found job 17075319 in provider checkpoint proc_lsst.multi INFO: found job 17075319 in provider checkpoint 2024-03-15 14:05:35 proc_lsst.multi:201 [INFO] cancelling 17075319 on provider checkpoint proc_lsst.multi INFO: cancelling 17075319 on provider checkpoint 2024-03-15 14:05:35 proc_lsst.multi:146 [INFO] found job 17075320 in provider checkpoint proc_lsst.multi INFO: found job 17075320 in provider checkpoint 2024-03-15 14:05:35 proc_lsst.multi:201 [INFO] cancelling 17075320 on provider checkpoint proc_lsst.multi INFO: cancelling 17075320 on provider checkpoint 2024-03-15 14:05:36 proc_lsst.multi:146 [INFO] found job 17075321 in provider checkpoint proc_lsst.multi INFO: found job 17075321 in provider checkpoint 2024-03-15 14:05:36 proc_lsst.multi:201 [INFO] cancelling 17075321 on provider checkpoint proc_lsst.multi INFO: cancelling 17075321 on provider checkpoint 2024-03-15 14:05:36 proc_lsst.multi:146 [INFO] found job 17075326 in provider checkpoint proc_lsst.multi INFO: found job 17075326 in provider checkpoint 2024-03-15 14:05:36 proc_lsst.multi:201 [INFO] cancelling 17075326 on provider checkpoint proc_lsst.multi INFO: cancelling 17075326 on provider checkpoint 2024-03-15 14:05:36 proc_lsst.multi:146 [INFO] found job 17075329 in provider checkpoint proc_lsst.multi INFO: found job 17075329 in provider checkpoint 2024-03-15 14:05:36 proc_lsst.multi:201 [INFO] cancelling 17075329 on provider checkpoint proc_lsst.multi INFO: cancelling 17075329 on provider checkpoint 2024-03-15 14:05:37 proc_lsst.multi:146 [INFO] found job 17075330 in provider checkpoint proc_lsst.multi INFO: found job 17075330 in provider checkpoint 2024-03-15 14:05:37 proc_lsst.multi:201 [INFO] cancelling 17075330 on provider checkpoint proc_lsst.multi INFO: cancelling 17075330 on provider checkpoint 2024-03-15 14:05:37 proc_lsst.multi:146 [INFO] found job 17075332 in provider checkpoint proc_lsst.multi INFO: found job 17075332 in provider checkpoint 2024-03-15 14:05:37 proc_lsst.multi:201 [INFO] cancelling 17075332 on provider checkpoint proc_lsst.multi INFO: cancelling 17075332 on provider checkpoint 2024-03-15 14:05:38 proc_lsst.multi:146 [INFO] found job 17075338 in provider checkpoint proc_lsst.multi INFO: found job 17075338 in provider checkpoint 2024-03-15 14:05:38 proc_lsst.multi:201 [INFO] cancelling 17075338 on provider checkpoint proc_lsst.multi INFO: cancelling 17075338 on provider checkpoint 2024-03-15 14:05:38 proc_lsst.multi:146 [INFO] found job 17075344 in provider checkpoint proc_lsst.multi INFO: found job 17075344 in provider checkpoint 2024-03-15 14:05:38 proc_lsst.multi:201 [INFO] cancelling 17075344 on provider checkpoint proc_lsst.multi INFO: cancelling 17075344 on provider checkpoint 2024-03-15 14:05:38 proc_lsst.multi:146 [INFO] found job 17075345 in provider checkpoint proc_lsst.multi INFO: found job 17075345 in provider checkpoint 2024-03-15 14:05:38 proc_lsst.multi:201 [INFO] cancelling 17075345 on provider checkpoint proc_lsst.multi INFO: cancelling 17075345 on provider checkpoint 2024-03-15 14:05:39 proc_lsst.multi:146 [INFO] found job 17075346 in provider checkpoint proc_lsst.multi INFO: found job 17075346 in provider checkpoint 2024-03-15 14:05:39 proc_lsst.multi:201 [INFO] cancelling 17075346 on provider checkpoint proc_lsst.multi INFO: cancelling 17075346 on provider checkpoint 2024-03-15 14:05:39 proc_lsst.multi:146 [INFO] found job 17075347 in provider checkpoint proc_lsst.multi INFO: found job 17075347 in provider checkpoint 2024-03-15 14:05:39 proc_lsst.multi:201 [INFO] cancelling 17075347 on provider checkpoint proc_lsst.multi INFO: cancelling 17075347 on provider checkpoint 2024-03-15 14:05:40 proc_lsst.multi:146 [INFO] found job 17075348 in provider checkpoint proc_lsst.multi INFO: found job 17075348 in provider checkpoint 2024-03-15 14:05:40 proc_lsst.multi:201 [INFO] cancelling 17075348 on provider checkpoint proc_lsst.multi INFO: cancelling 17075348 on provider checkpoint 2024-03-15 14:05:40 proc_lsst.multi:146 [INFO] found job 17075351 in provider checkpoint proc_lsst.multi INFO: found job 17075351 in provider checkpoint 2024-03-15 14:05:40 proc_lsst.multi:201 [INFO] cancelling 17075351 on provider checkpoint proc_lsst.multi INFO: cancelling 17075351 on provider checkpoint 2024-03-15 14:05:40 proc_lsst.multi:146 [INFO] found job 17075353 in provider checkpoint proc_lsst.multi INFO: found job 17075353 in provider checkpoint 2024-03-15 14:05:40 proc_lsst.multi:201 [INFO] cancelling 17075353 on provider checkpoint proc_lsst.multi INFO: cancelling 17075353 on provider checkpoint 2024-03-15 14:05:41 proc_lsst.multi:146 [INFO] found job 17075354 in provider checkpoint proc_lsst.multi INFO: found job 17075354 in provider checkpoint 2024-03-15 14:05:41 proc_lsst.multi:201 [INFO] cancelling 17075354 on provider checkpoint proc_lsst.multi INFO: cancelling 17075354 on provider checkpoint 2024-03-15 14:05:41 proc_lsst.multi:146 [INFO] found job 17075355 in provider checkpoint proc_lsst.multi INFO: found job 17075355 in provider checkpoint 2024-03-15 14:05:41 proc_lsst.multi:201 [INFO] cancelling 17075355 on provider checkpoint proc_lsst.multi INFO: cancelling 17075355 on provider checkpoint 2024-03-15 14:05:42 proc_lsst.multi:146 [INFO] found job 17075357 in provider checkpoint proc_lsst.multi INFO: found job 17075357 in provider checkpoint 2024-03-15 14:05:42 proc_lsst.multi:201 [INFO] cancelling 17075357 on provider checkpoint proc_lsst.multi INFO: cancelling 17075357 on provider checkpoint 2024-03-15 14:05:42 proc_lsst.multi:146 [INFO] found job 17075358 in provider checkpoint proc_lsst.multi INFO: found job 17075358 in provider checkpoint 2024-03-15 14:05:42 proc_lsst.multi:201 [INFO] cancelling 17075358 on provider checkpoint proc_lsst.multi INFO: cancelling 17075358 on provider checkpoint 2024-03-15 14:05:43 proc_lsst.multi:146 [INFO] found job 17075359 in provider checkpoint proc_lsst.multi INFO: found job 17075359 in provider checkpoint 2024-03-15 14:05:43 proc_lsst.multi:201 [INFO] cancelling 17075359 on provider checkpoint proc_lsst.multi INFO: cancelling 17075359 on provider checkpoint 2024-03-15 14:05:43 proc_lsst.multi:146 [INFO] found job 17075360 in provider checkpoint proc_lsst.multi INFO: found job 17075360 in provider checkpoint 2024-03-15 14:05:43 proc_lsst.multi:201 [INFO] cancelling 17075360 on provider checkpoint proc_lsst.multi INFO: cancelling 17075360 on provider checkpoint 2024-03-15 14:05:43 proc_lsst.multi:146 [INFO] found job 17075364 in provider checkpoint proc_lsst.multi INFO: found job 17075364 in provider checkpoint 2024-03-15 14:05:43 proc_lsst.multi:201 [INFO] cancelling 17075364 on provider checkpoint proc_lsst.multi INFO: cancelling 17075364 on provider checkpoint 2024-03-15 14:05:44 proc_lsst.multi:146 [INFO] found job 17075368 in provider checkpoint proc_lsst.multi INFO: found job 17075368 in provider checkpoint 2024-03-15 14:05:44 proc_lsst.multi:201 [INFO] cancelling 17075368 on provider checkpoint proc_lsst.multi INFO: cancelling 17075368 on provider checkpoint 2024-03-15 14:05:44 proc_lsst.multi:146 [INFO] found job 17075372 in provider checkpoint proc_lsst.multi INFO: found job 17075372 in provider checkpoint 2024-03-15 14:05:44 proc_lsst.multi:201 [INFO] cancelling 17075372 on provider checkpoint proc_lsst.multi INFO: cancelling 17075372 on provider checkpoint 2024-03-15 14:05:45 proc_lsst.multi:146 [INFO] found job 17075376 in provider checkpoint proc_lsst.multi INFO: found job 17075376 in provider checkpoint 2024-03-15 14:05:45 proc_lsst.multi:201 [INFO] cancelling 17075376 on provider checkpoint proc_lsst.multi INFO: cancelling 17075376 on provider checkpoint 2024-03-15 14:05:45 proc_lsst.multi:146 [INFO] found job 17075379 in provider checkpoint proc_lsst.multi INFO: found job 17075379 in provider checkpoint 2024-03-15 14:05:45 proc_lsst.multi:201 [INFO] cancelling 17075379 on provider checkpoint proc_lsst.multi INFO: cancelling 17075379 on provider checkpoint 2024-03-15 14:05:45 proc_lsst.multi:146 [INFO] found job 17075380 in provider checkpoint proc_lsst.multi INFO: found job 17075380 in provider checkpoint 2024-03-15 14:05:45 proc_lsst.multi:201 [INFO] cancelling 17075380 on provider checkpoint proc_lsst.multi INFO: cancelling 17075380 on provider checkpoint 2024-03-15 14:05:46 proc_lsst.multi:146 [INFO] found job 17075381 in provider checkpoint proc_lsst.multi INFO: found job 17075381 in provider checkpoint 2024-03-15 14:05:46 proc_lsst.multi:201 [INFO] cancelling 17075381 on provider checkpoint proc_lsst.multi INFO: cancelling 17075381 on provider checkpoint 2024-03-15 14:05:46 proc_lsst.multi:146 [INFO] found job 17075382 in provider checkpoint proc_lsst.multi INFO: found job 17075382 in provider checkpoint 2024-03-15 14:05:46 proc_lsst.multi:201 [INFO] cancelling 17075382 on provider checkpoint proc_lsst.multi INFO: cancelling 17075382 on provider checkpoint 2024-03-15 14:05:47 proc_lsst.multi:146 [INFO] found job 17075384 in provider checkpoint proc_lsst.multi INFO: found job 17075384 in provider checkpoint 2024-03-15 14:05:47 proc_lsst.multi:201 [INFO] cancelling 17075384 on provider checkpoint proc_lsst.multi INFO: cancelling 17075384 on provider checkpoint 2024-03-15 14:05:47 proc_lsst.multi:146 [INFO] found job 17075385 in provider checkpoint proc_lsst.multi INFO: found job 17075385 in provider checkpoint 2024-03-15 14:05:47 proc_lsst.multi:201 [INFO] cancelling 17075385 on provider checkpoint proc_lsst.multi INFO: cancelling 17075385 on provider checkpoint 2024-03-15 14:05:47 proc_lsst.multi:146 [INFO] found job 17075386 in provider checkpoint proc_lsst.multi INFO: found job 17075386 in provider checkpoint 2024-03-15 14:05:47 proc_lsst.multi:201 [INFO] cancelling 17075386 on provider checkpoint proc_lsst.multi INFO: cancelling 17075386 on provider checkpoint 2024-03-15 14:05:48 proc_lsst.multi:146 [INFO] found job 17075387 in provider checkpoint proc_lsst.multi INFO: found job 17075387 in provider checkpoint 2024-03-15 14:05:48 proc_lsst.multi:201 [INFO] cancelling 17075387 on provider checkpoint proc_lsst.multi INFO: cancelling 17075387 on provider checkpoint 2024-03-15 14:05:48 proc_lsst.multi:146 [INFO] found job 17075388 in provider checkpoint proc_lsst.multi INFO: found job 17075388 in provider checkpoint 2024-03-15 14:05:48 proc_lsst.multi:201 [INFO] cancelling 17075388 on provider checkpoint proc_lsst.multi INFO: cancelling 17075388 on provider checkpoint 2024-03-15 14:05:49 proc_lsst.multi:146 [INFO] found job 17075392 in provider checkpoint proc_lsst.multi INFO: found job 17075392 in provider checkpoint 2024-03-15 14:05:49 proc_lsst.multi:201 [INFO] cancelling 17075392 on provider checkpoint proc_lsst.multi INFO: cancelling 17075392 on provider checkpoint 2024-03-15 14:05:49 proc_lsst.multi:146 [INFO] found job 17075394 in provider checkpoint proc_lsst.multi INFO: found job 17075394 in provider checkpoint 2024-03-15 14:05:49 proc_lsst.multi:201 [INFO] cancelling 17075394 on provider checkpoint proc_lsst.multi INFO: cancelling 17075394 on provider checkpoint 2024-03-15 14:05:49 proc_lsst.multi:146 [INFO] found job 17075399 in provider checkpoint proc_lsst.multi INFO: found job 17075399 in provider checkpoint 2024-03-15 14:05:49 proc_lsst.multi:201 [INFO] cancelling 17075399 on provider checkpoint proc_lsst.multi INFO: cancelling 17075399 on provider checkpoint 2024-03-15 14:05:50 proc_lsst.multi:146 [INFO] found job 17075400 in provider checkpoint proc_lsst.multi INFO: found job 17075400 in provider checkpoint 2024-03-15 14:05:50 proc_lsst.multi:201 [INFO] cancelling 17075400 on provider checkpoint proc_lsst.multi INFO: cancelling 17075400 on provider checkpoint 2024-03-15 14:05:50 proc_lsst.multi:146 [INFO] found job 17075401 in provider checkpoint proc_lsst.multi INFO: found job 17075401 in provider checkpoint 2024-03-15 14:05:50 proc_lsst.multi:201 [INFO] cancelling 17075401 on provider checkpoint proc_lsst.multi INFO: cancelling 17075401 on provider checkpoint 2024-03-15 14:05:51 proc_lsst.multi:146 [INFO] found job 17075405 in provider checkpoint proc_lsst.multi INFO: found job 17075405 in provider checkpoint 2024-03-15 14:05:51 proc_lsst.multi:201 [INFO] cancelling 17075405 on provider checkpoint proc_lsst.multi INFO: cancelling 17075405 on provider checkpoint 2024-03-15 14:05:51 proc_lsst.multi:146 [INFO] found job 17075408 in provider checkpoint proc_lsst.multi INFO: found job 17075408 in provider checkpoint 2024-03-15 14:05:51 proc_lsst.multi:201 [INFO] cancelling 17075408 on provider checkpoint proc_lsst.multi INFO: cancelling 17075408 on provider checkpoint 2024-03-15 14:05:52 proc_lsst.multi:146 [INFO] found job 17075417 in provider checkpoint proc_lsst.multi INFO: found job 17075417 in provider checkpoint 2024-03-15 14:05:52 proc_lsst.multi:201 [INFO] cancelling 17075417 on provider checkpoint proc_lsst.multi INFO: cancelling 17075417 on provider checkpoint 2024-03-15 14:05:52 proc_lsst.multi:146 [INFO] found job 17075428 in provider checkpoint proc_lsst.multi INFO: found job 17075428 in provider checkpoint 2024-03-15 14:05:52 proc_lsst.multi:201 [INFO] cancelling 17075428 on provider checkpoint proc_lsst.multi INFO: cancelling 17075428 on provider checkpoint 2024-03-15 14:05:52 proc_lsst.multi:146 [INFO] found job 17075430 in provider checkpoint proc_lsst.multi INFO: found job 17075430 in provider checkpoint 2024-03-15 14:05:52 proc_lsst.multi:201 [INFO] cancelling 17075430 on provider checkpoint proc_lsst.multi INFO: cancelling 17075430 on provider checkpoint 2024-03-15 14:05:53 proc_lsst.multi:146 [INFO] found job 17075431 in provider checkpoint proc_lsst.multi INFO: found job 17075431 in provider checkpoint 2024-03-15 14:05:53 proc_lsst.multi:201 [INFO] cancelling 17075431 on provider checkpoint proc_lsst.multi INFO: cancelling 17075431 on provider checkpoint 2024-03-15 14:05:53 proc_lsst.multi:146 [INFO] found job 17075432 in provider checkpoint proc_lsst.multi INFO: found job 17075432 in provider checkpoint 2024-03-15 14:05:53 proc_lsst.multi:201 [INFO] cancelling 17075432 on provider checkpoint proc_lsst.multi INFO: cancelling 17075432 on provider checkpoint 2024-03-15 14:05:54 proc_lsst.multi:146 [INFO] found job 17075433 in provider checkpoint proc_lsst.multi INFO: found job 17075433 in provider checkpoint 2024-03-15 14:05:54 proc_lsst.multi:201 [INFO] cancelling 17075433 on provider checkpoint proc_lsst.multi INFO: cancelling 17075433 on provider checkpoint 2024-03-15 14:05:54 proc_lsst.multi:146 [INFO] found job 17075434 in provider checkpoint proc_lsst.multi INFO: found job 17075434 in provider checkpoint 2024-03-15 14:05:54 proc_lsst.multi:201 [INFO] cancelling 17075434 on provider checkpoint proc_lsst.multi INFO: cancelling 17075434 on provider checkpoint 2024-03-15 14:05:54 proc_lsst.multi:146 [INFO] found job 17075435 in provider checkpoint proc_lsst.multi INFO: found job 17075435 in provider checkpoint 2024-03-15 14:05:54 proc_lsst.multi:201 [INFO] cancelling 17075435 on provider checkpoint proc_lsst.multi INFO: cancelling 17075435 on provider checkpoint 2024-03-15 14:05:55 proc_lsst.multi:146 [INFO] found job 17075436 in provider checkpoint proc_lsst.multi INFO: found job 17075436 in provider checkpoint 2024-03-15 14:05:55 proc_lsst.multi:201 [INFO] cancelling 17075436 on provider checkpoint proc_lsst.multi INFO: cancelling 17075436 on provider checkpoint 2024-03-15 14:05:55 proc_lsst.multi:146 [INFO] found job 17075438 in provider checkpoint proc_lsst.multi INFO: found job 17075438 in provider checkpoint 2024-03-15 14:05:55 proc_lsst.multi:201 [INFO] cancelling 17075438 on provider checkpoint proc_lsst.multi INFO: cancelling 17075438 on provider checkpoint 2024-03-15 14:05:56 proc_lsst.multi:146 [INFO] found job 17075440 in provider checkpoint proc_lsst.multi INFO: found job 17075440 in provider checkpoint 2024-03-15 14:05:56 proc_lsst.multi:201 [INFO] cancelling 17075440 on provider checkpoint proc_lsst.multi INFO: cancelling 17075440 on provider checkpoint 2024-03-15 14:05:56 proc_lsst.multi:146 [INFO] found job 17075441 in provider checkpoint proc_lsst.multi INFO: found job 17075441 in provider checkpoint 2024-03-15 14:05:56 proc_lsst.multi:201 [INFO] cancelling 17075441 on provider checkpoint proc_lsst.multi INFO: cancelling 17075441 on provider checkpoint 2024-03-15 14:05:56 proc_lsst.multi:146 [INFO] found job 17075443 in provider checkpoint proc_lsst.multi INFO: found job 17075443 in provider checkpoint 2024-03-15 14:05:56 proc_lsst.multi:201 [INFO] cancelling 17075443 on provider checkpoint proc_lsst.multi INFO: cancelling 17075443 on provider checkpoint 2024-03-15 14:05:57 proc_lsst.multi:146 [INFO] found job 17075444 in provider checkpoint proc_lsst.multi INFO: found job 17075444 in provider checkpoint 2024-03-15 14:05:57 proc_lsst.multi:201 [INFO] cancelling 17075444 on provider checkpoint proc_lsst.multi INFO: cancelling 17075444 on provider checkpoint 2024-03-15 14:05:57 proc_lsst.multi:146 [INFO] found job 17075446 in provider checkpoint proc_lsst.multi INFO: found job 17075446 in provider checkpoint 2024-03-15 14:05:57 proc_lsst.multi:201 [INFO] cancelling 17075446 on provider checkpoint proc_lsst.multi INFO: cancelling 17075446 on provider checkpoint 2024-03-15 14:05:58 proc_lsst.multi:146 [INFO] found job 17074833 in provider astro proc_lsst.multi INFO: found job 17074833 in provider astro 2024-03-15 14:05:58 proc_lsst.multi:201 [INFO] cancelling 17074833 on provider astro proc_lsst.multi INFO: cancelling 17074833 on provider astro 2024-03-15 14:05:58 proc_lsst.multi:146 [INFO] found job 17074834 in provider astro proc_lsst.multi INFO: found job 17074834 in provider astro 2024-03-15 14:05:58 proc_lsst.multi:201 [INFO] cancelling 17074834 on provider astro proc_lsst.multi INFO: cancelling 17074834 on provider astro 2024-03-15 14:05:58 proc_lsst.multi:146 [INFO] found job 17074835 in provider astro proc_lsst.multi INFO: found job 17074835 in provider astro 2024-03-15 14:05:58 proc_lsst.multi:201 [INFO] cancelling 17074835 on provider astro proc_lsst.multi INFO: cancelling 17074835 on provider astro 2024-03-15 14:05:59 proc_lsst.multi:146 [INFO] found job 17074836 in provider astro proc_lsst.multi INFO: found job 17074836 in provider astro 2024-03-15 14:05:59 proc_lsst.multi:201 [INFO] cancelling 17074836 on provider astro proc_lsst.multi INFO: cancelling 17074836 on provider astro 2024-03-15 14:05:59 proc_lsst.multi:146 [INFO] found job 17074837 in provider astro proc_lsst.multi INFO: found job 17074837 in provider astro 2024-03-15 14:05:59 proc_lsst.multi:201 [INFO] cancelling 17074837 on provider astro proc_lsst.multi INFO: cancelling 17074837 on provider astro 2024-03-15 14:06:00 proc_lsst.multi:146 [INFO] found job 17074838 in provider astro proc_lsst.multi INFO: found job 17074838 in provider astro 2024-03-15 14:06:00 proc_lsst.multi:201 [INFO] cancelling 17074838 on provider astro proc_lsst.multi INFO: cancelling 17074838 on provider astro 2024-03-15 14:06:00 proc_lsst.multi:146 [INFO] found job 17074839 in provider astro proc_lsst.multi INFO: found job 17074839 in provider astro 2024-03-15 14:06:00 proc_lsst.multi:201 [INFO] cancelling 17074839 on provider astro proc_lsst.multi INFO: cancelling 17074839 on provider astro 2024-03-15 14:06:00 proc_lsst.multi:146 [INFO] found job 17074841 in provider astro proc_lsst.multi INFO: found job 17074841 in provider astro 2024-03-15 14:06:00 proc_lsst.multi:201 [INFO] cancelling 17074841 on provider astro proc_lsst.multi INFO: cancelling 17074841 on provider astro 2024-03-15 14:06:01 proc_lsst.multi:146 [INFO] found job 17074842 in provider astro proc_lsst.multi INFO: found job 17074842 in provider astro 2024-03-15 14:06:01 proc_lsst.multi:201 [INFO] cancelling 17074842 on provider astro proc_lsst.multi INFO: cancelling 17074842 on provider astro 2024-03-15 14:06:01 proc_lsst.multi:146 [INFO] found job 17074843 in provider astro proc_lsst.multi INFO: found job 17074843 in provider astro 2024-03-15 14:06:01 proc_lsst.multi:201 [INFO] cancelling 17074843 on provider astro proc_lsst.multi INFO: cancelling 17074843 on provider astro 2024-03-15 14:06:02 proc_lsst.multi:146 [INFO] found job 17074850 in provider astro proc_lsst.multi INFO: found job 17074850 in provider astro 2024-03-15 14:06:02 proc_lsst.multi:201 [INFO] cancelling 17074850 on provider astro proc_lsst.multi INFO: cancelling 17074850 on provider astro 2024-03-15 14:06:02 proc_lsst.multi:146 [INFO] found job 17074851 in provider astro proc_lsst.multi INFO: found job 17074851 in provider astro 2024-03-15 14:06:02 proc_lsst.multi:201 [INFO] cancelling 17074851 on provider astro proc_lsst.multi INFO: cancelling 17074851 on provider astro 2024-03-15 14:06:03 proc_lsst.multi:146 [INFO] found job 17074852 in provider astro proc_lsst.multi INFO: found job 17074852 in provider astro 2024-03-15 14:06:03 proc_lsst.multi:201 [INFO] cancelling 17074852 on provider astro proc_lsst.multi INFO: cancelling 17074852 on provider astro 2024-03-15 14:06:03 proc_lsst.multi:146 [INFO] found job 17074853 in provider astro proc_lsst.multi INFO: found job 17074853 in provider astro 2024-03-15 14:06:03 proc_lsst.multi:201 [INFO] cancelling 17074853 on provider astro proc_lsst.multi INFO: cancelling 17074853 on provider astro 2024-03-15 14:06:03 proc_lsst.multi:146 [INFO] found job 17074854 in provider astro proc_lsst.multi INFO: found job 17074854 in provider astro 2024-03-15 14:06:03 proc_lsst.multi:201 [INFO] cancelling 17074854 on provider astro proc_lsst.multi INFO: cancelling 17074854 on provider astro 2024-03-15 14:06:04 proc_lsst.multi:146 [INFO] found job 17074855 in provider astro proc_lsst.multi INFO: found job 17074855 in provider astro 2024-03-15 14:06:04 proc_lsst.multi:201 [INFO] cancelling 17074855 on provider astro proc_lsst.multi INFO: cancelling 17074855 on provider astro 2024-03-15 14:06:04 proc_lsst.multi:146 [INFO] found job 17074856 in provider astro proc_lsst.multi INFO: found job 17074856 in provider astro 2024-03-15 14:06:04 proc_lsst.multi:201 [INFO] cancelling 17074856 on provider astro proc_lsst.multi INFO: cancelling 17074856 on provider astro 2024-03-15 14:06:05 proc_lsst.multi:146 [INFO] found job 17074857 in provider astro proc_lsst.multi INFO: found job 17074857 in provider astro 2024-03-15 14:06:05 proc_lsst.multi:201 [INFO] cancelling 17074857 on provider astro proc_lsst.multi INFO: cancelling 17074857 on provider astro 2024-03-15 14:06:05 proc_lsst.multi:146 [INFO] found job 17074859 in provider astro proc_lsst.multi INFO: found job 17074859 in provider astro 2024-03-15 14:06:05 proc_lsst.multi:201 [INFO] cancelling 17074859 on provider astro proc_lsst.multi INFO: cancelling 17074859 on provider astro 2024-03-15 14:06:05 proc_lsst.multi:146 [INFO] found job 17074861 in provider astro proc_lsst.multi INFO: found job 17074861 in provider astro 2024-03-15 14:06:05 proc_lsst.multi:201 [INFO] cancelling 17074861 on provider astro proc_lsst.multi INFO: cancelling 17074861 on provider astro 2024-03-15 14:06:06 proc_lsst.multi:146 [INFO] found job 17074863 in provider astro proc_lsst.multi INFO: found job 17074863 in provider astro 2024-03-15 14:06:06 proc_lsst.multi:201 [INFO] cancelling 17074863 on provider astro proc_lsst.multi INFO: cancelling 17074863 on provider astro 2024-03-15 14:06:06 proc_lsst.multi:146 [INFO] found job 17074865 in provider astro proc_lsst.multi INFO: found job 17074865 in provider astro 2024-03-15 14:06:06 proc_lsst.multi:201 [INFO] cancelling 17074865 on provider astro proc_lsst.multi INFO: cancelling 17074865 on provider astro 2024-03-15 14:06:07 proc_lsst.multi:146 [INFO] found job 17074869 in provider astro proc_lsst.multi INFO: found job 17074869 in provider astro 2024-03-15 14:06:07 proc_lsst.multi:201 [INFO] cancelling 17074869 on provider astro proc_lsst.multi INFO: cancelling 17074869 on provider astro 2024-03-15 14:06:07 proc_lsst.multi:146 [INFO] found job 17074870 in provider astro proc_lsst.multi INFO: found job 17074870 in provider astro 2024-03-15 14:06:07 proc_lsst.multi:201 [INFO] cancelling 17074870 on provider astro proc_lsst.multi INFO: cancelling 17074870 on provider astro 2024-03-15 14:06:08 proc_lsst.multi:146 [INFO] found job 17074871 in provider astro proc_lsst.multi INFO: found job 17074871 in provider astro 2024-03-15 14:06:08 proc_lsst.multi:201 [INFO] cancelling 17074871 on provider astro proc_lsst.multi INFO: cancelling 17074871 on provider astro 2024-03-15 14:06:08 proc_lsst.multi:146 [INFO] found job 17074872 in provider astro proc_lsst.multi INFO: found job 17074872 in provider astro 2024-03-15 14:06:08 proc_lsst.multi:201 [INFO] cancelling 17074872 on provider astro proc_lsst.multi INFO: cancelling 17074872 on provider astro 2024-03-15 14:06:08 proc_lsst.multi:146 [INFO] found job 17074873 in provider astro proc_lsst.multi INFO: found job 17074873 in provider astro 2024-03-15 14:06:08 proc_lsst.multi:201 [INFO] cancelling 17074873 on provider astro proc_lsst.multi INFO: cancelling 17074873 on provider astro 2024-03-15 14:06:09 proc_lsst.multi:146 [INFO] found job 17074876 in provider astro proc_lsst.multi INFO: found job 17074876 in provider astro 2024-03-15 14:06:09 proc_lsst.multi:201 [INFO] cancelling 17074876 on provider astro proc_lsst.multi INFO: cancelling 17074876 on provider astro 2024-03-15 14:06:09 proc_lsst.multi:146 [INFO] found job 17074877 in provider astro proc_lsst.multi INFO: found job 17074877 in provider astro 2024-03-15 14:06:09 proc_lsst.multi:201 [INFO] cancelling 17074877 on provider astro proc_lsst.multi INFO: cancelling 17074877 on provider astro 2024-03-15 14:06:10 proc_lsst.multi:146 [INFO] found job 17074879 in provider astro proc_lsst.multi INFO: found job 17074879 in provider astro 2024-03-15 14:06:10 proc_lsst.multi:201 [INFO] cancelling 17074879 on provider astro proc_lsst.multi INFO: cancelling 17074879 on provider astro 2024-03-15 14:06:10 proc_lsst.multi:146 [INFO] found job 17074894 in provider escience_a40 proc_lsst.multi INFO: found job 17074894 in provider escience_a40 2024-03-15 14:06:10 proc_lsst.multi:201 [INFO] cancelling 17074894 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074894 on provider escience_a40 2024-03-15 14:06:10 proc_lsst.multi:146 [INFO] found job 17074895 in provider escience_a40 proc_lsst.multi INFO: found job 17074895 in provider escience_a40 2024-03-15 14:06:10 proc_lsst.multi:201 [INFO] cancelling 17074895 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074895 on provider escience_a40 2024-03-15 14:06:11 proc_lsst.multi:146 [INFO] found job 17074897 in provider escience_a40 proc_lsst.multi INFO: found job 17074897 in provider escience_a40 2024-03-15 14:06:11 proc_lsst.multi:201 [INFO] cancelling 17074897 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074897 on provider escience_a40 2024-03-15 14:06:11 proc_lsst.multi:146 [INFO] found job 17074898 in provider escience_a40 proc_lsst.multi INFO: found job 17074898 in provider escience_a40 2024-03-15 14:06:11 proc_lsst.multi:201 [INFO] cancelling 17074898 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074898 on provider escience_a40 2024-03-15 14:06:12 proc_lsst.multi:146 [INFO] found job 17074932 in provider escience_a40 proc_lsst.multi INFO: found job 17074932 in provider escience_a40 2024-03-15 14:06:12 proc_lsst.multi:201 [INFO] cancelling 17074932 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074932 on provider escience_a40 2024-03-15 14:06:12 proc_lsst.multi:146 [INFO] found job 17074933 in provider escience_a40 proc_lsst.multi INFO: found job 17074933 in provider escience_a40 2024-03-15 14:06:12 proc_lsst.multi:201 [INFO] cancelling 17074933 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074933 on provider escience_a40 2024-03-15 14:06:12 proc_lsst.multi:146 [INFO] found job 17074934 in provider escience_a40 proc_lsst.multi INFO: found job 17074934 in provider escience_a40 2024-03-15 14:06:12 proc_lsst.multi:201 [INFO] cancelling 17074934 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074934 on provider escience_a40 2024-03-15 14:06:13 proc_lsst.multi:146 [INFO] found job 17074935 in provider escience_a40 proc_lsst.multi INFO: found job 17074935 in provider escience_a40 2024-03-15 14:06:13 proc_lsst.multi:201 [INFO] cancelling 17074935 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074935 on provider escience_a40 2024-03-15 14:06:13 proc_lsst.multi:146 [INFO] found job 17074936 in provider escience_a40 proc_lsst.multi INFO: found job 17074936 in provider escience_a40 2024-03-15 14:06:13 proc_lsst.multi:201 [INFO] cancelling 17074936 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074936 on provider escience_a40 2024-03-15 14:06:14 proc_lsst.multi:146 [INFO] found job 17074937 in provider escience_a40 proc_lsst.multi INFO: found job 17074937 in provider escience_a40 2024-03-15 14:06:14 proc_lsst.multi:201 [INFO] cancelling 17074937 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074937 on provider escience_a40 2024-03-15 14:06:14 proc_lsst.multi:146 [INFO] found job 17074938 in provider escience_a40 proc_lsst.multi INFO: found job 17074938 in provider escience_a40 2024-03-15 14:06:14 proc_lsst.multi:201 [INFO] cancelling 17074938 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074938 on provider escience_a40 2024-03-15 14:06:14 proc_lsst.multi:146 [INFO] found job 17074939 in provider escience_a40 proc_lsst.multi INFO: found job 17074939 in provider escience_a40 2024-03-15 14:06:14 proc_lsst.multi:201 [INFO] cancelling 17074939 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074939 on provider escience_a40 2024-03-15 14:06:15 proc_lsst.multi:146 [INFO] found job 17074940 in provider escience_a40 proc_lsst.multi INFO: found job 17074940 in provider escience_a40 2024-03-15 14:06:15 proc_lsst.multi:201 [INFO] cancelling 17074940 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074940 on provider escience_a40 2024-03-15 14:06:15 proc_lsst.multi:146 [INFO] found job 17074942 in provider escience_a40 proc_lsst.multi INFO: found job 17074942 in provider escience_a40 2024-03-15 14:06:15 proc_lsst.multi:201 [INFO] cancelling 17074942 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074942 on provider escience_a40 2024-03-15 14:06:16 proc_lsst.multi:146 [INFO] found job 17074974 in provider escience_a40 proc_lsst.multi INFO: found job 17074974 in provider escience_a40 2024-03-15 14:06:16 proc_lsst.multi:201 [INFO] cancelling 17074974 on provider escience_a40 proc_lsst.multi INFO: cancelling 17074974 on provider escience_a40 2024-03-15 14:06:16 proc_lsst.multi:146 [INFO] found job 17075003 in provider escience_a40 proc_lsst.multi INFO: found job 17075003 in provider escience_a40 2024-03-15 14:06:16 proc_lsst.multi:201 [INFO] cancelling 17075003 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075003 on provider escience_a40 2024-03-15 14:06:16 proc_lsst.multi:146 [INFO] found job 17075018 in provider escience_a40 proc_lsst.multi INFO: found job 17075018 in provider escience_a40 2024-03-15 14:06:16 proc_lsst.multi:201 [INFO] cancelling 17075018 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075018 on provider escience_a40 2024-03-15 14:06:17 proc_lsst.multi:146 [INFO] found job 17075019 in provider escience_a40 proc_lsst.multi INFO: found job 17075019 in provider escience_a40 2024-03-15 14:06:17 proc_lsst.multi:201 [INFO] cancelling 17075019 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075019 on provider escience_a40 2024-03-15 14:06:17 proc_lsst.multi:146 [INFO] found job 17075020 in provider escience_a40 proc_lsst.multi INFO: found job 17075020 in provider escience_a40 2024-03-15 14:06:17 proc_lsst.multi:201 [INFO] cancelling 17075020 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075020 on provider escience_a40 2024-03-15 14:06:18 proc_lsst.multi:146 [INFO] found job 17075021 in provider escience_a40 proc_lsst.multi INFO: found job 17075021 in provider escience_a40 2024-03-15 14:06:18 proc_lsst.multi:201 [INFO] cancelling 17075021 on provider escience_a40 proc_lsst.multi INFO: cancelling 17075021 on provider escience_a40 2024-03-15 14:06:18 proc_lsst.multi:146 [INFO] found job 17074880 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074880 in provider escience_rtx6k 2024-03-15 14:06:18 proc_lsst.multi:201 [INFO] cancelling 17074880 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074880 on provider escience_rtx6k 2024-03-15 14:06:19 proc_lsst.multi:146 [INFO] found job 17074881 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074881 in provider escience_rtx6k 2024-03-15 14:06:19 proc_lsst.multi:201 [INFO] cancelling 17074881 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074881 on provider escience_rtx6k 2024-03-15 14:06:19 proc_lsst.multi:146 [INFO] found job 17074887 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074887 in provider escience_rtx6k 2024-03-15 14:06:19 proc_lsst.multi:201 [INFO] cancelling 17074887 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074887 on provider escience_rtx6k 2024-03-15 14:06:19 proc_lsst.multi:146 [INFO] found job 17074888 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074888 in provider escience_rtx6k 2024-03-15 14:06:19 proc_lsst.multi:201 [INFO] cancelling 17074888 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074888 on provider escience_rtx6k 2024-03-15 14:06:20 proc_lsst.multi:146 [INFO] found job 17074889 in provider escience_rtx6k proc_lsst.multi INFO: found job 17074889 in provider escience_rtx6k 2024-03-15 14:06:20 proc_lsst.multi:201 [INFO] cancelling 17074889 on provider escience_rtx6k proc_lsst.multi INFO: cancelling 17074889 on provider escience_rtx6k 2024-03-15 14:06:20 proc_lsst.multi:146 [INFO] found job 91028 in provider local proc_lsst.multi INFO: found job 91028 in provider local 2024-03-15 14:06:20 proc_lsst.multi:201 [INFO] cancelling 91028 on provider local proc_lsst.multi INFO: cancelling 91028 on provider local parsl.providers.local.local DEBUG: Terminating job/proc_id: 91028 parsl.providers.local.local WARNING: Failed to kill PID: 91028 and child processes on local 2024-03-15 14:06:21 proc_lsst.multi:50 [INFO] no new jobs since last cancel, resuming executor shutdown proc_lsst.multi INFO: no new jobs since last cancel, resuming executor shutdown parsl.executors.high_throughput.executor INFO: Attempting HighThroughputExecutor shutdown parsl.executors.high_throughput.executor INFO: Finished HighThroughputExecutor shutdown attempt parsl.dataflow.dflow INFO: Shut down executor multi parsl.dataflow.dflow INFO: Shutting down executor _parsl_internal parsl.executors.threads DEBUG: Shutting down executor, which involves waiting for running tasks to complete parsl.executors.threads DEBUG: Done with executor shutdown parsl.dataflow.dflow INFO: Shut down executor _parsl_internal parsl.dataflow.dflow INFO: Terminated executors parsl.dataflow.dflow INFO: DFK cleanup complete parsl.process_loggers DEBUG: Normal ending for cleanup on thread MainThread lsst.ctrl.bps.submit INFO: Completed submitting to a workflow management system: Took 1076.3012 seconds lsst.ctrl.bps.drivers INFO: Run 'DEEP_20210513_A1c_science#step0_20240315T134820Z' submitted for execution with id 'None' lsst.ctrl.bps.drivers INFO: Completed submit stage: Took 1076.3338 seconds; current memory usage: 0.371 Gibyte, delta: 0.030 Gibyte, peak delta: 0.030 Gibyte lsst.ctrl.bps.drivers INFO: Completed entire submission process: Took 1105.1558 seconds; current memory usage: 0.371 Gibyte, delta: 0.234 Gibyte, peak delta: 0.234 Gibyte lsst.ctrl.bps.drivers INFO: Peak memory usage for bps process 0.371 Gibyte (main), 0.371 Gibyte (largest child process) Run Id: None Run Name: DEEP_20210513_A1c_science#step0_20240315T134820Z parsl.dataflow.dflow INFO: python process is exiting, but DFK has already been cleaned up real 18m28.574s user 4m23.070s sys 2m8.234s